Senior Hadoop Platform Engineer- W2

1 week ago

Minnesota, United States Novia Infotech Full time

Must have

  • Apache Hadoop
  • Apache Hive
  • Apache Spark
  • Hadoop Distributed File System (HDFS)
  • Java
  • Linux

Nice To Have

  • Grafana
  • Python
  • trino

Role is specifically a Hadoop Platform Engineer

Engineer will have deep, hands-on experience across the Apache Hadoop ecosystem, focused on building and maintaining a reliable, scalable data platform; strong core Hadoop and Java expertise, enabling them to diagnose, optimize, and tune performance at the cluster and infrastructure levels. This role will also drive platform observability improvements, including standardizing monitoring, implementing health checks, and developing automated alerting systems—all to proactively identify and resolve issues before they impact users.

Description:

  • Design, build, and maintain a reliable, scalable, and high-performance Hadoop platform that supports large-scale data processing and analytics workloads.
  • Diagnose and optimize cluster-level performance issues across core Apache Hadoop components (HDFS, YARN, MapReduce, Hive, Spark, HBase) using deep Hadoop and Java expertise.
  • Develop and standardize monitoring and observability frameworks for the Hadoop ecosystem, ensuring proactive detection of system health issues.
  • Leverage strong Linux expertise (especially Ubuntu) to analyze system bottlenecks, perform kernel tuning, and optimize resource allocation for Hadoop workloads.
  • Implement automated health checks and alerting systems to reduce operational noise and minimize reliance on user-reported issues.
  • Collaborate with data engineering, platform, and infrastructure teams to tune resource utilization, improve job efficiency, and ensure cluster stability.
  • Establish and enforce operational standards for performance tuning, capacity planning, and version upgrades across the Hadoop platform.
  • Automate repetitive operational tasks and improve workflow efficiency using scripting languages (Python, Bash, or Scala)
  • Maintain and enhance data security, governance, and compliance practices within the Hadoop ecosystem.
  • Drive root cause analysis (RCA) and develop preventive measures for recurring production incidents.
  • Document best practices, operational runbooks, and configuration standards to ensure knowledge sharing and consistent platform management.

  • System Engineer

    1 week ago

    Minnesota, United States ITR Group Full time

    Position SummaryThe Systems Engineer is a senior-level engineer responsible for designing, building, and optimizing cloud-native platform services with an emphasis on AWS. This role involves deep hands-on engineering within AWS, automation development, API creation, and containerized workload orchestration.The ideal candidate brings strong cloud engineering...

  • Software Engineer

    7 days ago

    Minnesota, United States Hollstadt Consulting Full time

    Minnesota - DeveloperHollstadt OverviewHollstadt Consulting is a management and technology consulting firm dedicated to placing professionals at engagements where they will excel. When you work with us, you'll work with a refreshingly real company led and staffed by seasoned experts who are also down-to-earth, good people. We're committed to treating you...

  • Senior Product Development and Support Engineer

    7 days ago

    Minnesota, United States 3M Full time

    Job DescriptionSenior Product Development and Support EngineerCollaborate with Innovative 3Mers Around the WorldChoosing where to start and grow your career has a major impact on your professional and personal life, so it's equally important you know that the company that you choose to work at, and its leaders, will support and guide you. With a wide variety...

  • Senior Application Security Engineer

    2 weeks ago

    Minnesota, United States Greenfix Property Care Full time

    The pay range is $95, $171,000.00Pay is based on several factors which vary based on position. These include labor markets and in some instances may include education, work experience and certifications. In addition to your pay, Target cares about and invests in you as a team member, so that you can take care of yourself and your family. Target offers...

  • Senior Technical Product Owner

    2 weeks ago

    Minnesota, United States Hollstadt Consulting Full time

    Minnesota - Product OwnerHollstadt OverviewHollstadt Consulting is a management and technology consulting firm dedicated to placing professionals at engagements where they will excel. When you work with us, you'll work with a refreshingly real company led and staffed by seasoned experts who are also down-to-earth, good people. We're committed to treating you...

  • Software Engineer III

    2 weeks ago

    Minnesota, United States Hollstadt Consulting Full time

    Minnesota - DeveloperHollstadt OverviewHollstadt Consulting is a management and technology consulting firm dedicated to placing professionals at engagements where they will excel. When you work with us, you'll work with a refreshingly real company led and staffed by seasoned experts who are also down-to-earth, good people. We're committed to treating you...

  • Application Development Manager

    2 weeks ago

    Minnesota, United States TriCom Technical Services Full time

    Application Manager We are seeking someone who can lead two high-performing application development teams, providing people-first leadership while guiding the delivery of web/mobile and data integration solutions. This role is accountable for coaching and developing a seasoned group of engineers, owning team capacity and planning, and serving as a key...

  • Data Engineer

    1 week ago

    Minnesota, United States OM Housing Full time

    OverviewWe are seeking a highly capable Data Engineer to join our team and support enterprise-level data initiatives. The ideal candidate demonstrates strong technical expertise, analytical problem-solving abilities, and the ability to collaborate effectively with both technical and non-technical stakeholders.Position: Data EngineerLocation: Remote MN locals...

  • Sr. Software Engineer

    2 weeks ago

    Minnesota, United States Planet Pharma Full time

    Hybrid – Mounds View, MN(3 days/week onsite)12-Month Contract(Potential to Extend or Convert)We're seeking an experiencedSr. Software Engineerto join a high-performing R&D team supporting complex, regulated products that improve quality and reliability through advanced software testing and automation. The ideal candidate will have a strong background...

  • Senior Software Engineer

    2 weeks ago

    Minnesota, United States OCLC Full time

    Together we make breakthroughs possible.  At OCLC, we build technology with a purpose: to connect libraries and make knowledge accessible worldwide, because we believe that what is known must be shared. Our teams work with complex global datasets, AI and machine learning, hybrid cloud solutions, and other technologies that connect people and organizations...