Senior \/ Lead Hadoop Engineer

1 week ago

Charlotte, United States Compunnel Full time
Job Summary

We are seeking a highly experienced Senior / Lead Hadoop Engineer to lead complex technology initiatives focused on maintaining platform stability, automating monitoring, and providing 24/7 support for Hadoop clusters. The ideal candidate will have extensive experience in Hadoop administration, Unix scripting, and incident management. This role involves managing and optimizing the Hadoop ecosystem, coordinating with platform tenants and vendors, and ensuring platform resilience through backup, recovery, and patch management.

Key Responsibilities

Platform Stability and Incident Management:

• Lead complex technology initiatives focused on Hadoop platform stability and automation.

• Manage and troubleshoot platform incidents, serving as part of a 24/7 platform support team.

• Assess the impact of platform issues, prioritize triaging efforts, and partner with tenants, developers, and vendors.

• Perform root cause analysis of incidents and implement permanent solutions.

• Communicate progress and status updates to stakeholders and leadership.

Hadoop Cluster Monitoring and Troubleshooting:

• Monitor Hadoop cluster health and performance using tools such as Prometheus, Splunk, and Thousand Eyes.

• Automate monitoring opportunities and fine-tune alert systems for optimal performance.

• Use Autosys or similar scheduling tools to manage platform activities.

• Troubleshoot tenant-reported failures by analyzing logs to identify root causes.

Hadoop Configuration and Tuning:

• Periodically configure and tune Hadoop components (HDFS, YARN, Hive, HBase, Spark, HPOS).

• Schedule and track change requests for implementation.

• Collaborate with tenants and operating system administrators to manage patches and upgrades.

Backup, Recovery, and Patching Management:

• Ensure platform resilience through robust data backup and recovery strategies.

• Execute failover to disaster recovery sites and manage fallback processes.

• Plan and perform upgrades and patch management of Hadoop components.

• Validate cluster health post-patching and communicate with stakeholders.

• Identify automation opportunities to enhance patching processes.

Documentation and Reporting:

• Maintain detailed documentation of cluster configurations, troubleshooting processes, and procedures.

• Generate reports on cluster usage, performance metrics, and capacity utilization.

Collaboration and Support:

• Collaborate with platform tenants, development teams, and Hadoop vendors to troubleshoot complex issues.

• Work independently and in coordination with key technical experts to ensure production stability.

Required Qualifications:

• 8+ years of experience in technology incident and problem management.

• 4+ years of experience using monitoring tools (Prometheus, Splunk, SiteScope, Thousand Eyes).

• 6+ years of experience with Unix scripting.

• 2+ years of experience in Hadoop administration or a similar role.

• Strong knowledge of the Hadoop ecosystem and its components (Hive, HBase, Spark, HPOS).

• Experience with Autosys or a similar job scheduling tool.

Preferred Qualifications:

• Excellent problem-solving and troubleshooting skills.

• Strong communication and collaboration skills for working with cross-functional teams.

• Ability to work both independently and as part of a collaborative team.

Education: Bachelors Degree

  • Senior / Lead Hadoop Engineer

    1 week ago

    Charlotte, United States JobRialto Full time

    Job Summary We are seeking a highly experienced Senior / Lead Hadoop Engineer to lead complex technology initiatives focused on maintaining platform stability, automating monitoring, and providing 24/7 support for Hadoop clusters. The ideal candidate will have extensive experience in Hadoop administration, Unix scripting, and incident management. This role...

  • Senior / Lead Hadoop Engineer

    2 months ago

    Charlotte, United States NTT DATA Full time

    Req ID: 297836NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior / Lead Hadoop Engineer to join our team in Charlotte, North Carolina (US-NC), United States (US).Job Duties and...

  • Hadoop Technical Lead

    1 week ago

    Charlotte, North Carolina, United States JobRialto Full time

    Senior Hadoop Engineer Job SummaryJobRialto is seeking a highly experienced Senior Hadoop Engineer to lead complex technology initiatives focused on maintaining platform stability, automating monitoring, and providing 24/7 support for Hadoop clusters.Key ResponsibilitiesPlatform Stability and Incident Management:Lead complex technology initiatives focused on...

  • Senior Data Engineer

    5 days ago

    Charlotte, North Carolina, United States ApTask Full time

    About This Role:Candidates for the Senior Data Engineer - Big Data Hadoop position at ApTask will be responsible for designing, developing, and implementing large-scale data processing systems using Big Data Hadoop. The ideal candidate will have experience working with big data technologies, including Hadoop, PySpark, and Teradata, and will be able to...

  • Hadoop Developer

    1 month ago

    Charlotte, United States Collabera Full time

    About Collabera:Collabera is a leading global technology services and solutions provider committed to delivering high-quality, innovative solutions to our clients. Our diverse, global talent helps clients transform every aspect of their business and achieve exceptional results. We achieve success through collaboration and the use of our digital platforms....

  • Hadoop Developer

    4 weeks ago

    charlotte, United States Collabera Full time

    About Collabera:Collabera is a leading global technology services and solutions provider committed to delivering high-quality, innovative solutions to our clients. Our diverse, global talent helps clients transform every aspect of their business and achieve exceptional results. We achieve success through collaboration and the use of our digital platforms....

  • Hadoop Developer

    1 month ago

    charlotte, United States Collabera Full time

    About Collabera:Collabera is a leading global technology services and solutions provider committed to delivering high-quality, innovative solutions to our clients. Our diverse, global talent helps clients transform every aspect of their business and achieve exceptional results. We achieve success through collaboration and the use of our digital platforms....

  • Hadoop Developer

    5 days ago

    Charlotte, United States Apex Systems Full time

    Apex Systems is seeking a Hadoop Developer in Charlotte, NC for a client. If interested, please apply with your updated resume and a recruiter will reach out. Thank you!Must Haves:5+ years experience as a senior hands on Hadoop DeveloperPython, Spark, Spark SQL, PySpark Experience with MapRFamiliarity with containerization tools like Kubernetes and OpenShift

  • Big Data Hadoop Engineer

    7 days ago

    Charlotte, United States Aloden, Inc. Full time

    Technology and Data - Software Engineer 4 - Contingent (Big Data Hadoop Engineer) Location:Charlotte, NC - 28202 - Hybrid Roles (3 Days Onsite/2 DaysWFH) Job Descriptions: In this contingent resource assignment, you may: Consult on complex initiatives with broad impact and large-scale planning for Software Engineering. Review and analyze complex...

  • Big Data Hadoop Engineer

    1 week ago

    Charlotte, United States Aloden, Inc. Full time

    Technology and Data - Software Engineer 4 - Contingent (Big Data Hadoop Engineer) Location:Charlotte, NC - 28202 - Hybrid Roles (3 Days Onsite/2 DaysWFH) Job Descriptions: In this contingent resource assignment, you may: Consult on complex initiatives with broad impact and large-scale planning for Software Engineering. Review and analyze complex...

  • Big Data Hadoop Engineer

    4 weeks ago

    Charlotte, United States Aloden, Inc. Full time

    Technology and Data - Software Engineer 4 - Contingent (Big Data Hadoop Engineer) Location:Charlotte, NC - 28202 - Hybrid Roles (3 Days Onsite/2 DaysWFH) Job Descriptions: In this contingent resource assignment, you may: Consult on complex initiatives with broad impact and large-scale planning for Software Engineering. Review and analyze complex...

  • Senior Hadoop/Spark Developer Expert

    1 week ago

    Charlotte, North Carolina, United States CapB InfoteK Full time

    Job Title: Ab Initio Application Consultant Lead - Hadoop/Spark-ScalaAbout the Job:CapB InfoteK is seeking a talented Application Consultant Lead - Hadoop/Spark-Scala to join our team for one of our long-term, multi-year projects. If you have experience with Ab Initio and Hadoop and possess strong problem-solving skills, we encourage you to apply.Primary...

  • Hadoop Developer

    5 days ago

    Charlotte, United States Infosys Full time

    Job Title: Hadoop and Spark Lead DeveloperLocations: Charlotte, NC or Plano, TX or Jersey City, NJDuration: FulltimeSkills: Hadoop, Spark, Scala / Python, SQL, Unix Shell Scripting

  • Hadoop Developer

    6 days ago

    Charlotte, United States Experis Full time

    Position Title: Hadoop Developer Start Date: ASAP On-Site Location(s): Charlotte, NC On-Site Schedule: Hybrid # Of Openings: 1 Line of Business: Consumer Technology Contract Duration: 12-24 months (strong likelihood to convert to full-time) Candidate Citizenship Options: USC or GC - NO C2C *NO C2C* Group’s responsibility/Current Project and...

  • Hadoop Developer

    4 weeks ago

    Charlotte, NC, United States Collabera Full time

    About Collabera:Collabera is a leading global technology services and solutions provider committed to delivering high-quality, innovative solutions to our clients. Our diverse, global talent helps clients transform every aspect of their business and achieve exceptional results. We achieve success through collaboration and the use of our digital platforms....

  • Hadoop Developer

    2 days ago

    Charlotte, United States Motion Recruitment Partners LLC Full time

    Hadoop Developer Charlotte, NC Hybrid Contract $69.5/hr - $78.31/hr Outstanding long-term contract opportunity! A well-known Financial Services Company is looking for a Software Engineer in Charlotte, NC (Hybrid). Work with the brightest minds at one of the largest financial institutions in the world. This is a long-term contract opportunity that...

  • Senior Data Engineering Manager

    1 week ago

    Charlotte, North Carolina, United States TechnoGen Full time

    Job Title: Senior Data Engineering ManagerJob Summary:We are seeking a Senior Data Engineering Manager to lead our data engineering team at TechnoGen. The ideal candidate will have 12+ years of experience in data engineering and a strong background in SQL, Python, and Spark programming languages.Key Responsibilities:Lead multiple teams and drive technical...

  • Lead Hadoop/Spark-Scala

    5 months ago

    Charlotte, United States CapB InfoteK Full time

    For one of our long-term multiyear projects, we are looking for an Application Consultant - Lead Hadoop/Spark-Scala out of Charlotte, NC. Primary Skill- Ab Initio Secondary Skill - Hadoop 7+ experience with Spark/Scala, Hadoop, Java Good Knowledge in ETL Tools (Abinitio/talend/informatica) and Scheduling tools (Autosys preferred) Good Knowledge in Hadoop...

  • Java Developer w/ Hadoop

    1 week ago

    Charlotte, United States TechnoGen Full time

    Java Developer w/ Hadoop experienceCharlotte, NC (onsite 3x a week) - Local candidates only1 year + contractPay rate: Market /Flexible Client: Bank Top Skills • 5+ years of hands-on software development experience • Experience building web applications using J2EE technologies and frameworks like Spring • Experience building RESTful and consuming...

  • Senior Technology and Data Engineer

    7 days ago

    Charlotte, North Carolina, United States Aloden, Inc. Full time

    Job Title: Senior Technology and Data EngineerJob Summary:We are seeking a highly skilled Senior Technology and Data Engineer to join our team at Aloden, Inc. As a key member of our engineering team, you will be responsible for developing and implementing innovative technology solutions to drive business growth and improve operational efficiency.Key...