Hadoop PySpark Technical Lead

4 weeks ago


Atlanta, Georgia, United States Diverse Lynx Full time
Job Summary

As a Hadoop PySpark Technical Lead at Diverse Lynx LLC, you will be responsible for designing, developing, and scheduling data pipelines that handle large volumes of data within Service Level Agreements (SLAs).

Key Responsibilities

• Understand requirements and use cases to build efficient ETL solutions using Apache Spark, Python, and Kafka targeting Cloudera Data Platform.
• Analyze requirements and convert functional requirements into concrete technical tasks, providing reasonable effort estimates.
• Work closely with data analysts, modelers, and business users to understand data requirements and convert them into high-level and low-level designs and source-to-target documents.
• Design, develop, and schedule data pipelines that handle large volumes of data within SLAs, working with solution architects, technical managers, and admins to understand SLAs and limitations of systems.
• Expertise in processing large volumes of data aggregation using Spark, with knowledge of different performance improvement techniques and the ability to lead teams on optimization.
• Develop efficient data ingestion and data governance frameworks as per specification.
• Improve the performance of existing Spark-based data ingestion and aggregation pipelines to meet SLAs.

Requirements

• 10+ years of experience in Hadoop PySpark development and data engineering.
• Expertise in Cloudera Data Platform, Apache Spark, Python, and Kafka.
• Strong understanding of data pipelines, ETL solutions, and data governance frameworks.
• Ability to work independently and proactively with global teams to address project requirements and articulate issues and challenges with enough lead time to address project delivery risks.

About Diverse Lynx LLC

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence, and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.

  • Atlanta, Georgia, United States Diverse Lynx Full time

    Data Engineering Role:Hadoop PySpark Technical Lead required to manage and develop data pipelines and ETL solutions using Apache Spark, Python, and Kafka targeting Cloudera Data Platform.Key Responsibilities:Design, develop, and schedule data pipelines that handle large volumes of data within SLA.Work with Data Analysts, Business Users, and Solution...


  • Atlanta, Georgia, United States JobRialto Full time

    Job Summary: JobRialto is seeking an experienced Data Engineer with a strong focus on PySpark and Graph Databases (Neo4j, Neptune DB, or similar). The ideal candidate will have extensive experience in data engineering, development, and working with business and IT stakeholders. This role requires proficiency in big data technologies, cloud platforms, and...

  • Cloud Data Architect

    4 weeks ago


    Atlanta, Georgia, United States JobRialto Full time

    We are seeking an experienced Data Engineer with a strong focus on PySpark and Graph Databases (Neo4j, Neptune DB, or similar) to join JobRialto.Key Responsibilities:Collaborate with business and IT stakeholders to understand requirements and deliver solutions.Design and implement data models, database architecture, and schema design.Write and debug Python...

  • Data Engineer

    4 weeks ago


    Atlanta, Georgia, United States JobRialto Full time

    Job Summary:We are seeking an experienced Data Engineer with a strong focus on PySpark and Graph Databases (Neo4j, Neptune DB, or similar) to join our team at JobRialto.The ideal candidate will have extensive experience in data engineering, development, and working with business and IT stakeholders. This role requires proficiency in big data technologies,...

  • Senior Data Engineer

    4 weeks ago


    Atlanta, Georgia, United States BCG X Full time

    Job OverviewBCG X is seeking a Senior Data Engineer to lead the design, development, and operationalization of cutting-edge cloud-native big data solutions that enable innovative AI models and analytics.This role is a permanent position with one of our clients within the insurance industry. You will NOT be employed by BCG.As a Senior Data Engineer, you will...


  • Atlanta, Georgia, United States Bank of America Full time

    Job Description:At Bank of America, we are guided by a common purpose to help make financial lives better through the power of every connection. Our team is responsible for developing and maintaining models that identify potentially suspicious behaviors for further evaluation by our in-house investigation teams. This area has a unique appetite for a wide...

  • Data Architect

    1 month ago


    Atlanta, Georgia, United States NTT DATA Full time

    About the Role:We are seeking a highly skilled Data Architect to join our team at NTT DATA. As a Data Architect, you will be responsible for conceptualizing and creating an end-to-end vision for the seamless flow of data through successive stages.Key Responsibilities:Design and implement data solutions tailored to customer needs, including data ingestion,...


  • Atlanta, Georgia, United States TechnoGen Full time

    Job Description:As a Senior Salesforce Developer at TechnoGen, you will lead technical requirements sessions, architect and document technical solutions, and develop on the platform. You will design and develop interfaces between Salesforce and other systems, manage client expectations, and scope. This position acts as a liaison between the client and...

  • Senior Technical Lead

    4 weeks ago


    Atlanta, Georgia, United States Nagarro Full time

    Job Title: Senior Technical LeadJob Summary:Nagarro is seeking a highly skilled Senior Technical Lead to lead AEM Forms and AEM Sites projects. The ideal candidate will have a strong understanding of AEM architecture and components, as well as proficiency in Java, JavaScript, HTML, CSS, and related web technologies.Key Responsibilities:• Understand client...


  • Atlanta, Georgia, United States QuEST Global Services Pte. Ltd Full time

    Job Title: Technical Implementation LeadJob Summary:We are seeking a highly experienced Technical Implementation Lead to join our team at QuEST Global Services Pte. Ltd. As a Technical Implementation Lead, you will be responsible for supporting the preparation of information, knowledge, and experiences within the team and across departments.Key...

  • ITSM Technical Lead

    4 weeks ago


    Atlanta, Georgia, United States Bose Full time

    At Bose, we're committed to delivering valuable and reliable business and technology solutions. As an ITSM Technical Lead, you'll play a key role in driving this mission forward.Key Responsibilities:Engage with stakeholders to understand business needs and translate them into technical requirements.Lead requirement gathering sessions to capture functional...


  • Atlanta, Georgia, United States Centraprise Full time

    Job Title: Technical Product OwnerJob Summary:Centraprise is seeking a skilled Technical Product Owner to lead the development of our software products. The ideal candidate will have Airlines Industry experience and a strong understanding of Agile methodologies.Key Responsibilities:Provide clear documentation of progress and deliverables, including ownership...


  • Atlanta, Georgia, United States UPS Full time

    Job DescriptionThe Senior Software Development Manager is responsible for leading the design, development, and maintenance of software applications that provide business capabilities and solutions. This role requires a deep understanding of software engineering principles and the ability to apply them to deliver technical solutions on time and within...


  • Atlanta, Georgia, United States Kalos Consulting Full time

    About the RoleWe are seeking a highly skilled Healthcare Data Analytics Lead to join our team at Kalos Consulting. As a key member of our strategic analytics team, you will be responsible for developing and implementing data-driven solutions to drive business growth and improve healthcare outcomes.Key ResponsibilitiesLead the development of data analytics...


  • Atlanta, Georgia, United States Metric5 Full time

    Job Title: PeopleSoft Technical Team LeadWe are seeking a highly skilled PeopleSoft Technical Team Lead with extensive experience in PeopleSoft HCM 9.2. The ideal candidate will have a strong technical background, a deep understanding of PeopleSoft architecture, and the ability to lead technical teams in designing, developing, and supporting complex...


  • Atlanta, Georgia, United States Omni Inclusive Full time

    Job SummaryAt Omni Inclusive, we are seeking a highly skilled Senior Android Technical Lead to join our team. The ideal candidate will have 10+ years of experience in Android development, preferably in the healthcare domain. As a technical lead, you will be responsible for providing guidance to a team of developers, enhancing their technical capabilities,...


  • Atlanta, Georgia, United States Centraprise Full time

    Job Title: Technical Product OwnerLocation: Atlanta, GAJob Type: FulltimeJob Description: Must have Airlines Industry experienceAt Centraprise, we are seeking a skilled Technical Product Owner to lead our software development projects. The ideal candidate will have a strong background in Agile methodologies and experience working with business stakeholders....


  • Atlanta, Georgia, United States Centegix Full time

    Job OverviewCentegix is the industry leader in wearable safety technology for healthcare, education, government, and commercial workplaces. As a Technical Support Team Lead, you will play a critical role in ensuring our customers' technical issues are resolved quickly and accurately.Key Responsibilities:Customer Service: Manage customer issues with accuracy,...


  • Atlanta, Georgia, United States TEPHRA Full time

    Job Title: Workday Integrations Technical LeadLocation:Atlanta, GA.Number of positions:OneResponsibilities:We are seeking a Workday Integrations Technical Lead with experience in successful end-to-end medium to large implementations of Workday integrations. This individual will lead a team of Integration developers to mentor and deliver. As and when...


  • Atlanta, Georgia, United States Janus International Group Full time

    Job Overview We are seeking a skilled and experienced Lead Technical Architect for Frontend Development to join our team at Janus International Group. This is a challenging and rewarding opportunity for a seasoned professional to lead our frontend development efforts and drive the success of our web, mobile, and embedded software applications. Key...