Lead Data Engineer

2 weeks ago

Charlotte, NC, United States Diverse Lynx Full time
Job Description
Role: Lead Data Engineer
Location: Onsite/Charlotte, NC(4 days onsite)

Who are we looking for?
We're seeking an experienced data engineer to analyze large volume of different data set, identify the data gaps and address the same. Build automated future ready data reconciliation process in-place.
Technical Skills:
  • Lead the Analysis of Data Sources, ETL Processes, and Databases. Identify data and process gaps, and recommend data remediation and long-term process improvements.
  • Utilize deep expertise in various ETL tools (e.g., IBM DataStage), RDBMS (e.g., DB2, PostgreSQL, etc.), and different data formats (e.g., Complex XML, JSON, CSV) to drive analysis.
  • Leverage expertise in understanding and testing REST/SOAP Integrations.
  • Apply Python expertise to build tools for comparing data between heterogeneous sources [CSV, XML, JSON, RDBMS etc...] and automating the data validation process.
  • Demonstrate Strong Business Acumen and Stakeholder Communication Skills.
  • Translate complex business challenges into clear, data-driven solutions, effectively communicating technical concepts and project progress to both technical and non-technical stakeholders, including senior leadership.
  • Act as a key liaison between the data team and business units, providing data-backed insights and recommendations that directly influence business strategy.
  • Manage concurrent projects in a dynamic, research-oriented environment, ensuring timely delivery and high-quality outcomes.
  • Experience in the insurance domain is preferable.
  • AWS Data Engineer certification is good to have.
Key Responsibilities
  • Collaborate with business units and stakeholders to document business needs, processes, reporting requirements, and challenges.
  • Lead the design and development of robust data pipelines, utilizing deep understanding of existing ETL frameworks to build scalable and efficient solutions.
  • Analyze and understand diverse source systems and data formats to create and validate synthetic datasets for testing and development.
  • Develop automated data validation tools using Python to ensure data integrity across different datasets and generate comparison output reports.
  • Ensure industry best practices are followed in all aspects of the project.
  • Develop a robust development testing approach to minimize quality, integration, and user testing issues.
  • Leverage generative AI to enhance data analysis, accelerate development, streamline unit testing, and improve documentation.
  • Apply statistical analysis to large datasets, uncovering key patterns, identifying potential challenges, and generating actionable insights.
  • Partner with business leaders to understand their challenges and provide data-driven recommendations that improve processes and inform strategic decisions.

Qualification:
  • Somebody who has at least 12+ years of data engineering experience has played Lead Data Engineer role.
  • Bachelor's degree (or equivalent) in computer science, information technology, engineering, or related discipline
  • Education qualification: Any degree from a reputed college


Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.

  • Lead snowflake data engineer

    2 weeks ago

    Charlotte, NC, United States USM Full time

    Start Date: Interview Types Skills Snowflake Data Engin.. Visa Types Green Card, US Citiz.. I have an opening for a Lead Snowflake Data Engineer. Hybrid on site in Hartford, NC or Charlotte, NC Long term Must be on TEKsystems W2.. total pay is 75-78 per hour Need to be very senior level Data Engineer and need to have focused on snowflake the last...

  • Sr. Data Engineer

    2 weeks ago

    Charlotte, NC, United States Veracity Full time

    Job Title: Sr. Data Engineer Duration: 12+ Months Location: Charlotte, NC - Hybrid Role (3 days onsite in a week) Manager Notes: • Team Duties and business impact: The Risk Data Services is a horizontal function within Risk Technology organization and is responsible for delivering data consistently across Risk. The Risk Data Services team is seeking a Lead...

  • Staff Data Engineer

    2 days ago

    Charlotte, NC, United States Red Ventures Full time

    *Must be able to work Eastern Standard Time hours.This role is not open to visa sponsorship or transfer of visa sponsorship including those on OPT and STEM-EXT OPT, nor is it available to work corp-to-corp. Bankrate.com is looking for a Staff Data Engineer to shape the next generation of our data platform-powering analytics and insights that help millions...

  • Big Data Platform Engineer

    2 weeks ago

    Charlotte, NC, United States Veracity Full time

    Job Title: Big Data Platform Engineer Duration: 12 Months Location: Charlotte, NC - Hybrid RoleJob Summary: We are seeking a highly skilled and motivated Tools Administration & Engineering Specialist to join our Data private cloud. This role involves working closely with other technical staff to manage, support, and engineer platform tools across various big...

  • Big Data Platform Engineer

    1 week ago

    Charlotte, NC, United States Veracity Full time

    Job Title: Big Data Platform Engineer Duration: 12 Months Location: Charlotte, NC - Hybrid RoleJob Summary: We are seeking a highly skilled and motivated Tools Administration & Engineering Specialist to join our Data private cloud. This role involves working closely with other technical staff to manage, support, and engineer platform tools across various big...

  • AWS Data Engineer

    2 weeks ago

    Charlotte, NC, United States LSEG (London Stock Exchange Group) Full time

    Ready to build scalable data systems with real-world impact? Join us at FTSE Russell! We're establishing a team of engineers passionate about data, software craftsmanship, and substantial impact. You'll help to power the indices trusted by financial institutions worldwide. You'll work in an AWS environment, build robust pipelines, and work with product and...

  • AWS Data Engineer

    1 week ago

    Charlotte, NC, United States LSEG (London Stock Exchange Group) Full time

    Ready to build scalable data systems with real-world impact? Join us at FTSE Russell! We're establishing a team of engineers passionate about data, software craftsmanship, and substantial impact. You'll help to power the indices trusted by financial institutions worldwide. You'll work in an AWS environment, build robust pipelines, and work with product and...

  • Data Engineer

    3 days ago

    Charlotte, NC, United States Insight Global Full time

    We are seeking an experienced Data Engineer and Reporting Lead to oversee the maintenance and development of Tableau-based dashboards and reports. The ideal candidate will have a strong background in SQL, Python, and data workflows, with experience supporting end-to-end data flow processes. This role involves managing loss forecast reports for commercial...

  • Senior Data Engineer

    5 days ago

    Charlotte, NC, United States Macpower Digital Assets Edge Full time

    Job Summary: This job is responsible for driving data engineering efforts to deliver enterprise-wide capabilities and complex data solutions. Key responsibilities include directing code design and delivery tasks associated with the integration, cleaning, transformation, and control of data in operational and analytical data systems and working with the...

  • Data Engineer

    2 weeks ago

    Charlotte, NC, United States Purple Drive Full time

    Job Description: Data EngineerResponsibilities Data Pipelines: Design, build, and maintain scalable pipelines using Databricks and Spark for ETL (extraction, transformation, loading). Data Modeling: Develop and optimize databases and large-scale processing systems with robust data modeling. Version Control: Use GitHub for collaboration and codebase...