Lead Data Engineer

2 months ago

Kansas City, United States Capstone IT Staffing Full time

Capstone IT is seeking an experienced Lead Data Engineer to join one of clients' Data Platform team to build scalable enterprise data solutions that drive strategic business outcomes.


As the Lead Data Engineer, you will partner with Product Owners on strategic initiatives, understand business and functional requirements on projects, provide hands-on technical leadership to the engineering team with project execution and ensure deliverables are delivered with high quality and on time. You will help lead the iterative design and development of our cloud data warehouse. You will be responsible for designing, developing and maintaining data pipelines and data in our data warehouse to support Data Science, Business Intelligence and business operations. You will lead and guide the data engineering team, ensuring adherence to quality, standards, processes, and methodologies. Additionally, you will support the team in overcoming any technical challenges they encounter.


Duties & Responsibilities


  • Work closely with Product Manager and business stakeholders and develop a strong business understanding of our clients strategic and project level data needs.
  • Work with Product Owners to understand and translate business requirements into technical specifications and logical & physical data model requirements.
  • Design and extend data models working closely with Product and Data Architecture.
  • Design data pipelines that process both real time and batch data leveraging modern data engineering best practices.
  • Build, implement and test data pipelines using Databricks, Azure Data Factory, SSIS and other data integration tools.
  • Take technical ownership of projects and ensure deliverables meet scope, quality and timelines.
  • Provide technical leadership and guidance to the data engineering team, including mentoring, coaching, and fostering a collaborative work environment between all team members.
  • Support team members with troubleshooting and resolving complex technical issues and challenges.
  • Implement coding standards, conduct code reviews to ensure proper code quality level.
  • Implement effective source code management and CI/CD best practices to ensure data pipelines are automatically tested, integrated and deployed to production.
  • Partner with Data Architects to implement effective data integration and storage design patterns. Contribute to continuously improve existing data architecture and frameworks.
  • Assist in the development and implementation of data governance and management solutions.


Required Experience:


  • Bachelor’s Degree or higher in Computer Science, Engineering, or related field required.
  • 10+ years of experience in data engineering and data warehousing.
  • 5+ years of cloud data engineering and big data experience .


Required Knowledge, Skills And Abilities:


  • Experience with cloud data engineering and data warehouse technologies in Microsoft Azure, Google or AWS. Experience in Azure is highly preferred.
  • Strong programming skills in python and SQL for data processing and automation.
  • Strong technical skills in big data and Data integration technologies including Spark, Kafka, Databricks, Azure Data Factory, SSIS and SQL.
  • Experience with designing and building both cloud and on-prem data warehouses. Experience with Lake house architecture is highly preferred.
  • Experience in data modeling for transactional and analytical needs. Experience with NoSQL databases is preferred.
  • Development experience in Microsoft SQL Server including experience with tuning queries to optimize performance.
  • Skilled in Visual Studio with the use of source control applications such as Azure DevOps.
  • Experience with CI/CD and Jenkins.
  • In-depth knowledge of data management principles and best practices, including data governance, data security, data quality, and data integration.
  • Experience with delivering data solutions iteratively using the Agile Scrum framework and development processes.
  • Experience in providing guidance, mentorship, and technical support to a team of data engineers.
  • Demonstrated ability to troubleshoot and resolve complex technical issues related to data processing in a cloud big data infrastructure.
  • Exceptional analytical and conceptual thinking skills coupled with strong written and verbal communication skills.

  • Lead Data Engineer

    1 day ago

    Kansas City, United States Core BTS Full time

    Job DescriptionJob DescriptionTitle:Lead Data Engineer (Subcontract at a Core BTS Client)Through Core BTS Resource Management Services (RMS), we offer custom talent solutions to help our clients meet their evolving technology and business needs. We help effectively match the right technology professional to their organization, recruiting for contract,...

  • Data Engineering Lead

    3 weeks ago

    Kansas City, Missouri, United States CloudBC Labs Full time

    Job Summary:CloudBC Labs is hiring a talented Data Engineer to lead our cloud-based data engineering initiatives. The ideal candidate will have expertise in designing, developing, and deploying scalable data solutions using Azure services, such as Data Lake and Data Factory, as well as SQL Server and SSIS.About the Role:In this position, you will be...

  • Enterprise Data Engineer Lead

    3 weeks ago

    Kansas City, Missouri, United States Spring Venture Group Full time

    Job Description: Enterprise Data Engineer Lead About the Role:We are seeking an experienced Enterprise Data Engineer Lead to join our team at Spring Venture Group. As an Enterprise Data Engineer Lead, you will be responsible for leading the development and implementation of large-scale data engineering projects that support our business operations.About the...

  • Data Engineering Lead

    2 weeks ago

    Jersey City, New Jersey, United States Intellisoft Technologies Full time

    Job Title: Data Engineering LeadAbout the Role: We are seeking a highly skilled Data Engineering Lead to join our team at Intellisoft Technologies. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining scalable data pipelines and ETL/ELT processes using SSIS, Python, and API...

  • Data Engineer Lead

    2 weeks ago

    Texas City, Texas, United States Resource Informatics Group Full time

    Job Title: Cloud-Based Data Solutions ExpertWe are seeking an experienced Data Engineer Lead to join our team at Resource Informatics Group in Houston, Texas. The ideal candidate will have 11+ years of experience as a Data Engineer designing and maintaining data pipeline architectures.The estimated salary for this position is $170,000 - $200,000 per year,...

  • Data Engineering Lead

    1 week ago

    Jersey City, New Jersey, United States iShare Inc Full time

    iShare Inc. is seeking a highly skilled Data Engineering Lead to join our team in NY/NJ. As a Data Engineering Lead, you will be responsible for designing and implementing large data engineering programs that drive business growth.Key responsibilities include:Managing large accounts and driving account growth through data-driven insights.Collaborating with...

  • Data Engineer

    1 day ago

    Kansas City, United States HART, INC. Full time

    Job DescriptionJob DescriptionDescription:Company Overview:We are a leading healthcare technology company dedicated to transforming the way healthcare organizations leverage data to improve patient care and operational efficiency. As a Healthcare Data Engineer, you will play a vital role in developing and executing data transformation activities, enabling...

  • Data Engineering Lead

    1 week ago

    Maryland City, Maryland, United States iQuasar Full time

    Job Title: Data Engineering LeadThe iQuasar team is seeking an experienced Data Engineering Lead to design, develop, and implement scalable data solutions. The ideal candidate will have expertise in Azure Synapse Analytics and Microsoft Fabric.This role involves creating data pipelines, optimizing data lakes, ensuring high availability and security. Strong...

  • Data Engineer Lead

    2 weeks ago

    Jersey City, New Jersey, United States Inizio Partners Full time

    About the RoleInizio Partners is seeking a highly skilled Senior Data Engineer to join our team. This is a challenging and rewarding opportunity for an experienced professional who can lead data engineering initiatives and drive business growth.Key Responsibilities & Skillsets:Azure Databricks: Create data transformation logics, develop complex SQL queries,...

  • Cloud Data Engineering Lead

    2 weeks ago

    Ohio City, Ohio, United States CyberTec Full time

    Cloud Data Engineering LeadCyberTec is looking for a Cloud Data Engineering Lead to drive the design and development of large-scale data processing systems.The successful candidate will have expertise in cloud-based technologies, elastic architectures, and cost-effective solutions using Cosmos, Azure Data Factory, Azure Synapse, and Databricks platforms.Job...

  • Data Engineering Lead

    2 weeks ago

    Jersey City, New Jersey, United States Futran Tech Solutions Pvt. Ltd. Full time

    About the JobFutran Tech Solutions Pvt. Ltd. is a forward-thinking company looking for a talented Terraform Engineer to drive our data engineering initiatives.Job RequirementsThe ideal candidate will possess strong problem-solving skills, experience in working with gateways, and ability to design and develop BI dashboards and reports by connecting to various...

  • Technical Lead

    2 weeks ago

    Jersey City, New Jersey, United States Diverse Lynx Full time

    Diverse Lynx LLC is seeking a talented TECHNICAL LEAD - DATA ENGINEERING to lead our data engineering efforts. As a senior member of our team, you will be responsible for designing and implementing data pipelines using Python and data engineering tools. You will also collaborate with cross-functional teams to ensure successful project execution.About the...

  • Technical Lead for Data Engineering

    2 weeks ago

    Jersey City, New Jersey, United States TechnoGen Full time

    Job Title: Technical Lead for Data EngineeringWe are looking for an experienced Technical Lead for Data Engineering to join our team at TechnoGen. This is a long-term contract position with a salary range of $180,000 - $230,000 per year.About the Job:Lead evaluation sessions with external vendors, startups, and internal teams to drive outcomes-oriented...

  • Technical Lead

    2 weeks ago

    Jersey City, New Jersey, United States TechnoGen Full time

    Job Title: Technical Lead - Data EngineeringWe are seeking a skilled Technical Lead - Data Engineering to join our team at TechnoGen. This role requires 12+ years of experience in data engineering concepts and expertise in one or more programming languages such as Java, Python. The successful candidate will have hands-on experience delivering data pipelines...

  • Data Engineer Lead

    2 weeks ago

    Texas City, Texas, United States The University of Texas at Austin Full time

    About The Role We are seeking a highly skilled Data Engineer Lead to join our team at The University of Texas at Austin. As a key member of our organization, you will be responsible for leading the design, implementation, and maintenance of complex data pipelines using cloud-based infrastructure. With a strong background in data engineering and experience...

  • Data Engineering Lead

    2 weeks ago

    Jersey City, New Jersey, United States Compu-Vision Consulting Full time

    Job Description:Sr. Consultant profile with expertise in designing and developing scalable data architectures on Databricks Lakehouse platform.We are seeking a highly experienced professional with strong background in Property and Casualty (P&C) Industry to lead our data initiatives.Key Responsibilities:Design and implement data warehouses using Databricks...

  • Data Engineer III

    4 weeks ago

    Kansas City, United States Spring Venture Group Full time

    Company DescriptionWho We Are: Spring Venture Group is a leading digital direct-to-consumer sales and marketing company with product offerings focused on the senior market. We specialize in distributing Medicare Supplement, Medicare Advantage, and related products via our family of brands and dedicated team of licensed insurance agents. Powered by our unique...

  • Data Engineer III

    1 day ago

    Kansas City, United States Spring Venture Group Full time

    Job DescriptionJob DescriptionCompany DescriptionWho We Are: Spring Venture Group is a leading digital direct-to-consumer sales and marketing company with product offerings focused on the senior market. We specialize in distributing Medicare Supplement, Medicare Advantage, and related products via our family of brands and dedicated team of licensed insurance...

  • Azure Data Engineer Lead

    1 month ago

    Jersey City, United States SysMind Tech Full time

    Responsibilities Lead the Azure Data Ingestion and Integration activities from onshore for the on-off team of around 8 people Collaborate with the WAB Data Architects and Lead Data Engineers in data pipeline design Co-ordinate across the data engineering workstreams happening in parallel including reviews of work from the team Help PM/Scrum Master with...

  • Data Science and Engineering Lead

    2 weeks ago

    Foster City, California, United States InterSources Full time

    About the Role As Data Science and Engineering Lead, you will be responsible for leading our data science and engineering efforts and ensuring that our data infrastructure is scalable, secure, and reliable. With a minimum of 10 years of experience in data engineering and at least 3-4 years of lead experience, you will be an expert in designing and...