Lead Data Engineering

2 months ago


Charlotte, United States Compunnel Inc. Full time

Interview Process:

  1. hacker rank (30mins SQL / 30 mins python)
  2. initial interview
  3. 1 hour technical
  4. 3 hour panel


Must Have Skills: "And if someone is coming from a software background and then getting into data engineering, that's a perfect fit."

  • Looking for around 12+ years of experience
  • Data - python & SQL are a must have
  • 2-3 years retail experience
  • Databricks (azure preferred)
  • Azure tech stack experience
  • experience building best practices and guidelines
  • knowledge of reusability patterns -python, data bricks
  • frameworks
  • should understand basics of data modeling
  • experience with facts & dimensions of how tables are built


Nice to have:

  • snowflake, fabric


Key Role details:

  • will be primarily developing in this role
  • this role doesn't touch production support (have a team offshore in india)
  • will be migrating SQL server to Azure
  • goal is to have use cases flow in 5 minutes
  • each lead will end up mentoring a team of 3-4 devs under them . They will be part of building out their own team.


Responsibilities:

  • Apply advanced knowledge of Data Engineering principles, methodologies and techniques to design and implement data loading and aggregation frameworks across broad areas of the organization.
  • Gather and process raw, structured, semi-structured and unstructured data using batch and real-time data processing frameworks.
  • Implement and optimize data solutions in enterprise data warehouses and big data repositories, focusing primarily on movement to the cloud.
  • Drive new and enhanced capabilities to Enterprise Data Platform partners to meet the needs of product / engineering / business.
  • Experience building enterprise systems especially using Databricks, Snowflake and platforms like Azure, AWS, GCP etc
  • Leverage strong Python, Spark, SQL programming skills to construct robust pipelines for efficient data processing and analysis.
  • Implement CI/CD pipelines for automating build, test, and deployment processes to accelerate the delivery of data solutions.
  • Implement data modeling techniques to design and optimize data schemas, ensuring data integrity and performance.
  • Drive continuous improvement initiatives to enhance performance, reliability, and scalability of our data infrastructure.
  • Collaborate with data scientists, analysts, and other stakeholders to understand business requirements and translate them into technical solutions.
  • Implement best practices for data governance, security, and compliance to ensure the integrity and confidentiality of our data assets.


Qualifications:

  • Bachelor’s or master’s degree in computer science, Engineering, or a related field.
  • Proven experience (8+) in a data engineering role, with expertise in designing and building data pipelines, ETL processes, and data warehouses.
  • Strong proficiency in SQL, Python and Spark programming languages.
  • Strong experience with cloud platforms such as AWS, Azure, or GCP is a must.
  • Hands-on experience with big data technologies such as Hadoop, Spark, Kafka, and distributed computing frameworks.
  • Knowledge of data lake and data warehouse solutions, including Databricks, Snowflake, Amazon Redshift, Google BigQuery, Azure Data Factory, Airflow etc.
  • Experience in implementing CI/CD pipelines for automating build, test, and deployment processes.
  • Solid understanding of data modeling concepts, data warehousing architectures, and data management best practices.
  • Excellent communication and leadership skills, with the ability to effectively collaborate with cross-functional teams and drive consensus on technical decisions.
  • Relevant certifications (e.g., Azure, databricks, snowflake) would be a plus.


  • Data Engineer

    2 months ago


    Charlotte, North Carolina, United States NTT DATA Full time

    NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer (Python/PySpark) - Onsite to join our team in Charlotte, North Carolina (US-NC), United States (US).We are...

  • Lead Data Engineer

    1 month ago


    Charlotte, United States Abidi Solutions Full time

    Job DescriptionJob DescriptionJob Title: Lead Data EngineerLocation: Hybrid in Charlotte, NC Duration: 6 month contract to hireMust Haves:9+ years of experience in a data engineering role, with expertise in designing and building data pipelines, ETL processes, and data warehouses2+ years of experience working as a lead and developing/mentoring other...

  • Lead Data Engineer

    3 weeks ago


    Charlotte, United States Compunnel Inc. Full time

    Required Skills & Experience* 10+ years of experience in a data engineering role, with expertise in designing and building data pipelines, ETL processes, and data warehouses* 2+ years of experience working as a lead and developing/mentoring other engineers* Strong proficiency in SQL, Python and PySpark programming languages* Strong experience with the Azure...

  • Lead Data Engineer

    2 weeks ago


    Charlotte, United States Circle K Stores, Inc. Full time

    Job Description: As the Technical Lead Data Engineer, your primary responsibility will be to spearhead the design, development, and implementation of data solutions aimed at empowering our organization to derive actionable insights from intricate datasets. You will take the lead in guiding a team of data engineers, fostering collaboration with...

  • Lead Data Engineer

    3 months ago


    Charlotte, United States Circle K Full time

    Job Description:As the Technical Lead Data Engineer, your primary responsibility will be to spearhead the design, development, and implementation of data solutions aimed at empowering our organization to derive actionable insights from intricate datasets. You will take the lead in guiding a team of data engineers, fostering collaboration with...


  • Charlotte, North Carolina, United States Hunt Hive Talent Full time

    About Hunt Hive TalentWe are a leading talent recruiting agency dedicated to connecting top talent with exciting opportunities. Our mission is to empower businesses and professionals to reach new heights.About the RoleWe are seeking a highly skilled Data Engineering Lead to join our client's team. As a key member of the organization, you will be responsible...

  • Lead Data Engineer

    3 months ago


    Charlotte, United States Chatham Financial Full time

    Overview: We don't simply hire employees. We invest in them. When you work at Chatham, we empower you - offering professional development opportunities to help you grow in your career, no matter if you've been here for five months or 15 years. Chatham has worked hard to create a distinct work environment that values people, teamwork, integrity, and client...

  • Lead Data Engineer

    4 weeks ago


    Charlotte, United States Amtex Systems Inc. Full time

    Title: Lead Data EngineerLocation: Charlotte, NC(Hybrid)Duration: 12+ Months· Apply advanced knowledge of Data Engineering principles, methodologies and techniques to design and implement data loading and aggregation frameworks across broad areas of the organization.· Gather and process raw, structured, semi-structured and unstructured data using batch and...

  • Lead Data Engineer

    4 weeks ago


    Charlotte, United States Amtex Systems Inc. Full time

    Title: Lead Data EngineerLocation: Charlotte, NC(Hybrid)Duration: 12+ Months· Apply advanced knowledge of Data Engineering principles, methodologies and techniques to design and implement data loading and aggregation frameworks across broad areas of the organization.· Gather and process raw, structured, semi-structured and unstructured data using batch and...

  • Data Lead Engineer

    1 month ago


    Charlotte, United States Whistle Express Car Wash Full time

    Whistle Express is a leading and rapidly expanding express car wash operator in the U.S. Whistle Express operates car washes with locations in Alabama, Florida, Georgia, North Carolina, Ohio, South Carolina, Tennessee, and Virginia. Whistle is focused on building upon its established market position in the Southeast. We’re looking for people with a strong...

  • Data Lead Engineer

    1 month ago


    Charlotte, United States Whistle Express Car Wash Full time

    Whistle Express is a leading and rapidly expanding express car wash operator in the U.S. Whistle Express operates car washes with locations in Alabama, Florida, Georgia, North Carolina, Ohio, South Carolina, Tennessee, and Virginia. Whistle is focused on building upon its established market position in the Southeast. We’re looking for people with a strong...


  • Charlotte, North Carolina, United States Pantheon Data Full time

    Job OverviewCompany Background:Pantheon Data, a subsidiary of Kenific Holding, operates as a private enterprise in the Washington, DC area. Established in 2011, the company initially focused on acquisition and supply chain management services for the US Coast Guard. Over the past decade, Pantheon Data has diversified its offerings to include infrastructure...

  • Lead Data Engineer

    2 weeks ago


    Charlotte, United States American Red Cross Full time

    Please use Google Chrome or Mozilla Firefox when accessing Candidate Home. By joining the American Red Cross you will touch millions of lives every year and experience the greatness of the human spirit at its best. Are you ready to be part of the world's largest humanitarian network? Join us-Where your Career is a Force for Good! Job Description: WHY CHOOSE...

  • Lead Data Engineering

    2 months ago


    Charlotte, United States Compunnel Inc. Full time

    Interview Process:hacker rank (30mins SQL / 30 mins python)initial interview1 hour technical3 hour panelMust Have Skills: "And if someone is coming from a software background and then getting into data engineering, that's a perfect fit."Looking for around 12+ years of experienceData - python & SQL are a must have2-3 years retail experienceDatabricks (azure...

  • Lead Engineer

    3 weeks ago


    Charlotte, North Carolina, United States NTT DATA Services Full time

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Lead Engineer (Opentext Exstream Cloud Native) to join our team in Charlotte, North Carolina (US-NC), United States (US). Job...

  • Lead Data Engineer

    1 month ago


    Charlotte, United States WELLS FARGO BANK Full time

    About this role:Wells Fargo is seeking a Lead Data EngineerIn this role, you will:Lead complex initiatives with broad impact and act as key participant in large scale software planning for the Technology areaDesign, develop, and run tooling to discover problems in data and applications and report the issues to engineering and product leadershipReview and...

  • Lead Data Engineer

    2 months ago


    Charlotte, United States Compunnel Inc. Full time

    Role: Lead Data Engineer- W2 roleLocation: Charlotte, NCQualification: Must Have: Strong experience with AWS services on big data platforms.Strong Python experience in building Dash application in AWSSparkShell scripting and Linux knowledgeNice to have:Investment Knowledge preferredLooking for candidates who have worked in Investment/Finance spaceMachine...

  • Lead Data Engineer

    2 months ago


    Charlotte, United States Compunnel Inc. Full time

    Role: Lead Data Engineer- W2 roleLocation: Charlotte, NCQualification: Must Have: Strong experience with AWS services on big data platforms.Strong Python experience in building Dash application in AWSSparkShell scripting and Linux knowledgeNice to have:Investment Knowledge preferredLooking for candidates who have worked in Investment/Finance spaceMachine...

  • Lead Data Engineer

    3 weeks ago


    Charlotte, United States WELLS FARGO BANK Full time

    About this role:Wells Fargo is seeking a Lead Data Engineer experienced in Data visualization tools Microsoft Power BI and Tableau, and Windows, Oracle, and Teradata databases to join our Major Incident Management Command Center team. In this role, you will:Lead complex initiatives with broad impact and act as key participant in large scale software planning...

  • Technical Lead

    3 days ago


    Charlotte, United States Hunt Hive Talent Full time

    Buzz About Us: Welcome to Hunt Hive Talent Recruiting Agency where we are all about connecting amazing talent with fantastic opportunities! Our mission To help businesses and professionals soar to new heights. If youre ready for an adventure in the recruiting world look no further. Join our vibrant team and be part of something extraordinary! About the Role:...