Lead Data Engineer

4 months ago

Charlotte, United States Circle K Stores, Inc. Full time

Job Description:

As the Technical Lead Data Engineer, your primary responsibility will be to spearhead the design, development, and implementation of data solutions aimed at empowering our organization to derive actionable insights from intricate datasets. You will take the lead in guiding a team of data engineers, fostering collaboration with cross-functional teams, and spearheading initiatives geared towards fortifying our data infrastructure, CI/CD pipelines, and analytics capabilities.

Responsibilities:

  • Apply advanced knowledge of Data Engineering principles, methodologies and techniques to design and implement data loading and aggregation frameworks across broad areas of the organization.
  • Gather and process raw, structured, semi-structured and unstructured data using batch and real-time data processing frameworks.
  • Implement and optimize data solutions in enterprise data warehouses and big data repositories, focusing primarily on movement to the cloud.
  • Drive new and enhanced capabilities to Enterprise Data Platform partners to meet the needs of product / engineering / business.
  • Experience building enterprise systems especially using Databricks, Snowflake and platforms like Azure, AWS, GCP etc
  • Leverage strong Python, Spark, SQL programming skills to construct robust pipelines for efficient data processing and analysis.
  • Implement CI/CD pipelines for automating build, test, and deployment processes to accelerate the delivery of data solutions.
  • Implement data modeling techniques to design and optimize data schemas, ensuring data integrity and performance.
  • Drive continuous improvement initiatives to enhance performance, reliability, and scalability of our data infrastructure.
  • Collaborate with data scientists, analysts, and other stakeholders to understand business requirements and translate them into technical solutions.
  • Implement best practices for data governance, security, and compliance to ensure the integrity and confidentiality of our data assets.
  • Coaching and mentoring junior Data Engineers: Providing guidance, support, and technical expertise to junior members of the data engineering team.
Qualifications:
  • Bachelor's or master's degree in computer science, Engineering, or a related field.
  • Proven experience (8+) in a data engineering role, with expertise in designing and building data pipelines, ETL processes, and data warehouses.
  • Strong proficiency in SQL, Python and Spark programming languages.
  • Strong experience with cloud platforms such as AWS, Azure, or GCP is a must.
  • Hands-on experience with big data technologies such as Hadoop, Spark, Kafka, and distributed computing frameworks.
  • Knowledge of data lake and data warehouse solutions, including Databricks, Snowflake, Amazon Redshift, Google BigQuery, Azure Data Factory, Airflow etc.
  • Experience in implementing CI/CD pipelines for automating build, test, and deployment processes.
  • Solid understanding of data modeling concepts, data warehousing architectures, and data management best practices.
  • Excellent communication and leadership skills, with the ability to effectively collaborate with cross-functional teams and drive consensus on technical decisions.
  • Relevant certifications (e.g., Azure, databricks, snowflake) would be a plus.

#LI-KL1

Circle K is an Equal Opportunity Employer.
The Company complies with the Americans with Disabilities Act (the ADA) and all state and local disability laws. Applicants with disabilities may be entitled to a reasonable accommodation under the terms of the ADA and certain state or local laws as long as it does not impose an undue hardship on the Company. Please inform the Company's Human Resources Representative if you need assistance completing any forms or to otherwise participate in the application process.

Click below to review information about our company's use of the federal E-Verify program to check work eligibility:

In English

In Spanish

  • Cloud Data Engineer Position for Experienced Professionals

    7 days ago

    Charlotte, North Carolina, United States Pantheon Data Full time

    Job Title: Cloud Data EngineerAbout Us:Pantheon Data is a private company based in the Washington, DC area. Founded in 2011, we have grown to offer various services including infrastructure resiliency, contact center operations, and software engineering. Our customer base includes commercial clients and federal agencies.Position Overview:We are seeking an...

  • Advanced Deep Learning Engineer

    6 days ago

    Charlotte, North Carolina, United States Pantheon Data Full time

    Are you a skilled Machine Learning Engineer looking for a challenging role that combines technical expertise with business acumen? Pantheon Data, a leading provider of innovative solutions to the US government and commercial clients, is seeking a seasoned Deep Learning Engineer to join our team.About the RoleThis is a unique opportunity to leverage your...

  • Chief Data Systems Engineer

    7 days ago

    Charlotte, North Carolina, United States Pantheon Data Full time

    Job OverviewPantheon Data, a private company based in the Washington, DC area, is seeking an experienced Chief Data Systems Engineer.We are looking for someone to design and develop software applications that drive our business forward. Our ideal candidate will have experience working with embedded systems, data warehouses, and ETL...

  • Lead Data Engineer

    1 week ago

    Charlotte, United States TechnoGen Full time

    Lead Data Engineer for a contract onsite opportunity for a leading client located in Charlotte, NC Location: Charlotte, NC Looking for 12+ years of experience Must Have Skills 1) Strong proficiency in SQL, Python and Spark programming languages. 2) Strong experience with cloud platforms such as AWS, Azure, or GCP is a must. 3) Hands-on experience with big...

  • Big Data Engineering Lead

    6 days ago

    Charlotte, North Carolina, United States Motion Recruitment Full time

    We are seeking a highly skilled Big Data Engineering Lead to join our team at Motion Recruitment in Charlotte, NC. This full-time consulting assignment will be a long-term opportunity for the right candidate.The ideal candidate should have a strong understanding of big data development and experience working with Hadoop, Spark, Python, Scala, Unix, and SQL.A...

  • Lead Data Engineer

    1 week ago

    Charlotte, United States American Red Cross Full time

    Please use Google Chrome or Mozilla Firefox when accessing Candidate Home.By joining the American Red Cross you will touch millions of lives every year and experience the greatness of the human spirit at its best. Are you ready to be part of the world's largest humanitarian network? Join us-Where your Career is a Force for Good! Job Description: WHY CHOOSE...

  • Lead Azure Data Engineer

    4 weeks ago

    charlotte, United States Insight Global Full time

    POSITION: Lead Azure Data EngineerLOCATION: Charlotte, NC 28217 (Hybrid 3 times per week)PAY RANGE: $80.00 - $87.00 per hour W2DURATION: 6-Month Contract-To-Hire (100% career opportunity)MUST-HAVES:10+ years of experience in a data engineering role, with expertise in Azure, designing and building data pipelines, ETL processes, and data warehouses.2+ years of...

  • Lead Azure Data Engineer

    1 month ago

    charlotte, United States Insight Global Full time

    POSITION: Lead Azure Data EngineerLOCATION: Charlotte, NC 28217 (Hybrid 3 times per week)PAY RANGE: $80.00 - $87.00 per hour W2DURATION: 6-Month Contract-To-Hire (100% career opportunity)MUST-HAVES:10+ years of experience in a data engineering role, with expertise in Azure, designing and building data pipelines, ETL processes, and data warehouses.2+ years of...

  • Lead Azure Data Engineer

    1 month ago

    Charlotte, United States Insight Global Full time

    POSITION: Lead Azure Data EngineerLOCATION: Charlotte, NC 28217 (Hybrid 3 times per week)PAY RANGE: $80.00 - $87.00 per hour W2DURATION: 6-Month Contract-To-Hire (100% career opportunity)MUST-HAVES:10+ years of experience in a data engineering role, with expertise in Azure, designing and building data pipelines, ETL processes, and data warehouses.2+ years of...

  • Technical Lead

    2 days ago

    Charlotte, United States The Select Group Full time

    Job Details As the Technical Lead - Data Engineer, your primary responsibility will be to spearhead the design, development, and implementation of data solutions aimed at empowering our organization to derive actionable insights from intricate datasets. You will take the lead in guiding a team of data engineers, fostering collaboration with cross-functional...

  • Data Science Engineer

    2 days ago

    Charlotte, North Carolina, United States NTT DATA Full time

    About NTT DATA ServicesNTT DATA Services is a recognized leader in IT and business services, including cloud, data and applications. As part of NTT DATA, a $30 billion trusted global innovator with a combined global reach of over 80 countries, we help clients transform through business and technology consulting, industry and digital solutions, applications...

  • Senior Data Solutions Architect

    6 days ago

    Charlotte, North Carolina, United States Pantheon Data Full time

    Job OverviewPantheon Data is a private, small business based in the Washington, DC area.We are seeking a skilled Data Engineer with expertise in designing data-centric technical solutions.The ideal candidate will have hands-on experience with data modeling, data warehousing, and ETL pipelines.Analyze customer technology landscapes, business processes, and...

  • Senior / Lead Hadoop Engineer

    2 months ago

    Charlotte, United States NTT DATA Full time

    Req ID: 297836NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior / Lead Hadoop Engineer to join our team in Charlotte, North Carolina (US-NC), United States (US).Job Duties and...

  • Cybersecurity Lead

    3 weeks ago

    Charlotte, North Carolina, United States Pantheon Data Full time

    Job SummaryWe are seeking an experienced Cybersecurity Lead to oversee the security of our information systems and ensure compliance with all relevant regulations and standards.About UsPantheon Data is a private company based in the Washington, DC area. We provide a range of services, including infrastructure resiliency, contact center operations,...

  • Lead GCP Data Engineer

    1 week ago

    Charlotte, United States Mindlance Full time

    Job Descriptions: In this contingent resource assignment, you may: Consult on complex initiatives with broad impact and large-scale planning for Software Engineering. Review and analyze complex multi-faceted, larger scale or longer-term Software Engineering challenges that require in-depth evaluation of multiple factors including intangibles or unprecedented...

  • Data Engineer Lead

    6 months ago

    Charlotte, United States Omni Inclusive Full time

    Primary Skills: AWS Services (AWS platform and services knowledge including AWS Lambda, API Gateway) + Data Skills -Glue, Informatica/Kafka/AWS Data flow. Data Engineer with Informatica knowledge) Minimum ten (10) years technical experience in each of the following: • Design Proof of Technology using AWS platform and services for various client...

  • Lead Engineer

    1 week ago

    Charlotte, United States NTT DATA Full time

    Req ID:294180 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Lead Engineer (Opentext Exstream Cloud Native) to join our team in Charlotte, North Carolina (US-NC), United States...

  • Lead Engineer

    3 months ago

    Charlotte, United States NTT DATA Full time

    Req ID: 294180NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Lead Engineer (Opentext Exstream Cloud Native) to join our team in Charlotte, North Carolina (US-NC), United States...

  • Lead Information Security Engineer

    3 months ago

    Charlotte, United States NTT DATA Full time

    Req ID: 293769NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Lead Information Security Engineer (GCP/Terraform/FTE/Hybrid) to join our team in Charlotte, North Carolina (US-NC),...

  • Cloud Architect and Data Engineer Lead

    4 days ago

    Charlotte, North Carolina, United States Strategic Staffing Solutions Full time

    Job Description:Strategic Staffing Solutions is seeking an experienced Cloud Architect and Data Engineer Lead to join our team. The ideal candidate will have a strong background in designing and implementing data frameworks and hands-on experience building scalable data platforms on cloud platforms.Key Responsibilities:Deliver end-to-end products with cloud...