Data Engineer

3 weeks ago


Charlotte, United States Compunnel Inc. Full time

This is a W2 role.


Responsibilities:

The person on this role will play a crucial role in building scalable and cost-effective data pipelines, data lakes, and analytics systems.

Data Ingestion: Implement data ingestion processes to collect data from various sources, including databases, streaming data, and external APIs.

Data Transformation: Develop ETL (Extract, Transform, Load) processes to transform and cleanse raw data into a structured and usable format for analysis.

Data Storage: Manage and optimize data storage solutions, including Amazon S3, Redshift, and other AWS storage services.

Data Processing: Utilize AWS services like AWS Glue, Amazon EMR, and AWS Lambda to process and analyze large datasets.

Data Monitoring and Optimization: Continuously monitor and optimize data pipelines and infrastructure for performance, cost-efficiency, and scalability.

Integration: Collaborate with data scientists, analysts, and other stakeholders to integrate AWS-based solutions into data analytics and reporting platforms.

Documentation: Maintain thorough documentation of data engineering processes, data flows, and system configurations.

Scalability: Design AWS-based solutions that can scale to accommodate growing data volumes and changing business requirements.

Cost Management: Implement cost-effective solutions by optimizing resource usage and recommending cost-saving measures.

Troubleshooting: Diagnose and resolve AWS-related issues to minimize downtime and disruptions.


Qualifications: Must Have:

  • Strong experience with AWS services on big data platforms.
  • Strong Python experience in building Dash application in AWS
  • Spark
  • Shell scripting and Linux knowledge

Nice to have:

  • Investment Knowledge preferred
  • Looking for candidates who have worked in Investment/Finance space
  • Machine Learning Skills

Qualifications:

Educational Background: A bachelor's degree in computer science, information technology, or a related field is typically required.

A Master's degree is preferred with GPA of 3.8+

Problem-Solving: Strong analytical and problem-solving skills to address complex data engineering challenges.


  • Data Engineer

    1 week ago


    Charlotte, United States LHH Full time

    **Must be local to Charlotte area for a weekly hybrid commute**LHH is seeking a Data Engineer for a rapidly growing company based out the Charlotte, NC area. The ideal candidate will have 3+ years of experience with data engineering and a proven track record of implementing robust, scalable data solutions in Azure Cloud environments.Responsibilities:Design,...

  • Data Engineer

    1 week ago


    Charlotte, United States LHH Full time

    **Must be local to Charlotte area for a weekly hybrid commute**LHH is seeking a Data Engineer for a rapidly growing company based out the Charlotte, NC area. The ideal candidate will have 3+ years of experience with data engineering and a proven track record of implementing robust, scalable data solutions in Azure Cloud environments.Responsibilities:Design,...

  • Data Engineer

    7 days ago


    Charlotte, United States LHH Full time

    **Must be local to Charlotte area for a weekly hybrid commute**LHH is seeking a Data Engineer for a rapidly growing company based out the Charlotte, NC area. The ideal candidate will have 3+ years of experience with data engineering and a proven track record of implementing robust, scalable data solutions in Azure Cloud environments.Responsibilities:Design,...

  • Data Engineer

    3 weeks ago


    Charlotte, North Carolina, United States Belk Full time

    EDW Data Engineer is responsible for designing, maintaining, and optimizing data infrastructure for data collection, management, transformation, and access. Creates pipelines that convert raw data into usable formats for data scientists and other data consumers to utilize. Builds and leverages information loaded into the Belk Enterprise Data Warehouse (EDW)...


  • Charlotte, United States Trilogy International Full time

    Data Engineering Manager required for high-growth InsurTechMust be based in greater Charlotte, NC area (or willing to relocate) $180-200k base + Bonus + Benefits US Citizens/Green Card Holders invited to applyThe CompanyOur client is a dynamic and innovative organization at the forefront of revolutionizing the insurance industry. Their mission is to leverage...


  • Charlotte, United States Trilogy International Full time

    Data Engineering Manager required for high-growth InsurTechMust be based in greater Charlotte, NC area (or willing to relocate) $180-200k base + Bonus + Benefits US Citizens/Green Card Holders invited to applyThe CompanyOur client is a dynamic and innovative organization at the forefront of revolutionizing the insurance industry. Their mission is to leverage...

  • Senior Data Engineer

    3 weeks ago


    Charlotte, United States Judge Consulting Group Full time

    Job Description:As the Technical Lead Data Engineer, your primary responsibility will be to spearhead the design, development, and implementation of data solutions aimed at empowering our organization to derive actionable insights from intricate datasets. You will take the lead in guiding a team of data engineers, fostering collaboration with...

  • Senior Data Engineer

    3 weeks ago


    Charlotte, United States Judge Consulting Group Full time

    Job Description:As the Technical Lead Data Engineer, your primary responsibility will be to spearhead the design, development, and implementation of data solutions aimed at empowering our organization to derive actionable insights from intricate datasets. You will take the lead in guiding a team of data engineers, fostering collaboration with...


  • Charlotte, North Carolina, United States Mediavine Full time

    Mediavine is seeking an Engineering Manager to join our Data & Analytics Team to lead our data engineering efforts, focused on delivering impactful and innovative solutions in the Ad Tech Industry. This role is ideal for leaders enthusiastic about driving meaningful change and shaping the future of our technology, who embrace change and demand...

  • Data Engineer

    1 month ago


    Charlotte, North Carolina, United States Infinicsinc Full time

    Bachelors' degree in Computer Science Engineering Information Technology or related area 7+ years' experience on ETL Development Tools Spark / Java (preferred) Python Ab Initio Informatica Advanced experience & demonstrated proficiency in Spark. Experience with Apache Beam Cloud Data Flow Dataproc Cloud Composer and Big Query 3+ years' experience in Big Data...

  • Data Engineer

    1 month ago


    Charlotte, United States Infinicsinc.com Full time

    Bachelors' degree in Computer Science Engineering Information Technology or related area 7+ years' experience on ETL Development Tools Spark / Java (preferred) Python Ab Initio Informatica Advanced experience & demonstrated proficiency in Spark. Experience with Apache Beam Cloud Data Flow Dataproc Cloud Composer and Big Query 3+ years'...

  • Big Data Engineer

    2 days ago


    Charlotte, United States Emblue Full time

    Responsibilities Creates secure and high-quality production code and maintains algorithms that run synchronously with appropriate systems. Produces architecture and design artifacts for complex applications while being accountable for ensuring design constraints are met by software code development. Gathers analyzes synthesizes and develops...

  • Big Data Engineer

    5 days ago


    Charlotte, North Carolina, United States Emblue Full time

    Responsibilities Creates secure and high-quality production code and maintains algorithms that run synchronously with appropriate systems. Produces architecture and design artifacts for complex applications while being accountable for ensuring design constraints are met by software code development. Gathers analyzes synthesizes and develops visualizations...

  • GCP Data Engineer

    1 day ago


    Charlotte, United States Synechron Full time

    GCP Data EngineerType of Role: TechnologyLocation: Charlotte, North Carolina, United StatesNOTE: ONLY LOCAL Candidates need to apply who are in Charlotte, North Carolina, United StatesOverviewAs a Senior GCP Data Engineer, you will collaborate with stakeholders across the business to work on complex technical requirements. You will be responsible for...

  • GCP Data Engineer

    1 day ago


    Charlotte, United States Synechron Full time

    GCP Data EngineerType of Role: TechnologyLocation: Charlotte, North Carolina, United StatesNOTE: ONLY LOCAL Candidates need to apply who are in Charlotte, North Carolina, United StatesOverviewAs a Senior GCP Data Engineer, you will collaborate with stakeholders across the business to work on complex technical requirements. You will be responsible for...

  • GCP Data Engineer

    4 days ago


    Charlotte, United States Capgemini Full time

    Job Title :: GCP Data EngineerJob Location:: Charlotte , NCJob Description::Responsibilities::Develop complex data pipelines to support data initiativesDevelop reusable framework based data ingestion and transformation capabilities to rapidly design and deploy data pipelinesDesign build test and deploy new libraries frameworks or full systems for our core...

  • GCP Data Engineer

    3 weeks ago


    Charlotte, United States Capgemini Full time

    Job Title :: GCP Data EngineerJob Location:: Charlotte , NCJob Description::Responsibilities::Develop complex data pipelines to support data initiativesDevelop reusable framework based data ingestion and transformation capabilities to rapidly design and deploy data pipelinesDesign build test and deploy new libraries frameworks or full systems for our core...


  • Charlotte, United States Incedo Inc. Full time

    Job Title: Data Engineering Architect / DirectorJob Location: Charlotte, NCJob Type: Full TimePosition Overview:We are seeking a skilled and experienced Data Engineering Leader to join our team. The ideal candidate will have expertise in data streaming technologies, particularly Kafka, Kinesis and glue, and a strong background in the wealth management or...


  • Charlotte, United States Incedo Inc. Full time

    Job Title: Data Engineering Architect / DirectorJob Location: Charlotte, NCJob Type: Full TimePosition Overview:We are seeking a skilled and experienced Data Engineering Leader to join our team. The ideal candidate will have expertise in data streaming technologies, particularly Kafka, Kinesis and glue, and a strong background in the wealth management or...


  • Charlotte, United States Incedo Inc. Full time

    Job Title: Data Engineering Architect / DirectorJob Location: Charlotte, NCJob Type: Full TimePosition Overview:We are seeking a skilled and experienced Data Engineering Leader to join our team. The ideal candidate will have expertise in data streaming technologies, particularly Kafka, Kinesis and glue, and a strong background in the wealth management or...