Data Engineer

2 weeks ago

Atlanta, United States Compunnel Inc. Full time

We are seeking an experienced AWS Glue Specialist to join our team and lead the migration of ETL processes from IBM DataStage to AWS Glue. This role requires hands-on expertise in designing, developing, and optimizing ETL pipelines, with a focus on migrating data from various sources, both on-premises and cloud-based. The ideal candidate will have a proven ability to work independently and collaborate with multi-domain teams to ensure successful data integration and transformation.


Key Responsibilities

  • Lead the migration of ETL processes from IBM DataStage to AWS Glue, ensuring efficiency and reliability.
  • Design, develop, and optimize ETL pipelines using AWS Glue, incorporating best practices for performance, scalability, and cost-efficiency.
  • Analyze existing DataStage ETL workflows to identify and implement optimized solutions in the AWS environment.
  • Extract data from diverse sources (on-premises and cloud) and transform it into a cloud-compatible format using AWS Glue.
  • Collaborate with cross-functional teams to understand data requirements and ensure seamless data integration across multiple domains.
  • Independently troubleshoot and resolve technical challenges, ensuring data integrity and compliance with project timelines.
  • Document technical specifications, data flows, and transformation logic for ETL processes.
  • Monitor and maintain ETL pipelines to ensure continuous operation, making enhancements as needed.
  • Support the team with best practices in cloud data management and AWS Glue usage.

Qualifications

  • 5+ years of experience in ETL development, with a focus on data migration.
  • 3+ years of hands-on experience with AWS Glue, including job creation, workflow orchestration, and optimization.
  • Proven expertise in IBM DataStage and experience in migrating ETL jobs to cloud platforms, particularly AWS.
  • Strong knowledge of AWS services (S3, Lambda, CloudWatch, Redshift, RDS, etc.) and how they integrate with AWS Glue.
  • Experience with Python or PySpark for ETL scripting and transformations.
  • Familiarity with SQL and relational databases, as well as NoSQL databases.
  • Strong problem-solving skills with the ability to work independently and collaborate across multiple teams.
  • Excellent communication skills for interacting with stakeholders and documenting processes.

Preferred Skills

  • Experience with AWS Data Migration Services (DMS) or other data migration tools.
  • Knowledge of data warehousing concepts and best practices.
  • AWS certifications (e.g., AWS Certified Data Analytics, AWS Certified Solutions Architect) are a plus.
  • Familiarity with Agile methodologies and experience working in a fast-paced environment.

  • Data Center Engineering Operations Technician

    6 months ago

    Atlanta, United States Amazon Data Services, Inc. Full time

    At Amazon, we're working to be Earth's most Customer-centric Company and Earth's Best Employer. To get there, we need exceptionally talented, bright, and driven people. If you'd like to help us build and expand The Cloud, this is your chance to make history.Amazon Data Services Inc. is currently hiring for a Engineering Operations Tech (EOT) to support our...

  • Data Engineer

    1 month ago

    atlanta, United States Factspan Full time

    Job Title: Data EngineerJob Type: Full TimeJob Location: Atlanta, GAJob Description:You will be a core member of the Data Platform and Analytics Engineering team that is building the infrastructural foundations for data aggregation, management, and insights. Our data and engineering teams build products that power our operations (warehouse, transportation,...

  • Data Engineer

    1 month ago

    atlanta, United States Factspan Full time

    Job Title: Data EngineerJob Type: Full TimeJob Location: Atlanta, GAJob Description:You will be a core member of the Data Platform and Analytics Engineering team that is building the infrastructural foundations for data aggregation, management, and insights. Our data and engineering teams build products that power our operations (warehouse, transportation,...

  • Data Engineer

    1 month ago

    Atlanta, United States Factspan Full time

    Job Title: Data EngineerJob Type: Full TimeJob Location: Atlanta, GAJob Description:You will be a core member of the Data Platform and Analytics Engineering team that is building the infrastructural foundations for data aggregation, management, and insights. Our data and engineering teams build products that power our operations (warehouse, transportation,...

  • Data Engineer

    7 days ago

    Atlanta, United States Insight Global Full time

    We are actively seeking a skilled Data Engineer to join our innovative team in Atlanta. This is a contract-to-hire position with a decision expected by the end of the year. The ideal candidate will have extensive experience working with offshore teams and a strong background in data engineering and data science projects.Responsibilities:Collaborate with a...

  • Senior Data Engineer

    1 month ago

    atlanta, United States Saxon Global Full time

    Job Description: Senior Data EngineerLocation: Atlanta, GAType: Full-timeSalary : $130K/annumVisa : USC/GC/H4 EADNO OPT and H1B please9-10+ Years of candidate can apply for this role Role Overview:We are seeking a skilled Data Engineer with hands-on experience in Snowflake to join our dynamic team. In this role, you will play a crucial part in designing,...

  • Senior Data Engineer

    1 month ago

    Atlanta, United States Saxon Global Full time

    Job Description: Senior Data EngineerLocation: Atlanta, GAType: Full-timeSalary : $130K/annumVisa : USC/GC/H4 EADNO OPT and H1B please9-10+ Years of candidate can apply for this role Role Overview:We are seeking a skilled Data Engineer with hands-on experience in Snowflake to join our dynamic team. In this role, you will play a crucial part in designing,...

  • Senior Data Engineer

    1 month ago

    atlanta, United States Saxon Global Full time

    Job Description: Senior Data EngineerLocation: Atlanta, GAType: Full-timeSalary : $130K/annumVisa : USC/GC/H4 EADNO OPT and H1B please9-10+ Years of candidate can apply for this role Role Overview:We are seeking a skilled Data Engineer with hands-on experience in Snowflake to join our dynamic team. In this role, you will play a crucial part in designing,...

  • Data Engineer

    6 days ago

    Atlanta, United States S.P. Richards Company Full time

    Job DescriptionJob DescriptionSummary: Seeking a skilled Data Engineer to join our dynamic Business Intelligence team. Responsible for designing, developing, and maintaining our data infrastructure, ensuring reliable and efficient data flow and accessibility for analysis, decision-making, and internal data processes. Must possess a deep understanding of...

  • Data Engineer with ML Engineering Experience

    19 hours ago

    Atlanta, United States Highbrow LLC Full time

    Job DescriptionEducation and Experience· Bachelor's degree in Computer Science, Data Science, Engineering, or a related field. A Master's degree or relevant certifications (e.g., Google Professional Data Engineer) is a plus.· 5+ years of experience in data engineering, with at least 2-3 years of experience in machine learning engineering or deploying ML...

  • Lead Data Engineer

    3 weeks ago

    Atlanta, United States ATech Placement Full time

    Job DescriptionJob DescriptionWe are looking for an experienced Lead Data Engineer to mentor and lead the buildout and support of our Data Lake solutions. This person will be a hands-on partner who can work across various business units and technical teams to build the next generation data solutions for Company. The required skills are:· 7+ yrs data...

  • Data Engineering Lead

    2 weeks ago

    Atlanta, United States Booz Allen Full time

    Data Engineering LeadThe Opportunity:Ever-expanding technology like IoT, machine learning, and artificial intelligence means that there’s more structured and unstructured data available today than ever before. As a data engineer, you know that organizing big data can yield pivotal insights when it’s gathered from disparate sources. We need a big data...

  • Big Data Engineer

    2 weeks ago

    Atlanta, United States TEKsystems Full time

    Description: Overview Interested in building the next generation platform that handles petabytes of data and powers data insight creation using cutting edge data science technology? Excited about using Big Data, AI/Machine Learning and Data Analytics to deliver actionable insights? We are a dynamic team of top-notch engineers who are committed to make a...

  • AWS Data Engineer

    6 months ago

    Atlanta, United States Mueller Co. LLC Full time

    Job Summary : AWS-BI Developer with strong experience across the entire Cloud Data stack. The ideal candidate will have extensive experience in data pipelines (ELT/ETL), data warehousing and dimensional modeling, and curation of data sets for Data Scientists and Business Intelligence users. Experience in building Data Lake. This candidate will also have...

  • Big Data Engineer

    1 week ago

    Atlanta, United States TEKsystems Full time

    *Description:* Overview Interested in building the next generation platform that handles petabytes of data and powers data insight creation using cutting edge data science technology? Excited about using Big Data, AI/Machine Learning and Data Analytics to deliver actionable insights? We are a dynamic team of top-notch engineers who are committed to make a...

  • Senior Data Engineer

    2 months ago

    atlanta, United States Compunnel Inc. Full time

    Job Title: Data Engineer-Oracle/ETLLocation: Atlanta,GA | Boston, MA | Raleigh, NC| Coppell, TX Duration: 12+ MonthsMust Have's: • Strong Oracle SQL tuning and PL/SQL development (~70%)• Any ETL tool (Informatica, SSI, TeraData, SnapLogic) (~10%)• Familiarity with AutosysNice to Have's: • AWS/EKS• CI/CD tools (Jenkins, GitHub)•...

  • Senior Data Engineer

    2 months ago

    Atlanta, United States Compunnel Inc. Full time

    Job Title: Data Engineer-Oracle/ETLLocation: Atlanta,GA | Boston, MA | Raleigh, NC| Coppell, TX Duration: 12+ MonthsMust Have's: • Strong Oracle SQL tuning and PL/SQL development (~70%)• Any ETL tool (Informatica, SSI, TeraData, SnapLogic) (~10%)• Familiarity with AutosysNice to Have's: • AWS/EKS• CI/CD tools (Jenkins, GitHub)•...

  • Senior Data Engineer

    1 month ago

    atlanta, United States Compunnel Inc. Full time

    Job Title: Data Engineer-Oracle/ETLLocation: Atlanta,GA | Boston, MA | Raleigh, NC| Coppell, TX Duration: 12+ MonthsMust Have's: • Strong Oracle SQL tuning and PL/SQL development (~70%)• Any ETL tool (Informatica, SSI, TeraData, SnapLogic) (~10%)• Familiarity with AutosysNice to Have's: • AWS/EKS• CI/CD tools (Jenkins, GitHub)•...

  • Cloud Data Engineer Position

    1 month ago

    Atlanta, Georgia, United States Diverse Lynx Full time

    Job Title: Google Cloud Data EngineerJob Type: Contract RoleJob Description:As a Cloud Data Engineer at Diverse Lynx LLC, you will be responsible for designing, developing, and maintaining large-scale data processing systems on Google Cloud Platform. Your expertise in Google Cloud SDK and API scripting, as well as experience with Hadoop, Hive, MapReduce, and...

  • Sr. Data Engineer

    2 weeks ago

    Atlanta, United States Furniture.com America LLC Full time

    Job DescriptionJob DescriptionSr Data Engineer Furniture.com Furniture.com is a high growth consumer tech start up that is addressing fundamental challenges in the $200B US furniture space. The company is a B2B and B2C digital aggregator with a simple mission: To make finding furniture easy. For consumers, we make it easy to find the right furniture by...