Lead ETL Developer with PySpark

3 weeks ago


Minneapolis, United States Diverse Lynx Full time
Role: Lead ETL Developer with PySpark
Location: Minneapolis Downtown, MN (Onsite)
Job Type: Contract


Job Description:

ETL Developer (5+years) to create data pipeline ETL Jobs using AWS Glue and PySpark

Responsibilities:
  • Work with a scrum team(s) to deliver product stories according to priorities set by the business and the Product Owners. Interact with stakeholders
  • Provide knowledge transfer to other team members.
  • Creating and testing pipeline jobs locally using aws glue interactive session.
  • Performance tuning of PySpark jobs. AWS Athena to perform data analysis on Lake data populated into aws glue data catalog through aws glue crawlers.
Must Haves:
  • Responsible for designing, developing, and maintaining ETL processes to support data integration and business intelligence
  • initiatives. Need to closely work with stakeholders to understand data requirements and ensure efficient data flow and transformation using ETL tools and PySpark
  • Develop and implement ETL processes using with one of ETL tool and PySpark to extract, transform, and load data.4+ years of experience in ETL development with knowledge on Pyspark5+ years as an ETL Developer SQL expert AWS Glue WITH Python (PySpark) PySpark Data frame APISpark SQL
  • Knowledge in AWS services (e.g., DMS, S3, RDS, Redshift, Step Function)
Nice to Haves:
  • ETL development experience with tools e.g., SAP BODS, Informatica.
  • Good understanding of version control tools like Git, GitHub, TortoiseHg.
  • Financial services experience
  • Agile


Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.

  • Minneapolis, Minnesota, United States Diverse Lynx Full time

    Job Overview:We are looking for a highly experienced PySpark ETL Developer Lead to join our team at Diverse Lynx LLC. As a key member of our data pipelining team, you will be responsible for designing, developing, and maintaining efficient ETL processes using AWS Glue and PySpark.Key Responsibilities:Design and develop ETL processes using AWS Glue and...


  • Minneapolis, Minnesota, United States K&K Global Talent Solutions Inc. Full time

    Job OverviewWe are seeking a highly skilled Lead Data Integration Engineer to join our team at K&K Global Talent Solutions Inc. in this key role, you will be responsible for designing, developing, and maintaining ETL processes to support data integration and business intelligence initiatives.About the RoleThis is a full-time position with an estimated salary...


  • Minneapolis, Minnesota, United States Diverse Lynx Full time

    Job Description:Diverse Lynx LLC is seeking a highly skilled Senior Data Integration Specialist to lead our data pipelining efforts using PySpark. As a key member of our team, you will design, develop, and maintain efficient ETL processes to support data integration and business intelligence initiatives.Responsibilities:Work closely with stakeholders to...


  • Minneapolis, Minnesota, United States Accuro Group Full time

    We are seeking a highly skilled Data Integration Lead to join our team at Accuro Group in Minneapolis, MN. The ideal candidate will have 10+ years of experience in ETL development and a strong background in PySpark.Job DescriptionRole: Data Integration Lead with SparkLocation: Minneapolis, MN (Onsite)Experience: 10+ YearsDuration: Long-term contractJob...

  • ETL Lead

    2 weeks ago


    Minneapolis, United States ITC Infotech Full time

    Job Title: ETL Lead.Location: Minneapolis, MN. - This role would require you to be there in office for business hours for 2-3 days in a week.Job Type: Full-TimeAbout the Role:We are seeking an experienced ETL Developer to lead the conversion of DataStage ETL processes to SSIS, transitioning from Oracle to SQL Server source and target systems. The ideal...

  • ETL Lead

    2 months ago


    minneapolis, United States ITC Infotech Full time

    Job Title: ETL Lead.Location: Minneapolis, MN. - This role would require you to be there in office for business hours for 2-3 days in a week.Job Type: Full-TimeAbout the Role:We are seeking an experienced ETL Developer to lead the conversion of DataStage ETL processes to SSIS, transitioning from Oracle to SQL Server source and target systems. The ideal...

  • ETL Lead

    2 months ago


    Minneapolis, United States ITC Infotech Full time

    Job Title: ETL Lead.Location: Minneapolis, MN. - This role would require you to be there in office for business hours for 2-3 days in a week.Job Type: Full-TimeAbout the Role:We are seeking an experienced ETL Developer to lead the conversion of DataStage ETL processes to SSIS, transitioning from Oracle to SQL Server source and target systems. The ideal...


  • Minneapolis, Minnesota, United States Futran Tech Solutions Pvt. Ltd. Full time

    Job Description: We are looking for a skilled Azure Tech Lead with expertise in Pyspark to join our team at Futran Tech Solutions Pvt. Ltd. The ideal candidate will have hands-on experience in designing, implementing, and managing big data integration solutions using Azure and Apache Spark.The estimated salary for this role is $130-135K, which includes a...

  • ETL Developer

    4 weeks ago


    Minneapolis, United States Net2Source Inc. Full time

    Job Title: ETL DeveloperLocation: Minneapolis - MN - RemoteContract Required Skills: Python, Scala, SparkWe are seeking an experienced ETL Developer with 12+ years of experience to join our team.The ETL Developer will play a crucial role in designing and implementing scalable solutions that align with our business goals.ResponsibilitiesLead the design and...

  • ETL Developer

    2 weeks ago


    Minneapolis, United States Net2Source Inc. Full time

    Job Title: ETL DeveloperLocation: Minneapolis - MN - RemoteContract Required Skills: Python, Scala, SparkWe are seeking an experienced ETL Developer with 12+ years of experience to join our team.The ETL Developer will play a crucial role in designing and implementing scalable solutions that align with our business goals.ResponsibilitiesLead the design and...

  • ETL Developer

    2 weeks ago


    Minneapolis, United States Robert Half Full time

    Job DescriptionJob DescriptionWe are offering a long-term contract employment opportunity for a Development Engineer in the Services industry, located in Minneapolis, Minnesota. As a Development Engineer, you will be tasked with the creation, development, and implementation of ETL workflows, data processing tasks, and API integrations. Responsibilities:•...

  • ETL Developer

    6 days ago


    Minneapolis, United States VDart Full time

    Job Title: ETL Developer (SSIS)/API Integration Developer Location: Minneapolis, MN Duration: 6-month contract Job Responsibilities Design, develop and implement ETL workflows using SQL Server Integration Services (SSIS) to meet business and data processing needs. Integrate and process data from various sources like database tables, APIs, CSV, EXCEL, TXT,...

  • ETL Developer

    5 days ago


    Minneapolis, United States VDart Inc Full time

    Job DescriptionJob DescriptionJob Title: ETL Developer (SSIS)/API Integration Developer Location: Minneapolis, MNDuration: 6-month contractJob ResponsibilitiesDesign, develop and implement ETL workflows using SQL Server Integration Services (SSIS) to meet business and data processing needs.Integrate and process data from various sources like database tables,...


  • Minneapolis, Minnesota, United States Diverse Lynx Full time

    At Diverse Lynx LLC, we are committed to providing a fair and competitive compensation package to our employees.About the JobWe are seeking a highly skilled ETL Pipeline Developer to join our team. As a key member of our data integration team, you will be responsible for designing and maintaining ETL pipelines using Informatica PC/IDMC.Key...


  • Minneapolis, Minnesota, United States Diverse Lynx Full time

    Job Description:Data Integration SpecialistThe Role: As a Data Integration Specialist at Diverse Lynx, you will be responsible for designing, developing, and maintaining ETL processes to support data integration and business intelligence initiatives. You will work closely with stakeholders to understand data requirements and ensure efficient data flow and...

  • ETL Developer

    7 days ago


    Minneapolis, United States Insight Global Full time

    Job Description Our client, a global design practice, is looking for a ETL Developer to join their team of 8 individuals. The team consists of PowerBI Engineers, a ETL Programmer, a Business Analyst, and a Technical Manager. The ETL Developer will assist with the migration from Deltek Vision to Deltak VantagePoint, supporting enhancements, fine tuning, and...


  • Minneapolis, Minnesota, United States Syntricate Technologies Full time

    About Syntricate TechnologiesWe are a forward-thinking technology company dedicated to providing innovative solutions that propel our clients forward.Job DetailsThis is a full-time position, requiring a strong background in data engineering, software development, and team leadership.Responsibilities Lead the design, development, and implementation of...

  • Senior ETL Developer

    2 weeks ago


    Minneapolis, United States Robert Half Full time

    **this position is a 6 month contract role, hybrid in MinneapolisJob Duties:Take charge in designing, building and implementing ETL workflows using SQL Server Integration Services (SSIS) for data processing.Work with varying data sources like database tables, APIs, CSV, EXCEL, TXT, XML, JSON files etc.Create, optimize and manage TSQL, stored procedures,...

  • Senior ETL Developer

    2 weeks ago


    Minneapolis, United States Robert Half Full time

    **this position is a 6 month contract role, hybrid in MinneapolisJob Duties:Take charge in designing, building and implementing ETL workflows using SQL Server Integration Services (SSIS) for data processing.Work with varying data sources like database tables, APIs, CSV, EXCEL, TXT, XML, JSON files etc.Create, optimize and manage TSQL, stored procedures,...


  • Minneapolis, United States Robert Half Full time

    **this position is a 6 month contract role, hybrid in MinneapolisJob Duties:Take charge in designing, building and implementing ETL workflows using SQL Server Integration Services (SSIS) for data processing.Work with varying data sources like database tables, APIs, CSV, EXCEL, TXT, XML, JSON files etc.Create, optimize and manage TSQL, stored procedures,...