ETL Developer

1 month ago

Pittsburgh, United States Smart IT Frame LLC Full time

JD:


Role : ETL Engineer

Location: Remote

No. of years of experience: 8+ years’

Hardcore Technical - ETL

  • ETL experience Strong, 8+ years
  • Hadoop Strong, 8+ years
  • PySpark Strong, 8+ years
  • Spark Strong, 8+ years
  • Dask Required, should be able to set up DASK framework
  • JupyterHub Setup Experience required


Location: Pittsburgh preferred, but remote is fine


Job Description:

We are seeking an experienced ETL Engineer to join our team. The ideal candidate will have 8 to 10 years of experience in designing, developing, and optimizing ETL processes, with strong expertise in Hadoop, PySpark, Spark, and Dask. The role involves setting up and managing data workflows, ensuring data integrity and efficiency, and collaborating with cross-functional teams to meet business needs. If you are passionate about data engineering and have a track record of delivering high-quality ETL solutions, we encourage you to apply.

Key Responsibilities:

• ETL Processes: Design, develop, and optimize ETL pipelines to efficiently extract, transform, and load data from various sources.

• Hadoop: Implement and manage Hadoop-based solutions for large-scale data processing and storage, ensuring optimal performance and scalability.

• PySpark: Develop and maintain PySpark applications for processing and analyzing big data, leveraging Spark's capabilities for distributed computing.

• Spark: Utilize Apache Spark for data processing, including batch and streaming data applications, ensuring high performance and reliability.

• Dask: Set up and manage the Dask framework for parallel computing and distributed data processing, optimizing workflows and handling large-scale data tasks.

• JupyterHub Setup: Configure and maintain JupyterHub environments for collaborative data analysis and notebook sharing, ensuring a seamless user experience.

Must-Have Skills:

• ETL Expertise: Strong experience in designing and managing ETL processes, including data extraction, transformation, and loading.

• Hadoop: Proven proficiency with Hadoop ecosystem tools and technologies for big data processing and storage.

• PySpark: Extensive experience with PySpark for data processing, including writing and optimizing Spark jobs.

• Spark: Deep understanding of Apache Spark for both batch and real-time data processing.

• Dask: Hands-on experience setting up and managing the Dask framework for distributed computing and large-scale data processing.

• JupyterHub Setup: Experience configuring and maintaining JupyterHub for data analysis and notebook collaboration.

• Communication Skills: Strong verbal and written communication skills, with the ability to articulate complex technical concepts to diverse audiences.

• Independent Work: Ability to work independently, manage multiple tasks, and deliver high-quality results with minimal supervision.

Good-to-Have Skills:

• Cloud Platforms: Familiarity with cloud-based data platforms for deploying and managing big data solutions.

• Data Visualization: Experience with data visualization tools (e.g., Tableau, Power BI) for creating insightful visualizations and reports.

• Data Engineering Tools: Knowledge of additional data engineering tools and frameworks, including ETL and data integration technologies.

• Agile Methodologies: Experience with Agile development practices and methodologies for managing data projects and tasks.

Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Software Engineering, or a related field.

• 8 to 10 years of experience in data engineering, with strong expertise in ETL processes, Hadoop, PySpark, Spark, and Dask.

• Proven experience setting up and managing JupyterHub environments.

  • ETL Developer

    3 weeks ago

    Pittsburgh, United States Tata Consultancy Services Full time

    Job DescriptionRole: ETL DeveloperLocation: Pittsburgh PA / Cleveland OHWork Authorization: USC , GC, OPT EAD OnlyEXP 8 +Design, implement, and continuously expand data pipelines by performing extraction, transformation, and loading activities.Gather requirements and business process knowledge in order to transform the data in a way that's geared towards the...

  • Senior ETL Developer

    2 weeks ago

    pittsburgh, United States Tata Consultancy Services Full time

    Job DescriptionRole: ETL DeveloperDesign, implement, and continuously expand data pipelines by performing extraction, transformation, and loading activities.Gather requirements and business process knowledge in order to transform the data in a way that's geared towards the needs of end users.Maintain and improve already existing processes.Ensure that the...

  • Senior ETL Developer

    3 weeks ago

    pittsburgh, United States Tata Consultancy Services Full time

    Job DescriptionRole: ETL DeveloperDesign, implement, and continuously expand data pipelines by performing extraction, transformation, and loading activities.Gather requirements and business process knowledge in order to transform the data in a way that's geared towards the needs of end users.Maintain and improve already existing processes.Ensure that the...

  • Senior ETL Developer

    3 weeks ago

    Pittsburgh, United States Tata Consultancy Services Full time

    Job DescriptionRole: ETL DeveloperDesign, implement, and continuously expand data pipelines by performing extraction, transformation, and loading activities.Gather requirements and business process knowledge in order to transform the data in a way that's geared towards the needs of end users.Maintain and improve already existing processes.Ensure that the...

  • Senior Informatica ETL Developer

    4 weeks ago

    Pittsburgh, Pennsylvania, United States System One Full time

    Job Title: Senior Informatica ETL DeveloperJob Summary: We are seeking a highly skilled Senior Informatica ETL Developer to lead our offshore team. The ideal candidate will have 10+ years of experience in ETL development, including 5+ years of US experience, and expertise in Informatica, Oracle, and PL/SQL.Key Responsibilities:Lead junior offshore...

  • ETL Developer

    3 weeks ago

    Pittsburgh, PA, United States Tata Consultancy Services Full time

    Job DescriptionRole: ETL DeveloperLocation: Pittsburgh PA / Cleveland OHWork Authorization: USC , GC, OPT EAD OnlyEXP 8 +Design, implement, and continuously expand data pipelines by performing extraction, transformation, and loading activities.Gather requirements and business process knowledge in order to transform the data in a way that's geared towards the...

  • ETL Developer

    4 weeks ago

    Pittsburgh, Pennsylvania, United States Synechron Full time

    Job Title: ETL/Informatica Developer - PowerExchange and PowerCenterLocation: RemoteJob Type: Full-timeAbout the Role:We are seeking an experienced ETL/Informatica Developer to join our team. The successful candidate will have a strong background in Informatica PowerExchange and PowerCenter, as well as experience with data analysis and data...

  • ETL Lead

    3 weeks ago

    pittsburgh, United States Mastech Digital Full time

    Role: ETL Informatica Lead Location: Pittsburgh, PA (Hybrid) Job type: Full time/ Permanent role Responsibilities: Lead the design, development, and implementation of ETL processes using Informatica PowerCenter and related technologies. Collaborate with business stakeholders, data architects, and analysts to understand data requirements and translate them...

  • ETL Lead

    2 weeks ago

    Pittsburgh, United States Mastech Digital Full time

    Role: ETL Informatica Lead Location: Pittsburgh, PA (Hybrid) Job type: Full time/ Permanent role Responsibilities: Lead the design, development, and implementation of ETL processes using Informatica PowerCenter and related technologies. Collaborate with business stakeholders, data architects, and analysts to understand data requirements and translate them...

  • ETL Lead

    3 weeks ago

    Pittsburgh, United States Mastech Digital Full time

    Role: ETL Informatica Lead Location: Pittsburgh, PA (Hybrid) Job type: Full time/ Permanent role Responsibilities: Lead the design, development, and implementation of ETL processes using Informatica PowerCenter and related technologies. Collaborate with business stakeholders, data architects, and analysts to understand data requirements and translate them...

  • Senior ETL Developer

    3 weeks ago

    Pittsburgh, PA, United States Tata Consultancy Services Full time

    Job DescriptionRole: ETL DeveloperDesign, implement, and continuously expand data pipelines by performing extraction, transformation, and loading activities.Gather requirements and business process knowledge in order to transform the data in a way that's geared towards the needs of end users.Maintain and improve already existing processes.Ensure that the...

  • Senior ETL Engineer

    4 weeks ago

    Pittsburgh, Pennsylvania, United States Smart IT Frame LLC Full time

    Job Title: Senior ETL EngineerJob Summary:We are seeking a highly skilled Senior ETL Engineer to join our team at Smart IT Frame LLC. The ideal candidate will have 8 to 10 years of experience in designing, developing, and optimizing ETL processes, with strong expertise in Hadoop, PySpark, Spark, and Dask.Key Responsibilities: Design, develop, and optimize...

  • ETL/Informatica Developer

    4 weeks ago

    Pittsburgh, Pennsylvania, United States Synechron Full time

    Job Title: ETL/Informatica Developer - PowerExchange and PowerCenterLocation: Global OpportunitiesFull Time/ PermanentContact: The Role:Responsibilities:Develop and implement complex ETL processes using Informatica PowerExchange and PowerCenter, ensuring high-quality data integration and transformation.Collaborate with cross-functional teams to design and...

  • PowerExchange - ETL/Informatica Developer

    1 month ago

    pittsburgh, United States Synechron Full time

    Job Title: ETL/Informatica Developer - PowerExchange and PowerCenter Location: Pittsburgh, PA/Dallas, TXFull Time/ PermanentContact: Syed.Khaliq@Synechron.comThe Role:Responsibilities:Must have a minimum of 8 years of Informatica ETL development experience with good Oracle database background.Must have experience in Informatica Power Exchange and...

  • PowerExchange - ETL/Informatica Developer

    1 month ago

    Pittsburgh, United States Synechron Full time

    Job Title: ETL/Informatica Developer - PowerExchange and PowerCenter Location: Pittsburgh, PA/Dallas, TXFull Time/ PermanentContact: Syed.Khaliq@Synechron.comThe Role:Responsibilities:Must have a minimum of 8 years of Informatica ETL development experience with good Oracle database background.Must have experience in Informatica Power Exchange and...

  • PowerExchange - ETL/Informatica Developer

    1 month ago

    pittsburgh, United States Synechron Full time

    Job Title: ETL/Informatica Developer - PowerExchange and PowerCenter Location: Pittsburgh, PA/Dallas, TXFull Time/ PermanentContact: Syed.Khaliq@Synechron.comThe Role:Responsibilities:Must have a minimum of 8 years of Informatica ETL development experience with good Oracle database background.Must have experience in Informatica Power Exchange and...

  • ETL Lead

    3 weeks ago

    Pittsburgh, PA, United States Mastech Digital Full time

    Role: ETL Informatica Lead Location: Pittsburgh, PA (Hybrid) Job type: Full time/ Permanent role Responsibilities: Lead the design, development, and implementation of ETL processes using Informatica PowerCenter and related technologies. Collaborate with business stakeholders, data architects, and analysts to understand data requirements and translate them...

  • PL/SQL Developer

    1 month ago

    pittsburgh, United States ConsultUSA Full time

    Description:Our client has an immediate need for a PL/SQL DeveloperRequirements:Bachelor’s degree requiredMaster’s degree is a plus5 – 7 years of experience in Oracle PL/SQL Developer/DBA/DA in Design, Development & Support Roles, and as ETL DeveloperExperience in creating SQL Queries, PL/SQL packages, stored procedures, triggers, and cursorsExperience...

  • PL/SQL Developer

    1 month ago

    Pittsburgh, United States ConsultUSA Full time

    Description:Our client has an immediate need for a PL/SQL DeveloperRequirements:Bachelor’s degree requiredMaster’s degree is a plus5 – 7 years of experience in Oracle PL/SQL Developer/DBA/DA in Design, Development & Support Roles, and as ETL DeveloperExperience in creating SQL Queries, PL/SQL packages, stored procedures, triggers, and cursorsExperience...

  • Informatica Developer

    1 month ago

    pittsburgh, United States Tata Consultancy Services Full time

    Role : ETL informatica developerLocation : Pittsburgh, PAJob Type: Full TimeExperience Required: 9+ YearsJob Description:Design, implement, and continuously expand data pipelines by performing extraction, transformation, and loading activities.Gather requirements and business process knowledge in order to transform the data in a way that's geared towards the...