Senior ETL Engineer

5 days ago


Pittsburgh, Pennsylvania, United States Smart IT Frame LLC Full time
Job Title: Senior ETL Engineer

Job Summary:

We are seeking a highly skilled Senior ETL Engineer to join our team at Smart IT Frame LLC. The ideal candidate will have 8 to 10 years of experience in designing, developing, and optimizing ETL processes, with strong expertise in Hadoop, PySpark, Spark, and Dask.

Key Responsibilities:

Design, develop, and optimize ETL pipelines to efficiently extract, transform, and load data from various sources.

Implement and manage Hadoop-based solutions for large-scale data processing and storage, ensuring optimal performance and scalability.

Develop and maintain PySpark applications for processing and analyzing big data, leveraging Spark's capabilities for distributed computing.

Utilize Apache Spark for data processing, including batch and streaming data applications, ensuring high performance and reliability.

Set up and manage the Dask framework for parallel computing and distributed data processing, optimizing workflows and handling large-scale data tasks.

Configure and maintain JupyterHub environments for collaborative data analysis and notebook sharing, ensuring a seamless user experience.

Requirements:

Bachelor's or Master's degree in Computer Science, Data Engineering, Software Engineering, or a related field.

8 to 10 years of experience in data engineering, with strong expertise in ETL processes, Hadoop, PySpark, Spark, and Dask.

Proven experience setting up and managing JupyterHub environments.

Preferred Skills:

Cloud Platforms: Familiarity with cloud-based data platforms for deploying and managing big data solutions.

Data Visualization: Experience with data visualization tools (e.g., Tableau, Power BI) for creating insightful visualizations and reports.

Data Engineering Tools: Knowledge of additional data engineering tools and frameworks, including ETL and data integration technologies.

Agile Methodologies: Experience with Agile development practices and methodologies for managing data projects and tasks.

  • Pittsburgh, Pennsylvania, United States System One Full time

    Job Title: Senior Informatica ETL DeveloperJob Summary: We are seeking a highly skilled Senior Informatica ETL Developer to lead our offshore team. The ideal candidate will have 10+ years of experience in ETL development, including 5+ years of US experience, and expertise in Informatica, Oracle, and PL/SQL.Key Responsibilities:Lead junior offshore...

  • Senior Data Engineer

    1 month ago


    Pittsburgh, Pennsylvania, United States YinzCam, Inc. Full time

    About YinzCam, Inc.We're a leading provider of digital products for professional sports teams and leagues worldwide.The RoleWe're seeking a highly skilled Senior Data Engineer to join our team. As a key member of our data engineering team, you'll be responsible for collecting, organizing, and analyzing large datasets to drive business insights.Your primary...

  • Senior Data Engineer

    2 weeks ago


    Pittsburgh, Pennsylvania, United States YinzCam, Inc. Full time

    About YinzCam, Inc.We're a leading provider of digital products for professional sports teams and leagues worldwide.The RoleWe're seeking a highly skilled Senior Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining large-scale data pipelines to support our analytics and...


  • Pittsburgh, Pennsylvania, United States YinzCam, Inc. Full time

    About YinzCam, Inc.We are a leading provider of digital products for professional sports teams and leagues worldwide.The RoleAs a Senior Data Engineer, you will be responsible for collecting, organizing, and analyzing large datasets to provide actionable insights for our clients.Key ResponsibilitiesDesign and implement new ETL workflows using Python to...


  • Pittsburgh, Pennsylvania, United States YinzCam Full time

    About YinzCamYinzCam is a leading provider of digital products for professional sports teams and leagues worldwide. Our innovative solutions are used by over 200 teams, including clubs in the NFL, NBA, MLS, NHL, LigaMX, AFL, and more.The RoleWe are seeking a highly skilled Data Engineer to join our team. As a key member of our data engineering team, you will...

  • Data Engineer II

    3 weeks ago


    Pittsburgh, Pennsylvania, United States Edward Jones Full time

    Data Engineer II Job DescriptionWe are seeking a highly skilled Data Engineer II to join our team at Edward Jones. As a Data Engineer II, you will be responsible for designing, developing, and implementing large-scale data engineering projects using data warehousing ETL tools, such as Pentaho.Key Responsibilities:Design and develop ETL/ELT solutions based on...


  • Pittsburgh, Pennsylvania, United States System One Full time

    Job Title: Senior Oracle PL/SQL DeveloperWe are seeking a highly skilled Senior Oracle PL/SQL Developer to join our team at System One. As a key member of our development team, you will be responsible for designing, developing, and maintaining complex database systems and software platforms.Key Responsibilities:* Design and develop software systems using...


  • Pittsburgh, Pennsylvania, United States Saxon Global Full time

    Job Title: Senior Data ArchitectWe are seeking a highly skilled Senior Data Architect to join our team at Saxon Global. As a Senior Data Architect, you will be responsible for designing and implementing data models, architectures, and ETL processes to support analytics and reporting needs.### Responsibilities* Collaborate with business users, SMEs, and peer...


  • Pittsburgh, Pennsylvania, United States Saxon Global Full time

    Senior Database SpecialistWe are seeking a senior-level database specialist to collaborate with our BNY Mellon information security team on a 6-month contract. If you possess expert SQL querying skills and a good foundation in Python or Java programming languages, we encourage you to apply.Responsibilities:Develop and optimize complex SQL queries, views, and...


  • Pittsburgh, Pennsylvania, United States Saxon Global Full time

    Job SummaryWe are seeking a highly skilled Data Architect to join our team at Saxon Global. As a Data Architect, you will be responsible for designing and implementing data models, architectures, and ETL processes to support analytics and reporting needs of the organization.The ideal candidate will have a strong background in data modeling, data management,...


  • Pittsburgh, Pennsylvania, United States Saxon Global Full time

    Senior SQL Developer OpportunityWe are seeking a highly skilled Senior SQL Developer to join our team at Saxon Global. As a key member of our information security team, you will play a critical role in developing and optimizing complex SQL queries, views, and stored procedures on MS SQL Server.Responsibilities:Design and implement efficient SQL queries,...


  • Pittsburgh, Pennsylvania, United States Pike Corporation Full time

    Job DescriptionJob Title: Senior Electrical EngineerJob Summary:Pike Engineering is seeking a highly skilled Senior Electrical Engineer to join our team. As a Senior Electrical Engineer, you will be responsible for providing technical expertise and guidance in the identification, analysis, and resolution of complex engineering and technical problems.Key...


  • Pittsburgh, Pennsylvania, United States Hatch LLC Full time

    Job Title: Senior Mechanical EngineerWe are seeking a highly motivated Senior Mechanical Engineer to join our Project Delivery Group (PDG) at Hatch LLC. As a Senior Mechanical Engineer, you will lead and manage a team of engineers to deliver high-quality engineering services for projects within the USA Region and globally.Key Responsibilities:Lead and manage...


  • Pittsburgh, Pennsylvania, United States Altezas Full time

    Job Title: Senior Project EngineerAltezas LLC is seeking an experienced Senior Project Engineer to join our team in the Pittsburgh, PA area. As a Senior Project Engineer, you will be responsible for managing and delivering projects in the water, wastewater, gas, and telecommunications sectors.Responsibilities:Provide leadership and oversight to a diverse...


  • Pittsburgh, Pennsylvania, United States Hatch LLC Full time

    Hatch is seeking a Senior Structural Engineer to join our Project Delivery Group in Pittsburgh, PA.Job SummaryThe Senior Structural Engineer will provide leadership and technical expertise in the design of steel and concrete structures for industrial settings.Key Responsibilities:Lead teams of engineers and designers on complex projectsDevelop and implement...

  • Senior Civil Engineer

    2 weeks ago


    Pittsburgh, Pennsylvania, United States Toole Design Group Full time

    About the RoleWe are seeking a highly experienced Senior Engineer to join our team at Toole Design Group in Pittsburgh, PA. As a Senior Engineer, you will lead and support an interdisciplinary team to deliver multimodal transportation infrastructure projects.Key Responsibilities:Lead and mentor a team of engineers to deliver high-quality construction...


  • Pittsburgh, Pennsylvania, United States System One Full time

    System One is seeking a highly skilled Senior Geotechnical Engineer to lead and manage projects for an industry-leading client in the Pittsburgh area.Your key responsibilities will include:Directly managing projects from planning to completionServing as Engineer-of-Record on design and design-build projectsLeading and mentoring a team of engineers and...


  • Pittsburgh, Pennsylvania, United States talisman Full time

    Talisman Partners with Esteemed Consultancy to Hire Senior Traffic EngineerWe are partnering with a renowned consultancy to add a Senior Traffic Engineer to their team in Pittsburgh, PA. As a Senior Traffic Engineer, you will play a pivotal role in leading and managing transportation projects, including traffic analysis, roadway design, signal design, and...


  • Pittsburgh, Pennsylvania, United States International Search Partners Full time

    International Search Partners is seeking a highly skilled Senior Mechanical Engineer to join our team. The ideal candidate will have a strong background in mechanical engineering and experience in the design and implementation of thermal process systems.Key Responsibilities:The Senior Mechanical Engineer will be responsible for the design and development of...


  • Pittsburgh, Pennsylvania, United States NextDeavor Full time

    Senior Mechanical EngineerNextDeavor is seeking a highly skilled Senior Mechanical Engineer to join our team. As a key player in our engineering department, you will be responsible for managing multiple projects simultaneously, encompassing both design and construction aspects.Key Responsibilities:Creation of contract documents and specifications for...