Data Engineer

2 days ago


Pittsburgh, United States RapidBrains Full time

Job Post: Data Engineer

Experience: 8 - 11 Years

Location: Pittsburgh

Duration : 12 Months

Notice Period: 0 - 30 Days


Job Purpose

Design, construct, and maintain scalable data management systems using Azure Databricks, ensuring they meet end-user expectations. Supervise the upkeep of existing data infrastructure workflows and create data processing pipelines utilizing Databricks Notebooks, Spark SQL, Python, and other tools.


Key Responsibilities

  • Interpret business requirements and collaborate with internal resources and application vendors.
  • Design, develop, and maintain Databricks solutions and data quality rules.
  • Troubleshoot and resolve data-related issues.
  • Configure and create data models and quality rules to meet customer needs.
  • Work with multiple database platforms, including MSSQL and Oracle.
  • Optimize PySpark/Python code and SQL queries to improve performance.
  • Design and implement ETL pipelines and effective data models.
  • Document data pipeline architecture and processes.
  • Communicate effectively with business and technology stakeholders.
  • Deliver results under tight deadlines while adhering to quality standards.


Key Competencies

  • Experience: 8+ years in data engineering with expertise in Azure Databricks, MSSQL, LakeFlow, and Python.
  • Proficient in creating and optimizing data pipelines using Databricks Notebooks, Spark SQL, and PySpark.
  • Knowledge of Azure services like Azure Data Lake Storage and Azure SQL Data Warehouse.
  • Expertise in data warehousing, ETL pipeline development, and data governance.
  • Hands-on experience with data quality rules using Databricks and platforms like IDQ.
  • Strong problem-solving, analytical, and organizational skills.
  • Ability to work independently and collaboratively in cross-functional teams.

Skills & Requirements

  • Technical Skills: Azure Databricks, PySpark, SQL (MSSQL, Spark SQL), Azure Data Lake Storage, ETL, Data Modeling and Governance, Data Warehousing, Python.
  • Soft Skills: Strong communication, problem-solving, and attention to detail.


  • Data Engineer

    4 weeks ago


    Pittsburgh, Pennsylvania, United States Synergy Staffing Full time

    Job Title: Data EngineerJob Summary: We are seeking a skilled Data Engineer to design, build, and maintain data pipelines and data models using Snowflake and Azure technologies. The ideal candidate will work closely with data analysts, data scientists, and other stakeholders to ensure data is accessible, reliable, and ready for analysis.Key...

  • Data Engineer

    4 weeks ago


    Pittsburgh, United States Synergy Staffing Full time

    Job DescriptionJob DescriptionJob Title: Data EngineerLocation: Pittsburgh, PA(may have to come on-site a couple times a year)Job Type: Contract to Hire Job Summary: As a Data Engineer, you will be responsible for designing, building, and maintaining data pipelines and data models using Snowflake and Azure technologies. You will work closely with data...

  • Data Engineer

    4 weeks ago


    Pittsburgh, United States Synergy Staffing Full time

    Job DescriptionJob DescriptionJob Title: Data EngineerLocation: Pittsburgh, PA(may have to come on-site a couple times a year)Job Type: Contract to HireJob Summary: As a Data Engineer, you will be responsible for designing, building, and maintaining data pipelines and data models using Snowflake and Azure technologies. You will work closely with data...

  • Data Engineer

    4 weeks ago


    Pittsburgh, Pennsylvania, United States PNC Full time

    Job SummaryPNC is seeking a highly skilled Data Engineer to join our Enterprise Fraud Organization. As a key member of our team, you will be responsible for designing, building, and maintaining data services to support our business objectives. Key Responsibilities- Design and develop data services to meet business requirements- Collaborate with...

  • Senior Data Engineer

    4 weeks ago


    Pittsburgh, Pennsylvania, United States Highmark Health Full time

    Job Title: Senior Data EngineerJob Summary:Highmark Health is seeking a Senior Data Engineer to lead our data engineering team in delivering high-performance data solutions. As a key member of our team, you will design, build, and maintain well-engineered data solutions in various environments, including traditional data warehouses, Big Data solutions, and...

  • Data Engineer II

    4 days ago


    Pittsburgh, United States TeleTracking Technologies, Inc. Full time

    Job DescriptionJob DescriptionJob Title: Data Engineer IIOrganizational RelationshipReports to: Manager, Software EngineeringDirect Reports: NoneInteracts with: Internal Business UnitsEducation Degree Discipline Required or Preferred: Bachelors Computer Science, Software Engineering, or related Required Masters Computer Science, Software Engineering, or...


  • pittsburgh, United States Tredence Inc. Full time

    About Tredence­:Tredence focuses on last mile delivery of insights into actions by uniting its strengths in business analytics, data science, and software engineering. The largest companies across industries are engaging with Tredence and deploying its prediction and optimization solutions at scale –empowering end users to improve decision making....


  • pittsburgh, United States Tredence Inc. Full time

    About Tredence­:Tredence focuses on last mile delivery of insights into actions by uniting its strengths in business analytics, data science, and software engineering. The largest companies across industries are engaging with Tredence and deploying its prediction and optimization solutions at scale –empowering end users to improve decision making....

  • Senior Data Engineer

    2 weeks ago


    Pittsburgh, United States Tredence Inc. Full time

    About Tredence­:Tredence focuses on last mile delivery of insights into actions by uniting its strengths in business analytics, data science, and software engineering. The largest companies across industries are engaging with Tredence and deploying its prediction and optimization solutions at scale –empowering end users to improve decision making....

  • Senior Data Engineer

    4 weeks ago


    Pittsburgh, Pennsylvania, United States YinzCam Full time

    About YinzCamYinzCam is a leading provider of digital products for professional sports teams and leagues worldwide. Our innovative solutions are used by over 200 teams, including clubs in the NFL, NBA, MLS, NHL, LigaMX, AFL, and more.The RoleWe are seeking a highly skilled Data Engineer to join our team. As a key member of our data engineering team, you will...

  • Product Manager

    5 days ago


    Pittsburgh, United States Candela Search Full time

    About the Company - Product Management at DICK’S Sporting Goods specializes in delivering retail excellence along every aspect of the customer journey. Its mission focuses on crafting intuitive, easy-to-use products that span across the business.About the Role -The Data Engineering Group Product Manager will define the strategic vision for Data within the...

  • Product Manager

    5 days ago


    Pittsburgh, United States Candela Search Full time

    About the Company - Product Management at DICK’S Sporting Goods specializes in delivering retail excellence along every aspect of the customer journey. Its mission focuses on crafting intuitive, easy-to-use products that span across the business.About the Role -The Data Engineering Group Product Manager will define the strategic vision for Data within the...

  • Big Data Engineer

    4 weeks ago


    Pittsburgh, Pennsylvania, United States Tata Consultancy Services Full time

    Job SummaryWe are seeking a highly skilled Data Processing Specialist to join our team at Tata Consultancy Services. As a key member of our big data infrastructure team, you will be responsible for designing, developing, and documenting Hadoop applications.Key ResponsibilitiesDesign and develop Hadoop applications to process large datasetsInstall, configure,...


  • Pittsburgh, Pennsylvania, United States Jobot Full time

    About UsWe are an analytics company that was recently acquired by a giant in our industry. This has been a great moment for us because we still operate independently with the financial stability of a publicly traded company. We work in the public space and have built/ are building apps that will redefine the way public sentiment is captured and reported. We...


  • Pittsburgh, Pennsylvania, United States University of Pittsburgh Full time

    The University of Pittsburgh is seeking a highly skilled Data Warehouse Engineer/DBA to join its team. The ideal candidate will have a strong background in database administration and data engineering, with a minimum of five years of experience working in a MS SQL Server environment.The successful candidate will be responsible for designing and implementing...

  • Senior Data Engineer

    4 weeks ago


    Pittsburgh, Pennsylvania, United States YinzCam, Inc. Full time

    About YinzCam, Inc.We are a leading provider of digital products for professional sports teams and leagues worldwide.The RoleAs a Senior Data Engineer, you will be responsible for collecting, organizing, and analyzing large datasets to provide actionable insights for our clients.Key ResponsibilitiesDesign and implement new ETL workflows using Python to...

  • Data Engineer

    3 weeks ago


    Pittsburgh, Pennsylvania, United States Innovation Works, Inc. Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Innovation Works, Inc. As a Data Engineer, you will play a critical role in deploying machine learning models in production, assisting data scientists and machine learning engineers with coding and devops tasks.Key Responsibilities:Design and implement microservices to run...


  • Pittsburgh, Pennsylvania, United States FTI Consulting Full time

    About the RoleThe Datawarehouse / Business Intelligence Developer is responsible for designing and maintaining the overall lifecycle of Datawarehouse / BI solutions, all related infrastructure and services, in a 24 x 7 environment for the purposes of reporting, decision support and analysis.Core responsibilities include managing Azure Data Factory pipelines,...

  • Data Engineer

    4 weeks ago


    Pittsburgh, Pennsylvania, United States Syntricate Technologies Full time

    Job Requirements:At Syntricate Technologies, we are seeking a highly skilled Data Engineer - Cloud and Python Expert to join our team. The ideal candidate will have strong Databricks development experience with GCP and Python, as well as 5 years of experience as a developer with demonstrated skills in Databricks, Python/Pyspark, GCP: Big Query and...


  • Pittsburgh, Pennsylvania, United States University of Pittsburgh Full time

    Evaluating business requirements is a key responsibility of this role. The ideal candidate will have expertise in creating advanced data ingestion processes and modeling. They will also provide extensive support for databases and relevant services, ensuring data quality and delivery. This position requires a strong mentor who can train and assist lower-level...