Data Engineer

3 weeks ago

Philadelphia Pennsylvania, United States City of Philadelphia Full time
Job Description

Data Engineering at the City is a unique opportunity for meaningful, exciting work and professional development using state-of-the art technologies and software development best practices. This is a position on a diverse team of engineers, analysts, and GIS specialists, contributing to developing, maintaining, and optimizing our data pipelines and integration strategies, assisting departments and agencies in innovating, and enhancing the City’s service to its residents. 

The data engineer/integration specialist will develop an intimate understanding of the City’s diverse data and contribute to improving the City’s data engineering infrastructure, pipelines, models, and integrations. The team uses a blend of open source, custom developed and off-the-shelf tools, including Python, Bash, SQL, DBT, GIS, Docker, Terraform, Apache Airflow, Jenkins, Postgres, PostGIS, AWS, GitHub, MuleSoft as an iPaaS and SAS API provides like ArcGIS Online and CARTO.

Some prominent services the team is responsible for include DataBridge, OIT’s enterprise data warehouse, AIS, the City’s custom address information systems which supports public address search and internal geocoding, and Databridge-Airflow, a custom workflow generator and orchestrator for departmental integrations ((url removed)) and open data publishing ((url removed)). Some examples of OIT’s commitment to serving the public through the effective and innovative integration of City open data include:

(url removed)

(url removed)

(url removed)

(url removed)

www.phila.gov/solarmap

www.phila.gov/stressmap

We are looking for a candidate who is not only technically skilled but also passionate about leveraging data to improve City operations and community life. This team is fundamental to the innovation of City services that directly affect residents. If you are eager to contribute to a team that values innovation, efficiency, service-oriented work and data-driven decision-making, we encourage you to apply.

Essential Functions

Working with business partners of varying technical ability to understand how their data is produced, stored, and updated, designing enhancements to optimize enterprise integrations and public data accessibility.

Design and develop new and expand and refine existing data pipelines using Apache Airflow, custom Python-based software, and other tools to connect diverse systems of record, centralized databases, and SaaS environments, improving data platform components, data governance, master data management, data standardization, enrichment and quality assurance.

Play a key role in enterprise integration and open data initiatives, developing and supporting critical systems like the City’s DataBridge, Databridge-Airflow and AIS.

Write clear, descriptive technical documentation for systems/applications, partner with data stewards of various technical levels and communicate technical details to non-technical stakeholders.

Competencies, Knowledge, Skills and Abilities

Competencies:

Clear and concise writing and communication skills.

Creative problem-solving and critical thinking.

Ability to work with data from diverse domains.

Efficient time management and the ability to manage multiple workflows simultaneously.

Ability to seek innovative opportunities and continuous improvement.

Knowledge and Skills:

Proficient in Python

Strong SQL skills and experience with databases (Postgres preferred).

Familiarity with Docker, bash and minimal Linux server administration.

Experience with cloud services (AWS preferred).

Understanding of Git/GitHub for version control and CI/CD pipelines.

Experience or ability to learn Airflow, DBT, Terraform, Kubernetes, Mulesoft.

Experience working with spatial datasets a plus.

Experience using command line interfaces.

Abilities:

To reason about, model and manipulate complex datasets.

To maintain clean and secure data environments.

To work effectively in a hybrid (on-premises and cloud) environment.

To communicate complex technical concepts in understandable terms.

  • Data Engineer

    1 week ago

    Philadelphia, United States Mondo Full time

    Job DescriptionJob DescriptionApply now: Data Engineer, this is a 100% remote opportunity. The start date is ASAP for this long-term contract position.Job Title: Data EngineerLocation-Type: 100% remoteStart Date Is: ASAPDuration: Long-term contractRate: $55-$63 Per Hour W2 The estimate above represents the candidate's typical compensation range at the...

  • Machine Learning Engineer

    1 month ago

    Philadelphia, United States Data-Core Systems, Inc. Full time

    As a Machine Learning Engineer you will take a lead role in executing deep learning (CNN LSTM) clustering and classification algorithm fractal and spectral data analysis feature engineering and synthetic data generation.Duties and Responsibilities Execute deep learning (CNN LSTM) Clustering and classification algorithm Fractal and spectral data...

  • Machine Learning Engineer

    1 month ago

    Philadelphia, Pennsylvania, United States Data-Core Systems, Inc. Full time

    As a Machine Learning Engineer you will take a lead role in executing deep learning (CNN LSTM) clustering and classification algorithm fractal and spectral data analysis feature engineering and synthetic data generation.Duties and Responsibilities Execute deep learning (CNN LSTM) Clustering and classification algorithm Fractal and spectral data analysis...

  • AWS Data engineer

    3 weeks ago

    Philadelphia, United States ScaleneWorks INC Full time

    The Opportunity:· This position requires to work hybrid with 3-4 days a week in office (Philadelphia, PA).· Design, implement, and maintain scalable, high-performance data pipelines on AWS· Collaborate with cross-functional teams including data scientists, software engineers, and business analysts to understand data requirements and develop appropriate...

  • AWS Data engineer

    3 weeks ago

    Philadelphia, United States ScaleneWorks INC Full time

    The Opportunity:· This position requires to work hybrid with 3-4 days a week in office (Philadelphia, PA).· Design, implement, and maintain scalable, high-performance data pipelines on AWS· Collaborate with cross-functional teams including data scientists, software engineers, and business analysts to understand data requirements and develop appropriate...

  • AWS Data engineer

    3 weeks ago

    Philadelphia, United States ScaleneWorks INC Full time

    The Opportunity:· This position requires to work hybrid with 3-4 days a week in office (Philadelphia, PA).· Design, implement, and maintain scalable, high-performance data pipelines on AWS· Collaborate with cross-functional teams including data scientists, software engineers, and business analysts to understand data requirements and develop appropriate...

  • Data Engineering Manager

    4 days ago

    Philadelphia, United States Perpay Inc. Full time

    About the Role As a Data Engineering Manager at Perpay, you will lead data engineers and work closely with data scientists and analysts to help drive our vision of building data products and insights for our stakeholders and customers, accelerating Perpay's mission to create inclusive financial products that improve the lives of our members. We are actively...

  • Senior Data Engineer, Data Lake

    2 weeks ago

    Philadelphia, United States Penn Interactive Full time

    Penn Interactive (PI) is an interactive gaming company headquartered in Philadelphia. PI is the digital arm of PENN Entertainment (NASDAQ: PENN), the largest regional casino operator in the U.S.). Our mission is to challenge the norms of the gaming industry by building an immersive interactive gaming experience that is responsible, innovative, and fun. We...

  • Flight Data Engineer

    1 week ago

    Philadelphia, United States Leonardo Full time

    Leonardo Helicopters is a multinational company, producing helicopters deployed in more than 150 countries across the globe. Leonardo Helicopters’ U.S. headquarters has been in Philadelphia since 1980 and is home to a world-class production facility, maintenance center, training academy and stellar engineering team. We offer competitive compensation,...

  • Flight Data Engineer

    1 week ago

    Philadelphia, United States Leonardo Full time

    Leonardo Helicopters is a multinational company, producing helicopters deployed in more than 150 countries across the globe. Leonardo Helicopters’ U.S. headquarters has been in Philadelphia since 1980 and is home to a world-class production facility, maintenance center, training academy and stellar engineering team. We offer competitive compensation,...

  • Senior Data Engineer, Data Lake

    2 weeks ago

    Philadelphia, Pennsylvania, United States Penn Interactive Full time

    Penn Interactive (PI) is an interactive gaming company headquartered in Philadelphia. PI is the digital arm of PENN Entertainment (NASDAQ: PENN), the largest regional casino operator in the U.S.). Our mission is to challenge the norms of the gaming industry by building an immersive interactive gaming experience that is responsible, innovative, and fun. We...

  • Senior Data Engineer

    6 days ago

    Philadelphia, United States PENN Interactive Full time

    Penn Interactive (PI) is an interactive gaming company headquartered in Philadelphia. PI is the digital arm of PENN Entertainment (NASDAQ: PENN), the largest regional casino operator in the U.S.). Our mission is to challenge the norms of the gaming industry by building an immersive interactive gaming experience that is responsible, innovative, and fun. We...

  • Senior Data Engineer

    1 month ago

    Philadelphia, United States PENN Interactive Full time

    Penn Interactive (PI) is an interactive gaming company headquartered in Philadelphia. PI is the digital arm of PENN Entertainment (NASDAQ: PENN), the largest regional casino operator in the U.S.). Our mission is to challenge the norms of the gaming industry by building an immersive interactive gaming experience that is responsible, innovative, and fun. We...

  • Senior Data Engineer

    1 month ago

    Philadelphia, United States PENN Interactive Full time

    Penn Interactive (PI) is an interactive gaming company headquartered in Philadelphia. PI is the digital arm of PENN Entertainment (NASDAQ: PENN), the largest regional casino operator in the U.S.). Our mission is to challenge the norms of the gaming industry by building an immersive interactive gaming experience that is responsible, innovative, and fun. We...

  • Data Engineer

    1 day ago

    Philadelphia, United States Marvica Technologies Full time

    Job DescriptionJob DescriptionData Engineer Location: Philadelphia (Hybrid), Contract to hireExp: 8+ YearsResponsibilities: Develop data-driven solutions with Cloud based technologies to meet evolving business needs. Deliver impactful, scalable, and highly performant Data solution to our customers based in US, UK, Canada and India based customers Provide...

  • Sr Data Engineer

    2 weeks ago

    Philadelphia, United States codesbright Full time

    Responsibilities Develop and maintain database architectures data schemas and data integration processes. Conduct performance tuning troubleshooting and data quality checks to guarantee the accuracy reliability and availability of data. Collaborate with software engineers to integrate data-driven capabilities into applications. Stay up-to-date with...

  • Sr Data Engineer

    3 weeks ago

    Philadelphia, Pennsylvania, United States codesbright Full time

    Responsibilities Develop and maintain database architectures data schemas and data integration processes. Conduct performance tuning troubleshooting and data quality checks to guarantee the accuracy reliability and availability of data. Collaborate with software engineers to integrate data-driven capabilities into applications. Stay up-to-date with emerging...

  • Data Engineering Manager

    4 hours ago

    Philadelphia, United States Perpay - Career's Page Full time

    Job DescriptionJob DescriptionAbout Us:We believe everyone deserves a chance to improve their financial future. We're dedicated to building simple and inclusive financial products that help our members create healthy habits and achieve economic stability.Some things we're excited about:$500 million in spending power used by our membersIncreasing...

  • Lead Big Data Engineer

    6 days ago

    Philadelphia, United States Clarivate, plc Full time

    We are looking for a Lead Big Data Engineer to join our IMS team in Philadelphia, PA. This is an exciting new opportunity to support data excellence and research intelligence projects. If you have a solid background in Big Data design and development Data Engineer, Big Data, Engineer, Lead, Software, Technology, Data

  • Full Stack Data Engineer

    1 day ago

    Philadelphia, United States rockITdata Full time

    Job DescriptionJob DescriptionrockITdata, a veteran, minority, and woman-owned small business certified by the NWBOC, is a full-service consulting provider that specializes in providing management and IT services. We help companies build efficiencies, decrease cost, and drive better outcomes. We achieve these goals by leveraging industry leading cloud based...