Data Warehouse

Role: Data Warehouse - Data Engineering Lead

Location: Remote

Summary:

Looking for a senior developer responsible for direct technical oversight of a sub-team that will be working with the Data Warehouse team on designing, building and maintaining client's new data warehouse hosted in Google Cloud. This work includes all aspects of developing ETL/ELT data pipelines for the data warehouse. The Data Engineering Lead will work closely with other smart data engineers that enjoy being productive and making a difference in a friendly and pragmatic environment in which each person knows that part of their job is to make other's jobs easier.

Responsibilities:

• Provide fully remote technical leadership and oversight for the other developers working on the project in Eastern Europe, and coordinate with technical leads in California. Work hours are flexible, but need to have a few hours of overlap (as needed) with all 3 time zones (e.g. 9 am to 5 pm UTC).
• Provide advice and recommendations (in writing when appropriate) to the Data Warehouse Technical Lead regarding technical options and best practices for the data warehouse and related ETL/ELT pipelines.

• Design and build ETL/ELT pipelines using Composer (Airflow), Java Dataflow (Beam) and other technologies on the Google Cloud.

• Analyze source data and work with internal data consumers to determine which data is needed and how it should be represented in the output table schemas.
• Write and maintain related technical documentation (e.g. for data engineers, data analysts, security engineers, etc.).
• Perform thorough design and code reviews for other developers.

Minimum qualifications:

• 2 years - ETL/ELT to BigQuery.
• 2 years - Apache Airflow in Python.

Preferred qualifications:

• 3 years - ETL/ETL for data warehousing including data source investigation/analysis, target schema design and data pipeline design/implementation
• 3 years - developing for Google Cloud
• 3 years - SQL scripting
• 3 years - Java programming
• 2 years - Apache Beam in Java (preferably for Google Cloud Dataflow)
• 1 year - Google Cloud Composer in Python
• 1 year - Google Cloud Cortex
• Solid interpersonal skills
• Able to write clear, concise and well-reasoned technical explanations and documentation (in English)

Additional desirable qualifications:

• Experience with IntelliJ, Gradle, Google Cloud Storage, Google Cloud Datastream, Google Cloud Data Catalog, Google Looker, CI/CD, GitHub
• Certified as a Professional Data Engineer for Google Cloud