GCP Data Architect
3 days ago
United States
Tredence Inc.
Full-time
Free with email or Google
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
Free with email or Google
By continuing, you agree to our Terms & Privacy Policy.
We are seeking a highly skilled Data Engineer with expertise in Google Cloud Platform (GCP), particularly BigQuery, PySpark, and Dataflow, to design, build, and optimize scalable data pipelines. The role involves working closely with analytics, product, and engineering teams to ensure reliable data availability and performance.
🛠
Key Responsibilities
• Data pipeline development: Design, implement, and maintain ETL/ELT pipelines using GCP services (Dataflow, BigQuery, Pub/Sub, Cloud Storage).
• BigQuery optimization: Develop efficient SQL queries, optimize schema design, and manage partitioned and clustered tables.
• PySpark processing: Build scalable batch and streaming jobs for large datasets.
• Dataflow orchestration: Implement real-time and batch data processing pipelines.
• Data quality assurance: Ensure accuracy, consistency, and reliability of data across systems.
• Collaboration: Work with data scientists, analysts, and business stakeholders to deliver actionable insights.
• Documentation: Maintain clear technical documentation for pipelines, workflows, and data models.
Key Responsibilities
• Data pipeline development: Design, implement, and maintain ETL/ELT pipelines using GCP services (Dataflow, BigQuery, Pub/Sub, Cloud Storage).
• BigQuery optimization: Develop efficient SQL queries, optimize schema design, and manage partitioned and clustered tables.
• PySpark processing: Build scalable batch and streaming jobs for large datasets.
• Dataflow orchestration: Implement real-time and batch data processing pipelines.
• Data quality assurance: Ensure accuracy, consistency, and reliability of data across systems.
• Collaboration: Work with data scientists, analysts, and business stakeholders to deliver actionable insights.
• Documentation: Maintain clear technical documentation for pipelines, workflows, and data models.