GCP Data Engineer

2 weeks ago

Troy, United States HTC Global Services Full time

Job Description:

You will:

  • Work in collaborative environment including pairing and mobbing with other cross-functional engineers.
  • Work on a small agile team to deliver working, tested software.
  • Work effectively with fellow data engineers, product owners, data champions and other technical experts.
  • Demonstrate technical knowledge/leadership skills and advocate for technical excellence.
  • Develop exceptional Analytics data products using streaming, batch ingestion patterns in the Google Cloud Platform with solid Datawarehouse principles.
  • Be the Subject Matter Expert in Data Engineering and GCP tool technologies.


Skills Required:

  • Experience in working in an implementation team from concept to operations, providing deep technical subject matter expertise for successful deployment. Implement methods for automation of all parts of the pipeline to minimize labor in development and production.
  • Experience in analyzing complex data, organizing raw data and integrating massive datasets from multiple data sources to build subject areas and reusable data products.
  • Experience in working with architects to evaluate and productionalize appropriate GCP tools for data ingestion, integration, presentation, and reporting.
  • Experience in working with all stakeholders to formulate business problems as technical data requirement, identify and implement technical solutions while ensuring key business drivers are captured in collaboration with product management This includes designing and deploying a pipeline with automated data lineage. Identify, develop, evaluate and summarize Proof of Concepts to prove out solutions.
  • Test and compare competing solutions and report out a point of view on the best solution.
  • Integration between GCP Data Catalog and Informatica EDC.
  • Design and build production data engineering solutions to deliver pipeline patterns using Google Cloud Platform (GCP) services: BigQuery, DataFlow, Pub/Sub, BigTable, Data Fusion, DataProc, Cloud Composer, Cloud SQL, Compute Engine, Cloud Functions, and App Engine.



Experience Required:

  • In-depth understanding of Google’s product technology (or other cloud platform) and underlying architectures
  • 5+ years of analytics application development experience required - 5+ years of SQL development experience.
  • 3+ years of Cloud experience (GCP preferred) with solution designed and implemented at production scale.
  • Experience working in GCP based Big Data deployments (Batch/Real-Time) leveraging Terraform, Big Query, Big Table, Google Cloud Storage, PubSub, Data Fusion, Dataflow, Dataproc, Cloud Build, Airflow, Cloud Composer etc.
  • 2 + years professional development experience in Java or Python, and Apache Beam.
  • Experience developing with micro service architecture from container orchestration framework.
  • Extracting, Loading, Transforming, cleaning, and validating data - Designing pipelines and architectures for data processing.
  • 1+ year of designing and building Tekton pipelines


Education Required:

  • Bachelor’s degree in computer science or related scientific field

  • Data Engineer

    1 month ago

    Troy, United States QBrainX Remote Work Freelance Full time $60 - $70

    Job Title: Data Engineer (Snowflake + GCP)Location: 100% RemoteJob Type: ContractJob Overview: We are seeking a skilled Data Engineer with expertise in Snowflake and Google Cloud Platform (GCP) to join our data team. The ideal candidate will have experience in data modeling, ETL processes, and cloud-based data solutions to help us optimize our data...

  • GCP Data Engineer

    1 week ago

    Troy, Michigan, United States HTC Global Services Full time

    Job Title: GCP Data EngineerJob Summary:HTC Global Services is seeking a highly skilled GCP Data Engineer to join our team. As a GCP Data Engineer, you will be responsible for designing, building, and maintaining large-scale data processing systems on the Google Cloud Platform.Key Responsibilities: Collaborate with cross-functional teams to design and...

  • Senior Data Engineer

    2 weeks ago

    Troy, Michigan, United States J.D. Power Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at J.D. Power. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data systems and pipelines.Key Responsibilities:Design and implement data pipelines using GCP Dataflow and PalantirDevelop and maintain...

  • GCP Architect

    4 weeks ago

    Parsippany-Troy Hills, United States Tata Consultancy Services Full time

    Job DescriptionTechnical/Functional Skills Required:•Google Cloud Platform (GCP) services BigQuery, Dataprep, Dataflow etc. Or similar experience in AWS or MS Azure.•Experience in designing and implementing 2-3 GCP applications from ground up.•Experience reverse engineering enterprise application and implementing it in GCP.•Microsoft .Net,...

  • GCP Architect

    3 weeks ago

    Parsippany-Troy Hills, United States Centraprise Full time

    Job Description:Design and develop applications on Java/GCPConfigure and manage GCP services such as Compute Engine, App Engine, Cloud Storage, Cloud SQL, BigQuery, Cloud Functions, and Cloud Pub/SubDevelop and maintain cloud infrastructure using Terraform, Cloud Deployment Manager, and KubernetesImplement and manage CI/CD pipelines using Jenkins, Cloud...

  • AI Data Engineer

    3 weeks ago

    Troy, Michigan, United States Beacon Hill Full time

    Job Title: AI Big Data EngineerBeacon Hill is seeking a highly skilled AI Big Data Engineer to join our team. As an AI Big Data Engineer, you will play a critical role in designing, developing, and maintaining a Unified Data Platform.Main Responsibilities:Design and implement automated data pipelines to extract and process data from legacy platforms using...

  • AI Data Engineer

    4 weeks ago

    Troy, Michigan, United States MASHPOINT LLC Full time

    Job Title: AI Big Data EngineerMASHPOINT LLC is seeking a highly skilled AI Big Data Engineer to join our team. As a key member of our AI team, you will be responsible for designing, developing, and maintaining data pipelines, data lakes, and platforms that meet the analytics and business intelligence needs of our organization.Key Responsibilities:Design and...

  • GCP Cloud Architect

    3 weeks ago

    Parsippany-Troy Hills, United States Centraprise Full time

    Job Role: GCP Cloud ArchitectJob Type: Full Time/ Permanent Location : Parsippany, NJJob Description :Design and develop applications on Java/GCPConfigure and manage GCP services such as Compute Engine, App Engine, Cloud Storage, Cloud SQL, BigQuery, Cloud Functions, and Cloud Pub/SubDevelop and maintain cloud infrastructure using Terraform, Cloud Deployment...

  • Data Engineer

    3 weeks ago

    Troy, Michigan, United States Yoh Full time

    Job Title: Data Engineer - Unified Data PlatformWe are seeking a highly skilled Data Engineer to join our team at Yoh, a Day & Zimmermann company. As a Data Engineer, you will play a key role in designing, developing, and maintaining a Unified Data Platform that supports the analytics and business intelligence needs of our clients.Main...

  • AI Big Data Engineer

    2 weeks ago

    Troy, Michigan, United States Rose International Full time

    Job Title: AI Big Data EngineerWe are seeking an experienced AI, Big Data Engineer to join our team at Rose International. As a Data Engineer, you will focus on creating a Unified Data Platform, designing, developing, and maintaining data pipelines, data lakes, and data platforms that support the analytics and business intelligence needs of our clients.You...

  • AI Big Data Engineer

    4 weeks ago

    Troy, Michigan, United States Beacon Hill Full time

    Job Title: AI Big Data EngineerWe are seeking an experienced AI Big Data Engineer to join our team at Beacon Hill. As an AI Big Data Engineer, you will play a key role in designing, developing, and maintaining a Unified Data Platform.Main Responsibilities:Data Pipeline Development: Design and implement automated pipelines to extract and process data from...

  • AI Big Data Engineer

    3 weeks ago

    Troy, United States Beacon Hill Full time

    Summary: As an AI Big Data Engineer, you will focus on creating a Unified Data Platform. You will design, develop, and maintain data pipelines, data lakes, and data platforms to support analytics and business intelligence needs. This role involves working with cutting-edge technologies and tools like Spark, Kafka, AWS, Azure, and Kubernetes to manage...

  • AI Big Data Engineer

    4 weeks ago

    Troy, Michigan, United States Rose International Full time

    Job Title: AI Big Data EngineerWe are seeking an experienced AI, Big Data Engineer to join our team at Rose International. As a key member of our data engineering team, you will play a crucial role in designing, developing, and maintaining data pipelines, data lakes, and data platforms that support the analytics and business intelligence needs of our...

  • AI Big Data Engineer

    4 weeks ago

    Troy, United States Robert Half Full time

    Job DescriptionJob DescriptionJob Summary:As a Data Engineer, you will play a key role in building a Unified Data Platform that empowers our clients with advanced analytics and business intelligence. You will design, develop, and maintain data pipelines, data lakes, and platforms using cutting-edge technologies like Spark, Kafka, AWS, Azure, and Kubernetes...

  • AI Big Data Engineer

    4 weeks ago

    Troy, United States Robert Half Full time

    Job Summary:As a Data Engineer, you will play a key role in building a Unified Data Platform that empowers our clients with advanced analytics and business intelligence. You will design, develop, and maintain data pipelines, data lakes, and platforms using cutting-edge technologies like Spark, Kafka, AWS, Azure, and Kubernetes to tackle large-scale data...

  • AI Big Data Engineer

    4 weeks ago

    Troy, United States Robert Half Full time

    Job Summary:As a Data Engineer, you will play a key role in building a Unified Data Platform that empowers our clients with advanced analytics and business intelligence. You will design, develop, and maintain data pipelines, data lakes, and platforms using cutting-edge technologies like Spark, Kafka, AWS, Azure, and Kubernetes to tackle large-scale data...

  • Senior Data Engineer

    4 weeks ago

    Troy, United States A-Line Staffing Solutions Full time

    Job DescriptionJob Description    We Are actively building out an AI team. We are looking for people with a proven track record, who are willing to experiment with new ideas, invest time in them, fail-fast, and move on if they don't work out. We want team members who have shown a consistent interest in continuous learning, especially in Cloud...

  • AI Big Data Engineer

    4 weeks ago

    Troy, United States Mondo Full time

    Job Title: AI Big Data EngineerLocation-Type: Hybrid - Troy, MI - T/W/Th on-siteStart Date Is: 10/21/24Duration: 12+ month contractCompensation Range: $60-$75/hrRole Overview:As a Data Engineer with our client, you will contribute to building a Unified Data Platform, focusing on designing, developing, and maintaining data pipelines, data lakes, and platforms...

  • AI Big Data Engineer

    4 weeks ago

    Troy, United States Rose International Full time

    Date Posted: 10/04/2024Hiring Organization: Rose InternationalPosition Number: 472324Job Title: AI Big Data EngineerJob Location: Troy, MI, USA, 48098Work Model: HybridEmployment Type: TemporaryEstimated Duration (In months): 12Min Hourly Rate($): 75.00Max Hourly Rate($): 80.00Must Have Skills/Attributes: AWS, Azure, Databricks, GCP, Hadoop, Hive, Kafka,...

  • AI Big Data Engineer

    4 weeks ago

    Troy, United States Jobot Full time

    This Jobot Consulting Job is hosted by: Amanda PrestonAre you a fit? Easy Apply now by clicking the "Apply Now" button and sending us your resume.Salary: $65 - $73 per hourA bit about us:Automotive Information Systems Group Actively building out their AI teamWhy join us?Competitive Compensation PackageHybridAccelerated Growth PotentialJob DetailsJob...