Senior Data Engineer

3 days ago

Mountain View, CA, United States United IT Solutions Full time
Role : Senior Data Engineer - GCP

Location : Mountain View CA (Onsite)

What you'll do
  • Develop and enhance Python frameworks and libraries to support data processing, quality, lineage, governance, analysis, and machine learning operations.
  • Design, build, and maintain scalable and efficient data pipelines on GCP.
  • Implement robust monitoring, logging, and alerting systems to ensure the reliability and stability of data infrastructure.
  • Build scalable batch pipelines leveraging Big query, Dataflow and Airflow/Composer scheduler/executor framework on Google Cloud Platform
  • Building data pipelines, leveraging Scala, Pub Sub, Akka, Dataflow on Google Cloud Platform
  • Design our data models for optimal storage and retrieval and to meet machine learning modeling using technologies like Bigtable and Vertex Feature Store
  • Contribute to shared Data Engineering tooling & standards to improve the productivity and quality of output for Data Engineers across the company
Minimum Basic Requirements
  • Python Expertise: Write and maintain Python frameworks and libraries to support data processing and integration tasks.
  • Code Management: Use Git and GitHub for source control, code reviews, and version management.
  • GCP Proficiency: Extensive experience working with GCP services (e.g., BigQuery, Cloud Dataflow, Pub/Sub, Cloud Storage).
  • Python Mastery: Proficient in Python with experience in writing, maintaining, and optimizing data processing frameworks and libraries.
  • Software Engineering: Strong understanding of software engineering best practices, including version control (Git), collaborative development (GitHub), code reviews, and CI/CD.
  • Data Management: Deep knowledge of data modeling, ETL/ELT, and data warehousing concepts.
  • Problem-Solving: Excellent problem-solving skills with the ability to tackle complex data engineering challenges.
  • Communication: Strong communication skills, including the ability to explain complex technical details to non-technical stakeholders.
  • Data Science Stack: Proficiency in data analysis and familiarity with tools such as Jupyter Notebook, pandas, NumPy, and other Python data analysis libraries.
  • Frameworks/Tools: Familiarity with machine learning and data processing tools and frameworks such as TensorFlow, Apache Spark, and scikit-learn.
  • Bachelor's or master's degree in computer science, Engineering, Computer Information Systems, Mathematics, Physics, or a related field or software development training program
Preferred Qualifications
  • Experience in Scala, Java, and/or any functional language. We code primarily in Scala, so you'll be excited to either ramp or continue with such
  • Experience in microservices architecture, messaging patterns, and deployment models
  • Experience in API design and building robust and extendable client/server contracts

  • Senior Data Engineer

    5 days ago

    Mountain View, CA, United States Info Way Solutions Full time

    Job Title: Senior Data Engineer (High Priority) Location: Preferably Mountain View, CA; Hybrid Start Date: Immediate Duration: 6+ months Openings: 4 Senior Engineers About the Role We are seeking Senior Data Engineers to join an experienced team building robust data infrastructure to support high-impact analytics and business needs. This role leans heavily...

  • Senior Data Engineer

    22 hours ago

    Mountain View, CA, United States Info Way Solutions Full time

    Job Title: Senior Data Engineer (High Priority) Location: Preferably Mountain View, CA; Hybrid Start Date: Immediate Duration: 6+ months Openings: 4 Senior Engineers About the Role We are seeking Senior Data Engineers to join an experienced team building robust data infrastructure to support high-impact analytics and business needs. This role leans heavily...

  • Senior Data Engineer

    24 hours ago

    Mountain View, CA, United States Akraya Full time

    Job Description Primary Skills: Python (Expert), Java (Expert), Kafka (Advanced), AWS (Advanced), SQL (Expert) Contract Type: W2 Only Duration: 12+ Months Location: Sunnyvale, CA Pay Range: $85 - $90 per hour on W2 Job Summary: We are seeking a talented Senior Data Engineer to join our dynamic team. The ideal candidate will have a robust quantitative...

  • Senior Data Engineer

    2 weeks ago

    Mountain View, CA, United States ApTask Full time

    About Client: The client provides information technology (IT) services, including business outsourcing, infrastructure technology, and application services. The application service offered by the company includes application development, maintenance, and support. The markets served by the company are financial services and insurance, healthcare,...

  • Data Engineer

    2 weeks ago

    Mountain View, CA, United States DCM INFOTECH LIMITED Full time

    Position: Data Engineer Location: Mountain View, CA (Onsite) Duration: 6 Months + Possible extension Only Local candidates, please don't ask for relocation About the Role: We are seeking Senior Data Engineers to join an experienced team building robust data infrastructure to support high-impact analytics and business needs. This role leans heavily into...

  • Data Engineer

    3 days ago

    Mountain View, CA, United States DCM INFOTECH LIMITED Full time

    Position: Data Engineer Location: Mountain View, CA (Onsite) Duration: 6 Months + Possible extension Only Local candidates, please don't ask for relocation About the Role: We are seeking Senior Data Engineers to join an experienced team building robust data infrastructure to support high-impact analytics and business needs. This role leans heavily into...

  • Senior Software Engineer, Data Science

    4 days ago

    Mountain View, CA, United States General Motors Full time

    Job Description Senior Software Engineer, Data Science Hybrid: This role is categorized as hybrid. This means the successful candidate is expected to report to the Technical/Innovation Center in Warren (MI), Austin (TX), or Mountain View (CA) three times per week, at minimum. This position can be located at any of these locations. Why Join Us General Motors...

  • Senior Software Engineer, Data Science

    2 weeks ago

    Mountain View, CA, United States General Motors Full time

    Job Description Senior Software Engineer, Data Science Hybrid: This role is categorized as hybrid. This means the successful candidate is expected to report to the Technical/Innovation Center in Warren (MI), Austin (TX), or Mountain View (CA) three times per week, at minimum. This position can be located at any of these locations. Why Join Us General Motors...

  • Senior Software Engineer, Data Science

    1 day ago

    Mountain View, CA, United States General Motors Full time

    Job Description Senior Software Engineer, Data Science Hybrid: This role is categorized as hybrid. This means the successful candidate is expected to report to the Technical/Innovation Center in Warren (MI), Austin (TX), or Mountain View (CA) three times per week, at minimum. This position can be located at any of these locations. Why Join Us General Motors...

  • Data Engineer II

    5 days ago

    Mountain View, CA, United States Futran Tech Solutions Pvt. Ltd. Full time

    Position : Data Engineer Location - Mountain View CA Duration - 6 Months + Possible extension About the Role We are seeking Senior Data Engineers to join an experienced team building robust data infrastructure to support high-impact analytics and business needs. This role leans heavily into core data engineering - from pipeline development to data modeling...