Data Engineer

3 days ago

Sunnyvale, CA, United States Purple Drive Full time

****Preferred location is Sunnyvale, CA although will accept candidates from Bentonville, AR.

Job Description:

We are seeking a skilled Data Engineer with strong expertise in Big Data, Cloud platforms, and distributed data systems. The ideal candidate will have hands-on experience in designing, building, and optimizing data pipelines, API integrations, and real-time stream-processing systems.

Key Responsibilities:

  1. Design, develop, and optimize large-scale data pipelines and ETL workflows.
  2. Work with Java, Python, and Scala to build scalable data solutions.
  3. Develop APIs and integrate with systems using Node.js, GQL, and RESTful services.
  4. Implement big data solutions leveraging Hadoop, Hive, Spark (Scala), Presto/Trino, and Data Lake architectures.
  5. Deploy and manage workflows with Airflow, Luigi, Automic, and similar orchestration tools.
  6. Build and maintain real-time data streaming systems using Storm, Spark-Streaming, and Kafka.
  7. Utilize Vertex AI and Cloud services (AWS/GCP/Azure) for advanced analytics and ML integration.
  8. Ensure system reliability, scalability, and performance in distributed environments.
  9. Collaborate with cross-functional teams (data scientists, analysts, and engineers) to deliver high-quality data solutions.
  10. Apply best practices in CI/CD, Kubernetes-based deployments, and monitoring.
Required Skills:

  • Strong programming skills in Java, Python, and Scala.
  • Expertise in big data frameworks: Hadoop, Hive, Spark (Scala).
  • Hands-on with API development (REST, GraphQL, Node.js).
  • Experience with stream-processing tools: Kafka, Storm, Spark-Streaming.
  • Proficiency with workflow orchestration: Airflow, Luigi, Automic.
  • Knowledge of Presto/Trino and distributed SQL query engines.
  • Cloud experience (AWS, GCP, or Azure), with exposure to Vertex AI.
  • Strong understanding of Data Lake and data warehousing concepts.
  • Experience with Kubernetes for container orchestration.

  • Data Engineer

    6 days ago

    Sunnyvale, CA, United States Institute of Foundation Models Full time

    About the Institute of Foundation Models We are a dedicated research lab for building, understanding, using, and risk-managing foundation models. Our mandate is to advance research, nurture the next generation of AI builders, and drive transformative contributions to a knowledge-driven economy. As part of our team, you'll have the opportunity to work on the...

  • Data Engineer

    2 weeks ago

    Sunnyvale, CA, United States Institute of Foundation Models Full time

    About the Institute of Foundation Models We are a dedicated research lab for building, understanding, using, and risk-managing foundation models. Our mandate is to advance research, nurture the next generation of AI builders, and drive transformative contributions to a knowledge-driven economy. As part of our team, you'll have the opportunity to work on the...

  • Data Engineer

    6 days ago

    Sunnyvale, CA, United States Institute of Foundation Models Full time

    About the Institute of Foundation Models We are a dedicated research lab for building, understanding, using, and risk-managing foundation models. Our mandate is to advance research, nurture the next generation of AI builders, and drive transformative contributions to a knowledge-driven economy. As part of our team, you'll have the opportunity to work on the...

  • Data Engineer

    1 week ago

    Sunnyvale, CA, United States Metasys Technologies Full time

    Data Engineer Duration: 12 Months Sunnyvale, CA Qualifications: • 10-15 years of experience in data engineering or data analytics roles. • Proficiency in Python and SQL for building financial and operational data models. • Experience with Snowflake, Redis, and cloud-based data environments (AWS or GCP). • Understanding of data warehousing and...

  • Data Engineer

    7 days ago

    Sunnyvale, CA, United States Metasys Technologies Full time

    Data Engineer Duration: 12 Months Sunnyvale, CA Qualifications: • 10-15 years of experience in data engineering or data analytics roles. • Proficiency in Python and SQL for building financial and operational data models. • Experience with Snowflake, Redis, and cloud-based data environments (AWS or GCP). • Understanding of data warehousing and...

  • Data Engineer

    2 weeks ago

    Sunnyvale, CA, United States Syntricate Technologies Full time

    Data Engineer Austin, TX / Sunnyvale, CA onsite Contract Role Description: Posting Title Data Engineer (Python,Java, SQL,Spark, PySpark, Java Spark) Count: 2 Description Looking for a Data Engineer who has a passion for building scalable and high performing data pipelines and data services. Responsibilities: Design, implement, optimize, and maintain...

  • Data Engineer

    1 week ago

    Sunnyvale, CA, United States Syntricate Technologies Full time

    Data Engineer Austin, TX / Sunnyvale, CA onsite Contract Role Description: Posting Title Data Engineer (Python,Java, SQL,Spark, PySpark, Java Spark) Count: 2 Description Looking for a Data Engineer who has a passion for building scalable and high performing data pipelines and data services. Responsibilities: Design, implement, optimize, and maintain...

  • Data Engineer

    7 days ago

    Sunnyvale, CA, United States Syntricate Technologies Full time

    Data Engineer Austin, TX / Sunnyvale, CA onsite Contract Role Description: Posting Title Data Engineer (Python,Java, SQL,Spark, PySpark, Java Spark) Count: 2 Description Looking for a Data Engineer who has a passion for building scalable and high performing data pipelines and data services. Responsibilities: Design, implement, optimize, and maintain...

  • Data Engineer

    1 week ago

    Sunnyvale, CA, United States HCL Global Systems Full time

    looking for a highly energetic and collaborative Data Engineer Responsibilities: As a Senior Data Engineer, you will • Design and develop big data applications using the latest open source technologies. • Desired working in offshore model and Managed outcome • Develop logical and physical data models for big data platforms. • Automate workflows using...

  • Data Engineer

    7 days ago

    Sunnyvale, CA, United States HCL Global Systems Full time

    looking for a highly energetic and collaborative Data Engineer Responsibilities: As a Senior Data Engineer, you will • Design and develop big data applications using the latest open source technologies. • Desired working in offshore model and Managed outcome • Develop logical and physical data models for big data platforms. • Automate workflows using...