Data Engineering Leader

4 weeks ago


San Francisco, California, United States Windfall Full time

At Windfall, we're seeking a seasoned Data Engineer to spearhead the development of our core data asset. As a key member of our data team, you'll be responsible for designing and building scalable data pipelines that fuel our business.

This is a unique opportunity to leverage your expertise in distributed data processing and cloud-native technologies to drive business outcomes. You'll work closely with our data science team to integrate machine learning models and drive data-driven decision-making.

Key responsibilities include:

  • Designing and implementing data pipelines to ingest and merge massive datasets
  • Collaborating with our data science team to develop and deploy ML models
  • Building supporting data services and applications to monitor and orchestrate our data systems

Some of the technologies you'll work with include:

  • Cloud platform: GCP
  • Programming languages: Java, Python, and Kotlin
  • Data warehouse & databases: BigQuery, Postgres, Scylla/Cassandra
  • Distributed processing frameworks: Dataflow (Apache Beam) and Apache Spark
  • Orchestration: Airflow

Requirements:

  • 4-8 years of professional data engineering experience
  • Significant experience working with Apache Beam/Spark/Flink or MapReduce
  • Strong Object-oriented programming ability
  • Expert knowledge of distributed data processing
  • Familiarity with different datastores and their appropriate usages

Preferred qualifications include:

  • Proven experience taking a large project from ideation to production
  • Experience leading greenfield projects
  • Working knowledge of cloud-native data engineering infrastructure

We offer a comprehensive benefits package and a competitive salary range of $141k - $170k in California and $122k - $160k in Colorado.



  • San Francisco, California, United States Kikoff Full time

    Job Title: Data Engineering LeaderWe are seeking a highly skilled Data Engineering Leader to join our team at Kikoff. As a key member of our data engineering function, you will be responsible for designing, building, and scaling high-leverage data models, foundational datasets, and scalable infrastructure that enables analytics, modeling, and...


  • San Francisco, California, United States Windfall Full time

    We're seeking a highly skilled Data Engineering Leader to join our core data team at Windfall. As a key member of our team, you will be responsible for designing and building the core data asset that everything else at Windfall is built on top of.Key responsibilities include constructing data pipelines to ingest and merge billions of individual entities into...


  • San Francisco, California, United States Windfall Full time

    Windfall is seeking a seasoned Data Engineer to lead the development of our core data asset. As a key member of our data team, you will design and build scalable data pipelines to ingest and merge massive datasets, working closely with our data science and engineering teams.With a strong focus on collaboration and communication, you will ensure seamless...


  • San Jose, California, United States TechnoGen Full time

    We are seeking a highly skilled Data Engineering Leader to join our team at TechnoGen. As a key member of our Data Science and Engineering team, you will be responsible for designing, building, and launching efficient and reliable data pipelines to support our business needs.Key Responsibilities:Partner with leadership, engineers, and data scientists to...


  • San Francisco, California, United States California Water Data Consortium Full time

    The California Water Data Consortium is seeking a visionary leader to serve as its next Chief Executive Officer.The ideal candidate will be an excellent communicator and proven leader who demonstrates a spirit of entrepreneurship and can provide thought leadership on California's highest impact water data opportunities.The successful candidate will have at...


  • San Francisco, California, United States California Water Data Consortium Full time

    The California Water Data Consortium is seeking a visionary leader to serve as its next Chief Executive Officer.The ideal candidate will have a proven track record of strategic planning, program development, and team management.They will oversee and help expand a talented small team of professionals, which will manage regular meetings of the Board of...


  • San Francisco, California, United States Pendulum Full time

    Pendulum is a revolutionary company leading a global movement to improve physical and mental health by understanding, restoring, and enhancing the human microbiome.The company has made significant breakthroughs in the field, creating proprietary probiotic pipelines and a unique discovery platform to identify key bacterial strains and prebiotics.As a Director...

  • Data Science Leader

    1 month ago


    San Francisco, California, United States Robotics Technologies LLC Full time

    Job Title: Director of Data ScienceOur client, a leading company in San Francisco, is seeking a highly skilled Director of Data Science to lead their analytics team. The ideal candidate will have a strong background in data analysis, machine learning, and programming, with excellent leadership and communication skills.The Director of Data Science will be...


  • San Francisco, California, United States California Water Data Consortium Full time

    The California Water Data Consortium is seeking a visionary leader to serve as its next Chief Executive Officer.The ideal candidate will have a proven track record of strategic planning, programmatic execution, and financial management.They will oversee and help expand a talented team of professionals, which will manage regular meetings of the Board of...

  • Data Science Leader

    4 weeks ago


    San Francisco, California, United States LHH Recruitment Solutions Full time

    We are seeking a highly skilled Data Science Leader to join our Ads Data Science team. This individual will play a key role in shaping the future of our advertising strategy by developing and executing advanced machine learning models and algorithms.Key Responsibilities:Developing and executing advanced machine learning models and algorithms to address...


  • San Francisco, California, United States California Water Data Consortium Full time

    The California Water Data Consortium is seeking a visionary leader to serve as its next Chief Executive Officer.The ideal candidate will have a proven track record of strategic planning, programmatic execution, and financial management.They will oversee and help expand a talented team of professionals, which will manage regular meetings of the Board of...


  • San Francisco, California, United States Pendulum Full time

    Pendulum is revolutionizing the way we understand and interact with the human microbiome. As a Director of Data Engineering, you will play a pivotal role in building and maintaining our data platform and infrastructure, driving data-driven decision-making and AI/ML initiatives.Key Responsibilities:Lead and mentor a high-performing data engineering team,...

  • Data Engineer

    4 weeks ago


    San Francisco, California, United States Syntricate Technologies Full time

    Data EngineerJob Summary: We are seeking a highly skilled Data Engineer to join our team at Syntricate Technologies. As a Data Engineer, you will be responsible for designing, implementing, and maintaining large-scale data pipelines using Snowflake and other data engineering tools. You will work closely with our data team to ensure the accuracy and...


  • San Francisco, California, United States Robust Intelligence Full time

    About the RoleWe are seeking a highly skilled Data Platform Engineer to join our ML Data and Quality team at Robust Intelligence. As a key member of our team, you will be responsible for building the ML data infrastructure based on the requirements of the ML team and the product needs.Key Responsibilities:Design, develop, and deploy innovative AI solutions...

  • Data Engineer

    4 weeks ago


    San Francisco, California, United States Bayone Full time

    Job Title: Data EngineerCompany Overview:At Bayone, we strive to create an environment where people are valued, and differences are celebrated. Our purpose is to empower our teams and inspire our customers to become the best versions of themselves.The Opportunity:Our technology team works fast and smart, developing the latest in mobile technologies, scalable...

  • Data Engineer

    1 month ago


    San Francisco, California, United States Brex Full time

    Transforming Data into InsightsBrex is a leading AI-powered spend platform that helps companies spend with confidence. As a Data Engineer at Brex, you will play a critical role in transforming raw data into actionable insights for various departments across the organization.You will collaborate closely with Data Scientists, Software Engineers, and business...


  • San Francisco, California, United States Amazon Data Services, Inc. Full time

    About the RoleWe are seeking a highly skilled Data Center Operations Specialist to join our team at Amazon Data Services, Inc. As a key member of our Data Center Engineering Operations team, you will play a critical role in maintaining and operating our critical infrastructure systems to ensure 99.999% uptime of our cloud computing portfolio.Key...

  • Data Engineer

    4 weeks ago


    San Francisco, California, United States Airbyte Full time

    We are seeking a highly skilled and experienced Analytics Engineer to join the Data Team at Airbyte. In this role, you will work closely with product managers, engineers, and functional leads to build and maintain robust data pipelines and analytics infrastructure.Data Pipeline Development:Design, build, and maintain scalable and efficient data pipelines...

  • Data Engineer

    4 weeks ago


    San Francisco, California, United States Airbyte Full time

    Role OverviewAirbyte is seeking a highly skilled Data Engineer to join our Data Team. As a Data Engineer, you will be responsible for designing, building, and maintaining robust data pipelines and analytics infrastructure to support our product development initiatives.Key ResponsibilitiesDesign and develop scalable and efficient data pipelines using...

  • Engineering Leader

    4 weeks ago


    San Francisco, California, United States Anon Full time

    Define the Future of AI AuthenticationAnon is revolutionizing the identity layer for the automated internet. Our developer toolkit enables AI agents to securely integrate with any service or application on behalf of users, powering the next generation of AI applications.We've secured significant funding from top investors to pursue our vision of a world...