Data Infrastructure Engineer

6 days ago


San Francisco, California, United States Genmo Full time
Job Title

Data Infrastructure Engineer

Company Overview

We are Genmo, a research lab dedicated to building open, state-of-the-art models for video generation towards unlocking the right brain of AGI.

Salary

$250,000 - $350,000 per year, depending on experience.

Job Description

We're seeking an experienced Senior/Staff AI Infra Engineer to design, build, and scale our petabyte-scale data infrastructure. You'll be responsible for creating robust, scalable systems that manage and process enormous datasets crucial for training our advanced AI models.

Key Responsibilities:
  • Design highly scalable data infrastructure and systems to process petabyte-scale data stores.
  • Manage large-scale distributed processing jobs for ingesting and analyzing large-scale data sets for AI training.
  • Optimize our storage systems to maximize performance.
  • Build monitoring systems to ensure reliability of data infrastructure.
Qualifications:
  • Bachelors, Masters or PhD in Computer Science or related field.
  • Must have:
    • Extremely strong experience with Python.
    • Strong experience with a large-scale distributed computing frameworks (e.g., Spark, Ray).
    • Experience with a systems-level language (Rust, Go, C++).
  • Ideal candidates will have:
    • Familiarity with a lakehouse format such as Delta Lake.
    • Past experience working on machine learning is a very strong plus.
    • Strong past experience working with Kubernetes and cloud environments (AWS, GCP, Azure).
    • Past experience designing and then scaling a large-scale system from zero to one.
  • This is a senior/staff role. Candidates should have 5+ years of experience working with large scale systems.


  • San Francisco, California, United States San Francisco Standard Full time

    We're looking for a talented Data Infrastructure Engineer to help us design, build, and enhance our data infrastructure at The San Francisco Standard. As a key member of our team, you will play a critical role in developing and implementing robust data solutions that support strategic decisions and drive business growth through data-driven insights. With a...


  • San Francisco, California, United States Amazon Data Services, Inc. Full time

    Job Description">You will collaborate with engineers to design critical infrastructure systems, facilitate meetings to address design issues, meet milestones, and escalate as needed. You will also be responsible for driving process improvements, creating and tracking metrics related to cost, quality, and duration of design, and collecting data from...


  • San Francisco, California, United States Robust Intelligence Full time

    About the Role:As a Data Platform Engineer, you will be responsible for building the machine learning data infrastructure based on the requirements of the ML team and product needs. This includes data lineage, provenance, and validation. You will collaborate with the ML and research teams to understand data needs, help build novel data generation algorithms,...


  • San Francisco, California, United States Ellation, Inc. Full time

    We're seeking a highly skilled Staff Site Reliability Engineer to join our Data Engineering team at Ellation, Inc. This role is ideal for individuals with a strong background in site reliability engineering and a passion for ensuring the reliability, scalability, and performance of our data infrastructure.About the RoleThis position will be responsible for...


  • San Francisco, California, United States DoorDash USA Full time

    DoorDash is a leading food delivery and logistics company, and our Data Engineering team plays a crucial role in building database solutions to support various use cases. As a Staff Software Engineer, Data, you will be responsible for architecting and scaling our data reliability, infrastructure, automation, and tools to meet growing business needs.About the...


  • San Francisco, California, United States SherlockTalent Full time

    About the RoleWe are seeking an experienced Cloud Engineer to join our team as an AI Infrastructure and Data Engineer. This is a fantastic opportunity to work with innovative technologies and contribute to the development of scalable, reliable, and secure infrastructure.In this role, you will be responsible for designing and implementing cloud-based...


  • San Francisco, California, United States OpenAI Full time

    About the RoleThe Applied Data Platform team is responsible for designing, building, and operating the foundational data infrastructure that enables products and teams at OpenAI.Key Responsibilities:Design, build, and maintain data infrastructure systems such as distributed compute, data orchestration, distributed storage, streaming infrastructure while...


  • San Francisco, California, United States Unity Full time

    Job OverviewWe are seeking a Senior Data Engineer and Infrastructure Specialist to join our Data & ML Platform team at Unity.About the RoleIn this position, you will design and optimize large-scale data platforms and machine learning infrastructure systems for efficiency, reliability, and cost-effectiveness. You will also lead improvements in infrastructure...


  • San Francisco, California, United States Replica Inc. Full time

    **About Us:**We are Replica Inc., a privacy-centric urban data platform that delivers critical insights about the built environment. Our goal is to empower planners, scientists, analysts, and policymakers with better data, human context, and an intuitive design.Our team models travel behavior over time to show how people across the country live, move, and...


  • San Francisco, California, United States Unity Full time

    Welcome to Unity, the world's leading platform of tools for creators to build and grow real-time games, apps, and experiences across multiple platforms. As a highly skilled data and machine learning (ML) infrastructure engineer, you will play a crucial role in designing and optimizing large-scale data platforms and ML infrastructure systems for efficiency,...


  • San Francisco, California, United States Abridge Full time

    We are a growing team of practitioners, scientists, and engineers working together to empower people and make care more understandable.Abridge is at the forefront of leveraging AI to transform the healthcare industry. Our generative AI-powered products are revolutionizing the practice of medicine, and we're seeking a highly motivated Data Infrastructure...


  • San Francisco, California, United States Magic AI Full time

    Company OverviewMagic AI is a cutting-edge technology company dedicated to building safe Artificial General Intelligence (AGI) that accelerates humanity's progress on the world's most important problems.We believe the most promising path to safe AGI lies in automating research and code generation to improve models and solve alignment more reliably than...


  • San Francisco, California, United States ZipRecruiter Full time

    **Role Overview**">Abridge is a pioneering organization in the field of AI-powered healthcare solutions. We're seeking a highly skilled Data Infrastructure Specialist to join our growing team in San Francisco.The successful candidate will play a crucial role in building and optimizing large-scale data infrastructure to drive business decisions and machine...


  • San Francisco, California, United States Instrinsic Full time

    We are Intrinsic, a rapidly growing startup revolutionizing the way Trust & Safety teams protect their communities from abuse. Our mission is to empower these teams to focus on what matters most by streamlining their workflows and reducing manual reviews.As a Senior Data Engineer, you will play a critical role in designing and implementing our data platform...


  • San Francisco, California, United States Databricks Full time

    Senior Manager, Infrastructure Data Science OpportunityJob OverviewDatabricks is a cutting-edge data and AI infrastructure platform that empowers data teams to solve the world's most complex problems. We're seeking a seasoned Data Science Infrastructure Strategist to join our team and shape the future of our infrastructure through data-driven insights.This...

  • Electrical Engineer

    4 weeks ago


    San Francisco, California, United States Crusoe Energy Systems LLC Full time

    At Crusoe Energy Systems LLC, we are revolutionizing the field of artificial intelligence cloud infrastructure. Our company is pioneering vertically integrated, purpose-built AI infrastructure solutions that are trusted by Fortune 500 companies to power their most advanced AI applications.We are committed to aligning the future of computing with the future...


  • San Francisco, California, United States Databricks Inc. Full time

    Databricks Inc. is looking for a Data Science Infrastructure Leader to shape the future of Databricks infrastructure through data science.The successful candidate will tackle some of the most complex challenges related to capacity planning, performance optimization, reliability engineering, infrastructure efficiency, and customer experience. As a leader, you...


  • San Francisco, California, United States Monograph Full time

    At Monograph, we're on a mission to democratize data access and enable analytics capabilities across the organization. We're looking for a seasoned Senior Data Infrastructure Engineer to join our team and help us build scalable systems that power high-impact product features.


  • San Francisco, California, United States Unreal Gigs Full time

    Job Title: Data Infrastructure VisionaryWe are seeking a highly skilled Data Infrastructure Visionary to join our team at Unreal Gigs. As a key member of our data architecture team, you will be responsible for designing and implementing scalable, high-performance data infrastructure that supports business analytics, data science, and operational...


  • San Francisco, California, United States Genmo Full time

    We are Genmo, a research lab dedicated to building open, state-of-the-art models for video generation towards unlocking the right brain of AGI. Our team is extremely technical with leaders in distributed systems, GPU programming and large-scale training.Job OverviewWe're seeking an experienced Senior/Staff AI Infra Engineer to design, build, and scale our...