Senior ML Infrastructure Engineer

2 months ago


San Francisco, United States Recruiting From Scratch Full time
Who is Recruiting from Scratch: Recruiting from Scratch is a talent firm that focuses on placing the best candidate for our clients. Our team is 100% remote and we work with teams across North America, South America, and Europe to help them hire.

Senior ML Infrastructure Engineer | AI Infrastructure Scale-Up | SF Based Base: $180K - $300K + Equity (0.1-3%) | Visa Sponsorship Available

Are you excited about building the future of AI infrastructure? We're scaling our inference systems to handle millions of LLM requests daily, and we need exceptional talent to drive this growth.

The Role: We're seeking a Senior ML Infrastructure Engineer to architect and implement large-scale, fault-tolerant systems. You'll be joining a team that's pushing the boundaries of AI infrastructure, handling hundreds of millions of API calls daily.

What You'll Do:

  • Design and implement distributed systems for our inference network
  • Develop resource allocation models across heterogeneous hardware
  • Optimize network performance metrics (latency, throughput, availability)
  • Build robust monitoring and observability systems
  • Drive architectural decisions and best practices
  • Collaborate directly with founders and engineering teams

What You Bring:

  • 5+ years building high-performance, scalable distributed systems
  • Strong programming skills in TypeScript, Python, and either Go, Rust, or C++
  • Experience with Kubernetes/Nomad orchestration
  • Hands-on experience with AI tooling (ChatGPT, Claude, Cursor)
  • GPU programming and optimization skills (CUDA experience is a plus)
  • Startup experience (pre-seed to series A)

Bonus Points:

  • Experience with LLM inference engines (vLLM, TensorRT-LLM)
  • Track record of scaling distributed systems

Location & Details:

  • San Francisco, CA (In-person)
  • Full-time W-2 position
#J-18808-Ljbffr

  • San Francisco, California, United States Fieldguide Full time

    About Us: Fieldguide is a pioneering company that's revolutionizing the audit and advisory industry by leveraging cutting-edge Machine Learning (ML) technology. As a Senior Platform Engineer, Machine Learning, you'll be instrumental in building and maintaining the infrastructure that powers our ML solutions, enabling us to deliver impactful results to our...


  • San Francisco, United States Recruiting From Scratch Full time

    Who is Recruiting from Scratch : Recruiting from Scratch is a talent firm that focuses on placing the best candidate for our clients. Our team is 100% remote and we work with teams across North America, South America, and Europe to help them hire. Senior ML Infrastructure Engineer | AI Infrastructure Scale-Up | SF Based Base: $180K - $300K Equity (0.1-3%) |...


  • San Francisco, United States Recruiting from Scratch Full time

    Who is Recruiting from Scratch: Recruiting from Scratch is a talent firm that focuses on placing the best candidate for our clients. Our team is 100% remote and we work with teams across North America, South America, and Europe to help them hire. Senior ML Infrastructure Engineer | AI Infrastructure Scale-Up | SF Based Base: $180K - $300K + Equity (0.1-3%)...


  • San Francisco, CA, United States Recruiting From Scratch Full time

    Who is Recruiting from Scratch : Recruiting from Scratch is a talent firm that focuses on placing the best candidate for our clients. Our team is 100% remote and we work with teams across North America, South America, and Europe to help them hire. Senior ML Infrastructure Engineer | AI Infrastructure Scale-Up | SF Based Base: $180K - $300K + Equity...


  • San Francisco, California, United States Unity Full time

    Welcome to Unity, the world's leading platform of tools for creators to build and grow real-time games, apps, and experiences across multiple platforms. As a highly skilled data and machine learning (ML) infrastructure engineer, you will play a crucial role in designing and optimizing large-scale data platforms and ML infrastructure systems for efficiency,...


  • San Francisco, California, United States Magical Tome Full time

    About Magical TomeTome is a unified platform for enterprise sellers and account managers. Our mission is to simplify complex research and strategic planning for sellers by leveraging state-of-the-art models.We use our expertise in AI/ML to surface the most actionable knowledge about a customer from within internal systems as well as from public information...


  • San Francisco, California, United States CentML Full time

    About CentMLWe're a cutting-edge technology company dedicated to revolutionizing the field of artificial intelligence. Our goal is to make AI more accessible and affordable for everyone.Our TeamOur team consists of world-renowned experts in AI, compilers, and ML hardware who have led efforts at top tech companies like Amazon, Google, and Microsoft.Job...


  • San Francisco, United States Abridge AI Inc. Full time

    Abridge was founded in 2018 with the mission of powering deeper understanding in healthcare. Our AI-powered platform was purpose-built for medical conversations, improving clinical documentation efficiencies while enabling clinicians to focus on what matters most—their patients.Our enterprise-grade technology transforms patient-clinician conversations into...


  • San Francisco, United States ZipRecruiter Full time

    Job DescriptionAbridge was founded in 2018 with the mission of powering deeper understanding in healthcare. Our AI-powered platform was purpose-built for medical conversations, improving clinical documentation efficiencies while enabling clinicians to focus on what matters most—their patients.Our enterprise-grade technology transforms patient-clinician...


  • San Francisco, United States CentML Full time

    About Us We believe AI will fundamentally transform how people live and work. CentML's mission is to massively reduce the cost of developing and deploying ML models so we can enable anyone to harness the power of AI and everyone to benefit from its potential. Our founding team is made up of experts in AI, compilers, and ML hardware and has led efforts at...


  • San Francisco, United States CentML Full time

    About UsWe believe AI will fundamentally transform how people live and work. CentML's mission is to massively reduce the cost of developing and deploying ML models so we can enable anyone to harness the power of AI and everyone to benefit from its potential.Our founding team is made up of experts in AI, compilers, and ML hardware and has led efforts at...


  • San Francisco, California, United States CentML Full time

    About CentMLWe believe AI will fundamentally transform how people live and work. Our mission is to massively reduce the cost of developing and deploying ML models so we can enable anyone to harness the power of AI and everyone to benefit from its potential.Our founding team is made up of experts in AI, compilers, and ML hardware with extensive industry...

  • Senior AI/ML Engineer

    2 weeks ago


    San Francisco, California, United States Magical Tome Full time

    About Magical TomeMagical Tome is a unified platform for enterprise sellers and account managers. We use cutting-edge models to simplify complex research and strategic planning for sellers. Our system is tuned and customized by a team of experienced sellers, engineers, and researchers. We design and build Magical Tome in close partnership with our early...


  • San Francisco, United States Scale AI, Inc. Full time

    As a software engineer on the ML Infrastructure team, you will work on developing the platform for orchestrating post-training and model evaluation jobs. At Scale, we are constantly developing new data sources and running experiments to understand their impact on ML models. To support this effort, we are looking for engineers who are comfortable navigating...


  • San Francisco, California, United States Delphina Full time

    About DelphinaWe are on a mission to revolutionize the way data scientists work. Our vision is to empower teams to build powerful machine learning models quickly and efficiently, without the pain points associated with traditional tools.As a Founding ML Infrastructure Engineer at Delphina, you will be part of a team that has previously led large data science...


  • San Francisco, United States Relyance AI Full time

    As Relyance AI's Senior Software Engineer, ML, you will strategize, drive, and execute on the initiatives in NLP for information extraction from legal documents, ML/NLP for information extraction from code and general ML in code analysis, as well as overall AI backend initiatives. You will partner with cross-functional stakeholders to design and build...


  • San Francisco, United States Relyance AI Full time

    As Relyance AI's Senior Software Engineer, ML, you will strategize, drive, and execute on the initiatives in NLP for information extraction from legal documents, ML/NLP for information extraction from code and general ML in code analysis, as well as overall AI backend initiatives. You will partner with cross-functional stakeholders to design and build...

  • Data Engineering Lead

    1 month ago


    San Francisco, California, United States Rungalileo Full time

    About RungalileoRungalileo is a leading-edge company that specializes in developing cutting-edge Machine Learning systems. Our seasoned founding team has previously led product and engineering teams from 0 to $100M+ in revenue and from 0 to 1B+ users globally.We are committed to creating an inclusive culture driven by empathy, curiosity, and a passion for...


  • San Francisco, United States Harnham Full time

    SENIOR MACHINE LEARNING PLATFORM ENGINEER$195,000 - $220,000 BASE + BONUS + EQUITYREMOTE (US)ABOUT THE COMPANYThis innovative tech company is transforming its industry by creating seamless digital solutions that connect millions of users to real-life experiences. Operating across multiple platforms, it offers access to thousands of events, redefining how...


  • San Francisco, United States Delphina Full time

    About Delphina Today’s Data Scientists are in pain - spending their time manually wrangling data, building models through slow trial and error, taking on painstaking rewrites for deployment, and dealing with countless other frustrating bottlenecks. And the tools they are using for much of this work – e.g. Jupyter notebooks and Pandas – are over a...