Engineering Manager, AI Inference Systems

4 weeks ago


San Francisco CA, United States OpenAI Full time
About the Team

The Applied AI team safely brings OpenAI's technology to the world. We released ChatGPT, Plugins, DALL•E, and the APIs for GPT-4, GPT-3, embeddings, and fine-tuning. We also operate inference infrastructure at scale. There's a lot more on the immediate horizon.

We seek to learn from deployment and distribute the benefits of AI, while ensuring that this powerful tool is used responsibly and safely. Safety is more important to us than unfettered growth.

We serve end-users directly through ChatGPT, and serve developers through our APIs, which power product features that were never before possible.

About the Role

Model inference at OpenAI is powered through a single service we call our "Engine". The Engine wraps the PyTorch transformers which are GPT-4 and ChatGPT. We are looking for an engineering manager to help lead some of the critical work for this service and grow the team.

In this role, you will:
  • Own substantial portions of our inference stack
  • Ensure we have the ability to run GPT-4, ChatGPT, and future models at increasingly high scale with increasing efficiency
  • Hire world-class AI systems engineers in one of the most competitive hiring markets
  • Coordinate the inference needs of OpenAI's teams and products
  • Create a diverse, equitable, and inclusive culture that makes all feel welcome while enabling radical candor and the challenging of group think
You might thrive in this role if you:
  • Have 3+ years of experience in engineering management and 7+ years as an IC working with high scale distributed systems and ML systems.
  • Have experience with ML systems, particularly high scale distributed inference for modern LLMs.
  • Have experience with highly available, reliable, production grade systems at scale
  • Have familiarity with the latest AI research and working knowledge of how these systems are efficiently implemented
  • Care deeply about diversity, equity, and inclusion, and have a track record of building inclusive teams
  • Have experience closing extremely competitive candidates for your team, and the ability to craft and convey compelling visions of the future
  • Have a voracious and intrinsic desire to learn and fill in missing skills-and an equally strong talent for sharing learnings clearly and concisely with others
  • Are comfortable with ambiguity and rapidly changing conditions. You view changes as an opportunity to add structure and order when necessary
As technical context: at the heart of our infrastructure is a large-scale deployment of GPU nodes running in dozens of Kubernetes clusters across regions. Some core technologies we build with include Python, PyTorch, CUDA, Triton, Redis, Infiniband, NCCL, NVLink

This role is exclusively based in our San Francisco HQ. We offer relocation assistance to new employees.

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.

For US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records.

We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.

OpenAI Global Applicant Privacy Policy

At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology. #J-18808-Ljbffr

  • San Francisco, United States OpenAI Full time

    About the Team Our team brings OpenAI's most capable technology to the world through our products. Most recently, we released ChatGPT, GPT-4, the Whisper API, and DALL-E. We empower consumers and developers alike to use and access our start-of-the-art AI models, allowing them to do things that they've never been able to before. Across all product lines, we...


  • San Francisco, CA, United States OpenAI Full time

    About the TeamOur team brings OpenAI’s most capable technology to the world through our products. Most recently, we released ChatGPT, GPT-4, the Whisper API, and DALL-E. We empower consumers and developers alike to use and access our start-of-the-art AI models, allowing them to do things that they’ve never been able to before.Across all product lines, we...

  • Compiler Engineer

    2 days ago


    San Francisco, United States Untether AI Full time

    ***Please note: While our engineering HQ is in Toronto, this is a remote opportunity and we welcome applicants from anywhere in North America.*** Untether AI is building the world’s highest performance pure-digital AI inference startup. We’re a rapidly growing Toronto-based startup, with employees across Canada and the US, building next generation...


  • San Francisco, United States Snorkel AI, Inc. Full time

    We are looking for a Director of Engineering to lead our AI Platform team. Our AI Platform team builds innovative software systems to power the Snorkel Flow platform. This includes services to train and serve generative AI and machine learning models using novel data-centric techniques, libraries to support AI workflows for a variety of data modalities and...


  • San Francisco, United States Magic AI Corp. Full time

    Join us to build and safely deploy aligned, superhuman AI. We are building an AI pair programmer that feels like a full colleague inside your computer - capable, conversational, and reliable across domains. As a Software Engineer working on our large-scale training and inference infrastructure, you will architect and build resilient solutions for AI...


  • San Francisco, United States Linum Full time

    We’re looking for a founding AI Engineer to join our team to support our machine learning research and model training efforts. You will work in the core loop of implementing research papers, building datasets, training, and productizing. While most of the work will focus on these areas, we are a small team – and you will have to be flexible in picking up...


  • San Francisco, California, United States Linum Full time

    We're looking for a founding AI Engineer to join our team to support our machine learning research and model training efforts.You will work in the core loop of implementing research papers, building datasets, training, and productizing. While most of the work will focus on these areas, we are a small team – and you will have to be flexible in picking up...

  • AI Engineer

    2 weeks ago


    San Francisco, United States Cynch AI Full time

    Come Revolutionize Accounting with AI   We're a seed-stage AI startup led by a founding team of AI startup veterans on a mission to revolutionize the accounting industry. We're seeking an exceptional AI Engineer to help turn our vision into reality. We are combining reasoning, machine learning, and generative AI to augment and democratize the expertise...

  • AI Engineer

    3 weeks ago


    San Francisco, United States Cynch AI Full time

    Job DescriptionJob DescriptionCome Revolutionize Accounting with AI We're a seed-stage AI startup led by a founding team of AI startup veterans on a mission to revolutionize the accounting industry. We're seeking an exceptional AI Engineer to help turn our vision into reality. We are combining reasoning, machine learning, and generative AI to...

  • AI Engineer

    3 weeks ago


    San Francisco, United States Cynch AI Full time

    Come Revolutionize Accounting with AI  We're a seed-stage AI startup led by a founding team of AI startup veterans on a mission to revolutionize the accounting industry. We're seeking an exceptional AI Engineer to help turn our vision into reality. We are combining reasoning, machine learning, and generative AI to augment and democratize the expertise...


  • San Francisco, California, United States Spellbrush Full time

    The Role:Spellbrush, the world's leading generative AI studio behind niji・journey, is looking for an AI Infrastructure Engineer to join us in building out end-to-end ML infrastructure to run our models on all platforms.What you'll do:Design, implement and run our next-generation inference architecture for running all our models powering all platforms and...


  • San Francisco, United States Mistral AI Full time

    Mistral AI is hiring an expert in the role of serving and training large language models at high speed on GPUs. The role is based in San Francisco. The role will involve -Writing low-level code to take all advantage of high-end GPUs (H100) and max out their capacity -Rethinking various part of the generative model architecture to make them more suitable for...


  • San Francisco, United States Genai Works Full time

    About the team The Applied team at OpenAI safely brings cutting-edge technology to the world. We have released groundbreaking products such as ChatGPT, Plugins, DALL·E, and APIs for GPT-4, GPT-3, embeddings, and fine-tuning. Our team also manages large-scale inference infrastructure. With much more on the horizon, our impact continues to grow. Our customers...


  • San Francisco, United States Spellbrush Full time

    Job DescriptionJob DescriptionHere at Spellbrush, we're passionate about making a good anime game.We also happen to be the world's leading generative AI studio — we're the team behind niji・journey.We are currently investigating how AI can be used to help human artists perform masterpieces in the most complex medium of our times:...


  • San Francisco, United States Spellbrush Full time

    Job DescriptionJob DescriptionHere at Spellbrush, we're passionate about making a good anime game.We also happen to be the world's leading generative AI studio — we're the team behind niji・journey.We are currently investigating how AI can be used to help human artists perform masterpieces in the most complex medium of our times:...


  • San Francisco, California, United States Notsohuman Full time

    Perks: Netflix Subscription. Books Reimbursement Subscription. Hiring 150 EngineersFollow on Website soon to be launched:- Company Description is a leading AI Company in the space of providing AI Education and AI Courses to the general audience in an easy to remember byte sized videos in India. Our courses range from all things AI which include Machine...


  • San Francisco, California, United States Zep AI Full time

    Zep is building the long-term memory layer for the LLM application stack. We have a large and active open-source community and recently launched our cloud service. We are seeking an experienced ML Engineer to join our startup. As a critical member of our small, high-performance team, you will be responsible for model selection, evaluation, and performance,...


  • San Francisco, CA, United States OpenAI Full time

    About the TeamThe Applied Engineering team works across research, engineering, product, and design to bring OpenAI’s technology to consumers and businesses. You’ll join the team responsible for running the infrastructure that supports the models backing ChatGPT and the API. The systems we support include inference kubernetes clusters, GPU health,...


  • San Francisco, CA, United States Atlassian Full time

    Overview Working at AtlassianAs a Senior Software Engineer in the Central AI team, you will build and maintain the core infrastructure to allow machine learning engineers and data scientists to develop, train, evaluate, deploy, and operate Machine Learning models and pipelines. You will use your software development expertise to solve difficult problems,...

  • AI Engineer

    3 weeks ago


    San Francisco, United States Patterns Data Systems Inc. Full time

    We're a small team based in San Francisco with a few colleagues remote around the world. We're looking to grow our small team based in SF and looking to hire candidates already based in the bay area or willing to relocate. What we're looking for Someone to take ownership of all the generative AI features in Patterns --- generating answers, SQL queries,...