Software Engineer, Model Inference

1 week ago


San Francisco, United States OpenAI Full time

About the Team

Our team brings OpenAI’s most capable technology to the world through our products. Most recently, we released ChatGPT, GPT-4, the Whisper API, and DALL-E. We empower consumers and developers alike to use and access our state-of-the-art AI models, allowing them to do things that they’ve never been able to before.

Across all product lines, we ensure that these powerful tools are used responsibly. This is a key part of OpenAI’s path towards safely deploying broadly beneficial Artificial General Intelligence (AGI). Safety is more important to us than unfettered growth.

About the Role

We're looking for an engineer to join our team at OpenAI to help us scale up our critical inference infrastructure, which efficiently services every customer request to use our state-of-the-art AI models, including GPT-4 and DALL-E.

In this role, you will:

  • Work alongside machine learning researchers, engineers, and product managers to bring our latest technologies into production.
  • Introduce new techniques, tools, and architecture that improve the performance, latency, throughput, and efficiency of our deployed models.
  • Build tools to give us visibility into our bottlenecks and sources of instability and then design and implement solutions to address the highest priority issues.
  • Optimize our code and fleet of Azure VMs to utilize every FLOP and every GB of GPU RAM of our hardware.

You might thrive in this role if you:

  • Have an understanding of modern ML architectures and an intuition for how to optimize their performance, particularly for inference.
  • Own problems end-to-end, and are willing to pick up whatever knowledge you're missing to get the job done.
  • Have at least 3 years of professional software engineering experience.
  • Are an expert in core HPC technologies: InfiniBand, MPI, CUDA.
  • Understand how to overlap compute and communication to maximize utilization of scarce compute, memory, and bandwidth resources.
  • Have experience architecting, observing, and debugging production distributed systems.
  • Have a humble attitude, an eagerness to help your colleagues, and a desire to do whatever it takes to make the team succeed.
  • Have needed to rebuild or substantially refactor production systems several times over due to rapidly increasing scale.
  • Are self-directed and enjoy figuring out the most important problem to work on.
  • Have a good intuition for when off-the-shelf solutions will work, and build tools to accelerate your own workflow quickly if they won’t.

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.

#J-18808-Ljbffr

  • San Francisco, United States OpenAI Full time

    Software Engineer, Model Inference | OpenAICareersSoftware Engineer, Model InferenceApplied AI Engineering - San FranciscoAbout the TeamOur team brings OpenAI’s most capable technology to the world through our products. Most recently, we released ChatGPT, GPT-4, the Whisper API, and DALL-E. We empower consumers and developers alike to use and access our...


  • San Francisco, United States OpenAI Full time

    About the Team Our team brings OpenAI's most capable technology to the world through our products. Most recently, we released ChatGPT, GPT-4, the Whisper API, and DALL-E. We empower consumers and developers alike to use and access our start-of-the-art AI models, allowing them to do things that they've never been able to before. Across all product lines, we...


  • San Francisco, United States CentML Full time

    About Us We believe AI will fundamentally transform how people live and work. CentML's mission is to massively reduce the cost of developing and deploying ML models so we can enable anyone to harness the power of AI and everyone to benefit from its potential. Our founding team is made up of experts in AI, compilers, and ML hardware and has led...


  • San Francisco, California, United States Genmo Inc. Full time

    At Genmo Inc., we are a research lab dedicated to building state-of-the-art models for video generation. Our goal is to unlock the potential of Artificial General Intelligence (AGI).Job OverviewWe are seeking a senior/staff software engineer to join our inference team. This role involves designing and scaling our inference systems to support millions of...


  • San Francisco, United States CentML Inc. Full time

    About UsWe believe AI will fundamentally transform how people live and work. CentML's mission is to massively reduce the cost of developing and deploying ML models so we can enable anyone to harness the power of AI and everyone to benefit from its potential.Our founding team is made up of experts in AI, compilers, and ML hardware and has led efforts at...

  • AI Inference Engineer

    3 weeks ago


    San Francisco, United States Perplexity AI Full time

    Job DescriptionJob DescriptionWe are looking for an AI Inference to join our growing team. Our current stack is Python, C++, TensorRT-LLM, Kubernetes. You will have the opportunity to work on large-scale deployment of machine learning models for real-time inference.ResponsibilitiesDevelop APIs for AI inference that will be used by both internal and external...

  • Software Engineer

    3 weeks ago


    San Francisco, United States Genmo Inc. Full time

    We are Genmo, a research lab dedicated to building open, state-of-the-art models for video generation towards unlocking the right brain of AGI. Join us in shaping the future of AI and pushing the boundaries of what's possible in video generation.Role OverviewWe are looking for a senior/staff software engineer to join our inference team. In this role, you...


  • San Francisco, California, United States Tbwa ChiatDay Inc Full time

    We are seeking an experienced AI Inference Deployment Specialist to join our team at Skild AI. As a key member of our robotics team, you will be responsible for deploying cutting-edge AI models and optimizing their performance in real-world environments.Role OverviewIn this role, you will work closely with our cross-functional team to design and develop...

  • Software Engineer

    4 weeks ago


    San Francisco, United States Baseten Full time

    ABOUT BASETEN We’re a growing team of builders backed by top-tier investors, including IVP, Spark Capital, Greylock, and Sarah Guo at Conviction. ML teams at enterprises and category-defining AI-native companies like Descript, Bland.ai, Patreon, Writer, and Robust Intelligence use Baseten to power their core production workloads with best-in-class...


  • San Francisco, United States Untether AI Full time

    We are looking for best in class engineers to join our existing top-notch team. When you join Untether AI, you will be part of a team that designs, develops and verifies the software that interacts with our chip, collaborating with our hardware engineers and with fellow software engineers in the process. By creating software that fully realizes the...


  • San Francisco, United States Untether AI Full time

    We are looking for best in class engineers to join our existing top-notch team. When you join Untether AI, you will be part of a team that designs, develops and verifies the software that interacts with our chip, collaborating with our hardware engineers and with fellow software engineers in the process. By creating software that fully realizes the...

  • Software Engineer

    1 week ago


    San Francisco, United States Magic, Inc Full time

    Magic's mission is to build safe AGI that accelerates humanity’s progress on the world’s most important problems. We believe the most promising path to safe AGI lies in automating research and code generation to improve models and solve alignment more reliably than humans can alone. Our approach combines frontier-scale pre-training, domain-specific RL,...

  • Software Engineer

    1 week ago


    San Francisco, United States Krafted By KC Full time

    About CartesiaOur mission is to build the next generation of AI: ubiquitous, interactive intelligence that runs wherever you are. Today, not even the best models can continuously process and reason over a year-long stream of audio, video and text—1B text tokens, 10B audio tokens and 1T video tokens—let alone do this on-device.We're pioneering the model...

  • Software Engineer

    3 days ago


    San Francisco, United States ZipRecruiter Full time

    Job DescriptionMagic’s mission is to build safe AGI that accelerates humanity’s progress on the world’s most important problems. We believe the most promising path to safe AGI lies in automating research and code to improve models and solve alignment more reliably than humans can alone. Our approach combines frontier-scale pre-training, domain-specific...


  • San Francisco, United States Untether AI Full time

    We are looking for best in class engineers to join our existing top-notch team. When you join Untether AI, you will be part of a team that designs, develops and verifies the software that interacts with our chip, collaborating with our hardware engineers and with fellow software engineers in the process. By creating software that fully realizes the...


  • San Francisco, United States Quantum Technologies. LLC Full time

    Work Authorization:Minimum Required Skills: Hibernate, SaaS, Machine Learning, Python, ApplicationIf you are a Machine Learning Sr Software Engineer with experience, please read on! Immediately hiring an experienced Machine Learning professional. The Machine Learning team builds mathematical and behavioral models underpinning core solutions for actionable...


  • San Francisco, United States Openai Full time

    About the Team The Platform ML team builds the ML side of our state-of-the-art internal training framework used to train our cutting-edge models. We work on distributed model execution as well as the interfaces and implementation for model code, training, and inference. Our priorities are to maximize training throughput (how quickly we can train a new model)...


  • San Francisco, United States CentML Full time

    About UsWe believe AI will fundamentally transform how people live and work. CentML's mission is to massively reduce the cost of developing and deploying ML models so we can enable anyone to harness the power of AI and everyone to benefit from its potential.Our founding team is made up of experts in AI, compilers, and ML hardware and has led efforts at...


  • San Francisco, United States Weave Robotics Full time

    Weave is making Isaac, a general-purpose, personal robot that’s built for the home and will change how we live. We’re looking for high-performing people to join us and do exceptional work. Responsibilities In this role, you will be responsible for designing and implementing Isaac’s networking and low-latency inference stacks. Your work will ensure...


  • San Francisco, United States DoorDash USA Full time

    About the TeamThe Experimentation Platform team develops an industry-leading platform that enables data scientists, ML engineers and non-technical users to design, run and analyze experiments and conduct exploratory and causal analysis. At DoorDash, where we run thousands of experiments per month, our mission is to equip all decision makers with rigorous,...