Software Engineer, Model Inference

1 month ago

San Francisco, United States OpenAI Full time
About the Team

Our team brings OpenAI's most capable technology to the world through our products. Most recently, we released ChatGPT, GPT-4, the Whisper API, and DALL-E. We empower consumers and developers alike to use and access our start-of-the-art AI models, allowing them to do things that they've never been able to before.

Across all product lines, we ensure that these powerful tools are used responsibly. This is a key part of OpenAI's path towards safely deploying broadly beneficial Artificial General Intelligence (AGI). Safety is more important to us than unfettered growth.

About the Role

We're looking for an engineer to join our team at OpenAI to help us scale up our critical inference infrastructure, which efficiently services every customer request to use our state-of-the-art AI models, including GPT-4 and Dall-E.

In this role, you will:
  • Work alongside machine learning researchers, engineers, and product managers to bring our latest technologies into production.
  • Introduce new techniques, tools, and architecture that improve the performance, latency, throughput, and efficiency of our deployed models.
  • Build tools to give us visibility into our bottlenecks and sources of instability and then design and implement solutions to address the highest priority issues.
  • Optimize our code and fleet of Azure VMs to utilize every FLOP and every GB of GPU RAM of our hardware.
You might thrive in this role if you:
  • Have an understanding of modern ML architectures and an intuition for how to optimize their performance, particularly for inference.
  • Own problems end-to-end, and are willing to pick up whatever knowledge you're missing to get the job done.
  • Have at least 3 years of professional software engineering experience.
  • Are an expert in core HPC technologies: InfiniBand, MPI, CUDA.
  • Understand how to overlap compute and communication to maximize utilization of scarce compute, memory, and bandwidth resources.
  • Have experience architecting, observing, and debugging production distributed systems.
  • Have a humble attitude, an eagerness to help your colleagues, and a desire to do whatever it takes to make the team succeed.
  • Have needed to rebuild or substantially refactor production systems several times over due to rapidly increasing scale.
  • Are self-directed and enjoy figuring out the most important problem to work on.
  • Have a good intuition for when off-the-shelf solutions will work, and build tools to accelerate your own workflow quickly if they won't.


About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.

OpenAI Affirmative Action and Equal Employment Opportunity Policy Statement

For US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records.

We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.

OpenAI Global Applicant Privacy Policy

At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

  • Software Engineer, Model Inference

    3 weeks ago

    San Francisco, United States OpenAI Full time

    Software Engineer, Model Inference | OpenAICareersSoftware Engineer, Model InferenceApplied AI Engineering - San FranciscoAbout the TeamOur team brings OpenAI’s most capable technology to the world through our products. Most recently, we released ChatGPT, GPT-4, the Whisper API, and DALL-E. We empower consumers and developers alike to use and access our...

  • Software Engineer, Model Inference

    3 weeks ago

    San Francisco, United States OpenAI Full time

    About the TeamOur team brings OpenAI’s most capable technology to the world through our products. Most recently, we released ChatGPT, GPT-4, the Whisper API, and DALL-E. We empower consumers and developers alike to use and access our state-of-the-art AI models, allowing them to do things that they’ve never been able to before.Across all product lines, we...

  • Software Engineer, Model Inference

    6 days ago

    San Francisco, United States OpenAI Full time

    About the Team Our team brings OpenAI's most capable technology to the world through our products. Most recently, we released ChatGPT, GPT-4, the Whisper API, and DALL-E. We empower consumers and developers alike to use and access our start-of-the-art AI models, allowing them to do things that they've never been able to before. Across all product lines, we...

  • Senior Software Engineer

    4 weeks ago

    San Francisco, United States CentML Full time

    About Us We believe AI will fundamentally transform how people live and work. CentML's mission is to massively reduce the cost of developing and deploying ML models so we can enable anyone to harness the power of AI and everyone to benefit from its potential. Our founding team is made up of experts in AI, compilers, and ML hardware and has led...

  • Senior Inference Systems Engineer

    3 weeks ago

    San Francisco, California, United States Genmo Inc. Full time

    At Genmo Inc., we are a research lab dedicated to building state-of-the-art models for video generation. Our goal is to unlock the potential of Artificial General Intelligence (AGI).Job OverviewWe are seeking a senior/staff software engineer to join our inference team. This role involves designing and scaling our inference systems to support millions of...

  • Senior Software Engineer

    3 weeks ago

    San Francisco, United States CentML Inc. Full time

    About UsWe believe AI will fundamentally transform how people live and work. CentML's mission is to massively reduce the cost of developing and deploying ML models so we can enable anyone to harness the power of AI and everyone to benefit from its potential.Our founding team is made up of experts in AI, compilers, and ML hardware and has led efforts at...

  • Senior Software Engineer

    14 hours ago

    San Francisco, United States Baseten Full time

    ABOUT BASETEN We’re a growing team of builders backed by top-tier investors, including IVP , Spark Capital , Greylock , and Sarah Guo at Conviction . ML teams at enterprises and category-defining AI-native companies like Descript , Bland.ai , Patreon , Writer , and Robust Intelligence use Baseten to power their core production workloads with best-in-class...

  • AI Inference Software Architect

    7 days ago

    San Francisco, California, United States Untether AI Full time

    Software Architect for AI InferenceWe are seeking an exceptional Software Architect to join our team at Untether AI, where you will play a key role in designing and developing software that interacts with our innovative chip. As part of our top-notch team, you will collaborate closely with hardware engineers and fellow software engineers to create software...

  • AI Inference Engineer

    4 weeks ago

    San Francisco, United States Perplexity AI Full time

    Job DescriptionJob DescriptionWe are looking for an AI Inference to join our growing team. Our current stack is Python, C++, TensorRT-LLM, Kubernetes. You will have the opportunity to work on large-scale deployment of machine learning models for real-time inference.ResponsibilitiesDevelop APIs for AI inference that will be used by both internal and external...

  • **Machine Learning Engineer for Model Inference and Training**

    5 days ago

    San Francisco, California, United States Scale AI, Inc. Full time

    Job Description: We are seeking a skilled Machine Learning Engineer to join our team at Scale AI, Inc. As a member of our ML Infrastructure team, you will be responsible for developing and maintaining the platform for orchestrating post-training and model evaluation jobs.Key Responsibilities:Design and develop re-usable platforms for running in-house and...

  • Software Engineer

    11 hours ago

    San Francisco, United States Genmo Inc. Full time

    We are Genmo, a research lab dedicated to building open, state-of-the-art models for video generation towards unlocking the right brain of AGI. Join us in shaping the future of AI and pushing the boundaries of what's possible in video generation. Role Overview We are looking for a senior/staff software engineer to join our inference team. In this role, you...

  • Data Inference Specialist

    6 days ago

    San Francisco, California, United States Perplexity AI Full time

    We are seeking an experienced Data Inference Specialist to join our team at Perplexity AI.OverviewAt Perplexity AI, we've achieved tremendous growth and adoption since launching the world's first fully functional conversational answer engine. Our AI-powered search assistant has amassed 10 million monthly active users, with mobile apps installed over 1...

  • Software Engineer

    4 weeks ago

    San Francisco, United States Genmo Inc. Full time

    We are Genmo, a research lab dedicated to building open, state-of-the-art models for video generation towards unlocking the right brain of AGI. Join us in shaping the future of AI and pushing the boundaries of what's possible in video generation.Role OverviewWe are looking for a senior/staff software engineer to join our inference team. In this role, you...

  • Member of Technical Staff

    3 days ago

    San Francisco, United States Liquid AI Full time

    As we prepare to deploy our models across various device types, including GPUs, CPUs, and NPUs, we're seeking an expert who can optimize inference stacks tailored to each platform. We're looking for someone who can take our models, dive deep into the task, and return with a highly optimized inference stack-leveraging existing frameworks like ggml, vllm, and...

  • Member of Technical Staff

    4 days ago

    San Francisco, United States Liquid AI Full time

    As we prepare to deploy our models across various device types, including GPUs, CPUs, and NPUs, we're seeking an expert who can optimize inference stacks tailored to each platform. We're looking for someone who can take our models, dive deep into the task, and return with a highly optimized inference stack-leveraging existing frameworks like ggml, vllm, and...

  • Artificial Intelligence Software Engineer

    20 hours ago

    San Francisco, California, United States Perplexity AI Full time

    We are seeking a highly skilled AI Inference Engineer to join our team at Perplexity AI.As a member of our team, you will have the opportunity to work on large-scale deployment of machine learning models for real-time inference.The ideal candidate will have experience with ML systems and deep learning frameworks, as well as familiarity with common LLM...

  • AI Inference Systems Architect

    5 days ago

    San Francisco, California, United States Tbwa ChiatDay Inc Full time

    Perplexity PerksWe are seeking a skilled AI Inference Engineer to join our rapidly growing team in the San Francisco Bay area. With a base salary range of $190,000 - $240,000, this role offers an attractive compensation package.About the RoleIn this position, you will have the opportunity to work on large-scale deployments of machine learning models for...

  • AI Inference Deployment Specialist

    3 weeks ago

    San Francisco, California, United States Tbwa ChiatDay Inc Full time

    We are seeking an experienced AI Inference Deployment Specialist to join our team at Skild AI. As a key member of our robotics team, you will be responsible for deploying cutting-edge AI models and optimizing their performance in real-world environments.Role OverviewIn this role, you will work closely with our cross-functional team to design and develop...

  • Expert Machine Learning Engineer for Real-Time Inference

    5 days ago

    San Francisco, California, United States Perplexity AI Full time

    We are a fast-growing AI company looking for an expert machine learning engineer to join our team. Our current stack is Python, C++, TensorRT-LLM, and Kubernetes.You will have the opportunity to work on large-scale deployment of machine learning models for real-time inference. The ideal candidate should have experience with ML systems and deep learning...

  • Software Engineer

    1 month ago

    San Francisco, United States Baseten Full time

    ABOUT BASETEN We’re a growing team of builders backed by top-tier investors, including IVP, Spark Capital, Greylock, and Sarah Guo at Conviction. ML teams at enterprises and category-defining AI-native companies like Descript, Bland.ai, Patreon, Writer, and Robust Intelligence use Baseten to power their core production workloads with best-in-class...