Research Engineer, Safety Reasoning
2 weeks ago
The Safety Systems team is responsible for various safety work to ensure our best models can be safely deployed to the real world to benefit society and is at the forefront of OpenAI's mission to build and deploy safe AGI, driving our commitment to AI safety and fostering a culture of trust and transparency.
The Safety Reasoning Research team is poised at the intersection of short-term pragmatic projects and long-term fundamental research, prioritizing rapid system development while maintaining technical robustness. Key focus areas include improving foundational models’ ability to accurately reason about safety, values, and questions of cultural norms, refining moderation models, driving rapid policy improvements, and addressing critical societal challenges like election misinformation. As we venture into 2024, the team seeks talents adept in novel abuse discovery and policy iteration, aligning with our high-priority goals of multimodal moderation and ensuring digital safety.
About the Role
The role involves developing innovative machine learning techniques that push the limit of our foundation model’s safety understanding and capability. You will engage in defining and developing realistic and impactful safety tasks that, once improved, can be integrated into OpenAI's safety systems or benefit other safety/alignment research initiatives. Examples of safety initiatives include moderation policy enforcement, policy development using democratic input, and safety reward modeling. You will be experimenting with a wide range of research techniques not limited to reasoning, architecture, data, and multimodal.
In this role, you will:
- Conduct applied research to improve the ability of foundational models to accurately reason about questions of human values, morals, ethics, and cultural norms, and apply these improved models to practical safety challenges.
- Develop and refine AI moderation models to detect and mitigate known and emerging patterns of AI misuse and abuse.
- Work with policy researchers to adapt and iterate on our content policies to ensure effective prevention of harmful behavior.
- Contribute to research on multimodal content analysis to enhance our moderation capabilities.
- Develop and improve pipelines for automated data labeling and augmentation, model training, evaluation and deployment, including active learning processes, routines for calibration and validation data refresh, etc.
- Experiment and design an effective red-teaming pipeline to examine the robustness of our harm prevention systems and identify areas for future improvement.
Minimum Requirements:
- Possess 5+ years of research engineering experience and proficiency in Python or similar languages.
- Thrive in environments involving large-scale AI systems and multimodal datasets (a plus).
- Exhibit proficiency in the field of AI safety, focusing on topics like RLHF, adversarial training, robustness, fairness & biases, which is extremely advantageous.
- Show enthusiasm for AI safety and dedication to enhancing the safety of cutting-edge AI models for real-world use.
About OpenAI
OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.
We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.
For US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records.
We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.
At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.
#J-18808-Ljbffr-
Research Engineer, Safety Reasoning
12 hours ago
San Francisco, United States Openai Full timeAbout the Team The Safety Systems team is responsible for various safety work to ensure our best models can be safely deployed to the real world to benefit the society and is at the forefront of OpenAI's mission to build and deploy safe AGI, driving our commitment to AI safety and fostering a culture of trust and transparency. The Safety Reasoning Research...
-
Research Engineer
3 weeks ago
San Francisco, United States OpenAI Full timeBy applying to this role, you will be considered for Research Engineer roles across all teams at OpenAI.About the RoleAs a Research Engineer here, you will be responsible for building AI systems that can perform previously impossible tasks or achieve unprecedented levels of performance. We're looking for people with solid engineering skills,...
-
Research Engineer, Preparedness
2 weeks ago
San Francisco, United States OpenAI Full timeResearch Engineer, Preparedness | OpenAI | OpenAICareersResearch Engineer, PreparednessSafety Systems - San FranciscoAbout the teamThe Safety Systems team is responsible for various safety work to ensure our best models can be safely deployed to the real world to benefit society and is at the forefront of OpenAI's mission to build and deploy safe AGI,...
-
Research Engineer, Pre-training Architecture
2 weeks ago
San Francisco, United States OpenAI Full timeResearch Engineer, Pre-training Architecture | OpenAIResearch Engineer, Pre-training ArchitectureFoundations - San FranciscoAbout the TeamThe architecture team is responsible for advancing the neural network architecture of OpenAI’s flagship language models. Our work spans the entire spectrum of architecture development and deployment all the way from the...
-
Staff Research Engineer
2 days ago
San Francisco, United States Strativ Group Full timeStaff Research Engineer We are currently partnered with a well-backed AI Safety startup who are on a mission to improve humanity's understanding of complex AI models and improve the safety and reliability of AI on a global scale. The Founding team consists of experienced, successful entrepreneurs and pioneering leaders from some of the world's top Research...
-
Software Engineer, Safety
2 weeks ago
San Francisco, United States OpenAI Full timeAbout the TeamThe Applied AI team safely brings OpenAI's advanced technology to the world. We released the GPT-3 API, Codex (which powers GitHub Copilot), and DALL-E. More is coming very soon.We empower developers with APIs offering state-of-the-art AI capabilities, which power product features that were never before possible. We also build AI-driven...
-
Safety Manager
4 weeks ago
San Francisco, California, United States Total Safety Full timeTotal Safety is seeking a highly skilled and experienced professional to fill the role of Safety Manager.About the Role:Lead safety operations and ensure the overall safety of employees and customers within Total Safety.Develop and implement effective safety procedures and protocols to minimize risk and ensure compliance with industry standards.Collaborate...
-
Research Engineer, Alignment
16 hours ago
San Francisco, United States Openai Full timeAbout the Team The Alignment team at OpenAI is dedicated to ensuring that our AI systems are safe, trustworthy, and consistently aligned with human values, even as they scale in complexity and capability. Our work is at the cutting edge of AI research, focusing on developing methodologies that enable AI to robustly follow human intent across a wide range of...
-
Research Engineer, AI Security
2 weeks ago
San Francisco, United States OpenAI Full timeResearch Engineer, AI Security & PrivacyThe Safety Systems team is responsible for various safety work to ensure our best models can be safely deployed to benefit society. It is at the forefront of OpenAI's mission to build and deploy safe AGI, driving our commitment to AI safety and fostering a culture of trust and transparency.As cutting-edge AI models get...
-
Research Engineer, Security
2 weeks ago
San Francisco, United States OpenAI Full timeSecurity is at the foundation of OpenAI’s mission to ensure that artificial general intelligence benefits all of humanity.In addition to protecting OpenAI’s technology, people, and products, the Security Team has the mandate to explore the frontiers of AI and cybersecurity. Our research mission is to create a safer and more secure digital ecosystem by...
-
Research Engineer, Future of Computing
14 hours ago
San Francisco, United States Openai Full timeAbout the Team The Future of Computing is an Applied Research team focused on prototyping and exploring how (consumer) computing might holistically evolve as we advance on our mission of building AGI. About the Role As a Research Engineer on the Future of Computing team, you will work together with both the best research scientists in the world and the...
-
San Francisco, United States Amazon Full timePrincipal Applied Scientist, Automated Reasoning in IdentityJob ID: 2838634 | Amazon Web Services, Inc.Are you passionate about applying formal verification, program analysis, constraint-solving, and theorem proving to real world problems? Do you want to create products that help customers? If so, then we have an exciting opportunity for you. In this role,...
-
Research Engineer, Data Processing
2 weeks ago
San Francisco, United States OpenAI Full timeResearch Engineer, Data Processing (Foundations)At OpenAI, we strongly believe in the importance of data and have seen repeatedly how large of an impact focusing on data quality can yield across all of our projects. The Data Processing team brings this focus to our flagship GPT models, owning the pipelines for turning raw data into the high quality, diverse,...
-
Founding ML Research Engineer
3 weeks ago
San Francisco, United States ats.rippling.com- ATS Full timeBronco is an applied AI lab helping chipmakers keep Moore’s law going. Our mission is to build AI silicon engineers that can automate chip design and verification from initial spec to final tape-out. We are starting with the first AI Design Verification Engineer to help close the verification gap, which is where the bottleneck currently is and where we...
-
Research Engineer, AI Security
8 hours ago
San Francisco, United States Openai Full timeAbout the Team The Safety Systems team is responsible for various safety work to ensure our best models can be safely deployed to benefit society. It is at the forefront of OpenAI's mission to build and deploy safe AGI, driving our commitment to AI safety and fostering a culture of trust and transparency As the cutting edge AI models get deployed to the real...
-
San Francisco, United States Openai Full timeAbout the Team Our team is responsible for the post-training and alignment of the models behind ChatGPT and the API. We integrate various improvements from across the company into our RLHF process, ultimately producing the models used by hundreds of millions of users. About the Role We are looking for research scientists and research engineers to advance the...
-
Cloud Engineer
4 months ago
San Francisco, United States Cornerstone Research Full timeCloud EngineerJob Locations US-CA-San FranciscoID 2024-3688Category Information TechnologyOverviewCLOUD ENGINEER (HYBRID, ANY US OFFICE) If you are a technical professional looking for an opportunity to showcase your exceptional cloud expertise, then we would like to meet with you! The Cloud Engineer designs, deploys, manages, and supports cloud...
-
Cloud Engineer
3 weeks ago
San Francisco, United States Cornerstone Research Full timeCloud EngineerJob Locations: US-CA-San FranciscoID: 2024-3688Category: Information TechnologyOverviewCLOUD ENGINEER (HYBRID, ANY US OFFICE)If you are a technical professional looking for an opportunity to showcase your exceptional cloud expertise, then we would like to meet with you!The Cloud Engineer designs, deploys, manages, and supports cloud solutions...
-
Cloud Engineer
3 weeks ago
San Francisco, United States Cornerstone Research Full timeOverview CLOUD ENGINEER (HYBRID, ANY US OFFICE) If you are a technical professional looking for an opportunity to showcase your exceptional cloud expertise, then we would like to meet with you! The Cloud Engineer designs, deploys, manages, and supports cloud solutions running on AWS, Azure, and Google cloud platforms. Working with various functions in the IT...
-
Research Engineer, Societal Impacts
3 weeks ago
San Francisco, United States Menlo Ventures Full timeAbout Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the Role As a...
