Research Engineer, Collective Alignment
4 weeks ago
OpenAI's mission is to ensure that artificial general intelligence benefits all of humanity. OpenAI's Collective Alignment Team works on technical approaches to ensure that AGI is shaped democratically. We plan to do so by implementing a system for collecting and encoding public input on model behavior into our systems. While OpenAI has existing efforts to solve technical alignment, we also need systems to help us determine the answer to the question "What do we align to?"
Regulatory bodies are building up capacity to manage such systems, but may not be up-to-speed on the latest technology. Our best bet is to find more ways to ensure that we are appropriately managing risk and harms that our technologies may pose, including by increasing our ability to make good decisions about system development and deployment.
One way to increase our ability to make good decisions is to have input from the broader public. We are encouraged by the phenomenal work done by the recipients of our Democratic Inputs to AI grant that we launched in May, which was aimed at exploring and building in the space of governing AI behavior democratically. We selected 10 teams (out of nearly 1000 applicants) who were each awarded $100,000 to build, test, and share learnings from projects that explore how we can use democratic methods to decide the rules that govern AI systems.
We're excited to combine our research with ideas and prototypes developed by these teams in the coming months. We will implement a system that can find representative participants to provide their perspectives on a given topic, enable discussion and deliberation, and aggregate inputs into policies that are used to change our model behavior. And we're looking for excellent, highly motivated Research Engineers to help us.
About the role
We are seeking Research Engineers to help design and implement experiments for collective alignment research. Responsibilities may include:
- Writing performant and clean code for ML training.
- Independently running and analyzing ML experiments to diagnose problems and understand which changes are real improvements.
- Writing clean non-ML code, for example when building interfaces to let workers and participants interact with our models or pipelines for managing human data.
- Collaborating closely with a small team to balance the need for flexibility and iteration speed in research with the need for stability and reliability in a complex long-lived project.
- Understanding our high-level research roadmap to help plan and prioritize future experiments.
- Implement experiments to measure the effectiveness of different preference learning techniques, including RLHF
- Examine the impact of different aggregation methodologies on model behavior
- Curate large datasets of prompts and investigate coverage of boundary cases
- Exploring methods to understand and predict model behaviors, such as finding inputs causing anomalous circuits or catastrophic outputs
- Designing novel approaches for using LLMs in democratic inputs to AI research
- Are excited about OpenAI's mission of building safe, universally beneficial AGI and are aligned with OpenAI's charter
- Want to use your engineering skills to push the frontiers of what state-of-the-art language models can accomplish
- Possess a strong curiosity about the sociotechnical challenges around aligning and understanding ML models, and are motivated to use your career to address this challenge
- Enjoy fast-paced, collaborative, and cutting-edge research environments
- Have experience implementing ML algorithms (e.g., PyTorch)
- Can develop data visualization or data collection interfaces (e.g., JavaScript, Python)
- Want to ensure that humanity can shape future powerful AI systems
About OpenAI
OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.
We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.
For US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records.
We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.
OpenAI Global Applicant Privacy Policy
At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.
-
Research Engineer, Collective Alignment
2 months ago
San Francisco, United States OpenAI Full timeAbout the team OpenAI's mission is to ensure that artificial general intelligence benefits all of humanity. OpenAI's Collective Alignment Team works on technical approaches to ensure that AGI is shaped democratically. We plan to do so by implementing a system for collecting and encoding public input on model behavior into our systems. While OpenAI has...
-
Research Engineer, Collective Alignment
1 week ago
San Francisco, United States OpenAI Full timeAbout the team OpenAI's mission is to ensure that artificial general intelligence benefits all of humanity. OpenAI's Collective Alignment Team works on technical approaches to ensure that AGI is shaped democratically. We plan to do so by implementing a system for collecting and encoding public input on model behavior into our systems. While OpenAI has...
-
Research Engineer, Alignment Science
2 weeks ago
San Francisco, United States Karkidi Full timeYou want to build and run elegant and thorough machine learning experiments to help us understand and steer the behavior of powerful AI systems. You care about making AI helpful, honest, and harmless, and are interested in the ways that this could be challenging in the context of human-level capabilities. You could describe yourself as both a scientist and...
-
Research Engineer, Alignment Science
2 weeks ago
San Francisco, CA, United States Anthropic Limited Full timeAbout the role: You want to build and run elegant and thorough machine learning experiments to help us understand and steer the behavior of powerful AI systems. You care about making AI helpful, honest, and harmless, and are interested in the ways that this could be challenging in the context of human-level capabilities. You could describe yourself as both...
-
Sr. Research Engineer
4 days ago
San Antonio, United States Southwest Research Institute Full timeOrganization Description Providing design, development, and analysis on components, engines, transmissions, and vehicles, we also support research and modeling for fuel mixing, combustion, filtration, and fluid flow analysis. Objectives Of This Role Assist in research, development and testing activities for lithium-ion battery cells, modules and packs....
-
Research Engineer, Human-AI Interaction
2 months ago
San Francisco, CA, United States Genai Works Full timeThe Safety Systems team is responsible for various safety work to ensure our best models can be safely deployed to the real world to benefit the society and is at the forefront of OpenAI's mission to build and deploy safe AGI, driving our commitment to AI safety and fostering a culture of trust and transparency. The Human-AI Interaction (HAI) team is...
-
San Francisco, California, United States Ivy Exec Full timePosition: $300/hour Paid Market Research Study for Billing and Collections Managers in the Veterinary SectorMarket research studies are paid engagements Ivy Exec conducts with clients to get feedback on certain topics.For this study we are looking for financial professionals regarding their experience and feedback on virtual credit cards and accounts payable...
-
San Francisco, California, United States Ivy Exec Full timePosition: $300/hour Paid Market Research Study for Billing and Collections Managers in the Veterinary SectorMarket research studies are paid engagements Ivy Exec conducts with clients to get feedback on certain topics.For this study we are looking for financial professionals regarding their experience and feedback on virtual credit cards and accounts payable...
-
Student Researcher
3 days ago
San Jose, United States ByteDance Full timeEstablished in 2023, the ByteDance Doubao (Seed) Team is dedicated to building industry-leading AI foundation models. We aim to make world-leading research and foster both technological and social progress. With a long-term vision and a strong commitment to the AI field, the Team conducts research in a range of areas including natural language processing...
-
Research Engineer
2 weeks ago
San Francisco, CA, United States Anthropic Limited Full timeAbout the role: You want to help construct and rapidly iterate on machine learning experiments to help us improve the behavior of powerful AI systems through finetuning. You care about making AI helpful, honest, and harmless, and are interested in shaping model behavior to be more aligned with human values and goals. You could describe yourself as both a...
-
Research Engineer
2 weeks ago
San Francisco, United States Genmo Full timeJob Description Job Description Our mission Genmo makes it easy for anyone to create movies, as if it were magic. Using our web application, any user can create cinematic video using a simple text prompt. We imagine a world where high-quality cinematic video content is as plentiful as water. Our mission is to empower the next billion video creators to tell...
-
Research Engineer
1 week ago
San Francisco, United States Genmo Full timeJob Description Job Description Our mission Genmo makes it easy for anyone to create movies, as if it were magic. Using our web application, any user can create cinematic video using a simple text prompt. We imagine a world where high-quality cinematic video content is as plentiful as water. Our mission is to empower the next billion video creators to tell...
-
Research Engineer
2 weeks ago
San Francisco, United States Genmo Full timeJob DescriptionJob DescriptionOur missionGenmo makes it easy for anyone to create movies, as if it were magic. Using our web application, any user can create cinematic video using a simple text prompt.We imagine a world where high-quality cinematic video content is as plentiful as water. Our mission is to empower the next billion video creators to tell their...
-
Research Engineer
1 week ago
San Francisco, United States Genmo Full timeJob DescriptionJob DescriptionOur missionGenmo makes it easy for anyone to create movies, as if it were magic. Using our web application, any user can create cinematic video using a simple text prompt.We imagine a world where high-quality cinematic video content is as plentiful as water. Our mission is to empower the next billion video creators to tell their...
-
Software Engineer, Research
3 weeks ago
San Francisco, California, United States Wispr AI Full timeWispr is building a more natural way to interact with technology with neural interfaces. We have an elite team of engineers, product designers, and research scientists building magic.About Wispr: We've raised $25M from top-tier VCs like NEA and 8VC. Our angels and advisors include Chester Chipperfield (product lead for the first Apple Watch), Ben Jones (COO,...
-
Software Engineer, Research
1 week ago
San Francisco, California, United States Wispr AI Full timeWispr is building a more natural way to interact with technology with neural interfaces. We have an elite team of engineers, product designers, and research scientists building magic.About Wispr: We've raised $25M from top-tier VCs like NEA and 8VC. Our angels and advisors include Chester Chipperfield (product lead for the first Apple Watch), Ben Jones (COO,...
-
Research scientist biology engineering
2 weeks ago
San Francisco, CA, United States Anthropic Limited Full timeAbout the role: You want to help construct and rapidly iterate on machine learning experiments to help us improve the behavior of powerful AI systems through finetuning. You care about making AI helpful, honest, and harmless, and are interested in shaping model behavior to be more aligned with human values and goals. You could describe yourself as both a...
-
RESEARCH ENGINEER
2 months ago
San Antonio, United States Southwest Research Institute Full timeWho We Are:The Mechanics & Materials team in the Materials Engineering department performs advanced materials research to solve our clients’ most challenging technical problems related to aerospace and manufacturing applications. This position will support senior staff members and ongoing programs t...
-
RESEARCH ENGINEER
1 week ago
San Antonio, United States Southwest Research Institute Full timeWho We Are: The Liquid Propulsion Team performs R&D to solve challenges and advance technologies in machinery and power systems with a focus on liquid propulsion systems. We support actual launch, landing, lunar, and orbital vehicles and ground equipment, ranging from high-g to 0-g and supercritical to cryogenic conditions. Our technologies are advancing...
-
RESEARCH ENGINEER
2 weeks ago
San Antonio, United States Southwest Research Institute Full timeWho We Are: The Liquid Propulsion Team performs R&D to solve challenges and advance technologies in machinery and power systems with a focus on liquid propulsion systems. We support actual launch, landing, lunar, and orbital vehicles and ground equipment, ranging from high-g to 0-g and supercritical to cryogenic conditions. Our technologies are advancing...
