Research Engineer, Collective Alignment
1 month ago
OpenAI's mission is to ensure that artificial general intelligence benefits all of humanity. OpenAI's Collective Alignment Team works on technical approaches to ensure that AGI is shaped democratically. We plan to do so by implementing a system for collecting and encoding public input on model behavior into our systems. While OpenAI has existing efforts to solve technical alignment, we also need systems to help us determine the answer to the question "What do we align to?"
Regulatory bodies are building up capacity to manage such systems, but may not be up-to-speed on the latest technology. Our best bet is to find more ways to ensure that we are appropriately managing risk and harms that our technologies may pose, including by increasing our ability to make good decisions about system development and deployment.
One way to increase our ability to make good decisions is to have input from the broader public. We are encouraged by the phenomenal work done by the recipients of our Democratic Inputs to AI grant that we launched in May, which was aimed at exploring and building in the space of governing AI behavior democratically. We selected 10 teams (out of nearly 1000 applicants) who were each awarded $100,000 to build, test, and share learnings from projects that explore how we can use democratic methods to decide the rules that govern AI systems.
We're excited to combine our research with ideas and prototypes developed by these teams in the coming months. We will implement a system that can find representative participants to provide their perspectives on a given topic, enable discussion and deliberation, and aggregate inputs into policies that are used to change our model behavior. And we're looking for excellent, highly motivated Research Engineers to help us.
About the role
We are seeking Research Engineers to help design and implement experiments for collective alignment research. Responsibilities may include:
- Writing performant and clean code for ML training.
- Independently running and analyzing ML experiments to diagnose problems and understand which changes are real improvements.
- Writing clean non-ML code, for example when building interfaces to let workers and participants interact with our models or pipelines for managing human data.
- Collaborating closely with a small team to balance the need for flexibility and iteration speed in research with the need for stability and reliability in a complex long-lived project.
- Understanding our high-level research roadmap to help plan and prioritize future experiments.
- Implement experiments to measure the effectiveness of different preference learning techniques, including RLHF
- Examine the impact of different aggregation methodologies on model behavior
- Curate large datasets of prompts and investigate coverage of boundary cases
- Exploring methods to understand and predict model behaviors, such as finding inputs causing anomalous circuits or catastrophic outputs
- Designing novel approaches for using LLMs in democratic inputs to AI research
- Are excited about OpenAI's mission of building safe, universally beneficial AGI and are aligned with OpenAI's charter
- Want to use your engineering skills to push the frontiers of what state-of-the-art language models can accomplish
- Possess a strong curiosity about the sociotechnical challenges around aligning and understanding ML models, and are motivated to use your career to address this challenge
- Enjoy fast-paced, collaborative, and cutting-edge research environments
- Have experience implementing ML algorithms (e.g., PyTorch)
- Can develop data visualization or data collection interfaces (e.g., JavaScript, Python)
- Want to ensure that humanity can shape future powerful AI systems
About OpenAI
OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.
We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.
For US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records.
We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.
OpenAI Global Applicant Privacy Policy
At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.
-
Research Engineer, Collective Alignment
3 weeks ago
San Francisco, United States OpenAI Full timeAbout the team OpenAI's mission is to ensure that artificial general intelligence benefits all of humanity. OpenAI's Collective Alignment Team works on technical approaches to ensure that AGI is shaped democratically. We plan to do so by implementing a system for collecting and encoding public input on model behavior into our systems. While OpenAI has...
-
Research Engineer, Alignment Science
2 days ago
San Francisco, United States Karkidi Full timeYou want to build and run elegant and thorough machine learning experiments to help us understand and steer the behavior of powerful AI systems. You care about making AI helpful, honest, and harmless, and are interested in the ways that this could be challenging in the context of human-level capabilities. You could describe yourself as both a scientist and...
-
Research Engineer, Alignment Science
22 hours ago
San Francisco, CA, United States Anthropic Limited Full timeAbout the role: You want to build and run elegant and thorough machine learning experiments to help us understand and steer the behavior of powerful AI systems. You care about making AI helpful, honest, and harmless, and are interested in the ways that this could be challenging in the context of human-level capabilities. You could describe yourself as both...
-
Research Engineer, Human-AI Interaction
1 month ago
San Francisco, CA, United States Genai Works Full timeThe Safety Systems team is responsible for various safety work to ensure our best models can be safely deployed to the real world to benefit the society and is at the forefront of OpenAI's mission to build and deploy safe AGI, driving our commitment to AI safety and fostering a culture of trust and transparency. The Human-AI Interaction (HAI) team is...
-
Research Engineer
24 hours ago
San Francisco, CA, United States Anthropic Limited Full timeAbout the role: You want to help construct and rapidly iterate on machine learning experiments to help us improve the behavior of powerful AI systems through finetuning. You care about making AI helpful, honest, and harmless, and are interested in shaping model behavior to be more aligned with human values and goals. You could describe yourself as both a...
-
Research Engineer
2 days ago
San Francisco, United States Genmo Full timeJob Description Job Description Our mission Genmo makes it easy for anyone to create movies, as if it were magic. Using our web application, any user can create cinematic video using a simple text prompt. We imagine a world where high-quality cinematic video content is as plentiful as water. Our mission is to empower the next billion video creators to tell...
-
Research Engineer
3 days ago
San Francisco, United States Genmo Full timeJob DescriptionJob DescriptionOur missionGenmo makes it easy for anyone to create movies, as if it were magic. Using our web application, any user can create cinematic video using a simple text prompt.We imagine a world where high-quality cinematic video content is as plentiful as water. Our mission is to empower the next billion video creators to tell their...
-
Software Engineer, Research
4 days ago
San Francisco, California, United States Wispr AI Full timeWispr is building a more natural way to interact with technology with neural interfaces. We have an elite team of engineers, product designers, and research scientists building magic.About Wispr: We've raised $25M from top-tier VCs like NEA and 8VC. Our angels and advisors include Chester Chipperfield (product lead for the first Apple Watch), Ben Jones (COO,...
-
Research scientist biology engineering
24 hours ago
San Francisco, CA, United States Anthropic Limited Full timeAbout the role: You want to help construct and rapidly iterate on machine learning experiments to help us improve the behavior of powerful AI systems through finetuning. You care about making AI helpful, honest, and harmless, and are interested in shaping model behavior to be more aligned with human values and goals. You could describe yourself as both a...
-
Staff Research Associate I
1 week ago
San Francisco, United States Veterans Health Research Institute Full timePosition Definition: The Staff Research Associate I is responsible for participating in the day-to-day operations of routine, minimally complex clinical research programs and/or studies conducted by Principal Investigators (PIs) at the San Francisco VA. Performs a variety of administrative and operational duties involved in the collection, compilation,...
-
Research Assistant I
4 weeks ago
San Francisco, United States Veterans Health Research Institute Full timeUnder the general supervision of the Principal Investigator, perform specialized and routine procedures and techniques in controlled experiments in the Brain Imaging and EEG Laboratory. Position will aid in an ongoing research projects investigating brain structure and function in psychosis and depression. Essential Functions: * Collect and analyze EEG...
-
RESEARCH ENGINEER
1 month ago
San Antonio, United States Southwest Research Institute Full timeWho We Are:The Mechanics & Materials team in the Materials Engineering department performs advanced materials research to solve our clients’ most challenging technical problems related to aerospace and manufacturing applications. This position will support senior staff members and ongoing programs t...
-
Staff Research Associate II
1 week ago
San Francisco, United States Veterans Health Research Institute Full timePosition Definition: The incumbent will serve as the Study Coordinator for an investigator-initiated research study which uses ambulatory technology to understand sleep disturbance in PTSD. The SRA II will collaborate on other studies, including the STAR study, as needed. This research is being conducted by researchers from the San Francisco VA Medical...
-
Research Engineer, Human-AI Interaction
1 month ago
San Francisco, CA, United States Genai Works Full timeAbout the Team The Safety Systems team is responsible for various safety work to ensure our best models can be safely deployed to the real world to benefit the society and is at the forefront of OpenAI's mission to build and deploy safe AGI, driving our commitment to AI safety and fostering a culture of trust and transparency. The Human-AI...
-
RESEARCH ENGINEER
2 days ago
San Antonio, United States Southwest Research Institute Full timeWho We Are: The Liquid Propulsion Team performs R&D to solve challenges and advance technologies in machinery and power systems with a focus on liquid propulsion systems. We support actual launch, landing, lunar, and orbital vehicles and ground equipment, ranging from high-g to 0-g and supercritical to cryogenic conditions. Our technologies are advancing...
-
Senior Research Engineer
2 days ago
San Francisco, United States CytoVale Full timeCompany Description Headquartered in San Francisco, CA, Cytovale is a dynamic, late-stage startup focused on developing a faster, more insightful way to diagnose fast-moving and immune-mediated diseases. Cytovale's IntelliSep test is pioneering a new way of quickly and accurately analyzing white blood cells to stratify a patient's risk of sepsis. The test...
-
Founding Research Engineer
2 days ago
San Francisco, United States Primeintellect Full timeAt Prime Intellect, we are on a mission to accelerate open and decentralized AI progress by enabling anyone to contribute compute, code or capital to train powerful, open models. Our ultimate goal? Openly accessible AGI that benefits everyone. But we can't do it alone and we want to do this together with you. We are building the infrastructure for...
-
Clinical Research Administrative
1 month ago
San Francisco, United States SF Research Institute Inc. Full timeResearch experience is a plus NOT a remote position. Must be able work on site. *Position Overview:* The administrative staff is the go-between for management, staff members, study participants, vendors, consultants, partnerships and other outside organizations. They play a large role in supporting the daily operations. Besides support, the administrative...
-
Senior Research Engineer
5 days ago
San Francisco, United States CytoVale Full timeCompany Description Headquartered in San Francisco, CA, Cytovale is a dynamic, late-stage startup focused on developing a faster, more insightful way to diagnose fast-moving and immune-mediated diseases. Cytovale's IntelliSep test is pioneering a new way of quickly and accurately analyzing white blood cells to stratify a patient's risk of sepsis. The test...
-
Research Engineer
3 days ago
San Francisco, United States Autodesk Full timeJob Requisition ID # 23WD74103 Position OverviewWe are looking for a Research Engineer, Mechatronics & Software Research to develop and implement research prototypes for the Autodesk Research organization.Our dedicated team, Autodesk Research, seeks to explore the "new possible" what's possible in the future which means we're looking as far as 10 years out,...