AI Abuse

2 days ago

San Francisco CA United States OpenAI Full time
OpenAI

The Intelligence and Investigations team seeks to rapidly detect and disrupt abuse in AI and generative AI technologies to ensure their safe use. We are dedicated to identifying emerging abuse trends, analyzing risks, and working with our internal partners to implement effective mitigation strategies to protect against misuse. Our efforts contribute to OpenAI's overarching goal of developing AI that benefits humanity.

About the Role

As an AI Abuse and Threat Intelligence Analyst focused on Intel & Investigations at OpenAI, you will play a pivotal role in ensuring the safety of AI.

We’re looking for people who combine online safety experience and risk management with outstanding analytical and problem-solving skills. The ideal candidate will be able to clearly articulate complex abuse concepts and collaborate effectively with diverse technical and business teams. We value professionals dedicated to promoting the responsible use of AI. Adaptability and a commitment to continuous learning in the rapidly evolving field of AI are also key traits for this role.

This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees.

In this role, you will:

  1. Drive insights and recommendations through comprehensive qualitative and quantitative analysis of emerging risks in AI and online platforms, translating complex patterns into actionable safety insights.
  2. Collaborate on safety reporting tools by contributing technical expertise in Python (essential) and SQL (desirable), helping to scale and automate safety insights for diverse internal and external stakeholders.
  3. Support safety enhancements on OpenAI’s platforms, leveraging experience with React and TypeScript to assist in building robust safety features.
  4. Conduct and evaluate targeted risk assessments and simulations, working with cross-functional teams to assess and mitigate AI misuse scenarios, while ensuring insights are effectively communicated and actionable.
  5. Build tooling for internal and external users to help track abuse on first-party and third-party platforms.

You might thrive in this role if you:

  1. Have experience in performing in-depth qualitative and quantitative analysis to inform safety strategies, particularly within online content and AI applications.
  2. Possess technical skills in Python, and modern front-end development technologies like React, and TypeScript, with a track record of contributing to projects aimed at risk assessment, safety reporting, or building technical systems that detect abuse at scale.
  3. Demonstrate a deep understanding of AI and generative AI technologies and the potential abuse scenarios, with a proactive approach to identifying and addressing emerging threats.
  4. Exhibit excellent communication skills, capable of synthesizing complex findings into clear, concise recommendations for various audiences.

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.

For US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records.

We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.

At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

#J-18808-Ljbffr

  • AI Abuse

    4 weeks ago

    San Francisco, United States OpenAI Full time

    OpenAIThe Intelligence and Investigations team seeks to rapidly detect and disrupt abuse in AI and generative AI technologies to ensure their safe use. We are dedicated to identifying emerging abuse trends, analyzing risks, and working with our internal partners to implement effective mitigation strategies to protect against misuse. Our efforts contribute to...

  • Software Engineer, GenAI Violations

    2 days ago

    San Francisco, CA, United States Scale AI, Inc. Full time

    About Scale At Scale AI, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, helping fuel the most exciting advancements in AI, including: generative AI, defense applications, and autonomous vehicles. With our recent Series F round, we're accelerating the abundance of frontier data to...

  • Senior AI Safety Researcher

    7 days ago

    San Francisco, California, United States Openai Full time

    At OpenAI, we are dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. Our team is seeking a highly skilled AI Abuse and Threat Intelligence Analyst focused on Intel & Investigations to play a pivotal role in ensuring the safety of AI.We are looking for individuals who combine online safety experience and risk...

  • Software Engineer, Anti Fraud

    3 weeks ago

    San Francisco, United States OpenAI Full time

    About the teamThe Applied team safely brings OpenAI's technology to the world. We released ChatGPT; Plugins; DALL·E; and the APIs for GPT-4, GPT-3, embeddings, and fine-tuning. We also operate inference infrastructure at scale. There's a lot more on the immediate horizon.Our customers build fast-growing businesses around our APIs, which power product...

  • Software Engineer, GenAI Violations

    4 weeks ago

    San Francisco, United States Scale AI, Inc. Full time

    About ScaleAt Scale AI, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, helping fuel the most exciting advancements in AI, including: generative AI, defense applications, and autonomous vehicles. With our recent Series F round, we're accelerating the abundance of frontier data to pave...

  • Fullstack Software Engineer, GenAI Violations

    4 weeks ago

    San Francisco, United States Scale AI, Inc. Full time

    About ScaleAt Scale AI, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, helping fuel the most exciting advancements in AI, including: generative AI, defense applications, and autonomous vehicles. With our recent Series F round, we're accelerating the abundance of frontier data to pave...

  • Data Engineer

    1 week ago

    San Francisco, California, United States Scale AI, Inc. Full time

    About Scale AI, Inc.We accelerate the development of AI applications at Scale AI. Our mission is to pave the road to Artificial General Intelligence (AGI) through frontier data abundance. We offer deep capabilities and offerings for public and private evaluations, building on our prior model evaluation work with enterprise customers and governments.About...

  • Senior Software Engineer, GenAI Violations

    4 weeks ago

    San Francisco, United States Scale AI, Inc. Full time

    About ScaleAt Scale AI, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, helping fuel the most exciting advancements in AI, including: generative AI, defense applications, and autonomous vehicles. With our recent Series F round, we're accelerating the abundance of frontier data to pave...

  • Substance Abuse Counselor

    2 months ago

    Los Angeles, CA, United States United American Indian Involvement Full time

    Job Location:Los Angeles, CA 90026 Reports to (title):Healthcare Clinical Operations DirectorDepartment:Robert Sundance Family Wellness Center Supervises:N/AWorking title:Substance Abuse Counselor Classification:Non-ExemptSalary Range:$21.00 - $31.50 hourly Status:RegularHours/Week:40 Effective Date:February 2022Type of Position:Full-time Revised...

  • ML Technical Product Manager

    2 days ago

    San Francisco, CA, United States Wallarm Inc. Full time

    Short facts about us: We are a global remote-first team of 100+ people on 4 continents and in 10+ countries. We have been protecting our clients since 2014. The company has raised over $10M in investments. More than 200 customers around the world, including Fortune 500, Nasdaq, and high-growth startups choose Wallarm to protect their API and web...

  • Software Engineer, AI

    4 weeks ago

    Redwood City, CA, United States BlueArc Full time

    Company:BlueArc is a seed-stage venture-backed start-up building AI automated systems that deeply research fraud, policy abuse, and anti-money laundering (AML) risks of businesses. We provide automated and high-quality risk decisioning for tech and financial services companies who have risky business customers.Market opportunity:2-5% of the global economy is...

  • Software Engineer, Safety

    1 day ago

    San Francisco, CA, United States OpenAI Full time

    About the Team The Applied AI team safely brings OpenAI's advanced technology to the world. We released the GPT-3 API , Codex (which powers GitHub Copilot ), and DALL-E . More is coming very soon. We empower developers with APIs offering state-of-the-art AI capabilities, which power product features that were never before possible. We also build AI-driven...

  • Research Engineer, Safety Reasoning

    1 day ago

    San Francisco, CA, United States OpenAI Full time

    The Safety Systems team is responsible for various safety work to ensure our best models can be safely deployed to the real world to benefit society and is at the forefront of OpenAI's mission to build and deploy safe AGI, driving our commitment to AI safety and fostering a culture of trust and transparency. The Safety Reasoning Research team is poised at...

  • Software Engineer, Trust

    1 day ago

    San Francisco, CA, United States Anthropic Limited Full time

    About the role: We are looking for software engineers to help build safety and oversight mechanisms for our AI systems. As a trust and safety software engineer, you will work to monitor models, prevent misuse, and ensure user well-being. This role will focus on building systems to detect unwanted model behaviors and prevent disallowed use of models. You...

  • ML Technical Product Manager

    2 weeks ago

    San Francisco, United States Wallarm Full time

    Short facts about us: We are a global remote-first team of 100+ people on 4 continents and in 10+ countries. We have been protecting our clients since 2014. The company has raised over $10M in investments. More than 200 customers around the world, including Fortune 500, Nasdaq, and high-growth startups choose Wallarm to protect their API and web...

  • Technical Product Manager

    2 weeks ago

    San Francisco, United States Wallarm Inc. Full time

    Short facts about us:We are a global remote-first team of 100+ people on 4 continents and in 10+ countries.We have been protecting our clients since 2014.The company has raised over $10M in investments.More than 200 customers around the world, including Fortune 500, Nasdaq, and high-growth startups choose Wallarm to protect their API and web applications.The...

  • Technical Product Manager

    2 weeks ago

    San Francisco, United States Wallarm Inc. Full time

    Short facts about us:We are a global remote-first team of 100+ people on 4 continents and in 10+ countries.We have been protecting our clients since 2014.The company has raised over $10M in investments.More than 200 customers around the world, including Fortune 500, Nasdaq, and high-growth startups choose Wallarm to protect their API and web applications.The...

  • Strategic AI Policy Lead

    7 days ago

    San Jose, California, United States Adobe Full time

    About AdobeAt the forefront of digital innovation, Adobe is a pioneering company that empowers individuals and businesses to create transformative experiences. Our mission revolves around harnessing the power of technology to shape a better world.Salary InformationThe estimated annual salary for this position ranges from $101,600 to $200,700, based on U.S....

  • Engineering Manager, Detection

    2 days ago

    San Francisco, CA, United States ADVANCED ENGINEERING GROUP PC Full time

    Anthropic is an AI safety and research company that’s working to build reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our customers and for society as a whole. Our interdisciplinary team has experience across ML, physics, policy, business and product. Responsibilities: Lead a team of engineers building systems...

  • Software Engineer, Trust

    2 days ago

    San Francisco, CA, United States Trust A-tech Full time

    About Anthropic Anthropic is an AI safety and research company that’s working to build reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our customers and for society as a whole. Our interdisciplinary team has experience across ML, physics, policy, business and product. Responsibilities: Develop monitoring systems...