AI Safety and Evaluation Specialist

5 days ago


San Francisco, California, United States Scale AI Full time

Scale AI's Safety and Evaluation Lab (SEAL) is seeking a skilled Research Scientist/Engineer to join our team. You will be responsible for developing state-of-the-art LLM-based automated rating systems leveraging the Scale ecosystem for customized training data.

This is an exciting opportunity to work with a talented team on challenging research problems in evaluation and red teaming. You will have the chance to collaborate with industry leaders and publish your research findings.

The ideal candidate has:

  • A commitment to promoting safe, secure, and trustworthy AI deployments in the industry.
  • Practical experience conducting technical research collaboratively, with proficiency in frameworks like PyTorch, JAX, or TensorFlow.
  • A track record of published research in machine learning, particularly in generative AI.
  • At least three years of experience addressing sophisticated ML problems, whether in a research setting or in product development.

Nice to have:

  • Hands-on experience with open-source LLM fine-tuning or involvement in bespoke LLM fine-tuning projects using PyTorch/JAX.
  • Experience in crafting evaluations or a background in data science roles related to LLM technologies.

We offer a competitive salary range of $176,000-$300,000 USD, equity based compensation, and benefits including comprehensive health, dental, and vision coverage, retirement benefits, a learning and development stipend, and generous PTO. This role may also be eligible for additional benefits such as a commuter stipend.

],
  • Senior AI Engineer

    5 days ago


    San Francisco, California, United States Scale AI Full time

    Job DescriptionIn this role, you will play a critical part in understanding the capabilities and safeguarding large language models (LLMs). Our Safety and Evaluation Lab (SEAL) aims to build robust evaluation products and tackle challenging research problems in evaluation and red teaming.Design and implement robust evaluation benchmarks to measure and...


  • San Francisco, California, United States Scale AI, Inc. Full time

    About the OpportunityScale AI, Inc. is at the forefront of revolutionizing how organizations build and deploy artificial intelligence (AI). As part of our mission to accelerate this transition across every industry, we're seeking a skilled Senior Software Engineer to join our GenAI Safety & Evaluation product team.The ideal candidate will have expertise in...


  • San Francisco, California, United States Center for AI Safety Full time

    We are seeking a talented research engineer intern to join our team at the Center for AI Safety. As a research engineer intern, you will work closely with our researchers on cutting-edge projects in AI safety, leveraging your skills in machine learning and software development.Company OverviewThe Center for AI Safety is a leading research and field-building...


  • San Francisco, California, United States Scale AI Full time

    Unlock the Future of AI with Scale AIWe're pushing the boundaries of artificial intelligence at Scale AI, and we need your expertise to make it happen. As a key member of our Safety and Evaluation Lab (SEAL), you'll play a vital role in developing cutting-edge evaluation products and tackling complex research problems.About the RoleAs an AI Safety...


  • San Francisco, California, United States Virtue AI Full time

    Virtue AI seeks an experienced Product Manager to lead our efforts in AI safety and security. This role will involve collaborating with leaders to develop and execute the GTM strategy, working closely with Engineering teams to develop and improve AI products, and tracking trends in AI security and safety.The ideal candidate will have 3+ years of technical...


  • San Francisco, California, United States Scale AI Full time

    About the RoleWe are looking for an Ai Training Specialist to join our team at Scale AI. In this role, you will be responsible for developing and implementing training programs that enhance the skills and knowledge of our human contributors. You will work closely with cross-functional teams to design and deliver training content that meets the needs of our...


  • San Francisco, California, United States Scale AI Full time

    Unlocking Human Potential through AI TrainingWe are seeking a talented AI Instructional Specialist to join our team and develop innovative training solutions that unlock human potential. As an integral part of our organization, you will work closely with subject matter experts, product developers, and training stakeholders to create engaging and effective...


  • San Francisco, California, United States Scale AI Full time

    Role Overview:">Scale AI is seeking a highly skilled AI Research Scientist/Research Engineer to join our Safety, Evaluations and Analysis Lab (SEAL) team. As a key member of this pioneering research effort, you will play a crucial role in developing robust evaluation products and tackling challenging research problems in evaluation and red teaming.">About...


  • San Francisco, California, United States AI Tech Suite Full time

    About AI Tech SuiteAt AI Tech Suite, we're pioneering the future of artificial intelligence and machine learning. Our cutting-edge solutions empower businesses to harness the full potential of data-driven insights.We're a dynamic team of innovators, united by our passion for revolutionizing industries through AI-powered technologies.Job SummaryWe're seeking...


  • San Francisco, California, United States Scale AI Full time

    Company Overview:    At Scale AI, we are at the forefront of the transformative shift from traditional software to artificial intelligence (AI). Our mission is to expedite this process across various industries by empowering organizations to build and deploy AI more efficiently. Our products play a crucial role in powering cutting-edge language models,...


  • San Francisco, California, United States Together AI Full time

    Company Overview:At Together AI, we believe open and transparent AI systems will drive innovation and create the best outcomes for society. Our team has been behind technological advancements such as FlashAttention, Hyena, FlexGen, and RedPajama.Job Description:We are seeking an experienced MLOps engineer to develop systems and APIs that enable our customers...


  • San Francisco, California, United States OpenAI Full time

    At OpenAI, we are dedicated to developing AI that benefits humanity. As an AI Abuse and Threat Intelligence Analyst, you will play a pivotal role in ensuring the safety of our AI systems.About the RoleWe are seeking a highly skilled professional with experience in online safety and risk management to join our Intelligence and Investigations team. The ideal...

  • AI Safety Expert

    4 weeks ago


    San Francisco, California, United States OpenAI Full time

    Role OverviewWe are seeking a highly skilled AI Safety Expert to join our team at OpenAI.Job SummaryThe ideal candidate will have extensive experience in machine learning research engineering, ML observability and monitoring, creating large language model-enabled applications, and/or another technical domain applicable to AI risk.This role requires a deep...


  • San Francisco, California, United States Scale AI Full time

    About the RoleWe're looking for an entrepreneurial Software Engineer who can take an ambiguous scope and lead the execution of outcomes. You'll be given the opportunity to build products and drive millions of dollars in revenue.You'll also get widespread exposure to the forefront of the AI race as Scale sees it in enterprises, startups, governments, and...


  • San Francisco, California, United States Scale AI Full time

    About ScaleAt Scale AI, our mission is to accelerate the development of AI applications. For years, we've been leading the way in AI data foundry, helping fuel exciting advancements in AI, including generative AI, defense applications, and autonomous vehicles. With our recent Series F round, we're accelerating the abundance of frontier data to pave the road...


  • San Francisco, California, United States Scale AI Full time

    About ScaleScale AI is a pioneering company that's revolutionizing the way organizations build and deploy AI. With a strong mission to accelerate AI development, we provide innovative data solutions that fuel the most exciting advancements in AI.Our team is committed to making AI more accessible, powering the world's most advanced LLMs, generative models,...

  • AI Development Lead

    2 weeks ago


    San Francisco, California, United States Scale AI Full time

    About ScaleAt Scale AI, we are dedicated to accelerating the development of AI applications.We believe that the transition from traditional software to AI is one of the most significant shifts of our time.Our mission is to make this happen faster across every industry, and our team is revolutionizing how organizations build and deploy AI.About Data EngineOur...


  • San Francisco, California, United States Scale AI Full time

    About Job: This role is a great fit for those who want to accelerate the development of AI applications. Our company is scaling up fast and we are looking for a skilled software engineer to join our team. As a member of our team, you'll be working on building systems that monitor and flag quality issues with large-scale data collections. You will use machine...

  • AI Data Engineer

    6 days ago


    San Francisco, California, United States Scale AI Full time

    About ScaleWe are a leading AI data foundry, accelerating the development of AI applications. Our mission is to make this happen faster across every industry.Our Generative AI Data EnginePowers the world's most advanced LLMs and generative models through world-class RLHF (Reinforcement Learning with Human Feedback), human data generation, model evaluation,...


  • San Francisco, California, United States Scale AI Full time

    About Scale AIScale AI is at the forefront of accelerating the development of AI applications. With a strong foundation in data, we power the world's most advanced LLMs and generative models through cutting-edge RLHF, human data generation, model evaluation, safety, and alignment.Job Description:Growth Team OverviewThe Growth team plays a pivotal role in...