Language Model Robustness Specialist

3 weeks ago


New York, New York, United States Scale AI Full time
About the Role:

Scale AI is seeking a highly skilled researcher to join our team. As part of our Safety, Evaluations and Analysis Lab (SEAL), you will play a key role in advancing the field of AI safety and evaluation.

The ideal candidate will have a strong background in machine learning, deep learning, natural language processing, or computer vision, with a focus on agent safety, evaluation, alignment, and related fields. You should have prior experience or a track record of research publications in these areas.

In this position, you will work on various projects focused on the development and application of advanced techniques for evaluating and improving the safety and reliability of large language models. These projects may include:
  • Developing and applying novel metrics and benchmarks for measuring the safety and reliability of LLMs
  • Designing and implementing efficient and effective methods for scaling up the evaluation and testing of LLMs
  • Collaborating with cross-functional teams to develop and deploy AI-powered solutions that enhance the safety and reliability of LLMs

Requirements:

To be successful in this role, you will need:
  1. A current enrollment in a BS/MS/PhD program with a focus on Machine Learning, Deep Learning, Natural Language Processing, or Computer Vision, with a graduation date in Fall 2025 or Spring 2026
  2. Prior experience or a track record of research publication in agent safety, evaluation, alignment, or a related field
  3. Experience with one or more general-purpose programming languages, including Python, Javascript, or similar
  4. Ability to speak and write in English fluently
  5. Availability for a Summer 2025 internship starting in May/June

Why Work at Scale AI?

We offer a dynamic and inclusive work environment, with a talented team of researchers and engineers who are passionate about advancing the field of AI safety and evaluation. We also offer a competitive salary of $85,000 per year, plus benefits including health insurance, retirement plans, and paid time off.

  • New York, New York, United States Brains Workgroup, Inc. Full time

    Job DescriptionWe are seeking a skilled Financial Model Specialist to join our team at Brains Workgroup, Inc. The ideal candidate will possess a strong understanding of how banking and trading businesses generate revenue, the associated risks, and regulatory requirements surrounding model development and validation.About the Role:Develop and implement robust...


  • New York, New York, United States Normal Computing Full time

    Unlock the Power of AI EngineeringWe are seeking a highly skilled Data Scientist to join our team at Normal Computing. As an AI Engineer, you will play a crucial role in developing cutting-edge machine learning applications that transform complex technical documents into code.About the RoleThis is an exciting opportunity to contribute to groundbreaking...


  • New York, New York, United States Michael Page Full time

    Michael Page is seeking a skilled Financial Modeling Specialist to join their team. As a Financial Modeling Specialist, you will be responsible for analyzing and monitoring risk metrics for equity tax-managed strategies, ensuring compliance with investment goals. You will also apply your programming skills to automate and enhance risk reporting and analysis...


  • New York, New York, United States Firsthand Full time

    Job OverviewWe're seeking an AI Specialist engineer to join our team and lead the application of our product's latest LLM/AI technology. As an AI/ML engineer, you'll be responsible for pioneering experimenting, developing, and deploying cutting-edge machine learning and large language models for production systems. You'll monitor and evaluate the performance...


  • New York, New York, United States Underground Administration Full time

    Job DescriptionAs a Quantitative Modeling Specialist at Underground Administration, you will be responsible for developing and implementing advanced quantitative models to analyze and mitigate risk in the mortgage industry. Your expertise in C++ and knowledge of interest rate derivatives models will enable us to make informed decisions and optimize our...


  • New York, New York, United States Flagstar Bank Full time

    About the OpportunityWe are looking for a highly motivated Quantitative Financial Modeling Specialist to join our team at Flagstar Bank. As a key contributor, you will be responsible for developing and maintaining advanced financial models used in risk measurement and management.Key ResponsibilitiesDevelop and maintain quantitative financial models used in...


  • New York, New York, United States GNY Insurance Companies Full time

    Job Overview:GNY Insurance Companies seeks a skilled Machine Learning and Predictive Analytics Specialist to join our team. This role involves developing and maintaining advanced analytics tools using machine learning and predictive analytics techniques.Responsibilities:Data Analysis: Collect, analyze, and clean large datasets.Predictive Modeling: Develop...


  • New York, New York, United States Skale Co. Full time

    Job DescriptionWe're looking for a highly experienced Machine Learning Engineer to join our team at Skale Co.The ideal candidate will have a deep understanding of NLP, specifically experience with fine-tuning large language models.Key responsibilities include:Developing and implementing NLP solutions for search, personalisation, and ads.Fine-tuning and...


  • New York, New York, United States Bloomberg Full time

    Key ResponsibilitiesThis role involves training, tuning, and evaluating Large Language Models using vast amounts of high-quality data. The ideal candidate will demonstrate technical leadership by owning cross-team projects and staying current with the latest research in AI, NLP, and LLMs. They will also represent Bloomberg at scientific conferences and...


  • New York, New York, United States Amazon Full time

    Salary and BenefitsThe estimated annual salary for this position is $114,100, reflecting the cost of labor across various US geographic markets. Amazon offers a total compensation package, including equity, sign-on payments, and benefits, in addition to a full range of medical, financial, and other benefits.Job RequirementsThis role requires a Masters degree...


  • New York, New York, United States Harnham Full time

    Director of Machine Learning PositionAbout UsHarnham is a pioneering tech partner that brings together cutting-edge AI and open-source infrastructure to create innovative data models. We're looking for a seasoned Director of Machine Learning to join our team.Job SummaryThis role involves driving the development of novel Large Language Models (LLMs) from...


  • New York, New York, United States KPMG Full time

    **About Us**KPMG is a dynamic firm at the forefront of innovation in tax services. We empower our employees to thrive in a collaborative environment, where they can leverage our extensive network of specialists and cutting-edge technologies to drive business solutions for high-profile clients.We are seeking an experienced Manager to join our Economic and...


  • New York, New York, United States BizTek People, Inc. | APA International Placement Consultants Full time

    BizTek People, Inc. | APA International Placement Consultants seeks a Senior Risk Model Specialist to serve as local model owner and monitor credit risk related models for the Bank.ResponsibilitiesConduct regular performance monitoring of credit risk models based on established plans and communicate with team leads, model validation teams, and...


  • New York, New York, United States Bloomberg Full time

    Required Skills and Qualifications\The ideal candidate will have practical and subject matter experience in Natural Language Processing, a Ph.D. in ML, NLP, or a relevant field, or MSc in CS, ML, Math, Statistics, Engineering, or related fields, and 2+ years of relevant work experience.\Additionally, the candidate should have 5+ years of experience in Python...


  • New York, New York, United States Harnham Full time

    We're seeking a talented Lead Large Language Model Developer to join our team at Harnham. As a founding member of this innovative startup, you'll have the opportunity to shape the direction of AI research and development. Your primary responsibility will be to build novel machine learning and LLM models for use in open-source environments.About the...


  • New York, New York, United States Brains Workgroup Full time

    Job Description">Company OverviewBrains Workgroup is a leading provider of financial services, seeking a talented Financial Model Developer to join our team.Salary and BenefitsWe offer a competitive compensation package with excellent benefits and a target bonus. This hybrid commute position comes with 4 days per week in the office.ResponsibilitiesThe ideal...


  • New York, New York, United States Amazon Full time

    Join Our TeamAs a Data Scientist in our Topline Forecasting team, you will have the opportunity to drive business growth and decision-making at Amazon. We are looking for a skilled professional with expertise in forecasting models, data analysis, and machine learning.ResponsibilitiesDevelop and improve forecasting models using advanced techniques.Analyze...


  • New York, New York, United States Scale AI Full time

    Shape the Future of AI with Scale AI">We're looking for talented individuals to join our team as Machine Learning Research Interns and contribute to the development of cutting-edge AI solutions.">About the Opportunity">This role offers a unique chance to apply your expertise in machine learning, deep learning, and natural language processing to challenging...


  • New York, New York, United States Mizuho Bank Ltd Full time

    Job SummaryMizuho Financial Group, Inc. is a leading global bank with a strong presence in the Americas. We are seeking a highly skilled Financial Model Developer to join our team.The ideal candidate will have a strong background in finance and economics, with expertise in developing and implementing financial models. The successful applicant will be...


  • New York, New York, United States Vimeo Full time

    Senior Analytics Engineer Job DescriptionWe're searching for a highly skilled Senior Data Engineer - Analytics to join our team. This role involves working closely with Data Analysts and Data Scientists to build and maintain robust data models that support business decision-making. Main Responsibilities:Design and implement scalable data models that support...