Machine Learning

1 week ago

Sunnyvale, CA, United States Apple Full time

Role Number: 200612047-3956

Summary

Do you have a passion for computer vision and solving deep learning problems? The Video Engineering Data Analytics and Quality group is seeking an expert in evaluating machine learning and deep learning models, including foundation models and multimodal systems.

This role will play a critical part in crafting robust evaluation frameworks, using both traditional statistical methods and modern techniques like LLM-as-a-Judge The ideal candidate combines strong analytical thinking, expertise in Python, and advanced knowledge of statistical methodologies and data quality standards.

This role involves collaboration with teams at Apple passionate about developing foundation models, including ML engineers, data scientists, and ML Infrastructure engineers to deliver amazing user experiences

Description

Develop robust methodologies to assess the performance of foundation models (e.g., LLMs, vision-language models, etc.) across diverse tasks. Leverage LLMs as judges to perform subjective and open-ended model evaluations (e.g., for summarization, reasoning, or multimodal generation tasks). Build, curate, and lead evaluation datasets and benchmarks. Advanced proficiency in at least one scripting language, preferably Python. Collaborate with research, engineering, and product teams to define evaluation goals aligned with user experience and product quality. Conduct failure analysis and uncover edge cases to improve model robustness. Contribute to our tools and infrastructure to automate and scale evaluation processes.

Minimum Qualifications

  • BS and a minimum of 10 years relevant industry experience.

  • Strong experience in evaluating supervised, unsupervised, and deep learning models.

  • Hands-on experience evaluating LLMs (e.g., GPT, Claude, PaLM) and using them as scoring/judging mechanisms.

  • Familiarity with multimodal models (e.g., image + text, video + audio) and related evaluation challenges.

  • Proficiency in Python and libraries such as NumPy, pandas, scikit-learn, PyTorch, or TensorFlow.

  • Solid understanding of statistical testing, sampling, confidence intervals, and metrics (e.g., precision/recall, BLEU, ROUGE, FID, etc.).

  • Strong documentation skills, including the ability to write technical reports and present to non-technical audiences.

Preferred Qualifications

  • Experience working with open-source evaluation tools like OpenEval, ELO-based ranking, or LLM-as-a-Judge frameworks.

  • Familiarity with prompt engineering, few-shot or zero-shot evaluation techniques.

  • Experience evaluating generative models (e.g., text generation, image generation).

  • Prior contributions to ML benchmarks or public evaluations.

  • Strong interpersonal skills.

Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant (https://www.eeoc.gov/sites/default/files/2023-06/22-088_EEOC_KnowYourRights6.12ScreenRdr.pdf) .

  • Machine Learning Engineer

    1 week ago

    Sunnyvale, CA, United States Institute of Foundation Models Full time

    About the Institute of Foundation Models We are a dedicated research lab for building, understanding, using, and risk-managing foundation models. Our mandate is to advance research, nurture the next generation of AI builders, and drive transformative contributions to a knowledge-driven economy. As part of our team, you'll have the opportunity to work on the...

  • Machine Learning Scientist

    7 hours ago

    Sunnyvale, CA, United States Scopely Full time

    Niantic’s Machine Learning Team seeks a Machine Learning Scientist to join our AI/ML team. This role focuses on crafting and implementing machine learning-powered features for our geo-location-based mobile games. You will collaborate closely with product managers, game designers, engineers, and data scientists to craft intelligent game mechanics that...

  • Machine Learning Compiler Engineer

    2 weeks ago

    Sunnyvale, CA, United States Apple Full time

    At Apple, we're on the cutting edge of delivering transformative experiences through Artificial Intelligence. If you're passionate about pushing the boundaries of AI and hardware optimization, we want you to join our team! As a Machine Learning Compi Machine Learning, Engineer, Computer Engineer, Computer Science, Technology, Hardware

  • Machine Learning Compiler Engineer

    2 weeks ago

    Sunnyvale, CA, United States Apple Full time

    At Apple, we're on the cutting edge of delivering transformative experiences through Artificial Intelligence. If you're passionate about pushing the boundaries of AI and hardware optimization, we want you to join our team! As a Machine Learning Compi Machine Learning, Engineer, Computer Engineer, Computer Science, Technology, Hardware

  • Machine Learning Compiler Engineer

    2 weeks ago

    Sunnyvale, CA, United States Apple Full time

    At Apple, we're on the cutting edge of delivering transformative experiences through Artificial Intelligence. If you're passionate about pushing the boundaries of AI and hardware optimization, we want you to join our team! As a Machine Learning Compi Machine Learning, Engineer, Computer Engineer, Computer Science, Technology, Hardware

  • Machine Learning Compiler Engineer

    2 weeks ago

    Sunnyvale, CA, United States Apple Full time

    At Apple, we're on the cutting edge of delivering transformative experiences through Artificial Intelligence. If you're passionate about pushing the boundaries of AI and hardware optimization, we want you to join our team! As a Machine Learning Compi Machine Learning, Engineer, Computer Engineer, Computer Science, Technology, Hardware

  • Machine Learning Compiler Engineer

    2 weeks ago

    Sunnyvale, CA, United States Apple Full time

    At Apple, we're on the cutting edge of delivering transformative experiences through Artificial Intelligence. If you're passionate about pushing the boundaries of AI and hardware optimization, we want you to join our team! As a Machine Learning Compi Machine Learning, Engineer, Computer Engineer, Computer Science, Technology, Hardware

  • Machine Learning Compiler Engineer

    2 weeks ago

    Sunnyvale, CA, United States Apple Full time

    At Apple, we're on the cutting edge of delivering transformative experiences through Artificial Intelligence. If you're passionate about pushing the boundaries of AI and hardware optimization, we want you to join our team! As a Machine Learning Compi Machine Learning, Engineer, Computer Engineer, Computer Science, Technology, Hardware

  • Machine Learning Compiler Engineer

    2 weeks ago

    Sunnyvale, CA, United States Apple Full time

    At Apple, we're on the cutting edge of delivering transformative experiences through Artificial Intelligence. If you're passionate about pushing the boundaries of AI and hardware optimization, we want you to join our team! As a Machine Learning Compi Machine Learning, Engineer, Computer Engineer, Computer Science, Technology, Hardware

  • Machine Learning Compiler Engineer

    2 weeks ago

    Sunnyvale, CA, United States Apple Full time

    At Apple, we're on the cutting edge of delivering transformative experiences through Artificial Intelligence. If you're passionate about pushing the boundaries of AI and hardware optimization, we want you to join our team! As a Machine Learning Compi Machine Learning, Engineer, Computer Engineer, Computer Science, Technology, Hardware