AIML - Senior Manager, ML Product Evaluation Engineering

22 hours ago


Cupertino CA United States Apple Inc. Full time
AIML - Senior Manager, ML Product Evaluation Engineering

Machine Learning and AI

Would you like to play a part in shipping groundbreaking technology for large-scale systems, natural language, and artificial intelligence? Join the Siri ML Systems Evaluation Engineering (MLSEE) team at Apple We interact with various engineering teams to identify issues and test requirements, drive the quality of Siri features, and create frameworks that support evaluation and qualification. You will work with the people who craft the intelligent assistant that helps millions of people get things done — just by asking

As a senior manager, you will play a critical role in driving the evaluation methods of various platforms and product areas for Siri. You will drive coverage and fidelity of testing and evaluation for pre-ship hill climbing and readiness for a wide range of AIML products, including Apple Intelligence features. You will be crafting new and leveraging existing methodologies and frameworks to validate the end-to-end quality of new features. Working alongside SW and ML engineering product teams as a critical partner, you will utilize a suite of available libraries and frameworks - test automation, LLM-based evaluators, and failure analysis tools - and drive requirements for improving those. You will be responsible for benchmarking performance for new features/platforms and driving improvement recommendations.

Minimum Qualifications
  • Minimum of 5+ years of leadership experience
  • Experience with Machine Learning development and evaluation
  • Experience with building and shipping products at scale
  • Proven and consistent track record of forming partnerships to solve complex technical problems at scale, ideally for products with a global customer base
  • MS/PhD in Machine Learning, Computer Science, or a related field
Preferred Qualifications
  • Passion for product and customer experience
  • Strong experience leading a breadth of product areas simultaneously
  • Extensive experience delivering large-scale products or platforms, leveraging cross-functional partnerships to drive outcomes across multiple organizations
  • Excellent understanding of an ML-based product lifecycle
  • Demonstrated depth of knowledge and application of statistics-based evaluation methodologies or user success metrics
  • Strong experience with systems engineering; in-depth understanding of interdependencies of ML and SW components
  • Strong history of delivering innovation
  • Experience managing a team of expert engineers
  • Ability to develop a long-term vision and execute strategies at scale while maintaining a grasp of technical details for better decision-making
  • Experience with VP-level presentations and reporting

Apple is an equal opportunity employer that is committed to inclusion and diversity. We take affirmative action to ensure equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics.

#J-18808-Ljbffr

  • Cupertino, CA, United States Apple Inc. Full time

    Do you want to play a part in building groundbreaking technology for large-scale systems, natural language, big data, and artificial intelligence? Join the AIML team at Apple and help us improve the user experience of Siri and other AIML products, helping millions of people get things done across the globe! Description In this senior leadership role, you...


  • Cupertino, CA, United States Apple Inc. Full time

    Do you want to play a part in building groundbreaking technology for large-scale systems, natural language, big data, and artificial intelligence? Join the AIML team at Apple, and help us as we improve the user experience of Siri and AIML products, helping millions of people get things done across the globe! Description In this senior technical leadership...


  • Cupertino, CA, United States Apple Full time

    AIML - Senior ML Engineering Manager, Evaluation & Analytics Cupertino, California, United States Machine Learning and AI We are seeking a Senior Machine Learning Manager with a proven track record of leading projects in cross-functional and interdisciplinary settings, along with strong technical expertise. Your highly skilled team will be responsible for...


  • Cupertino, CA, United States Apple Full time

    AIML - Senior Front End Platform Engineer - Full Stack, ML Systems and Evaluation Engineering Cupertino, California, United States Machine Learning and AI Apple is where individual imaginations gather together, committing to the values that lead to great work. Every new product we build, service we create, or Apple Store experience we deliver is the result...


  • Cupertino, United States Apple Inc. Full time

    AIML - Software Development Automation Engineering Lead, ML Systems Evaluation EngineeringDoes the opportunity to play a part in building groundbreaking technology for large-scale systems, natural language and artificial intelligence excite you? Do you want to expand the experience of Siri and other AIML products to new products that will help millions get...


  • Cupertino, CA, United States Apple Inc. Full time

    Software Engineer, ML Product Evaluation Engineering Machine Learning and AI Apple is where individual imaginations gather together, committing to the values that lead to great work. Every new product we build, service we create, or Apple Store experience we deliver is the result of us making each other’s ideas stronger. That happens because every one of...


  • Seattle, WA, United States Apple Full time

    AIML - Senior ML Engineering Manager, Evaluation & Analytics Seattle, Washington, United States Machine Learning and AI We are seeking a Senior Machine Learning Manager with a proven track record of leading projects in cross-functional and interdisciplinary settings, along with strong technical expertise. Your highly skilled team will be responsible for...


  • Cupertino, CA, United States Apple Inc. Full time

    AIML - ML Research Engineer or Scientist Our team is looking for a roboticist to work with a collaborative team of researchers within Apple’s AIML organization. This team is working to leverage and build upon groundbreaking ML robotics research, thereby enabling development of generalizable and reliable robot systems. Description Duties associated with...


  • Santa Clara, CA, United States Apple Full time

    AIML - Senior Software Engineer - ML Platform & Technology (MLPT) Santa Clara, California, United States Machine Learning and AI Join us in enabling the next generation of intelligent experiences in Apple’s products and services with the latest advancements in Generative AI and Large Language Models! We are seeking a highly skilled Senior Software Engineer...

  • AIML Resident

    20 hours ago


    Cupertino, CA, United States Apple Full time

    Summary Imagine what you could do here! At Apple, great ideas have a way of becoming great products, services, and customer experiences very quickly. Combining groundbreaking machine learning research with next-generation hardware, our teams take user experiences to the next level! Description Apple’s AIML residency is a year-long program inviting...


  • Cupertino, CA, United States Apple Full time

    AIML - Senior Machine Learning Manager, Privacy ML Cupertino, California, United States Machine Learning and AI Imagine what you could do here. At Apple, new ideas have a way of becoming great products, services, and customer experiences very quickly. Bring passion and dedication to your job and there's no telling what you could accomplish. Apple delivers...


  • Cupertino, CA, United States Apple Inc. Full time

    Apple is where individual imaginations gather together, committing to the values that lead to great work. Every new product we build, service we create, or Apple Store experience we deliver is the result of us making each other’s ideas stronger. That happens because every one of us shares a belief that we can make something wonderful and share it with the...


  • Santa Clara, CA, United States Apple Inc. Full time

    AIML - Senior Software Engineer - ML Platform & Technology (MLPT) Join us in enabling the next generation of intelligent experiences in Apple’s products and services with the latest advancements in Generative AI and Large Language Models! We are seeking a highly skilled Senior Software Engineer to join our dynamic team. This role will focus on designing,...


  • Seattle, WA, United States Apple Inc. Full time

    AIML - Senior Software Engineer, ML Platform & Technology Join us in enabling the next generation of intelligent experiences in Apple’s products and services with the latest advancements in Generative AI and Large Language Models!We are seeking a highly skilled Senior Software Engineer to join our dynamic team. This role will focus on designing,...


  • Cupertino, CA, United States Apple Inc. Full time

    AIML - Sr Machine Learning Engineer, Data and ML Innovation As part of Apple's AI and Machine Learning org, we encourage and create groundbreaking technology for large-scale ML systems, computer vision, natural language processing, and multi-modal understanding. The Data and Machine Learning Innovation (DMLI) team is looking for a passionate Machine...


  • Cupertino, United States Apple Full time

    AIML - Sr Machine Learning Engineer, Data and ML Innovation Cupertino, California, United States Machine Learning and AI Do you want to play a part in the next revolution in large language models, contribute to products that are redefining mobile and desktop computing, and work with the people who built the intelligent assistant and search products that help...


  • Cupertino, California, United States Apple Inc. Full time

    AIML-Sr. Machine Learning Engineer, MeasurementAre you ready to shape the future with cutting-edge technology? As a Senior Machine Learning Software Engineer, you'll have the opportunity to design, develop, and deploy intelligent systems that solve real-world problems and redefine possibilities. In this role, you'll harness the power of data to build...


  • Seattle, WA, United States Apple Full time

    AIML - Senior Full-Stack Software Engineer - ML Platform & Technology Seattle, Washington, United States Machine Learning and AI Would you like to join us in building the next generation of machine learning platform powering Apple's intelligent experiences across hardware, software, and service products? We are looking for a senior full stack software...


  • Cupertino, CA, United States Apple Inc. Full time

    AIML- Senior Backend Software Engineer, Measurement At Apple, we create world-class innovative products that seamlessly combine cutting-edge hardware with intelligent software experiences, powered by advanced machine learning technologies. The Measurement org is responsible for building platforms and tools to measure, understand, and proactively enhance...


  • Cupertino, CA, United States Apple Full time

    AIML - Staff Machine Learning Engineer - ML Efficiency, ML Platform & Technology Cupertino, California, United States Machine Learning and AI Do you want to shape the platform that enables the next generation of intelligent experiences on Apple products & services? In Apple’s Machine Learning Platform Technology & Infra team we have built the platform that...