AI/HPC Systems Performance Engineer

1 month ago

Menlo Park, California, United States Facebook Full time
Meta's AI Training and Inference Infrastructure is growing exponentially to support ever increasing uses cases of AI. This results in a dramatic scaling challenge that our engineers have to deal with on a daily basis. We need to build and evolve our network infrastructure that connects myriads of training accelerators like GPUs together. In addition, we need to ensure that the network is running smoothly and meets stringent performance and availability requirements of RDMA workloads that expects a loss-less fabric interconnect. To improve performance of these systems we constantly look for opportunities across stack: network fabric and host networking, comms lib and scheduling infrastructure.

AI/HPC Systems Performance Engineer Responsibilities:

  • Active member of a multi-disciplinary team to develop solutions for large scale training systems.
  • Responsible for the overall performance of the communication system, including performance benchmarking, monitoring and troubleshooting production issues.
  • Identify potential performance issues across the stack: comms lib, RDMA transport, host networking, scheduling and network fabric. Develop and deploy innovative solutions to address the performance issues.
Minimum Qualifications:

  • Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience.
  • BS/MS/PhD in relevant fields (EE, CS), with 4+ years work experience.
  • Experience with using communication libraries, such as MPI, NCCL, and UCX.
  • Experience with developing, evaluating and debugging host networking protocols such as RDMA.
  • Experience with triaging performance issues in complex scale-out distributed applications.
Preferred Qualifications:

  • Understanding of AI training workloads and demands they exert on networks.
  • Understanding of RDMA congestion control mechanisms on IB and RoCE Networks.
  • Understanding of the latest artificial intelligence (AI) technologies.
  • Experience with machine learning frameworks such as PyTorch and TensorFlow.
  • Experience in developing systems software in languages like C++.
Meta is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. Meta participates in the E-Verify program in certain locations, as required by law. Please note that Meta may leverage artificial intelligence and machine learning technologies in connection with applications for employment.

Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at accommodations-

  • Senior Production Systems Engineer, Fleet AI Systems

    5 days ago

    Menlo Park, California, United States META Full time

    Job DescriptionMeta is seeking an experienced Production Systems Engineer to join our Release to Production (RTP) team. Our servers and data centers are the foundation upon which our rapidly scaling infrastructure operates efficiently to deliver our innovative services.Key ResponsibilitiesDevelop and Implement Robust Hardware Infrastructure Practices:...

  • Systems Engineering Specialist, AI Fleet Operations

    2 weeks ago

    Menlo Park, California, United States META Full time

    Meta is on the lookout for a Systems Engineering Specialist to become a vital part of our Release to Production (RTP) team. Our infrastructure, comprising servers and data centers, is crucial for the seamless operation of our rapidly expanding services. The RTP team oversees the Hardware Lifecycle of all Meta servers, engaging in hands-on system and hardware...

  • AI Engineering Project Manager

    5 days ago

    Menlo Park, California, United States January, Inc. Full time

    About January, Inc.January, Inc. is a pioneering company in the field of precision health, dedicated to empowering individuals to take control of their metabolic wellbeing. Our innovative approach combines continuous glucose monitoring with heart rate, sleep, activity, and food tracking to provide real-time insights into how the body metabolizes food.Job...

  • Senior Site Reliability Engineer

    1 week ago

    Menlo Park, California, United States Character Technologies Full time

    About the Position:As a pivotal member of our Site Reliability Engineering team at Character Technologies, you will play a crucial role in managing our extensive infrastructure, which encompasses thousands of nodes, vast amounts of data, and millions of active users daily. Your primary focus will be to ensure the reliability, scalability, and performance of...

  • Director of Engineering

    1 month ago

    Menlo Park, California, United States January AI Full time

    Founded by veteran business executive, Noosheen Hashemi, and Director of Genomics and Personalized Medicine at Stanford University, Dr. Michael Snyder, January AI is the first "precision health" company that combines continuous glucose monitoring along with heart rate, sleep, activity and food tracking, to enable you to see the impact of your diet and...

  • Senior Solutions Engineer for AI Innovations

    2 weeks ago

    Menlo Park, California, United States HCLTech Full time

    Position OverviewWe are seeking a Senior Solutions Engineer to leverage advanced AI and machine learning methodologies in the development and deployment of innovative generative AI solutions utilizing HCLTech's Llama and other cutting-edge LLMs.Key ResponsibilitiesGain a comprehensive understanding of HCLTech's AI and Llama frameworks, along with their...

  • Artificial Intelligence Engineer

    2 weeks ago

    Menlo Park, California, United States AI Technologies LLC. Full time

    Position Overview:Field of Expertise: Machine LearningPosition: Machine Learning EngineerContract Length: 8 MonthsEmployer: To Be Discussed LaterWe are seeking a proficient Machine Learning Engineer to contribute to our innovative team at AI Technologies LLC. This role encompasses engagement with advanced technologies in the realm of machine learning systems...

  • Artificial Intelligence Engineer

    2 weeks ago

    Menlo Park, California, United States AI Technologies LLC. Full time

    Position Overview:Field of Expertise: Machine LearningRole: Machine Learning EngineerContract Length: 8 MonthsEmployer: To Be Discussed LaterWe are seeking a proficient Machine Learning Engineer to contribute to our innovative team at AI Technologies LLC. This role entails engaging with advanced technologies in the realm of machine learning systems and...

  • Generative AI Engineering Project Manager

    5 days ago

    Menlo Park, California, United States January, Inc. Full time

    About January, Inc.January, Inc. is a pioneering company in the field of precision health, dedicated to empowering individuals to take control of their metabolic wellbeing. Our innovative approach combines continuous glucose monitoring with heart rate, sleep, activity, and food tracking to provide real-time insights into how the body metabolizes food.Job...

  • Machine Learning Engineer, AI Innovator

    3 days ago

    Menlo Park, California, United States Robinhood Full time

    About the RoleWe are seeking a highly skilled Machine Learning Engineer to join our team at Robinhood, a leading fintech company dedicated to democratizing finance for all. As a key member of our Gen AI team, you will play a crucial role in accelerating the development and expansion of products powered by Gen AI.Key ResponsibilitiesLLM Development and...

  • Staff Data Engineer

    2 days ago

    Menlo Park, California, United States Character Full time

    About UsAt Character, our mission is to empower individuals with Artificial General Intelligence (AGI). Our vision is to enable people to utilize our technology at any moment, every day.We are a full-stack AI company with a globally scaled direct-to-consumer platform. As a leading personal AI platform, we are uniquely centered around people, allowing users...

  • Research Scientist, Systems ML

    2 months ago

    Menlo Park, California, United States Modern Mechanical Systems, Inc. Full time

    Meta is seeking a software engineer to join our AI & Systems Co–Design team to drive the definition of our next–generation compute and storage architectures. This person will work cross–functionally with internal software and platforms engineering teams to understand the workloads and infrastructure requirements. They will drive technology...

  • Generative AI Solutions Project Lead

    2 weeks ago

    Menlo Park, California, United States Debtsy Full time

    Debtsy:Debtsy is at the forefront of innovation, dedicated to transforming financial technology through advanced artificial intelligence solutions. Our mission is to empower individuals and businesses with intelligent tools that enhance decision-making and optimize performance.We are currently seeking a Generative AI Engineering Project Manager to lead and...

  • Senior Project Manager

    5 days ago

    Menlo Park, California, United States January, Inc. Full time

    About January, Inc.January, Inc. is a pioneering company in the field of precision health, dedicated to empowering individuals to take control of their metabolic wellbeing. Our innovative approach combines continuous glucose monitoring with heart rate, sleep, activity, and food tracking to provide real-time insights into how the body metabolizes food.Job...

  • Generative AI Solutions Project Lead

    2 weeks ago

    Menlo Park, California, United States Debtsy Full time

    Debtsy:Debtsy is at the forefront of technological advancement, focusing on innovative solutions that harness the power of generative AI. Our mission is to revolutionize how businesses operate by integrating cutting-edge AI technologies into their workflows. We are currently seeking a Generative AI Engineering Project Manager to lead and manage the...

  • Meta's Fleet AI Systems Production Systems Engineer

    5 days ago

    Menlo Park, California, United States META Full time

    About the RoleWe are seeking a highly skilled Production Systems Engineer to join our Release to Production (RTP) team at Meta. As a key member of our team, you will play a critical role in ensuring the efficient operation of our rapidly scaling infrastructure, which is the foundation upon which our innovative services are delivered.Key...

  • Generative AI Solutions Project Lead

    2 weeks ago

    Menlo Park, California, United States Debtsy Full time

    Debtsy: Debtsy is at the forefront of revolutionizing financial solutions through innovative technology. Our mission is to empower individuals and businesses with cutting-edge tools that enhance financial decision-making and improve overall financial health. We are currently seeking a Generative AI Engineering Project Manager to lead and orchestrate the...

  • Product Designer

    8 hours ago

    Menlo Park, California, United States Character Technologies Full time

    About the RoleCharacter Technologies is revolutionizing the future of open-ended interactions between humans and intelligent agents. We're seeking an experienced and talented Product Designer to join our mission and help shape the future of Consumer AI.Key ResponsibilitiesDevelop high-fidelity user interfaces and user experience designs across desktop and...

  • Product Designer

    3 days ago

    Menlo Park, California, United States Character Technologies Full time

    About the RoleCharacter Technologies is revolutionizing the future of open-ended interactions between humans and intelligent agents. We're seeking an experienced and talented Product Designer to join our mission and help shape the future of Consumer AI.Key ResponsibilitiesDevelop high-fidelity user interfaces and user experience designs across desktop and...

  • AI Systems Engineer

    2 weeks ago

    Menlo Park, California, United States Ethereum Technologies LLC Full time

    Position OverviewJob ID: J36993Specialization: Machine LearningJob Title: Machine Learning EngineerContract Duration: 8 MonthsEmployment Type: W-2 (Consultant must be on our company payroll. C2C is not permitted)Key Responsibilities:Demonstrated experience in programming for multicore CPUs and vector cores is preferred.Fundamental knowledge of machine...