Current jobs related to Software Engineer, LLM Infrastructure - Cupertino - ETCHED LLC

  • NLP Solutions Software Engineer

    3 weeks ago

    Cupertino, California, United States Apple Full time

    Job SummaryAs a highly skilled NLP Solutions Software Engineer at Apple, you will play a crucial role in building AI-driven solutions that solve pressing business challenges. Your primary responsibilities will include developing LLM components for use in generative AI applications, collaborating with internal design teams and the AIML organization to...

  • NLP Solutions Software Engineer

    4 days ago

    Cupertino, California, United States Apple Full time

    Role OverviewAs a member of Apple's Silicon Technologies group, you will play a key role in building AI-driven solutions that address pressing business challenges. Your primary responsibilities will include developing LLM components for use in generative AI applications, collaborating with internal design teams and the AIML organization to understand...

  • NLP Solutions Software Engineer

    4 weeks ago

    Cupertino, California, United States Apple Full time

    Role SummaryWe are seeking an experienced NLP Solutions Software Engineer to join our Silicon Technologies group at Apple. As a key member of our team, you will play a critical role in building AI-driven solutions that solve pressing business challenges.Key ResponsibilitiesDevelop LLM components for use in generative AI applications.Collaborate with internal...

  • Software Engineer

    4 weeks ago

    Cupertino, California, United States Apple Full time

    Job Title: Software Engineer - Cloud InfrastructureJoin Apple's iCloud Efficiency team as a Software Engineer - Cloud Infrastructure and contribute to making our software and services more efficient and sustainable.About the RoleWe are seeking an experienced Software Engineer to help us drive key initiatives to improve Apple Services through Service...

  • Software Development Engineer

    2 days ago

    Cupertino, California, United States Apple Full time

    Job DescriptionAt Apple, we're looking for a talented Software Development Engineer to join our Apple Services Engineering (ASE) team. As a key member of our iCloud Services SRE team, you'll be responsible for designing and running systems and infrastructure that will delight millions of customers.Key Responsibilities:Support and maintain ML services by...

  • Software Development Engineer, Annapurna Labs

    23 hours ago

    Cupertino, California, United States Amazon Full time

    About the RoleWe are seeking an experienced software engineer with expertise in low-latency networking to optimize customer experience by designing systems that enable scaling network-intensive workloads over thousands of CPUs, GPUs, and TPUs. This role is on the forefront of AI/ML, where we spend a good deal of the day optimizing the networking for the...

  • Senior Software Engineer

    4 weeks ago

    Cupertino, California, United States Apple Full time

    Job Title: Senior Software Engineer - AIML ObservabilityWe are seeking a highly skilled Senior Software Engineer to join our AIML Observability team at Apple. As a key member of our team, you will design and build cloud-native solutions for Siri, Search, and other AIML products.About the RoleThis is an exciting opportunity to work on large-scale cloud-native...

  • Software Engineering Manager

    4 days ago

    Cupertino, California, United States Apple Full time

    Job DescriptionAs the Software Engineering Manager for Apple's Software Localization team, you will lead a diverse team of Applied ML engineers in designing, implementing, and qualifying ML localization features, processes, and tooling across various Apple products.The team's primary focus is on exploring and applying Large Language Models (LLMs) for...

  • Senior Software Engineer

    4 weeks ago

    Cupertino, California, United States Apple Full time

    Job Title: Senior Software Engineer - Security and InfrastructureJoin Apple's Data Platform team as a Senior Software Engineer - Security and Infrastructure. We're looking for a talented engineer to help us build a secure and reliable data platform that powers analytics, experimentation, and ML feature engineering for Siri, Search, and other ML...

  • Software Development Engineer, Core Network Infrastructure

    4 days ago

    Cupertino, California, United States Amazon Full time

    About the RoleWe are seeking an experienced Software Development Engineer to join our Core Network Infrastructure team at Amazon Web Services (AWS). As a key member of our team, you will be responsible for designing, implementing, and deploying new features to improve the scale and availability of our network infrastructure.You will work closely with our...

  • Machine Learning Engineer

    4 weeks ago

    Cupertino, California, United States Apple Full time

    Job SummaryWe are seeking a highly skilled Machine Learning Engineer to join our LLM Optimization team at Apple. As a key member of our AI and Machine Learning organization, you will have the opportunity to work on groundbreaking technology for large-scale ML systems, computer vision, natural language processing, and multi-modal understanding.Key...

  • Data Infrastructure Engineer

    4 weeks ago

    Cupertino, California, United States Apple Full time

    Job Title: Sr Software Data Infrastructure EngineerWe are seeking a highly skilled Sr Software Data Infrastructure Engineer to join our team at Apple. As a key member of our Data and ML Innovation organization, you will play a critical role in driving product impact through measurement and evaluation.Key Responsibilities:Design and implement a unified and...

  • Senior Software Development Engineer

    4 weeks ago

    Cupertino, California, United States Apple Full time

    Job SummaryAs a Staff Engineer on the Data Solution Platform team at Apple, you will play a key role in accelerating the adoption of the Apple Data Platform by developing data solutions, including advanced data insights, unified search powered by knowledge bases, and the seamless integration of the latest AI technologies to enhance workflows for large data...

  • Senior Software Engineer

    4 weeks ago

    Cupertino, California, United States Apple Full time

    About the RoleWe are seeking a highly skilled Senior Software Engineer to join our Apple Services Engineering (ASE) team. As a key member of our team, you will play a critical role in designing, developing, and deploying high-performance systems that handle millions of queries every day.As a senior engineer on our team, you will advance our data...

  • Senior Software Development Engineer

    2 weeks ago

    Cupertino, California, United States Apple Full time

    Job SummaryAs a Senior Software Development Engineer on the Data Solution Platform team at Apple, you will play a key role in accelerating the adoption of the Apple Data Platform by developing data solutions, including advanced data insights, unified search powered by knowledge bases, and the seamless integration of the latest AI technologies to enhance...

  • Senior Software Engineer

    4 weeks ago

    Cupertino, California, United States Apple Full time

    Job SummaryWe are seeking a highly skilled Senior Software Engineer - Data Platform to join our team at Apple. As a key member of our Data Platform organization, you will be responsible for designing, building, and operating large-scale data processing systems in the public cloud.Our team is responsible for enabling analytics, experimentation, and ML feature...

  • Senior Software Engineer

    4 weeks ago

    Cupertino, California, United States Apple Full time

    Job SummaryWe are seeking a highly skilled Senior Software Engineer to join our Apple Maps Data Infrastructure team. As a key contributor, you will be responsible for building capabilities across a spectrum of technologies in a hybrid-cloud environment.Key ResponsibilitiesDesign and develop innovative solutions for large-scale data processing and machine...

  • Senior Software Development Engineer, Machine Learning Infrastructure Team

    4 days ago

    Cupertino, California, United States Amazon Full time

    About the RoleWe are seeking a highly skilled Senior Software Development Engineer to join our Machine Learning (ML) Infrastructure team. As a key member of this team, you will be responsible for designing and developing the tools and infrastructure that support the success of our ML and High Performance Computing (HPC) technologies.As a Senior Software...

  • Senior Cloud Infrastructure Engineer

    4 days ago

    Cupertino, California, United States Amazon Full time

    About the RoleWe are seeking an experienced Senior Cloud Infrastructure Engineer to join our Core Networking Team at Amazon Web Services (AWS). As a key member of our team, you will be responsible for designing, implementing, and operating large-scale cloud infrastructure to support our customers' needs.As a Senior Cloud Infrastructure Engineer, you will...

  • Automation Engineer

    3 weeks ago

    Cupertino, California, United States Intelliswift Software Full time

    Job Title: Quality Engineer III - AutomationJob Summary:We're seeking a highly skilled Quality Engineer to join our team at Intelliswift Software. As a Quality Engineer III - Automation, you will be responsible for developing and executing automated tests, building and maintaining the testing infrastructure to ensure our shipping features continue to work as...

Software Engineer, LLM Infrastructure

3 months ago

Cupertino, United States ETCHED LLC Full time
About Etched

Etched is building AI chips that are hard-coded for individual model architectures. Our first product (Sohu) only supports transformers, but has an order of magnitude more throughput and lower latency than a B200. With Etched ASICs, you can build products that would be impossible with GPUs, like real-time video generation models and extremely deep chain-of-thought reasoning.

Software Engineer, LLM Infrastructure

Transformer ASICs, like those built by Etched, dramatically improve time-to-first-token latency. For a large model like Llama-3-70B with 2048 input tokens, the TTFT will be single-digit milliseconds (we will announce performance figures publicly at our launch).

However, single-digit millisecond latency means nothing if the rest of the serving stack takes 100+ ms, and customers actually use it (or adopt the optimizations into their own stack). You will help make both of these happen.

You will work with our software team to build software for continuous batching, and write world-class interactive documentation (like Pytorch's Run in Colab feature) to show customers how it works. You will get this software working on our pre-silicon platform, and port it over to work on the physical chips once they are done being fabbed. You will find creative, new ways to improve this latency - can we speculatively decode the user's inputs? Can we pre-empt sequences if we run out of KV cache space and re-compute them later? Can we cache common pre-fills?

Representative projects:
  • Working with emulators like Palladium to develop software for chips while they are being fabricated
  • Developing algorithms for balancing prefill and completion tokens when serving LLMs
  • Profiling network latency when responding to prompts to help eliminate it in our test environment
  • Develop ways for customers to work with our pre-silicon infrastructure and understand how their workloads will run on it.
  • Build tools for Jupyter notebooks to connect to emulated and physical Etched systems
You may be a good fit if you:
  • Have 3+ years of software engineering experience
  • Are good at math, and good at communicating mathematical ideas
  • Pick up slack, even if it goes outside your job description
  • Are results-oriented, and bias towards shipping products
  • Want to learn more about machine learning research
We encourage you to apply even if you do not believe you meet every single qualification.

Strong candidates may also have experience with:
  • Palladium emulation
  • Real-time audio and video communication
  • GPU kernel profiling and low-level programming
  • Transformer optimizations, such as FlashAttention
  • Palladium emulation
How we're different:

Etched believes in the Bitter Lesson. We think most of the progress in the AI field has come from using more FLOPs to train and run models, and the best way to get more FLOPs is to build model-specific hardware. Larger and larger training runs encourage companies to consolidate around fewer model architectures, which creates a market for single-model ASICs.

We are a fully in-person team in Cupertino, and greatly value engineering skills. We do not have boundaries between engineering and research, and we expect all of our technical staff to contribute to both as needed.

Benefits:
  • Full medical, dental, and vision packages, with 100% of premium covered, 90% for dependents
  • Housing subsidy of $2,000/month for those living within walking distance of the office
  • Daily lunch and dinner in our office
  • Relocation support for those moving to Cupertino