Current jobs related to Large Scale Distributed System Architect - San Francisco, California - Openai


  • San Francisco, California, United States Recruiting from Scratch Full time

    Distributed Systems ArchitectWe are looking for a Distributed Systems Architect to design and implement large-scale, fault-tolerant systems for AI infrastructure.This role involves architecting and implementing distributed systems for our inference network, developing resource allocation models across heterogeneous hardware, and optimizing network...


  • San Francisco, California, United States Naptha AI Full time

    We are seeking an exceptional Infrastructure Leader to join our team at Naptha AI. As a key member of our infrastructure team, you will be responsible for designing and implementing the systems that will power the next generation of AI agent networks. This includes architecting systems for efficient agent communication and coordination, building robust,...


  • San Jose, California, United States Hireio, Inc. Full time

    Job OverviewHireio, Inc. is a cutting-edge technology company that empowers businesses to make data-driven decisions.We are seeking an experienced Tech Lead Software Engineer for Data Platform to join our Experimentation and Evaluation team.Estimated Salary: $150,000 - $200,000 per yearJob DescriptionAs a software engineer in this role, you will have the...


  • San Jose, California, United States Mindlance Full time

    Distributed Systems Architect PositionWe are seeking an experienced Distributed Systems Architect to join our team. As a Distributed Systems Architect, you will be responsible for designing and building large-scale distributed systems. You will work closely with our engineering team to ensure that our systems are secure, scalable, and highly available.Key...


  • San Jose, California, United States Tik Tok Full time

    Job DescriptionSMB Engineering Team OverviewThe SMB engineering team is responsible for developing software solutions that empower small and medium businesses to grow and succeed. We're looking for talented engineers who share our passion for innovation and collaboration.About the RoleThis role involves designing and implementing large-scale distributed...


  • San Jose, California, United States TikTok Full time

    Transforming Data into Actionable InsightsWe are seeking a highly skilled Data Architect to join our team in crafting and implementing a storage solution for offline data in TikTok's recommendation system, catering to over a billion users. The ideal candidate will possess a deep understanding of designing and implementing large-scale data architectures, with...


  • San Francisco, California, United States Tbwa ChiatDay Inc Full time

    Role OverviewWe are seeking a highly skilled Distributed ML Systems Engineer to join our team. This individual will be responsible for designing and building large-scale machine learning systems that power our AI initiatives.This role involves developing fault-tolerant distributed systems that handle high-load and high-performance requirements. The ideal...

  • Network Architect

    4 weeks ago


    San Francisco, California, United States Broadcom Corporation Full time

    About the Role:We are seeking a Network Architect - Distributed Systems professional to join our team at Broadcom Corporation. As a key member of our VCF Division, you will have the opportunity to work on bleeding-edge network virtualization technologies, network overlays, and layer-2 switching. The VCF networking management stack is responsible for...


  • San Francisco, California, United States Scale AI, Inc. Full time

    Job DescriptionOur MissionAt Scale AI, Inc., we are committed to pushing the boundaries of AI research and development. We are seeking a highly experienced Director of Agents Research to join our team and lead our research initiatives in developing large-scale model training systems.Key Areas of FocusAgent architecture development and scaffoldingData...


  • San Francisco, California, United States OpenAI Full time

    About the RoleWe are looking for a talented Software Engineer to join our Data Acquisition team at OpenAI. The ideal candidate will have 4+ years of industry experience in software development, with a strong background in large stateful distributed systems and data processing.The successful candidate will have expertise in Kubernetes, Infrastructure-as-Code...


  • San Mateo, California, United States Snowflake Computing Full time

    About Us:Snowflake Computing is a leading provider of cloud-based data warehousing and analytics solutions. Our team is dedicated to scaling our globally distributed Snowflake infrastructure, driving key initiatives and innovations in real-time streaming ingestion, data replication, disaster recovery, and more.We're passionate about our people, customers,...


  • San Francisco, California, United States Succinct Full time

    Innovate with Succinct, a pioneer in blockchain scaling, interoperability, and privacy solutions. As a Senior Software Engineer, you'll play a crucial role in developing our distributed proving cluster for SP1 and prover network in our San Francisco office.About the RoleThis position requires expertise in architecting and maintaining a highly available...


  • San Jose, California, United States Tik Tok Full time

    About the RoleWe are looking for a highly skilled Senior Backend Software Engineer to join our User Growth Team. As a key member of the team, you will be responsible for designing and developing large-scale software systems that power TikTok's apps, leveraging data to inform product decisions and drive business outcomes.Your Key ResponsibilitiesDesign and...


  • San Francisco, California, United States Figma Full time

    Required Skills and QualificationsTo succeed in this role, you'll need to have strong technical skills and experience working with large-scale distributed systems. Specifically, you should have:* 6+ years of experience building and scaling distributed systems and online services* Strong coding skills with proficiency in programming languages such as Go,...


  • San Jose, California, United States Tik Tok Full time

    **Job Description**We are seeking a Fullstack DevOps Engineer to join our team at TikTok. As a key member of our recommendation engineering team, you will play a crucial role in building the next-generation TT4B recommendation system. This system will provide customized recommendations to advertisers across various platforms, including Ads Manager, Business...


  • San Francisco, California, United States Databricks Full time

    Your Skills: A Bachelor's or Master's degree in Computer Science or a related field.10+ years of production-level experience in Java, Scala, C++, or similar languages.Proficiency in architecting, developing, deploying, and operating large-scale distributed systems.Familiarity with cloud technologies like AWS, Azure, GCP, Docker, and Kubernetes.Experience...


  • San Jose, California, United States HireIO Inc Full time

    Your OpportunityHireIO Inc is seeking a seasoned Machine Learning Engineer Lead to drive the innovation and growth of our digital advertising products. As a key member of our team, you will lead the design and implementation of large-scale ad systems that power millions of transactions daily. We are looking for a talented individual who has a strong...


  • San Francisco, California, United States Genmo Full time

    Role OverviewWe are seeking an experienced Senior/Staff AI Infra Engineer to design, build, and scale our petabyte-scale data infrastructure.Key ResponsibilitiesDesign and Implementation: Create highly scalable data infrastructure and systems to process petabyte-scale data stores.Distributed Processing Jobs: Manage large-scale distributed processing jobs for...


  • San Francisco, California, United States AI Talent Flow Full time

    Job OverviewA high-performance distributed systems architect is required to join the team at AI Talent Flow, responsible for building scalable and reliable data platforms that power our cloud service. As a key member of our engineering team, you will work closely with cross-functional teams to design, develop, and operate our open-source Chroma data plane.In...


  • San Francisco, California, United States AI Talent Flow Full time

    Job OverviewWe are seeking a highly skilled Distributed Systems Architect to join our team at AI Talent Flow. As a key member of our engineering team, you will be responsible for designing and developing scalable and reliable distributed systems that support AI applications at scale.The successful candidate will have experience in building correct,...

Large Scale Distributed System Architect

3 weeks ago


San Francisco, California, United States Openai Full time
Senior Software Engineer Position Overview
The Data Acquisition team within OpenAI's Foundations organization is responsible for all aspects of data collection to support model training operations. This team manages web crawling and GPTBot services and works closely with Data Processing, Architecture, and Scaling teams. We are seeking a skilled Senior Software Engineer to join our Data Acquisition team.

About the Position
As a Senior Software Engineer, Data Acquisition, you will own and lead engineering projects in the area of data acquisition, including web crawling, data ingestion, and search. You will collaborate with other sub-teams to ensure smooth data flow and system operability. The role requires developing and deploying highly scalable distributed systems capable of handling petabytes of data.

Responsibilities
Develop and deploy highly scalable distributed systems capable of handling petabytes of data Collaborate with other sub-teams to ensure smooth data flow and system operability Work closely with the legal team to handle any compliance or data privacy-related matters Architect and implement algorithms for data indexing and search capabilities Build and maintain backend services for data storage, including work with key-value databases and synchronization

Qualifications
A BS/MS/PhD in Computer Science or a related field is required, along with 6+ years of industry experience in software development. Experience with large web crawlers is a plus. Strong expertise in large stateful distributed systems and data processing is essential. Proficiency in Kubernetes and Infrastructure-as-Code concepts is also required. A willingness to try new approaches and technologies, as well as strong communication skills, are highly valued.

Estimated Salary: $200,000 - $280,000 per year.