Senior Software Engineer, Data Acquisition

2 weeks ago

San Francisco, United States OpenAI Full time

Senior Software Engineer, Data Acquisition

Overview:

The Data Acquisition team within the Foundations organization at OpenAI is responsible for all aspects of data collection to support our model training operations. Our team manages web crawling and GPTBot services and works closely with Data Processing, Architecture, and Scaling teams. We are looking for a skilled Senior Software Engineer to join our Data Acquisition team.

Responsibilities:

  • Own and lead engineering projects in the area of data acquisition including web crawling, data ingestion, and search.

  • Collaborate with other sub-teams, such as Data Processing, Architecture, and Scaling, to ensure smooth data flow and system operability.

  • Work closely with the legal team to handle any compliance or data privacy-related matters.

  • Develop and deploy highly scalable distributed systems capable of handling petabytes of data.

  • Architect and implement algorithms for data indexing and search capabilities.

  • Build and maintain backend services for data storage, including work with key-value databases and synchronization.

  • Deploy solutions in a Kubernetes Infrastructure-as-Code environment and perform routine system checks.

  • Conduct and analyze experiments on data to provide insights into system performance.

Qualifications:

  • BS/MS/PhD in Computer Science or a related field.

  • 6+ years of industry experience in software development.

  • Experience with large web crawlers a plus

  • Strong expertise in large stateful distributed systems and data processing.

  • Proficiency in Kubernetes, and Infrastructure-as-Code concepts.

  • Willingness and enthusiasm for trying new approaches and technologies.

  • Ability to handle multiple tasks and adapt to changing priorities.

  • Strong communication skills, both written and verbal.

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.

OpenAI Affirmative Action and Equal Employment Opportunity Policy Statement

For US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records.

We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via thislink.

OpenAI Global Applicant Privacy Policy

At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

Compensation Range: $310K - $385K

Apply for this Job

#J-18808-Ljbffr

  • Senior Software Engineer, Data Acquisition

    2 weeks ago

    San Francisco, United States OpenAI Full time

    Senior Software Engineer, Data AcquisitionOverview:The Data Acquisition team within the Foundations organization at OpenAI is responsible for all aspects of data collection to support our model training operations. Our team manages web crawling and GPTBot services and works closely with Data Processing, Architecture, and Scaling teams. We are looking for a...

  • Senior Software Engineer, Data Acquisition

    4 weeks ago

    San Francisco, United States OpenAI Full time

    Senior Software Engineer, Data Acquisition Overview: The Data Acquisition team within the Foundations organization at OpenAI is responsible for all aspects of data collection to support our model training operations. Our team manages web crawling and GPTBot services and works closely with Data Processing, Architecture, and Scaling teams. We are looking for a...

  • Software Engineer, Data Acquisition

    2 weeks ago

    San Francisco, United States OpenAI Full time

    Overview: The Data Acquisition team within the Foundations organization at OpenAI is responsible for all aspects of data collection to support our model training operations. Our team manages web crawling and GPTBot services and works closely with Data Processing, Architecture, and Scaling teams. We are looking for a skilled Software Engineer to join our Data...

  • Software Engineer, Data Acquisition

    2 weeks ago

    San Francisco, United States OpenAI Full time

    The Data Acquisition team within the Foundations organization at OpenAI is responsible for all aspects of data collection to support our model training operations. Our team manages web crawling and GPTBot services and works closely with Data Processing, Architecture, and Scaling teams. We are looking for a skilled Software Engineer to join our Data...

  • Software Engineer, Data Acquisition

    4 weeks ago

    San Francisco, United States OpenAI Full time

    Overview: The Data Acquisition team within the Foundations organization at OpenAI is responsible for all aspects of data collection to support our model training operations. Our team manages web crawling and GPTBot services and works closely with Data Processing, Architecture, and Scaling teams. We are looking for a skilled Software Engineer to join our Data...

  • Software Engineer, Data Acquisition

    1 week ago

    San Francisco, United States OpenAI Full time

    Overview:The Data Acquisition team within the Foundations organization at OpenAI is responsible for all aspects of data collection to support our model training operations. Our team manages web crawling and GPTBot services and works closely with Data Processing, Architecture, and Scaling teams. We are looking for a skilled Software Engineer to join our Data...

  • Software Engineer, Data Acquisition

    4 days ago

    San Francisco, United States OpenAI Full time

    Overview:The Data Acquisition team within the Foundations organization at OpenAI is responsible for all aspects of data collection to support our model training operations. Our team manages web crawling and GPTBot services and works closely with Data Processing, Architecture, and Scaling teams. We are looking for a skilled Software Engineer to join our Data...

  • Senior Software Engineer

    4 months ago

    San Francisco, United States Fathom Full time

    Fathom is on a mission to use AI to understand and structure the world's medical data, starting by making sense of the terabytes of clinician notes contained within the electronic health records of the world's largest health systems. Our deep learning engine automates the translation of patient records into the billing codes used for healthcare provider...

  • Senior Software Engineer

    4 weeks ago

    San Francisco, United States Discord Full time

    Discord is used by over 200 million people every month for many different reasons, but there's one thing that nearly everyone does on our platform: play video games. Over 90% of our users play games, spending a combined 1.5 billion hours playing thousands of unique titles on Discord each month. Discord plays a uniquely important role in the future of gaming....

  • Senior Software Engineer, Data Platform

    3 weeks ago

    San Francisco, United States Social Finance (SoFi) Full time

    Employee Applicant Privacy NoticeWho we are:Shape a brighter financial future with us. Together with our members, we're changing the way people think about and interact with personal finance. We're a next-generation financial services company and national bank using innovative, mobile-first technology to help our millions of members reach their goals. The...

  • Senior Software Engineer

    3 weeks ago

    San Francisco, United States Discord Full time

    This position is US based only.Discord is about giving people the power to create space to find belonging in their lives. We want to make it easier for you to talk regularly with the people you care about. We want you to build genuine relationships with your friends and communities close to home or around the world. Original, reliable, playful, and...

  • Senior Software Engineer, Data Platform

    4 months ago

    San Francisco, United States Social Finance Ltd Full time

    Employee Applicant Privacy Notice Who we are: Shape a brighter financial future with us. Together with our members, we're changing the way people think about and interact with personal finance. We're a next-generation financial services company and national bank using innovative, mobile-first technology to help our millions of members reach their goals. The...

  • Senior Data Engineer

    2 weeks ago

    san francisco, United States Jacobs Full time

    LocationSan Francisco, California, United StatesCapabilitiesDigital Design and Data AnalyticsOffice SetupHybrid, RemoteJob ID#19216 Market Digital and Data At Jacobs, we're challenging today to reinvent tomorrow by solving the world's most critical problems for thriving cities, resilient environments, mission-critical outcomes, operational advancement,...

  • Senior Software Engineer

    3 weeks ago

    San Francisco, United States Acceler8 Talent Full time

    Senior Software Engineer (AI Infrastructure / MLOps)Location: San Francisco (3 days per week in office)Introduction:We are seeking a Senior Software Engineer (AI Infrastructure / MLOps) to join a pioneering AI startup focused on enhancing data quality for machine learning. This role offers the chance to work on large-scale web applications and tackle complex...

  • Senior Software Engineer

    1 month ago

    san francisco, United States Acceler8 Talent Full time

    Senior Software Engineer (AI Infrastructure / MLOps)Location: San Francisco (3 days per week in office)Introduction:We are seeking a Senior Software Engineer (AI Infrastructure / MLOps) to join a pioneering AI startup focused on enhancing data quality for machine learning. This role offers the chance to work on large-scale web applications and tackle complex...

  • Senior Software Engineer

    4 months ago

    San Francisco, United States Acceler8 Talent Full time

    Senior Software Engineer (AI Infrastructure / MLOps)Location: San Francisco (3 days per week in office)Introduction:We are seeking a Senior Software Engineer (AI Infrastructure / MLOps) to join a pioneering AI startup focused on enhancing data quality for machine learning. This role offers the chance to work on large-scale web applications and tackle complex...

  • Senior Software Engineer

    2 months ago

    san francisco, United States Acceler8 Talent Full time

    Senior Software Engineer (AI Infrastructure / MLOps)Location: San Francisco (3 days per week in office)Introduction:We are seeking a Senior Software Engineer (AI Infrastructure / MLOps) to join a pioneering AI startup focused on enhancing data quality for machine learning. This role offers the chance to work on large-scale web applications and tackle complex...

  • Senior Software Engineer, Data Infrastructure

    1 week ago

    San Francisco, United States Tempus Ex Full time

    Senior Software Engineer, Data InfrastructureUnited States (Remote)About UsInfinite Athlete’s mission is to build an operating system for sports that powers infinite innovation and makes sports better for the fan, the game, and the athlete. Our goal is to create a single technological foundation across all major sports upon which innovative sports...

  • Senior Software Engineer

    2 months ago

    San Francisco, United States Jobot Full time

    Job DescriptionJob DescriptionB2C Series-A startup is looking for a Senior Software Engineer to join their growing team in San Fran!This Jobot Job is hosted by: Sydney WeaverAre you a fit? Easy Apply now by clicking the "Apply Now" buttonand sending us your resume.Salary: $150,000 - $200,000 per yearA bit about us:A well-funded Series-A startup, backed by...

  • Staff/Principal Software Engineer, Data

    1 week ago

    San Francisco, United States Tbwa ChiatDay Inc Full time

    World is a network of real humans, built on privacy-preserving proof-of-human technology, and powered by a globally inclusive financial network that enables the free flow of digital assets for all. It is built to connect, empower, and be owned by everyone.This opportunity would be with Tools for Humanity.About the AI & Biometrics Team:The AI & Biometrics...