Technical Lead, Large Scale Data Systems
2 days ago
We are seeking an experienced Senior Software Engineer to lead our Data Acquisition team. The ideal candidate will have a strong background in large-scale distributed systems and data processing.
The successful candidate will own and lead engineering projects in the area of data acquisition, including web crawling, data ingestion, and search. They will collaborate with other sub-teams, such as Data Processing, Architecture, and Scaling, to ensure smooth data flow and system operability.
Key responsibilities include:
- Owning and leading engineering projects in data acquisition
- Collaborating with other teams for smooth data flow and system operability
- Working with the legal team on compliance and data privacy matters
- Developing and deploying highly scalable distributed systems
- Architecting and implementing algorithms for data indexing and search
- Building and maintaining backend services for data storage
- Deploying solutions in a Kubernetes environment
Qualifications include:
- BS/MS/PhD in Computer Science or a related field
- 6+ years of industry experience in software development
- Experience with large web crawlers is a plus
- Strong expertise in large stateful distributed systems and data processing
- Proficiency in Kubernetes and Infrastructure-as-Code concepts
- Ability to handle multiple tasks and adapt to changing priorities
About OpenAI: We are an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. Our mission is to safely deploy AI systems through our products. We value diversity and do not discriminate based on race, religion, national origin, gender, sexual orientation, age, veteran status, disability, or any other protected status.
We estimate the salary for this position to be around $175,000 - $225,000 per year, depending on location and experience.
-
San Francisco, California, United States Early Warning® Full timeAbout the RoleWe are seeking a seasoned Cloud Engineering Lead to join our team at Early Warning Services, LLC.This is a senior technical individual contributor position responsible for leading and mentoring large-scale engineering projects across the organization. As a trusted expert in cloud engineering, you will drive best practices, architectures, and...
-
San Francisco, California, United States Databricks Full timeDatabricks is passionate about empowering data teams to tackle the world's most complex problems.We are looking for a Technical Lead to spearhead our infrastructure development efforts, driving strategic initiatives and shaping the future of our platform. As a key member of our Platform Engineering team, you will have a profound impact on the success of our...
-
San Francisco, California, United States OpenAI Full timeAbout OpenAIOpenAI is a pioneering AI research and deployment company dedicated to harnessing the power of artificial intelligence for the betterment of humanity. Our mission is to create safe and beneficial AI systems that elevate human capabilities.We are committed to fostering an inclusive and diverse work environment, where talented individuals from...
-
Large Scale Data Infrastructure Specialist
6 days ago
San Francisco, California, United States Anthropic Full timeAbout AnthropicWe are a leading AI research organization dedicated to developing safe, ethical, and powerful artificial intelligence. Our mission is to ensure that transformative AI systems are aligned with human interests.Our Pretraining team is seeking a highly skilled Research Engineer to join our efforts in developing the next generation of large...
-
San Francisco, California, United States ZipRecruiter Full timeJob DescriptionWe are seeking a highly skilled Staff Backend Software Engineer to join our Data E-Commerce department.In this role, you will have the opportunity to design and implement large-scale, distributed systems that solve complex business problems. You will be responsible for managing the end-to-end lifecycle of services, from design and development...
-
San Francisco, California, United States Databricks Full timeAt Databricks, we are passionate about helping data teams solve complex problems. Our Platform Engineering team provides the core infrastructure that powers our platform. We are rebuilding our platform from the ground up to meet growing needs.We are seeking a Senior Staff Software Engineer to join this team and spearhead our strategy, lead high-stakes...
-
Data Engineer for Large-Scale Data Platforms
6 days ago
San Francisco, California, United States Tbwa ChiatDay Inc Full timeCompany Overview\rTbwa Chiat/Day Inc is a leading community of communities, built on shared interests and trust. It's home to the most open and authentic conversations on the internet, with 100,000+ active communities and approximately 82M+ daily active unique visitors.\rWe're now looking for a talented individual to accelerate our data-centric culture and...
-
Data Engineer
6 days ago
San Francisco, California, United States Scale AI, Inc. Full timeScale AI, Inc.We're seeking an experienced Data Engineer to lead our AI research and development efforts in Generative AI. As a Machine Learning Research Engineer, you will design next-generation data pipelines and supervision strategies in collaboration with our customers to accelerate progress in machine learning research.You will:Collaborate closely with...
-
San Francisco, California, United States Pinterest Full timeAbout PinterestPinterest is a platform where millions of people around the world come to find new ideas every day. It's where they get inspiration, dream about new possibilities, and plan for what matters most. Our mission is to help those people find their inspiration and create a life they love.Role OverviewIn this role, you'll be challenged to take on...
-
San Francisco, California, United States Descript Full timeAbout Descript">Descript is a pioneering company that's revolutionizing the way people edit video and audio content. Our mission is to create a simple, intuitive, fully-powered editing tool for the age of AI.Role Overview">We're seeking an experienced Technical Lead to join our team. As a key member of our Client Platform org, you'll play a vital role in...
-
San Francisco, California, United States Scale AI, Inc. Full timeScale AI, Inc.We are seeking a highly skilled Machine Learning Engineer to join our team at Scale AI, Inc. as a key member of our research and development efforts in large language models. This is an exciting opportunity to work with the industry's leading foundation model labs, design next generation data pipelines, and contribute technically and...
-
San Francisco, California, United States Pinterest Full timeAbout Pinterest:Pinterest is a global platform where millions of people come to find inspiration, plan their lives and create a better world.Our mission is to help these individuals achieve their goals and live a life they love. As a Staff Software Engineer on our Core Retrieval Infrastructure team, you'll be part of shaping this mission by building...
-
IT Project Lead
7 days ago
San Francisco, California, United States Pager Full timePagerDuty is a global leader in digital operations management, revolutionizing how critical work gets done. We're seeking an experienced IT Project Lead to manage large-scale programs with a wide impact radius.This role involves identifying stakeholders, driving roadmaps and strategies, leading delivery, reporting status, and ensuring quality. You will work...
-
San Francisco, California, United States Pinterest Full timePrincipal Machine Learning Engineer, Trust EngineeringKey ResponsibilitiesDrive the technical strategy and direction of Pinterest's Trust Engineering org to ensure technology aligns with business requirements.Develop strong partnerships with product teams to understand and address future technology needs and current developer pain points.Champion large-scale...
-
San Francisco, California, United States Google Full timeAbout the JobWe're seeking a skilled Cloud Engineer to join our Site Reliability Engineering (SRE) team. As an SRE, you'll play a critical role in designing, analyzing, and troubleshooting large-scale distributed systems.Key ResponsibilitiesEngage in the entire lifecycle of services, from inception and design to deployment, operation, and refinement.Support...
-
Lead Data Science Architect
6 days ago
San Francisco, California, United States Windfall Data, Inc. Full timeWindfall Data, Inc. is seeking a seasoned data professional to play a pivotal role in developing and scaling our foundational predictive models.This is a highly impactful position where you will influence the core of our data product offerings and collaborate with cross-functional teams to maintain the highest standards of model performance.We're on a...
-
San Diego, California, United States BAE Systems USA Full timeCompany Overview">BAE Systems is a leading provider of advanced defense, security, and aerospace solutions. With a rich history and commitment to innovation, we are dedicated to protecting those who protect us.">Job Summary">We are seeking highly skilled software developers to join our team of experts in developing large-scale systems. As a key member of our...
-
San Mateo, California, United States Roblox Full timeAt Roblox, we're building a platform that empowers our global community of developers and creators to bring any experience to life. As a Senior Software Engineer on the Moderation Services Team, you'll be part of a team that's 10x-ing our scale, growing our platform capabilities, and reducing the time it takes for creators or teams to safeguard their...
-
San Francisco, California, United States FrontApp Inc. Full timeAt Front, we're building a platform that enables businesses to deliver exceptional customer service at scale. Our team is responsible for designing and developing the APIs that power dozens of partner integrations, thousands of custom workflows, and millions of API requests per day.About the RoleWe're looking for an experienced engineer to join our platform...
-
San Mateo, California, United States Sony Interactive Entertainment America Full timeLead Distributed Computing Engineer at Sony Interactive Entertainment AmericaWe are seeking a highly skilled and motivated Distributed Computing Engineering Manager to lead our team of engineers in building and maintaining large-scale, distributed systems. In this role, you will drive the development of robust, scalable solutions that power our core...