Data Infrastructure Engineer

4 weeks ago


San Francisco, United States TWILIO Full time

Responsibilities

In this role, you’ll:

Dive into our dataset and design, implement, and scale data pre/post-processing pipelines. Work on applied ML solutions in the areas of query processing, data mining, cleaning, normalizing and modeling. Design and build data platforms & frameworks for processing high volumes of data, in real-time as well as batch, that will be used across engineering teams. Build data processing streams for cleaning and modeling text data for LLMs Shape the decision-making process for data architecture to effectively deliver insights to the business. Research and evaluate new technologies in the big data space to guide our continuous improvement  Collaborate with multi-functional teams to help tune the performance of large data applications Partner with Data Engineering, IT, and business units to ensure the creation of high-quality data solutions that are consistent, clean, and integrated. Conduct data exploration and profiling using analysis, design, and presentation tools. Identify and assess business impacts resulting from strategic initiatives. Deliver analytics solutions to support key business decisions and enhance operational efficiency.

Qualifications 

Not all applicants will have skills that match a job description exactly. Twilio values diverse experiences in other industries, and we encourage everyone who meets the required qualifications to apply. While having “desired” qualifications make for a strong candidate, we encourage applicants with alternative experiences to also apply. If your career is just starting or hasn't followed a traditional path, don't let that stop you from considering Twilio. We are always looking for people who will bring something new to the table

Required:

Experience using big data technologies (Airflow, Spark, pySpark, Presto, Athena) AWS experience Demonstrated prior experience in creating data pipelines for text data sets NLP/ large language models Experience building and maintaining ETL (managing high-quality reliable ETL pipelines) Previous experience in building data pipelines for conversational AI APIs

Location 

This role will be remote, and based in the United States. This role will be remotebut is not eligible to be hired in San Francisco, CA, Oakland, CA, San Jose, CA, or the surrounding areas.

What We Offer

There are many benefits to working at Twilio, including, in addition to competitive pay, things like generous time-off, ample parental and wellness leave, healthcare, a retirement savings program, and much more. Offerings vary by location.



  • San Francisco, United States Acceler8 Talent Full time

    Data Infrastructure EngineerAre you an experienced Data Infrastructure Engineer ready to tackle challenges in AI and data processing at an unprecedented scale? Our organization is looking for a skilled professional to join our dynamic team. Here, technology and innovation intersect to create smarter solutions that redefine human-computer interactions.About...


  • San Francisco, United States Acceler8 Talent Full time

    Data Infrastructure EngineerAre you an experienced Data Infrastructure Engineer ready to tackle challenges in AI and data processing at an unprecedented scale? Our organization is looking for a skilled professional to join our dynamic team. Here, technology and innovation intersect to create smarter solutions that redefine human-computer interactions.About...


  • San Francisco, United States Acceler8 Talent Full time

    Data Infrastructure EngineerAre you an experienced Data Infrastructure Engineer ready to tackle challenges in AI and data processing at an unprecedented scale? Our organization is looking for a skilled professional to join our dynamic team. Here, technology and innovation intersect to create smarter solutions that redefine human-computer interactions.About...


  • San Francisco, California, United States Databricks Full time

    While candidates in the listed locations are encouraged for this role, we are open to remote candidates in other locations.As a Data Infrastructure Engineer on the security data infrastructure team you will help build Lakehouse for Security organization. You will build reliable, large-scale, multi-geo data pipelines to support detecting threats(internal and...


  • San Francisco, CA, United States Adept AI Labs Full time

    Adept is working to advance a people-centric approach to AI that optimizes for what’s actually most useful for people and their work. You can see this approach in the technology we’re building: models that are trained to use software and take actions just as a person would.We’ve recently raised a $350M Series B led by General Catalyst and Spark, on top...


  • San Francisco, United States forhyre.com Full time

    Job Description Job Description Do you enjoy solving technical issues, empathize with customer user experiences and want to keep up with the latest tech? We are looking for a Cloud Infrastructure Engineer that will work with talented software engineering and support teams to deploy, maintain and ensure reliability of our applications in a fast paced...


  • San Francisco, CA, United States OpenAI Full time

    About the TeamYou’ll join the team that’s behind OpenAI’s data infrastructure that powers critical engineering, product, alignment teams that are core to the work we do at OpenAI. The Streaming Infrastructure team within Data Platform is responsible for building and maintaining our streaming platform. This platform plays a crucial role in facilitating...


  • San Francisco, California, United States Roadio (Formerly Streetlogic) Full time

    Streetlogic is a team of cyclists on a mission to make biking safer. Our product gives more people the confidence to bike - leading to more livable cities and a cleaner environment.We're building a light-weight, Advanced Driver Assistance System (ADAS) for ebikes, using a vision-first approach to detect incoming collisions and give riders early warning and...


  • San Francisco, United States VAST Data Full time

    VAST Data is looking for a Senior Systems Engineer to join our growing team! This is a great opportunity to be part of the fastest-growing infrastructure company in history. In just a few short years, we’ve shaken up the industry by challenging traditional architecture models and introduced a revolutionary set of storage possibilities through our...


  • San Francisco, California, United States Databricks Full time

    While candidates in the listed locations are encouraged for this role, we are open to remote candidates in other locations.At Databricks Information Technology, we are a product led organization transforming the way we work from how easy it is to use our IT services to the applications we develop that help us scale seamlessly in face of incredible growth.The...


  • San Francisco, California, United States Fathom Full time

    Fathom is on a mission to use AI to understand and structure the world's medical data, starting by making sense of the terabytes of clinician notes contained within the electronic health records of the world's largest health systems. Our deep learning engine automates the translation of patient records into the billing codes used for healthcare provider...


  • San Francisco, California, United States Fathom Full time

    Fathom is on a mission to use AI to understand and structure the world's medical data, starting by making sense of the terabytes of clinician notes contained within the electronic health records of the world's largest health systems. Our deep learning engine automates the translation of patient records into the billing codes used for healthcare provider...


  • San Francisco, United States JobRialto Full time

    Description: Demonstrated technical expertise across a variety of technology platforms and solutions Ability to work effectively independently or in a team environment Strong analytical, collaboration, leadership, critical thinking, multitasking, and time management skills Ability to influence successfully in a highly matrix or virtual organization...

  • Data Engineer

    2 weeks ago


    San Francisco, United States VRChat Inc Full time

    Join the VRChat team! VRChat offers a first-of-its kind, game-changing platform that provides an endless collection of social immersive experiences and gives the power of creation to its robust community. With over 250,000 worlds and growing, VRChat’s vision is to allow users to bring their imaginations to life and help shape the metaverse anywhere in the...


  • San Francisco, United States Coalesce Full time

    Job DescriptionJob DescriptionWe are seeking a seasoned Director of Infrastructure to lead our platform engineering team and drive the scaling of our infrastructure to support our aggressive growth targets. This role requires a strategic leader with hands-on technical skills, proven experience scaling infrastructure in high-growth environments, and a solid...


  • San Francisco, United States Hazel Health, Inc. Full time

    Hazel partners with schools and families to provide physical and mental virtual health care that helps students feel better and get back to learning. As telehealth becomes more and more relevant in the lives of children, Hazel is experiencing tremendous company growth. Our innovative response to our nation’s call for equitable, affordable, and safe virtual...

  • Sr Data Engineer

    5 days ago


    San Francisco, United States Varite Full time

    Pay rate range: $70-90/hr. True Job Title-Senior Infrastructure Automation Engineer Location-this is a hybrid position but we can consider remote candidates with exceptional technical expertise. This role will be subject to level 2 screening. In this role, you will get an opportunity to broadly apply your engineering skills across various technology...

  • Sr Data Engineer

    2 weeks ago


    San Francisco, United States Varite Full time

    Pay rate range: $80-90/hr. True Job Title-Senior Infrastructure Automation Engineer Location-this is a hybrid position but we can consider remote candidates with exceptional technical expertise. This role will be subject to level 2 screening. In this role, you will get an opportunity to broadly apply your engineering skills across various technology...


  • San Mateo, California, United States Observe Full time

    The Infrastructure team at Observe is responsible for developing, scaling, and maintaining development and production infrastructure. We're a small team with a wide domain, and we value collaboration, willingness to learn, and the ability to solve immedate problems quickly while building towards a long-term vision. If you think this sounds interesting, you...

  • Data Center Manager

    2 weeks ago


    San Francisco, United States Amazon Data Services, Inc. Full time

    Amazon Web Services (AWS) is growing rapidly, and we are looking for a Data Center Manager to join our expanding Infrastructure Operations teamAWS is a fast paced technology company and a leader in the world of data centers. The position will help ensure overall availability and reliability to meet or exceed defined service levels of data center operations....