Engineer, Data Platform

2 weeks ago


San Francisco, United States PineCone Full time

About Pinecone

Pinecone is on a mission to build the search and database technology to power AI applications for the next decade and beyond. Our fully managed vector database makes it easy to add vector search to AI applications. Since creating the “vector database” category, demand has grown incredibly fast and it shows in our user base.

We are a distributed team with clusters in New York, San Francisco, Tel-Aviv, and Manchester.

About The Role

Pinecone is seeking a skilled and highly motivated Engineer for our internal Data Platform team to drive the development and maintenance of our data infrastructure, ensuring the efficient orchestration, governance, quality, and accessibility of data across the organization. As a Senior Engineer in the Data Team, you will play a critical role in building and optimizing our data ecosystem to enable data delivery, literacy, insights, and data science work at scale.

You will work in a fast-paced and rewarding environment that demands the highest quality work with minimal supervision. And as we all do a little bit of everything, you will also be a strong generalist, work directly with executive leadership, and mentor new data engineers and scientists.

Responsibilities

Design and Build Data Infrastructure

Architect and develop scalable and efficient data infrastructure, encompassing orchestration, metric store, feature store, governance, data quality, alerting infrastructure, and reverse ETL processes.

Enable Data Quality and Governance

Establish robust data quality framework/tooling and governance processes to maintain high data quality and integrity throughout the data lifecycle.

Collaborate with Data Science and Engineering Teams

Collaborate with data science teams to understand their requirements and ensure the availability and accessibility of data for modeling, experimentation, and analysis.

What We Look For

A passion for technology 5+ years of experience with SQL and Python 5+ years of experience with designing and developing high performance systems BS in Computer Science, Math, a related technical field or equivalent experience Strong foundations in databases, warehousing, data infrastructure, ELT/ETL Proficient in building and optimizing data infrastructure using modern technologies and frameworks (e.g., Kafka, Airflow, API Integrations, CI/CD, Terraform, etc.).

Bonus Points

Experience with orchestration platforms Experience with Data Governance infrastructure (RBAC, Data Quality, Alerting, etc) Expertise working with cloud-based data warehouse solutions (BigQuery, Snowflake) Knowledge and experience with code deployment and K8s resource management

#J-18808-Ljbffr



  • San Francisco, United States Motion Recruitment Full time

    Our large crypto company is looking for a contract Data Platform Engineer. This is a remote contract position. Contract Duration: 12-Months Required Skills & Experience 4+ years of relevant work experience. Fluent SQL skills for ETL/ELT, data quality, and reporting. Strong Python or Spark backend development skills. General experience working...


  • San Francisco, California, United States Everyday Agents Full time

    About WovenConsumer Travel is Ripe for DisruptionToday's travelers are seeking personalized, immersive, and authentic experiences that reflect their unique tastes and interests. They want to be inspired, to feel the thrill of discovery, and to seamlessly benefit from technology throughout their journeys. Yet, the current travel landscape is a maze of...


  • San Francisco, California, United States Everyday Agents Full time

    About WovenConsumer Travel is Ripe for DisruptionToday's travelers are seeking personalized, immersive, and authentic experiences that reflect their unique tastes and interests. They want to be inspired, to feel the thrill of discovery, and to seamlessly benefit from technology throughout their journeys. Yet, the current travel landscape is a maze of...


  • San Jose, United States Bayone Full time

    Location: San Francisco, Seattle, LA or PST Enterprise Data Platform to enable timely, effective and safe sharing of data to multiple engineering, operations and business teams for building world class data products Responsibilities Build data ingestion and processing pipelines to enable data analytics and data science use-cases in areas of digital...


  • San Francisco, United States Everyday Agents Full time

    About Woven Consumer Travel is Ripe for DisruptionToday's travelers are seeking personalized, immersive, and authentic experiences that reflect their unique tastes and interests. They want to be inspired, to feel the thrill of discovery, and to seamlessly benefit from technology throughout their journeys. Yet, the current travel landscape is a maze of...


  • San Francisco, CA, United States Nextdoor Full time

    #TeamNextdoorNextdoor is where you connect to the neighborhoods that matter to you so you can belong. Our purpose is to cultivate a kinder world where everyone has a neighborhood they can rely on.Neighbors around the world turn to Nextdoor daily to receive trusted information, give and get help, get things done, and build real-world connections with those...


  • San Francisco, United States Snorkel AI, Inc. Full time

    We are looking for a Director of Engineering to lead our AI Platform team. Our AI Platform team builds innovative software systems to power the Snorkel Flow platform. This includes services to train and serve generative AI and machine learning models using novel data-centric techniques, libraries to support AI workflows for a variety of data modalities and...

  • Platform Engineer

    4 weeks ago


    San Francisco, California, United States Watchtower AI Full time

    "As an early Platform Engineer at Watchtower, you'll enable us to deliver our platform reliably, securely, and at massive scale. You'll help architect low latency, real-time microservices that process & detect sensitive data at scale.Responsibilities:Building highly-available and secure authentication and API servicesMaintaining and evolving mission-critical...


  • San Francisco, CA, United States Discord Full time

    Discord is about giving people the power to create space to find belonging in their lives. We want to make it easier for you to talk regularly with the people you care about. We want you to build genuine relationships with your friends and communities close to home or around the world. Original, reliable, playful, and relatable. These are the values that...


  • San Francisco, CA, United States Discord Full time

    Discord is about giving people the power to create space to find belonging in their lives. We want to make it easier for you to talk regularly with the people you care about. We want you to build genuine relationships with your friends and communities close to home or around the world. Original, reliable, playful, and relatable. These are the values that...


  • San Francisco, United States Nightfall AI Full time

    Nightfall makes safeguarding sensitive data for every application simple and seamless. Organizations, from startups to global brands, trust Nightfall’s software platform and APIs to discover, classify, and protect sensitive data. We are looking for an exceptional engineering technical leader to join our growing team at Nightfall. As a senior, hands-on,...


  • San Francisco, United States Juniper Square Full time

    About Juniper Square Our mission is to unlock the full potential of private markets. Privately owned assets like commercial real estate, private equity, and venture capital make up half of our financial ecosystem yet remain inaccessible to most people. We are digitizing these markets, and as a result, bringing efficiency, transparency, and access to one of...


  • San Francisco, United States VAST Data Full time

    VAST Data is looking for a Senior Systems Engineer to join our growing team! This is a great opportunity to be part of the fastest-growing infrastructure company in history. In just a few short years, we’ve shaken up the industry by challenging traditional architecture models and introduced a revolutionary set of storage possibilities through our...


  • San Francisco, United States OpenAI Full time

    About the Team The Research Platform Analytics team designs, builds, and operates the critical foundational data and analytics infrastructure that enables research at OpenAI. Our goal is one, and one only: accelerate the progress of research towards AGI. We do this by owning critical components of the research training stack, starting from key data...


  • San Mateo, United States Zoox Full time

    Zoox is on a mission to make personal transportation safer, cleaner, and more enjoyable for everyone. We are transforming mobility-as-a-service by developing a fully autonomous, purpose-built robotaxi designed for AI to drive and humans to enjoy. We are still in the early stages of deploying our robotaxis on public roads, and it is a great time to join Zoox...

  • Data Engineer

    1 week ago


    San Francisco, United States VRChat Inc Full time

    Join the VRChat team! VRChat offers a first-of-its kind, game-changing platform that provides an endless collection of social immersive experiences and gives the power of creation to its robust community. With over 250,000 worlds and growing, VRChat’s vision is to allow users to bring their imaginations to life and help shape the metaverse anywhere in the...


  • San Jose, United States HireIO Inc Full time

    Team Introduction: Our mission in experimentation and evaluation team is to build the next-gen A/B testing platform, that empowers the company to make data-driven decision for the products. The supported scenarios include recommendation, push, ads, search, mobile app, UI interaction and service upgrades etc. Our platform's capabilities cover the entire...


  • San Francisco, United States Uncountable Inc Full time

    Thank you for your interest in Uncountable Engineering! Description ‍Uncountable is seeking experienced platform engineers who are passionate about user experience and scaling web applications.  Our goal is to revolutionize industrial research and development.  We're looking for motivated engineers who can help to build a state of the art development...

  • Data Engineer

    1 week ago


    San Francisco, United States Triune Infomatics Inc Full time

    Role: Data Engineer Location: Hybrid in Pleasanton, CA Duration: 6 months Areas of Responsibility Include: • Build test and implement Data Engineering - ETL, Data Integrations, Data Governance, Data Quality. • Build reusable Data Engineering assets and frameworks. • Identify opportunities to use existing data from multiple sources to inform data...

  • Data Engineer

    2 days ago


    San Francisco, United States Bayone Full time

    Your role at Sephora: As a Sr. Software Engineer you will design and implement innovative analytical solutions and work alongside the product engineering team, evaluating new features and architecture. Reporting to the Engineering Manager, Data Platform, you will work closely with other team members like data architects and business analysts to understand...