Data Platform Engineer

2 weeks ago


Remote, Oregon, United States Stability AI Full time

Location: Remote US
About the role:

We are looking for a talented Data Platform Engineer with a focus on scaling efficient distribution workloads. You will work alongside a growing multidisciplinary team of talented Research Scientists and Machine Learning Engineers to improve and scale the efficiency within our computing capacity. Stability AI operates a very large HPC cluster for training foundational AI models across several modalities.
In this role, you will contribute to groundbreaking projects that redefine visual storytelling through advanced generative modeling techniques, optimizing and managing large-scale distributed workloads to drive project efficiency and success.

Responsibilities:

  • Clean, normalize, and preprocess data in a scalable, parallelizable way to prepare it for ingestion into our machine learning model training pipelines while ensuring of data quality
  • Building and maintaining highly scalable distributed workloads
  • Build data pipelines to ingest and process data (e.g. images and text) for feeding into ML models
  • AWS Resource Management
  • Keep up-to-date with papers / methods regarding how to improve data quality and/or curate data for Image, Video, LLMs etc.

Qualifications:

  • Proven background within large scale distributed workloads
  • Experience with large scale data loading for machine learning training runs
  • Knowledge of cloud computing platforms, and experience with managing associated resources. AWS is strongly preferred but open to other cloud platforms
  • Experience with Python + Pytorch, Deep learning, Computer Vision
  • Experience with multiprocessing and multithreading python workloads
  • Experience with parallel dataframe manipulation using Pyspark / Ray
  • Proficiency in HPC cluster management tools and technologies
  • Excellent communication skills to effectively collaborate with users, solve issues, and provide guidance
  • Attention to detail and the ability to document processes and solutions effectively

Compensation

The salary range for this role is between $130,000 and $190,000. Individual pay within the range is based on factors like job-related skills and experience. Total compensation also includes stock options and benefits.

Equal Employment Opportunity:

We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or other legally protected statuses.



  • Remote, Oregon, United States TrueML Full time

    Your Role:TrueAccord is looking for an Engineering Manager, Data Platform who will lead a team of data engineers to architect, develop, and maintain data pipelines using the latest technologies.Ideally, you have a proven track record in managing a group of world-class data engineers with demonstrated growth in their career development. You possess hands-on...


  • Remote, Oregon, United States TrueML Full time

    Your Role:TrueAccord is looking for an Engineering Manager, Data Platform who will lead a team of data engineers to architect, develop, and maintain data pipelines using the latest technologies.Ideally, you have a proven track record in managing a group of world-class data engineers with demonstrated growth in their career development. You possess hands-on...


  • Remote, Oregon, United States TrueML Full time

    Why TrueML? TrueML is a mission-driven financial software company that aims to create better customer experiences for distressed borrowers. Consumers today want personal, digital-first experiences that align with their lifestyles, especially when it comes to managing finances. TrueML's approach uses machine learning to engage each customer digitally and...


  • Remote, Oregon, United States Stack Overflow Full time

    Every developer has a tab open on Stack Overflow. We are one of the most popular websites in the world - a community-based space focused on increasing productivity, decreasing cycle times, accelerating time to market, and protecting institutional knowledge. Innovation is at the heart of everything we do. We embrace collaboration, transparency, and believe in...


  • Remote, Oregon, United States Stack Overflow Full time

    Every developer has a tab open on Stack Overflow. We are one of the most popular websites in the world - a community-based space focused on increasing productivity, decreasing cycle times, accelerating time to market, and protecting institutional knowledge. Innovation is at the heart of everything we do. We embrace collaboration, transparency, and believe in...

  • Program Data Analyst

    2 months ago


    Remote, Oregon, United States Pattern Data Full time

    Who We AreThe Pattern Data platform was created for the rapid analysis of thousands of medical records. Fueled by AI, our platform efficiently reviews and categorizes documents at a speed 10 times faster, significantly reducing validation time by highlighting pertinent information crucial for litigation in medical analysis and personal injury cases. Our...

  • Program Data Analyst

    2 weeks ago


    Remote, Oregon, United States Pattern Data Full time

    Who We AreThe Pattern Data platform was created for the rapid analysis of thousands of medical records. Fueled by AI, our platform efficiently reviews and categorizes documents at a speed 10 times faster, significantly reducing validation time by highlighting pertinent information crucial for litigation in medical analysis and personal injury cases. Our...

  • Program Data Analyst

    4 weeks ago


    Remote, Oregon, United States Pattern Data Full time

    Who We AreThe Pattern Data platform was created for the rapid analysis of thousands of medical records. Fueled by AI, our platform efficiently reviews and categorizes documents at a speed 10 times faster, significantly reducing validation time by highlighting pertinent information crucial for litigation in medical analysis and personal injury cases. Our...


  • Remote, Oregon, United States Cruise Full time

    We're Cruise, a self-driving service designed for the cities we love.We're building the world's most advanced self-driving vehicles to safely connect people to the places, things, and experiences they care about. We believe self-driving vehicles will help save lives, reshape cities, give back time in transit, and restore freedom of movement for many.In our...


  • Remote, Oregon, United States Cruise Full time

    We're Cruise, a self-driving service designed for the cities we love.We're building the world's most advanced self-driving vehicles to safely connect people to the places, things, and experiences they care about. We believe self-driving vehicles will help save lives, reshape cities, give back time in transit, and restore freedom of movement for many.In our...


  • Remote, Oregon, United States Level Home Full time

    About The Company:Founded by ex-Apple Product and Engineering leaders, Level is redefining the smart community with technology that is simple, intuitive, useful, and invisible. Together, our platform and services provide premium experiences for all those who live, work, and own multifamily communities.At Level, we take a unique approach to designing products...

  • Data Engineer

    4 weeks ago


    Remote, Oregon, United States Jellyfish Full time

    As a member of Jellyfish Research, you will work closely with data scientists to understand and facilitate data needs for high-impact research, analysis and reporting. You will optimize, expand or redesign our current data platforms, including storage and pipeline architecture, to ingest, combine, and aggregate data from multiple sources. You'll also build...

  • Platform Engineer

    1 month ago


    Remote, Oregon, United States Vetcove Full time

    Join our growing team at Vetcove, and help us transform how veterinarians buy the supplies they need to keep America's pets healthy. Vetcove's eCommerce and mobile platforms enable veterinary practices to compare and buy from all of their vendors in one place. Our community of more than 17,000 veterinary hospitals employs tens of thousands of veterinarians,...

  • Platform Engineer

    2 weeks ago


    Remote, Oregon, United States Vetcove Full time

    Join our growing team at Vetcove, and help us transform how veterinarians buy the supplies they need to keep America's pets healthy. Vetcove's eCommerce and mobile platforms enable veterinary practices to compare and buy from all of their vendors in one place. Our community of more than 17,000 veterinary hospitals employs tens of thousands of veterinarians,...

  • Platform Engineer

    4 weeks ago


    Remote, Oregon, United States Vetcove Full time

    Join our growing team at Vetcove, and help us transform how veterinarians buy the supplies they need to keep America's pets healthy. Vetcove's eCommerce and mobile platforms enable veterinary practices to compare and buy from all of their vendors in one place. Our community of more than 17,000 veterinary hospitals employs tens of thousands of veterinarians,...

  • Databricks Engineer

    4 weeks ago


    Remote, Oregon, United States NTT DATA Services Full time

    Databricks EngineerNTT DATA is a team of more than 190,000 diverse professionals, operating in more than 50 countries throughout the world. The sectors where we have activities include: telecommunications, finance, industry, utilities, energy, public administration and health.Our mission? Offer technological solutions, business, strategy, development and...

  • Databricks Engineer

    1 month ago


    Remote, Oregon, United States NTT DATA Services Full time

    Databricks EngineerNTT DATA is a team of more than 190,000 diverse professionals, operating in more than 50 countries throughout the world. The sectors where we have activities include: telecommunications, finance, industry, utilities, energy, public administration and health.Our mission? Offer technological solutions, business, strategy, development and...


  • Remote, Oregon, United States WEKA Full time

    WEKA is leading a paradigm shift in how data is stored, managed, and processed in the cloud and AI era. We are a pre-IPO, growth-stage company on a hyper-growth trajectory that has raised over $275M in capital and is backed by dozens of world-class venture capital and strategic investors.Our flagship product, the WEKA Data Platform, is a software-based,...


  • Remote, Oregon, United States WEKA Full time

    WEKA is leading a paradigm shift in how data is stored, managed, and processed in the cloud and AI era. We are a pre-IPO, growth-stage company on a hyper-growth trajectory that has raised over $275M in capital and is backed by dozens of world-class venture capital and strategic investors.Our flagship product, the WEKA Data Platform, is a software-based,...


  • Remote, Oregon, United States Hypori Inc. Full time

    Hypori Inc, a leading provider of SaaS cybersecurity solutions, is transforming secure mobility for federal and commercial customers, including the United States Army. Hypori's secure virtual workspace enables users to access critical data and apps from any mobile device without compromising user privacy. From commercial IP to national security level intel,...