Senior Data Engineer

4 weeks ago

San Francisco, California, United States Nightfall AI Full time

Nightfall makes safeguarding sensitive data for every application simple and seamless. Organizations, from startups to global brands, trust Nightfall's software platform and APIs to discover, classify, and protect sensitive data.


We're looking for a Senior Data Engineer to enhance Nightfall's core data models and infrastructure supporting our real-time and data annotation pipelines, analytics warehouse, MLOps, real-time monitoring, and business intelligence reporting.

At Nightfall, we use ML/NLP to detect and protect sensitive personal, health, financial, and credential information in real time. You will contribute directly to the core product. Working with Engineers, Data Scientists, and Data Analysts, you will ensure high technical standards for data management and drive Nightfall's short and long-term data strategies.

This is a hybrid role based out of our San Francisco office.

Responsibilities

  • Design, build, and own the core data models and infrastructure to manage data across Nightfall's analytics stack - AWS, Kafka, Glue, S3, Snowflake, Cassandra, Cockroach DB, Tableau, and Datadog.
  • Collaborate closely with Data Analysts to unlock the data needed to power dashboards, ad-hoc analyses, monitoring, and high-quality decision-making across the company.
  • Collaborate closely with ML Engineering and Data Science to scale our data pipelines for machine learning, analytics, and data annotation.
  • Design and implement a data quality monitoring system with source-to-target data validations and anomaly detection.
  • Collaborate with Security and Engineering to meet data governance requirements.

Qualifications

  • SQL Expert. Solid experience in Python or similar.
  • Experience (3+ years) with analytical data warehouses/lakes. AWS/Snowflake preferred.
  • Experience (3+ years) developing data pipelines supporting ML/NLP projects. Experience with popular MLOps and data annotation platforms is a plus.
  • Experience (3+ years) analyzing complex datasets using SQL, optimizing queries, working with Data Analysts to automate dashboards (Tableau preferred), and performing ad-hoc analyses that lead to actionable business insights.
  • Bachelor's degree in Computer Science or related field. MS/PhD a plus.
  • Experience with source code repositories and cloud monitoring tools (Datadog preferred).
  • Possess excellent communication skills, strong analytical skills, and the ability to work independently and as part of a team.
  • The ability to thrive in a dynamic environment, being flexible and willing to jump in and do whatever it takes to be successful.
About Nightfall:

Nightfall is a cybersecurity startup dedicated to helping organizations secure and manage their sensitive data. As a leading enterprise technology company, our product affects the personal data that people entrust businesses to store & process with care every day. Critical data in modern organizations is often sprayed across a broad set of cloud data silos, and it's a herculean task for security & compliance teams to monitor, manage, and protect this highly sensitive data. Via machine learning, our product makes it easy for organizations to discover, classify, and protect this sensitive data across their cloud footprint - such as their corporate SaaS, data infrastructure, and even their own apps. In doing so, we prevent data leakage, provide unprecedented data visibility & protection across the cloud, and enable compliance. We're a technology startup founded in San Francisco, well-funded by leading institutional investors like Bain and Venrock, and a cadre of security & IT leaders from Okta, Salesforce, Atlassian, Splunk, FireEye, and more. Learn more on our website or by reaching out via email at

  • Senior Data Integration Engineer

    3 weeks ago

    San Francisco, California, United States Trace3, Inc Full time

    Our major field office locations include Denver, Indianapolis, Grand Rapids, Lexington, Los Angeles, Louisville, Texas, San Francisco.Juice - The "Stuff" it takes to be a Needle MoverWe look forward to the goal, mentallymapping outevery checkpoint on the pathway to success, and visualizing what the final destination looks and feels like.We're looking to add...

  • Senior Data Engineer

    4 weeks ago

    San Francisco, California, United States Glow Full time

    About GlowAt Glow, we believe that insurance can provide peace of mind to small business owners so they can pursue their dreams. We are building a digital insurance platform that ensures small businesses have the right coverage for all their insurance needs at the lowest cost, not just when they purchase, but every year. We recently completed a $22M Series A...

  • Senior Data Engineer

    4 weeks ago

    San Francisco, California, United States Scroll Capital Full time

    Lead Data Engineer - Python Expert$45k – $90k • 0.1% – 0.25%About Our CompanyWe are a VC backed Fintech Startup in the Bay Area.The RoleStaff/Lead Software EngineerThis is a core and very important role for the company. You'll have a lot of agency in architecting and implementing a corner store of modern financial infrastructure.You'll be enhancing and...

  • Senior Flutter Engineer

    4 weeks ago

    San Francisco, California, United States Motion Recruitment Full time

    This company develops technology that protects the integrity, confidentiality, and privacy of healthcare data while enabling consumers to comprehend it. They create systems that exhibit an accurate picture of the underlying data, are auditable and automated as far as feasible, and, most importantly, are responsive to the needs of our end users.They're...

  • Senior Machine Learning Engineer

    3 weeks ago

    San Francisco, California, United States Syntegra Full time

    Syntegra has developed a state-of-the-art method to generate patient-level synthetic dataset for healthcare applications. By utilizing transformer models (like GPT-2 and GPT-3), our synthetic data engine enables us to create realistic equivalents of any healthcare dataset that accurately preserves the analytic characteristics of the original, while...

  • Senior DevOps Engineer

    3 weeks ago

    San Francisco, California, United States Quid Full time

    Senior DevOps EngineerAs a Senior DevOps engineer for the Quid product you will build software and processes to enable engineers to self-service the operation of NetBase Quid at scale and support 24x7 operations. Our developers are your customers. Your goal is to continuously assess and ease pain points of fellow engineers and of our Cloud infrastructure. We...

  • Data Engineer

    4 weeks ago

    San Francisco, California, United States iTvorks Inc Full time

    Required Skills Data engineer main skills Good knowledge of Data warehousing with expertise in data modeling and ETL. Advanced skills in SQL and HQL Should be good at these core tech AWS/EMR/Spark Hadoop Hive Vertica Qlikview Tableau Should have exp in CI/CD

  • Staff Data Engineer

    2 weeks ago

    San Francisco, California, United States Pachama Full time

    Who we are.Pachama is a mission-driven company looking to restore nature to help address climate change. Pachama brings the latest technology in remote sensing and AI to the world of forest carbon in order to enable forest conservation and restoration to scale. Pachama's core technology harnesses satellite imaging with artificial intelligence to measure...

  • Sr. Data Engineer

    3 weeks ago

    San Francisco, California, United States iTvorks Inc Full time

    Role Sr. Data Engineer (8+ years) Location SFO CA Duration Long TermExperience & Skills Extensive experience with Hadoop (or similar) Ecosystem (Map Reduce HDFS Hive Spark Pig HBase) Proficient in at least one of the SQL languages (MySQL PostgreSQL SqlServer Oracle) Good understanding of SQL Engine and able to conduct advanced performance tuning Strong...

  • Founding Data Platform Engineer

    2 weeks ago

    San Francisco, California, United States Everyday Agents Full time

    About WovenConsumer Travel is Ripe for DisruptionToday's travelers are seeking personalized, immersive, and authentic experiences that reflect their unique tastes and interests. They want to be inspired, to feel the thrill of discovery, and to seamlessly benefit from technology throughout their journeys. Yet, the current travel landscape is a maze of...

  • Founding Data Platform Engineer

    4 weeks ago

    San Francisco, California, United States Everyday Agents Full time

    About WovenConsumer Travel is Ripe for DisruptionToday's travelers are seeking personalized, immersive, and authentic experiences that reflect their unique tastes and interests. They want to be inspired, to feel the thrill of discovery, and to seamlessly benefit from technology throughout their journeys. Yet, the current travel landscape is a maze of...

  • Senior Engineer

    4 weeks ago

    San Francisco, California, United States The League Full time

    Company DescriptionThe League is a pre-series A mobile social dating app startup backed by IDG Ventures, xSeed Capital, Cowboy Ventures, Structure Capital, Sherpa Ventures, and many notable angels. The Founder is a Stanford MBA (ex-Google, ex-Salesforce) with a strong product sense (engineering degree from Carnegie Mellon) and a fierce determination to...

  • Data Infrastructure Security Engineer

    4 weeks ago

    San Francisco, California, United States Databricks Full time

    While candidates in the listed locations are encouraged for this role, we are open to remote candidates in other locations.As a Data Infrastructure Engineer on the security data infrastructure team you will help build Lakehouse for Security organization. You will build reliable, large-scale, multi-geo data pipelines to support detecting threats(internal and...

  • Senior Software Engineer

    3 weeks ago

    San Francisco, California, United States Ridepanda Full time

    Job functionsYou will work with the product and design team with input from other parts of the company. You will be critical in helping us build the core technologies and features that enable the company to scale quickly to serve as many customers and partners/clients as possible.You will be responsible for architecting scalable, modular solutions and...

  • Senior Software Engineer

    4 weeks ago

    San Francisco, California, United States Bunkerhill Health Full time

    About the roleWe are seeking a talented and enthusiastic Senior Software Engineer to join our dynamic team. As a Senior Software Engineer, you will work closely with our team to develop, test, and maintain software solutions that meet our clients' needs.Responsibilities:Collaborate with cross-functional teams to understand project requirements and translate...

  • Senior Software Engineer, Infrastructure

    4 weeks ago

    San Francisco, California, United States Fathom Full time

    Fathom is on a mission to use AI to understand and structure the world's medical data, starting by making sense of the terabytes of clinician notes contained within the electronic health records of the world's largest health systems. Our deep learning engine automates the translation of patient records into the billing codes used for healthcare provider...

  • Senior Software Engineer, Infrastructure

    4 weeks ago

    San Francisco, California, United States Fathom Full time

    Fathom is on a mission to use AI to understand and structure the world's medical data, starting by making sense of the terabytes of clinician notes contained within the electronic health records of the world's largest health systems. Our deep learning engine automates the translation of patient records into the billing codes used for healthcare provider...

  • Senior Android Engineer

    3 weeks ago

    San Francisco, California, United States Motion Recruitment Full time

    The goal of this organization is to establish a society where individuals are the only ones in charge of their personal information. They support businesses in quickly developing and implementing branded privacy and cybersafe solutions for their clientele, as well as enabling regular users to make decisions about when, where, and with whom to share their...

  • Senior System Engineer

    4 weeks ago

    San Francisco, California, United States Neuro42 Full time

    The next evolution of MRI is hereneuro42 is a Bay Area startup developing point-of-care MRI and robotics for diagnosis and interventions of neurological conditions.Senior Systems Engineer.Perform systems engineering and integration for the design, development, testing, and certification of MRI equipment.Provide hands-on system support for MRI prototype...

  • AIML - Sr Engineering Program Manager, Data

    2 weeks ago

    San Francisco, California, United States Apple Full time

    SummaryImagine what you could do here. At Apple, new ideas have a way of becoming extraordinary products, services, and customer experiences very quickly. Do you love taking on challenges that create a positive impact? If so, then we want to speak with youAs a member of the AIML Data EPM group, you will be responsible for Apple's ability to use data to power...