Principal Data Engineer

4 weeks ago

San Francisco, California, United States Index Exchange Full time
About the Role

We are seeking a highly experienced Principal Data Engineer to lead our data systems team and drive strategic outcomes for our organization. As a key member of our technology team, you will be responsible for providing strategic leadership across multiple domains, collaborating with cross-functional teams to achieve business objectives.

Key Responsibilities
  • Provide strategic leadership across multiple domains, collaborating with cross-functional teams to achieve business objectives.
  • Foster a culture of innovation, continuous improvement, and collaboration within the data systems team.
  • Advocate for strategic directions that reduce our complexity and tech stack footprint.
  • Demonstrate deep expertise in our core data streaming components, including Kafka, Spark, Hadoop, Presto/Trino, Flink, and Redis in a low-latency environment.
  • Architect relational data structures for performance, scale, and continuity.
  • Drive technical excellence by staying abreast of industry trends, evaluating emerging technologies, and making recommendations for adoption.
  • Lead initiatives with other cross-functional Principal engineering staff that are supporters and consumers of your data systems.
  • Ensure robust operational processes are in place to guarantee stability and scalability of database environments.
  • Develop and implement best practices for database administration, monitoring, and maintenance using open-source observability and instrumentation tools.
  • Develop and enhance testing and production release procedures with an automation-first core principle to end-to-end delivery.
  • Collaborate effectively with cross-functional teams, including developers, system administrators, and data scientists, to optimize database performance and functionality.
  • Communicate complex technical concepts to non-technical stakeholders, fostering understanding and alignment across the organization.
  • Participate in quarterly planning events to present your roadmaps and ensure strong alignment with the objectives and the broader technology teams.
  • Lead efforts in problem identification, troubleshooting, and resolution to minimize downtime and enhance our Exchange reliability.
  • Lead retrospective/rca engagements for data systems to ensure we are on a continuous improvement path.
Requirements
  • 3-5 years of proven experience as a Principal Engineer or similar role.
  • In-depth expertise with streaming data components Kafka, Spark, Flink, Hadoop, Presto/Trino, Redis.
  • Demonstrated leadership, mentoring, and collaboration skills balancing strategic and tactical needs.
  • Excellent problem-solving and troubleshooting abilities, diagnosing at depth for a globally distributed infrastructure with low-latency requirements.
  • Effective communication skills, both for technical and non-technical audiences.
What We Offer

We offer a comprehensive compensation package, including a base salary of USD 236,000 to 320,000, comprehensive health, dental, and vision plans, time off and flexible work schedules, retirement plan with a 5% company match, stock options and equity packages, generous parental leave, monthly wellness stipend, community engagement opportunities, and a workplace that supports a diverse, equitable, and inclusive environment.

We are an equal employment opportunity employer and welcome applications from diverse candidates. If you are a seasoned Data Systems Engineer with a passion for driving strategic outcomes and ensuring technical excellence, we invite you to apply and be a key player in shaping the future of our global data environments.

  • Principal Data Engineer

    4 weeks ago

    San Francisco, California, United States UNITY Full time

    Job Title: Principal Data Engineer - Machine Learning ExpertWe are seeking a highly skilled Principal Data Engineer to lead our data engineering efforts in machine learning and AdTech. The ideal candidate will have a strong background in machine learning and a proven track record in the AdTech sector.The successful candidate will be responsible for...

  • Principal Software Engineer

    3 weeks ago

    San Francisco, California, United States project44 Full time

    About the RoleAs a Principal Software Engineer - Data Engineering at project44, you will have opportunities to work on the latest technologies to streamline Machine Learning & AI Operations, build scalable data infrastructure, and democratize data access.Key Responsibilities:Work with software architecture and design as part of your job, leveraging and...

  • Principal Software Engineer Data

    3 weeks ago

    San Francisco, California, United States Autodesk Full time

    Job SummaryAutodesk is seeking a highly skilled Principal Software Engineer to join our Network team. As a key member of our team, you will be responsible for the development and maintenance of our graph database, which powers our pre-construction bidding application.Key ResponsibilitiesImplement new features for end-users and data consumersIntegrate new...

  • Principal Software Engineer

    4 weeks ago

    San Francisco, California, United States Autodesk Full time

    Job SummaryAutodesk's Platform Services and Emerging Technology (PSET) team is seeking a highly skilled Principal Software Engineer to lead the development and operation of a robust and scalable data platform.As a Principal Software Engineer, you will design, develop, and operate large-scale data lake infrastructure, partnering closely with Product teams to...

  • Principal Data Scientist

    4 weeks ago

    San Francisco, California, United States DaVita Full time

    Job Title: Principal Data ScientistAt SoFi, we're shaping a brighter financial future with our members. We're a next-generation financial services company and national bank using innovative, mobile-first technology to help our millions of members reach their goals.We're seeking a Principal Data Scientist with strong engineering skills to lead the domain of...

  • Principal Software Engineer

    3 weeks ago

    San Francisco, California, United States Salesforce Inc Full time

    At Salesforce, we're seeking a highly skilled Principal Member of Technical Staff to join our team in San Francisco, CA.The ideal candidate will have a Master's degree in Computer Science, Engineering, or a related quantitative discipline, and two years of experience in software development, data engineering, or a related field.Key responsibilities...

  • Principal Data Scientist with Strong Engineering Skills

    3 weeks ago

    San Francisco, California, United States Social Finance Ltd Full time

    Shape the Future of Financial ServicesAt Social Finance Ltd, we're on a mission to revolutionize the way people interact with personal finance. As a Principal Data Scientist with strong engineering skills, you'll play a critical role in shaping our decision science and experimentation domain.About the RoleWe're seeking a highly skilled professional to lead...

  • Principal Data Architect

    3 weeks ago

    San Francisco, California, United States SmithRx Full time

    Job OverviewSmithRx is revolutionizing the pharmacy benefit management industry with a cutting-edge platform that delivers real-time insights, cost efficiencies, and exceptional customer experiences. As we continue to expand, we are seeking an experienced Principal Data Engineer to drive innovation and lead the technology strategy for modern data...

  • Principal Civil Engineer

    4 weeks ago

    San Francisco, California, United States City of Arcadia, CA Full time

    Job Title: Principal Civil EngineerJoin the City of Arcadia, CA team as a Principal Civil Engineer and take on a challenging role that requires expertise in civil engineering and project management.About the Role:We are seeking a highly skilled and experienced Principal Civil Engineer to lead our Engineering Division in the Public Works Services Department....

  • Database Engine Principal

    4 weeks ago

    San Francisco, California, United States SingleStore, Inc. Full time

    Job Title: Principal Engineer, Database EngineAt SingleStore, Inc., we're seeking a highly skilled Principal Engineer to join our Database Engine team. As a key member of our team, you will be responsible for designing, developing, and maintaining the core database engine that powers our cloud-native database.Key Responsibilities:Lead technical decisions and...

  • Principal Automation Engineer, Site Services

    4 weeks ago

    San Francisco, California, United States F. Hoffmann-La Roche Gruppe Full time

    Job Summary:The Principal Automation Engineer is a key member of the Site Operations organization, responsible for providing technical support to ensure the safe, efficient, and reliable operation of automation systems owned and supported by Site Services at Genentech's South San Francisco and Dixon sites.Key Responsibilities:As a Technical System Owner, the...

  • Principal Software Engineer

    4 weeks ago

    San Francisco, California, United States Nextdoor Full time

    About the RoleNextdoor is seeking a highly skilled Principal Software Engineer to join our Data Platform team. As a key member of our team, you will be responsible for designing, building, and maintaining a highly scalable data platform that will power the development of Nextdoor for the foreseeable future.The ideal candidate will have a deep understanding...

  • Principal Automation Engineer

    3 weeks ago

    San Francisco, California, United States F. Hoffmann-La Roche Gruppe Full time

    As a key member of the Site Operations organization, the Principal Automation Engineer plays a crucial role in the Facility Systems & Plant Engineering (FSPE) team, providing technical support to ensure the safe, efficient, and reliable operation of automation systems owned and supported by Site Services at Genentech's South San Francisco and Dixon...

  • Principal Software Engineer

    4 weeks ago

    San Francisco, California, United States Fivetran Full time

    About the RoleFivetran is a leading provider of cloud-based data integration solutions. We're seeking a highly skilled Principal Software Engineer to lead the technical architecture, design, and development of our Data Lake product offering.The successful candidate will have a deep understanding of high-performance relational data management systems, as well...

  • Principal Systems Engineer

    4 weeks ago

    San Francisco, California, United States DataSF Full time

    Job SummaryWe are seeking a highly skilled Principal Systems Engineer to join our team at DataSF. As a key member of our IT department, you will be responsible for designing, implementing, and maintaining large complex systems and networks.Key ResponsibilitiesOversee the performance of day-to-day operational support for server, storage, or network...

  • Principal Systems Engineer

    4 weeks ago

    San Francisco, California, United States DataSF Full time

    Job SummaryWe are seeking a highly skilled Principal Systems Engineer to join our team at DataSF. As a key member of our IT department, you will be responsible for designing, implementing, and maintaining large complex systems and networks.Key ResponsibilitiesOversee the performance of day-to-day operational support for server, storage, or network...

  • Principal Machine Learning Engineer

    4 weeks ago

    San Francisco, California, United States Unity Technologies Full time

    Job Title: Principal Machine Learning EngineerUnity Technologies is seeking a highly skilled Principal Machine Learning Engineer to join our Ads Applied Research team. As a key member of our team, you will be responsible for driving the overall direction of machine learning for our Ads platform, ensuring it remains at the forefront of industry...

  • Principal Energy Engineer

    4 weeks ago

    San Francisco, California, United States Roche Holdings Inc. Full time

    Job Summary:We are seeking a highly experienced and skilled Principal Energy Engineer to lead and support energy management initiatives across our organization.The successful candidate will spearhead projects focused on energy efficiency, retro-commissioning, heat pump design and implementation, and sustainability strategies for new building projects.In...

  • Principal Software Engineer, Core Systems

    3 weeks ago

    San Francisco, California, United States Reddit Full time

    Job Title: Principal Software Engineer, Core SystemsAbout the Role:We are seeking a highly skilled Principal Software Engineer to join our Core Platform team at Reddit. As a key member of our engineering organization, you will be responsible for designing and evolving our Core Systems to support the continued scale and growth of Reddit.Responsibilities:Own...

  • Principal Civil Engineer

    4 weeks ago

    San Francisco, California, United States City of Arcadia, CA Full time

    Job Title: Principal Civil EngineerWe are seeking a highly skilled and experienced Principal Civil Engineer to join our team at the City of Arcadia, CA. As a key member of our Public Works Services Department, you will be responsible for managing the Engineering Division and overseeing various engineering projects.Key Responsibilities:Direct and manage the...