Current jobs related to Principal Engineer, Central Data Lake Lead - San Francisco, California - Autodesk

  • Principal Engineer, Central Data Lake Lead

    6 days ago

    San Francisco, California, United States Autodesk Full time

    Job Title: Principal Engineer, Central Data Lake LeadAutodesk's Platform Services and Emerging Technology (PSET) team is seeking a highly skilled Principal Engineer to lead the development and operation of a robust and scalable data platform.This exciting role involves automating large-scale cloud infrastructure services, designing and developing highly...

  • Lead Data Engineer

    2 days ago

    San Francisco, California, United States Unreal Gigs Full time

    Job Title: Lead Big Data EngineerAt Unreal Gigs, we're seeking a highly skilled Lead Big Data Engineer to join our team. As a key member of our data engineering team, you'll be responsible for designing, building, and maintaining our big data infrastructure and pipelines.Key Responsibilities:Lead a team of big data engineers, providing guidance, mentorship,...

  • Senior Software Engineer

    2 weeks ago

    San Francisco, California, United States Avature Full time

    About the RoleWe are seeking a highly skilled Senior Software Engineer to join our Portfolio Enterprise Data Lake team at Avature. As a key member of our engineering team, you will be responsible for designing and building scalable, high-performance data lakes using modern technologies such as Apache Iceberg, Trino, and Spark-SQL.Key...

  • Principal Machine Learning Engineer

    2 weeks ago

    San Francisco, California, United States Atlassian Full time

    About the RoleWe are seeking a highly skilled Principal Machine Learning Engineer to join our Central AI team at Atlassian. As a key member of our organization, you will play a pivotal role in constructing the fundamental infrastructure, data pipeline, frameworks, models, and other capabilities to expedite AI feature development throughout the entire...

  • Big Data Engineering Lead

    2 weeks ago

    San Francisco, California, United States Unreal Gigs Full time

    Job Title: Lead Big Data EngineerAt Unreal Gigs, we're seeking a highly skilled Lead Big Data Engineer to join our team and drive the future of data-driven innovation.Job SummaryWe're looking for a seasoned engineer with expertise in big data technologies and a track record of leading successful data projects. As the Lead Big Data Engineer, you'll be...

  • Cloud Data Engineering Lead

    2 weeks ago

    San Francisco, California, United States Unreal Gigs Full time

    Cloud Data Engineering LeadUnreal Gigs is a trailblazer in leveraging the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge cloud data solutions that harness the scalability, flexibility, and power of cloud platforms.Position OverviewAs the Cloud Data Engineering Lead, you'll...

  • Cloud Data Engineering Lead

    6 days ago

    San Francisco, California, United States Unreal Gigs Full time

    Lead Cloud Data EngineerUnreal Gigs is a trailblazer in leveraging the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge cloud data solutions that harness the scalability, flexibility, and power of cloud platforms.Position OverviewAs the Lead Cloud Data Engineer, you'll...

  • Principal Data Architect

    2 days ago

    San Francisco, California, United States SmithRx Full time

    Transforming Pharmacy Benefit Management with Data Engineering ExpertiseAt SmithRx, we're revolutionizing the pharmacy benefit management industry with cutting-edge technology and innovative solutions. As a Principal Data Engineer, you'll play a pivotal role in driving this transformation by designing and developing robust data architectures that support...

  • Principal Electrical Engineer

    2 days ago

    San Francisco, California, United States FELLOW Full time

    Job Title: Principal Electrical EngineerWe are seeking a highly skilled Principal Electrical Engineer to join our team at Fellow, a leading manufacturer of premium coffee gear. As a key member of our engineering team, you will play a pivotal role in designing and developing our electrical systems for our product line.Responsibilities:Own the electrical...

  • Lead Data Systems Architect

    1 week ago

    San Francisco, California, United States Unreal Gigs Full time

    Job Title: Lead Data Systems EngineerUnreal Gigs is seeking a highly skilled and experienced Lead Data Systems Engineer to join our team. As a key member of our data engineering team, you will be responsible for leading the design, implementation, and optimization of our data systems infrastructure.Key Responsibilities:Technical Leadership: Provide strategic...

  • Principal Data Science Leader

    2 weeks ago

    San Francisco, California, United States Lee Hecht Harrison Nederland B.V. Full time

    About the RoleWe are seeking a seasoned Principal Machine Learning Engineer to join our team at Lee Hecht Harrison Nederland B.V. in San Francisco, CA. This is a hybrid schedule position that requires extensive experience in machine learning and a strong background in the AdTech industry.Key ResponsibilitiesLead Data Engineering Initiatives: Spearhead the...

  • Lead Principal Software Engineer

    3 weeks ago

    San Francisco, California, United States Crusoe Full time

    Job OverviewCrusoe Energy is dedicated to harnessing value from stranded energy resources through innovative computational solutions.Our mission aligns the long-term interests of environmental sustainability with the future of global computing infrastructure. As data centers face an ever-increasing energy demand, we are committed to ensuring that this energy...

  • Principal Software Architect

    3 weeks ago

    San Francisco, California, United States Hinge-Health Full time

    About the RoleWe are seeking a highly skilled and experienced Principal Software Architect to join our team at Hinge Health. As a key member of our engineering team, you will play a pivotal role in guiding and accelerating the evolution of our AI and related Infrastructure initiatives.Key ResponsibilitiesTechnical Leadership: Provide technical leadership and...

  • Principal Machine Learning Engineer

    6 days ago

    San Francisco, California, United States Atlassian Full time

    About the RoleWe are seeking a highly skilled Principal Machine Learning Engineer to join our Central AI team in Bellevue, WA. As a key member of our organization, you will play a pivotal role in constructing the fundamental infrastructure, data pipeline, frameworks, models, and other capabilities to expedite AI feature development throughout the entire...

  • Lead Data Scientist

    2 weeks ago

    San Francisco, California, United States LHH Full time

    About LHH:We are a leading provider of recruitment and HR solutions, seeking a highly skilled Principal Data Scientist to join our team in San Francisco, CA (hybrid). We are looking for an experienced data professional with a strong background in machine learning and a proven track record in the AdTech sector. The ideal candidate will lead our data...

  • Lead Software Engineer

    4 weeks ago

    San Francisco, California, United States Databricks Full time

    Position OverviewWe are seeking a Front End or Full Stack Lead Software Engineer/Principal Software Engineer to join the Engineering team at Databricks. This role will report to an Engineering Leader and is integral to our engineering organization, focusing on one of the key products within Databricks. You will collaborate with teams dedicated to developing...

  • Principal Machine Learning Systems Engineer

    2 weeks ago

    San Francisco, California, United States Atlassian Full time

    About the RoleWe are seeking a highly skilled Principal Machine Learning Systems Engineer to join our Central AI team at Atlassian. As a key member of our organization, you will play a critical role in building and maintaining the core infrastructure for machine learning engineers and data scientists to develop, train, evaluate, deploy, and operate Machine...

  • Principal Machine Learning System Engineer

    6 days ago

    San Francisco, California, United States Atlassian Full time

    About the RoleWe are seeking a highly skilled Principal Machine Learning System Engineer to join our Central AI team at Atlassian. As a key member of our organization, you will play a critical role in building and maintaining the core infrastructure for machine learning engineers and data scientists to develop, train, evaluate, deploy, and operate machine...

  • Data Center Engineer/Principal Data Center Engineer

    2 weeks ago

    San Diego, California, United States Northrop Grumman Full time

    Job Title: Data Center Engineer/Principal Data Center EngineerWe are seeking a highly skilled Data Center Engineer/Principal Data Center Engineer to join our team at Northrop Grumman. This role will be responsible for integrating and testing software/hardware systems in complex IT systems.Key Responsibilities:Developing engineering documentation for system...

  • Principal SRE Data Infrastructure

    2 days ago

    San Jose, California, United States F5 Full time

    Job Title: Principal SRE Data InfrastructureF5 is seeking a highly skilled Principal SRE Data Infrastructure to join our team. As a key member of our engineering organization, you will be responsible for designing, building, and operating large-scale cloud-based and on-prem data systems ensuring scalability, reliability, and security.Responsibilities:Design...

Principal Engineer, Central Data Lake Lead

3 months ago

San Francisco, California, United States Autodesk Full time

Job Requisition ID #

24WD77185

Autodesk's Platform Services and Emerging Technology (PSET) team is hiring a Principal Software Engineer with experience automating large-scale cloud infrastructure services. In this exciting role, you will help us automate a robust and scalable data platform used by numerous teams across the company. Reporting to our Software Engineering Manager for Platform Infrastructure, you'll help us tackle tough problems and improve our platform's reliability, resiliency, and scalability. If you enjoy developing high quality and resilient systems and want to make a big impact with data at Autodesk, we want to meet you

As a Lead for our Central Data Lake, you will:

  • Design, develop, and operate highly reliable large-scale data lake
  • Partner closely with Product teams to understand requirements and design new capabilities that go directly into customer's hands
  • Build and scale data infrastructure that powers batch and real-time data processing of billions of records daily
  • Automate and administer big data applications and services on cloud
  • Help drive observability into the health of our data infrastructure and understanding of system behavior
  • Guide initiatives to enable best practices across infrastructure, deployments, automation, and accessibility
  • Set technical directions and influence cross-functional teams
  • Ensure operational excellence of the services and meet the commitments to our customers regarding reliability, availability, and performance
  • Develop and implement security best practices at the data, application, infrastructure, and network layers

Minimum Qualifications

  • 10+ years of hands-on direct internal experience in large-scale data intensive distributed systems, especially in query engines, object storage, data warehouse, data lake, data analytics, SQL/NoSQL databases, distributed file systems and data platform infrastructure
  • Experience with Apache Iceberg, Parquet, Spark, Hive, Presto, Trino, and Avro
  • Experience provisioning and managing orchestration tools like Airflow
  • Excellent coding skills and experience in Python, Unix shell scripting
  • Experience building, implementing and automating APIs
  • Experience building code-driven infrastructure on AWS through end-to-end automation including integration tests and unit tests
  • Experience with Self-service platform offering by building APIs to deploy Cloud resources
  • Demonstrated experience with EKS, EMR, S3, IAM, Networking and other AWS services
  • Experience with observability tools (such as, Hive, Spark, Presto, Jenkins, CloudOS, Datadog)
  • Experience communicating solutions and resolutions with technical and non-technical audiences
  • Willingness to participate in professional development activities to stay current on industry knowledge and excitement about breaking the status quo by seeking innovative solutions
  • Experience with public clouds (AWS, Azure, GCP)
  • Computer Science degree (or equivalent experience will be considered)

#LI-CD2

Learn More

About Autodesk

Welcome to Autodesk Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.

We take great pride in our culture here at Autodesk – our Culture Code is at the core of everything we do. Our values and ways of working help our people thrive and realize their potential, which leads to even better outcomes for our customers.

When you're an Autodesker, you can be your whole, authentic self and do meaningful work that helps build a better future for all. Ready to shape the world and your future? Join us

Benefits

From health and financial benefits to time away and everyday wellness, we give Autodeskers the best, so they can do their best work. Learn more about our benefits in the U.S. by visiting

Salary transparency

Salary is one part of Autodesk's competitive compensation package. For U.S.-based roles, we expect a starting base salary between $141,400 and $228,800. Offers are based on the candidate's experience and geographic location, and may exceed this range. In addition to base salaries, we also have a significant emphasis on annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.

Equal Employment Opportunity

At Autodesk, we're building a diverse workplace and an inclusive culture to give more people the chance to imagine, design, and make a better world. Autodesk is proud to be an equal opportunity employer and considers all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender, gender identity, national origin, disability, veteran status or any other legally protected characteristic. We also consider for employment all qualified applicants regardless of criminal histories, consistent with applicable law.

Diversity & Belonging

We take pride in cultivating a culture of belonging and an equitable workplace where everyone can thrive. Learn more here:

Are you an existing contractor or consultant with Autodesk?

Please search for open jobs and apply internally (not on this external site).

Welcome to Autodesk Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.

We take great pride in our culture here at Autodesk – our Culture Code is at the core of everything we do. Our values and ways of working help our people thrive and realize their potential, which leads to even better outcomes for our customers.

When you're an Autodesker, you can be your whole, authentic self and do meaningful work that helps build a better future for all. Ready to shape the world and your future? Join us

Not quite ready to apply? Join our talent community ) to stay up to date on new job opportunities and the latest Autodesk news.