Senior Data Engineer, Big Data

4 weeks ago

Palo Alto, California, United States ZipRecruiter Full time
About the Job

We're seeking a skilled Data Processing Engineer to join our team. As a key member of our engineering team, you'll be responsible for designing and implementing data processing pipelines using big data technologies like Spark and Flink.

Key Responsibilities:

  • Design and implement data processing pipelines using big data technologies
  • Collaborate with cross-functional teams to develop and deploy data-driven applications
  • Develop a strong understanding of the business, industry, codebase, and/or systems

Requirements:

  • 3+ years of professional software development experience with a focus on big data technologies
  • Experience with Hadoop, Spark, Hive, and/or other big data technologies
  • Comprehensive computer science fundamentals in coding, object-oriented programming, data structures, and algorithms
  • Experience with containerization technologies like Docker and/or Kubernetes

Preferred Qualifications:

  • 5+ years of professional software development experience, with a focus on big data technologies
  • BS/MS/PhD in Mathematics, Computer Science, Physics, related technical field or equivalent practical experience
  • Experience with data integration tools like Apache Kafka, Flume, and NiFi
  • Familiarity with data storage technologies such as Delta Lake, HBase, Cassandra, and MongoDB
  • Familiarity with Apache Hudi, Apache Beam, Apache Flink, Google Cloud Dataflow, Amazon Kinesis Data Analytics, and/or Azure Databricks

As a member of our team, you'll enjoy:

  • Competitive compensation
  • Exceptional benefits package
  • Flexible Vacation & Paid Time Off
  • Employer-matched 401(k) plan

#LI-Remote

The US base salary range for this full-time position is $105,000-$145,000. Our salary ranges are determined by role, level, and location, and the range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position across all US locations. Within the range, individual pay is determined by work location, role-related knowledge and skills, depth of experience, relevant education or training, and additional role-related considerations.

Depending on the position offered, equity, bonuses, commission, or other forms of compensation may also be provided as part of a total compensation package, in addition to a full range of medical, financial, and other benefits.

ZipRecruiter is proud to be an equal opportunity employer and provides equal employment opportunities (EEO) to all employees and applicants without regard to race, color, religion, sex, national origin, age, disability, veteran status, sexual orientation, gender identity or genetics.

Privacy Notice: For information about ZipRecruiter's collection and processing of job applicant personal data for this job, please see our Privacy Notice at: https://www.ziprecruiter.com/careers/job-applicant-privacy-notice

About ZipRecruiter

ZipRecruiter is the fastest growing employment marketplace. We have helped over 1 million businesses and 100 million job seekers find their next perfect match through partnerships with the best job boards on the web, curated email alerts, award-winning mobile apps, and the world's best search algorithm for jobs.

  • Senior Big Data Software Engineer

    4 weeks ago

    Palo Alto, California, United States Predactiv Full time

    Job OverviewAbout Predactiv: A Big Data PioneerWe're a cutting-edge big data company that owns online behavior data of 1 billion+ users globally. Our audience intelligence platform is built on innovative big data technologies. We're seeking a skilled software engineer to join our team and contribute to our exciting projects and platform development.Key...

  • Big Data Software Engineer

    4 weeks ago

    Palo Alto, California, United States Predactiv Full time

    About Us:At Predactiv, we're a big data company that owns online behavior data of 1 billion+ users globally. We're developing an audience intelligence platform using cutting-edge big data technologies. We're looking for a skilled software engineer to contribute to our exciting projects and platform development.You'll be responsible for designing and...

  • Senior Data Engineer

    4 weeks ago

    Palo Alto, California, United States Capital One Full time

    We are seeking a Data Engineer who is passionate about marrying data with emerging technologies. As a Capital One Data Engineer, you'll have the opportunity to be on the forefront of driving a major transformation within Capital One.Key Responsibilities:Be a part of a team designing and building Enterprise Level scalable, low-latency, fault-tolerant...

  • Senior Data Engineer

    4 weeks ago

    Palo Alto, California, United States Acceler8 Talent Full time

    Job Title: Senior Data Engineer - AI Training DataWe are seeking a Senior Data Engineer - AI Training Data to join our team at Acceler8 Talent. This role is ideal for an engineer passionate about building innovative systems and optimizing large-scale datasets for advanced AI applications.About the CompanyOur organization is dedicated to reshaping the...

  • Senior Data Engineer

    4 weeks ago

    Palo Alto, California, United States Tesla Full time

    Job DescriptionTesla is seeking a highly skilled Sr. Software Engineer, Data Platform to join our team of experts building the company's data platforms powered by all Tesla products.As a key member of our engineering team, you will be responsible for designing, developing, and maintaining scalable distributed systems that process terabytes of IoT data...

  • Senior Backend Data Engineer

    4 weeks ago

    Palo Alto, California, United States Luma AI Full time

    Job SummaryWe are seeking a highly skilled Senior Backend Data Engineer to join our applied research team at Luma AI. As a key member of our team, you will design, build, and automate infrastructure for processing large-scale data across multiple clusters of thousands of GPUs.Responsibilities Design and implement efficient data processing systems for...

  • Senior Data Engineer

    4 weeks ago

    Palo Alto, California, United States Machinify Full time

    Job Title: Staff/Lead Data EngineerMachinify is a leading provider of AI-powered software products that transform healthcare claims and payment operations. Our revolutionary AI-platform enables us to develop and deploy industry-specific products that increase the speed and accuracy of claims processing by orders of magnitude.Job SummaryWe are seeking a...

  • Senior Data Engineer, Distributed Systems

    3 weeks ago

    Palo Alto, California, United States Tesla Full time

    As a Senior Data Engineer at Tesla, you will play a key role in building and improving our industry-leading, scalable distributed systems. Our team is responsible for powering a cutting-edge platform that processes terabytes of IoT data daily, working with Engineering teams on making our products better in every way. From storage to access to data query, our...

  • Senior Data Scientist

    4 weeks ago

    Palo Alto, California, United States Machinify Full time

    Job OverviewMachinify is a leading provider of AI-powered software products that transform healthcare claims and payment operations. Our revolutionary AI-platform enables us to develop and deploy industry-specific products that increase the speed and accuracy of claims processing by orders of magnitude.We are seeking a Data Scientist, a proven AI...

  • Senior Simulation and Data Modeling Quality Engineer

    4 weeks ago

    Palo Alto, California, United States Tech Tammina Full time

    Job Summary:As a Senior Simulation and Data Modeling Quality Engineer at Tech Tammina, you will lead the testing of our core meso-scale agent-based simulation environment. This environment is used to develop and demonstrate algorithms to control and optimize goods delivery. You will work with product and engineering teams to understand new features and...

  • Senior Data Analyst

    4 weeks ago

    Palo Alto, California, United States Rivian Full time

    About RivianRivian is a pioneering company that's revolutionizing the automotive industry with its innovative electric adventure vehicles. We're on a mission to keep the world adventurous forever, and we're seeking a talented Senior Software Analyst to join our Self-Driving Data Team.Job SummaryWe're looking for a detail-oriented, data-driven problem solver...

  • Senior Data Engineer

    4 weeks ago

    Palo Alto, California, United States Machinify Full time

    Job OverviewMachinify is a leading provider of AI-powered software products that transform healthcare claims and payment operations. Our revolutionary AI-platform enables us to develop and deploy industry-specific products that increase the speed and accuracy of claims processing by orders of magnitude.We are seeking a Senior Data Engineer to build and own...

  • Staff Data Engineer, Data, Autonomy

    3 weeks ago

    Palo Alto, California, United States Rivian Full time

    About RivianRivian is on a mission to keep the world adventurous forever. This goes for the emissions-free Electric Adventure Vehicles we build, and the curious, courageous souls we seek to attract.We are looking for a seasoned Staff Data Engineer who will play a critical role in advancing our data infrastructure, ensuring it meets the highest standards of...

  • Senior Product Data Scientist

    4 weeks ago

    Palo Alto, California, United States Celonis Full time

    About the RoleWe're seeking a highly skilled Senior Product Data Scientist to join our Product Analytics & Data Sciences team at Celonis. As a key member of our Engineering organization, you will play a critical role in developing and communicating data-informed insights that drive strategic and tactical decision-making in a hyper-growth environment.Key...

  • Data Scientist

    3 weeks ago

    Palo Alto, California, United States Glean Full time

    About GleanWe're on a mission to make knowledge work faster and more humane. We believe that AI will fundamentally transform how people work. In the future, everyone will work in tandem with expert AI assistants who find knowledge, create and synthesize information, and execute work. These assistants will free people up to focus on the higher-level, creative...

  • Data Scientist

    4 weeks ago

    Palo Alto, California, United States Glean Full time

    About GleanWe're on a mission to revolutionize the way companies harness knowledge and AI. Our team believes that AI will fundamentally transform the way people work, and we're building a system of intelligence that will make knowledge work faster and more humane.Our platform brings all of a company's knowledge together, understands it at a deep level,...

  • Data Engineering Lead

    4 weeks ago

    Palo Alto, California, United States Machinify, Inc. Full time

    Machinify, Inc. is revolutionizing the healthcare industry with its AI-powered software products. We're seeking a highly skilled Data Engineering Lead to join our team and drive the development of critical data pipelines.The ideal candidate will have a deep understanding of data engineering principles and experience building production-ready data pipelines....

  • Senior Data Analyst

    4 weeks ago

    Palo Alto, California, United States BioSpace, Inc. Full time

    Job Title: Senior Data AnalystAt BioSpace, Inc., we are seeking a highly skilled Senior Data Analyst to join our Commercial Department. As a key member of our team, you will be responsible for collecting, curating, and analyzing commercial data to inform business decisions.Key Responsibilities:Collaborate with Commercial team to synthesize insights and drive...

  • Senior Data Scientist

    4 weeks ago

    Palo Alto, California, United States Qualified Health Full time

    Lead Our Technical TeamWe are seeking a highly skilled and experienced Senior Machine Learning Engineer to drive the adoption of best practices in machine learning operations (MLOps) and lead our technical team. Key responsibilities include:Collaborating with cross-functional teams to align technical solutions with business goals.Designing, developing, and...

  • Data Science and Engineering Leadership Opportunity

    3 weeks ago

    Palo Alto, California, United States Rivian Full time

    About Rivian, a company revolutionizing the EV industry with its innovative Electric Adventure Vehicles, is seeking a seasoned leader to spearhead its Data Science and Engineering initiative. This role is a critical component of our mission to drive insights and strategic decision-making across our Service organization. The successful candidate will lead a...