Data Engineer

3 days ago

Oakland, California, United States Elicit Full time
About Elicit

Elicit is an innovative AI research assistant that leverages language models to empower researchers in making informed decisions. Our mission is to increase the amount of good reasoning in the world by providing a scalable and reliable platform for literature review and research tasks.

Our Vision

We envision a future where Elicit is the go-to platform for researchers, offering a comprehensive and up-to-date database of scholarly sources. Our goal is to make Elicit the most complete and accurate database of academic literature, enabling researchers to make better decisions and drive progress in their fields.

Job Description

We are seeking an experienced Data Engineer to join our team and help us achieve our mission. As a Data Engineer at Elicit, you will be responsible for designing and implementing robust, scalable solutions to handle our growing data needs while maintaining high performance and data quality.

Key Responsibilities:

  • Design and implement efficient data pipelines to process, deduplicate, and index hundreds of millions of research papers
  • Optimize Spark jobs and data pipelines to handle large amounts of data efficiently
  • Develop processes to handle regular updates from multiple academic paper sources, ensuring efficient deduplication
  • Maintain and improve data quality by implementing robust data quality management processes

Requirements:

  • 5+ years of experience as a data engineer building core datasets and supporting business verticals with high data volumes
  • Strong proficiency in Python (5+ years experience)
  • Experience with architecting and optimizing large data pipelines with Spark
  • Strong SQL skills, including understanding of aggregation functions, window functions, UDFs, self-joins, partitioning, and clustering approaches
  • Experience with Parquet file formats and other columnar data storage formats
  • Strong data quality management skills
  • Ability to balance technical expertise with creative problem-solving

What We Offer:

  • Competitive salary and equity package
  • Flexible work environment with remote work options
  • Opportunity to work on a cutting-edge AI research assistant platform
  • Collaborative and dynamic team environment

How to Apply:

If you are a motivated and experienced data engineer looking to join a dynamic team, please submit your application, including your resume and a cover letter, to [insert contact information].

  • Data Engineer

    2 days ago

    Oakland, California, United States Rose International Full time

    {"header": "Job Title: Data Engineer", "subheader": "Join Rose International as a Data Engineer and contribute to the development of innovative data solutions.", "description": "As a Data Engineer at Rose International, you will be responsible for designing and implementing robust data pipelines to integrate various data sources, including OSIsoft PI...

  • Data Quality Engineer Senior

    3 days ago

    Oakland, California, United States Blue Shield of California Full time

    About the RoleWe are seeking a highly skilled Senior Data Quality Engineer to join our Data Services Quality Engineering team at Blue Shield of California. As a key member of our team, you will be responsible for ensuring the quality and integrity of our data assets.Key ResponsibilitiesDevelop and execute test scenarios and scripts to validate data quality...

  • Data Quality Engineer Senior

    3 days ago

    Oakland, California, United States Blue Shield of California Full time

    About the RoleBlue Shield of California is seeking a highly skilled Senior Data Quality Engineer to join our Data Services Quality Engineering team. As a key member of our team, you will be responsible for ensuring the quality and integrity of our data assets.Key ResponsibilitiesDevelop and execute test scenarios and scripts to validate data quality and...

  • Data Engineer

    3 weeks ago

    Oakland, California, United States Rose International Full time

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at Rose International. As a Data Engineer, you will play a critical role in designing and implementing robust data pipelines to integrate various data sources, including OSIsoft PI systems, into a centralized analytics platform.Key Responsibilities:Design and implement...

  • Data Quality Engineer, Senior

    3 days ago

    Oakland, California, United States Blue Shield of California Full time

    About the RoleWe are seeking a highly skilled Data Quality Engineer, Senior to join our team at Blue Shield of California. As a key member of our Data Services Quality Engineering team, you will play a critical role in ensuring the quality and integrity of our data.Key ResponsibilitiesDevelop and execute test scenarios and scripts to validate data...

  • Data Quality Engineer Senior

    2 weeks ago

    Oakland, California, United States Blue Shield of California Full time

    About the RoleWe are seeking a highly skilled Data Quality Engineer Senior to join our team at Blue Shield of California. As a key member of our Data Services Quality Engineering team, you will play a critical role in ensuring the quality and integrity of our data.Key ResponsibilitiesDevelop and execute test scenarios and scripts to validate data...

  • Data Quality Engineer Senior

    3 days ago

    Oakland, California, United States Blue Shield of California Full time

    About the RoleWe are seeking a highly skilled Data Quality Engineer Senior to join our team at Blue Shield of California. As a key member of our Data Services Quality Engineering team, you will play a critical role in ensuring the quality and integrity of our data.Key ResponsibilitiesDevelop and execute test scenarios and scripts to validate data quality and...

  • Data Engineer

    2 days ago

    Oakland, California, United States Rose International Full time

    Job SummaryRose International is seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing and implementing robust data pipelines to integrate various data sources, including OSIsoft PI systems, into a centralized analytics platform.Key ResponsibilitiesDesign and implement data pipelines to integrate...

  • Data Engineer

    2 days ago

    Oakland, California, United States Diverse Lynx Full time

    Job SummaryData Engineers at Diverse Lynx LLC design, implement, and maintain large-scale data processing systems. This role involves developing and managing ETL processes, generating dashboards and visualizations, and ensuring data security and governance. The ideal candidate will have experience with Microsoft Azure and Databricks, as well as strong...

  • Data Architect

    2 days ago

    Oakland, California, United States Axelon Full time

    Data EngineerAt Axelon, we're seeking a skilled Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing and implementing robust data pipelines to integrate various data sources into a centralized analytics platform. Your expertise in OSIsoft PI, SQL, and NoSQL databases will be instrumental in...

  • Senior Network Engineer

    1 month ago

    Oakland, California, United States MGT Full time

    About the RoleMGT Consulting Group is a national leader in public sector management consulting and services, delivering diverse business consulting services to a wide range of public sector groups. Our mission is to impact communities for good.We are seeking a Senior Network Engineer to assist with a data center migration project, requiring expertise in Data...

  • Data Scientist

    2 weeks ago

    Oakland, California, United States V2 Innovations Full time

    Job Title: Data EngineerWe are seeking a highly skilled Technical Specialist with expertise in Engineering or Computer Sciences to join our team at V2 Innovations.Key Responsibilities:Proficiency in Python and developing REST APIs.Strong experience in Python, SQL, Scala, or R for large scale data analysis.Familiarity with Spark for data processing.Previous...

  • Staff Data Scientist

    1 week ago

    Oakland, California, United States Credit Karma Full time

    Champion Operational Excellence in Data ScienceCredit Karma is a mission-driven company that champions financial progress for our members globally. We're looking for a Staff Data Scientist - Data Science Ops to drive operational excellence across organizations, leveraging AI-powered innovation and collaboration.Key Responsibilities:Design and maintain...

  • Financial Data Analyst

    2 days ago

    Oakland, California, United States Dolls Kill Full time

    About the Role:We're seeking a skilled Financial Data Analyst to join our team at Dolls Kill. As a key member of our data-driven organization, you'll play a crucial role in managing complex projects, collaborating with cross-functional teams, and leveraging data to inform business decisions.Key Responsibilities:Develop and maintain a deep understanding of...

  • Data Architect

    1 month ago

    Oakland, California, United States Resource Informatics Group Full time

    Job Title: Tableau ArchitectWe are seeking a highly skilled Tableau Architect to join our team at Resource Informatics Group. As a key member of our data analytics team, you will be responsible for designing and implementing a centralized Business Intelligence platform that will serve all groups and users in the department.Key Responsibilities:Design and...

  • Financial Data Analyst

    4 weeks ago

    Oakland, California, United States Dolls Kill Full time

    About the RoleDolls Kill is a global fashion brand that empowers individuals to express their unique style. With over 10 million social media followers, we're a leader in the fashion industry. Our team is growing rapidly, and we're seeking a skilled Financial Data Analyst to join our crew.Key ResponsibilitiesDevelop and maintain a deep understanding of the...

  • Data Architect

    2 weeks ago

    Oakland, California, United States Resource Informatics Group Full time

    Job Title: Tableau ArchitectWe are seeking a skilled Tableau Architect to join our team at Resource Informatics Group for a multi-year engagement.Role:As a Tableau Architect, you will lead the creation of a strategic enterprise data architecture, partnering with internal stakeholders to define principles, standards, and guidelines for data flow, aggregation,...

  • Data Architect

    3 weeks ago

    Oakland, California, United States Saxon Global Full time

    Job Title: Tableau ArchitectJob Summary:We are seeking a highly skilled Tableau Architect to lead the creation of our strategic enterprise data architecture. As a key member of our team, you will partner with internal stakeholders to define the principles, standards, and guidelines for data flow, data aggregation, migration, data curation, data model, data...

  • SAP Data Lead

    4 weeks ago

    Oakland, California, United States PG&E Corporation Full time

    Job Title: Principal SAP Data LeadPG&E Corporation is seeking a highly skilled Principal SAP Data Lead to join our team. As a key member of our Information Systems Technology Services organization, you will be responsible for defining how data will be stored, consumed, integrated, and managed by different IT systems.Key Responsibilities:Oversee all Data...

  • SAP Data Architect

    1 month ago

    Oakland, California, United States Finezi Technologies Pvt Ltd Full time

    Job Title: SAP Data LeadJob Summary:Finezi Technologies Pvt Ltd is seeking a highly skilled SAP Data Lead to join our team. As a key member of our organization, you will be responsible for defining how data will be stored, consumed, integrated, and managed by different IT systems.Key Responsibilities:Oversee all Data activities during the detailed design,...