Sr. Data Engineer
3 days ago
Job Title: Sr. Data Engineer
Job Duration: 6 months contract
Job Location: San Francisco, CA, Hybrid (San Fran)/2 days a week between Tues through Thurs
Our Client is looking for an experienced Data Engineer to support the Marketing Solutions execution of complex data development and database efficiency, with a deep focus on improving the data quality, accuracy and timeliness for marketing related products. This role will interact directly with Engineering teams to optimize data operations for product metrics to surface insights at terabyte scale, globally. As our data and business needs increase, we continue to iterate and evolve our processes, best-practice documentation, and deep-dive retrospectives to inform our go/no go decision making. Ultimately to be successful, the individual should feel comfortable advising a team of data professionals, navigating our tech stack, and recommending solutions to complex data problems. Our team is highly collaborative, open and honest, and brings a low-ego, high impact mindset to work
Responsibilities:
Design, develop, and manage data pipelines and workflows to enable efficient and accurate data processing using Trino SQL/Spark SQL warehoused in HDFS datasets.
Effectively performs code designs and reviews/approves test cases.
Implement data quality checks and audits to maintain high data accuracy and integrity.
Produces elegant and efficient designs, high performance, and scalable code that allows for easy extension to future needs.
Collaborate with cross-functional teams, especially data engineering, to understand data requirements and implement robust data solutions.
Work closely with data domain experts to gather data requirements, translate business needs into technical specifications, and communicate data insights effectively for sales representative workflow efficiency.
Optimize data storage for performance and scalability, ensuring efficient data Extraction, Transformation and Load (ETL).
Develop and maintain documentation related to data pipelines, QA, metrics, and data policy as it relates to best practice, compliance and GDPR.
Stay up to date with industry best practices and emerging trends in data engineering and analytics, including Generative AI as it impacts our data operations.
Qualifications:
2+ years in using SQL and experience optimizing SQL databases for performance (Trino SQL, or Spark).
Demonstrated experience in managing data pipelines (like HDFS), data repository (like GitHub), workflows (like Apache Airflow), and ETL (best practice coding).
Ability to communicate complex technical concepts to both technical and non-technical individuals.
Experience working with multiple stakeholders, setting project priorities and delivering on Objectives and Key Results (OKRs).
Experience automating script changes in Python.
Preferred Qualifications:
BA/BS in engineering, computer science, or related technical field (such as statistics, or data science).
Excellent analytical skills, designing data workflows and analyzing data for anomalies, or setting data quality thresholds via automated solutions.
Familiarity with data governance principles
Program Manager experience
Demonstrated experience in managing data pipelines in HDFS
Experience running a scrum team and using Jira.
Spark SQL
Suggested Skills:
Data Analysis
Project Management
Data Engineering
-
Sr. Data Engineer
4 weeks ago
San Francisco, United States SGF Global Full timeJob Title: Sr. Data Engineer Job Title: Sr. Data Engineer (Remote for California based team) California -USA SGF Global is looking for a Sr. Data Engineer for our client based in California USA. Requisitions: You're a talented, creative, and motivated engineer who loves developing powerful, stable, and intuitive apps - and you're excited to...
-
Sr. Data Engineer
5 days ago
San Francisco, United States SGF Global Full timeJob Title: Sr. Data Engineer Job Title: Sr. Data Engineer (Remote for California based team) California -US SGF Global is looking for a Sr. Data Engineer for our client based in California USA. Requisitions: You're a talented, creative, and motivated engineer who loves developing powerful, stable, and intuitive apps - and you're excited to...
-
Sr. Data Engineer
1 month ago
San Francisco, United States Windfall Full timeWindfall is seeking a Sr. Data Engineer to join our core data team. As a Sr. Data Engineer on our core data team, you will be building out the core data asset that everything else at Windfall is built on top of. Communication and collaboration are at the heart of Windfall, and you will work closely with our product, data science, and other engineering teams....
-
Sr. Data Engineer
1 day ago
San Francisco, United States Smart IT Frame LLC Full timeJob Title: Sr. Data EngineerLocation: San Francisco, CA (Hybrid)Work Type: Full TimeJob Description:-Create and maintain optimal data pipeline architecture Build data pipelines that transform raw, unstructured data into formats that data analyst can use to for analysis Assemble large, complex data sets that meet functional / non-functional business...
-
Sr. Data Engineer
1 day ago
San Francisco, United States Smart IT Frame LLC Full timeJob Title: Sr. Data EngineerLocation: San Francisco, CA (Hybrid)Work Type: Full TimeJob Description:-Create and maintain optimal data pipeline architecture Build data pipelines that transform raw, unstructured data into formats that data analyst can use to for analysis Assemble large, complex data sets that meet functional / non-functional business...
-
eTeam | Sr. Data Engineer
2 weeks ago
san francisco, United States eTeam Full timeJob Title: Sr. Data Engineer Job Duration: 6 months contractJob Location: San Francisco, CA, Hybrid (San Fran)/2 days a week between Tues through ThursOur Client is looking for an experienced Data Engineer to support the Marketing Solutions execution of complex data development and database efficiency, with a deep focus on improving the data quality,...
-
Sr. Backend Engineer
14 hours ago
San Francisco, United States Hashicorp Full timeAbout Engineering at HashiCorpOn the HashiCorp engineering team, we build the Infrastructure Cloud which allows enterprises to take a unified approach to Infrastructure and Security Lifecycle Management:Infrastructure Lifecycle Management: Build / Deploy / ManageTerraform allows you to use infrastructure as code to provision and manage any infrastructure...
-
Sr. Data Scientist
2 days ago
San Francisco, United States Abacus Full timeTitle Data Analyst-III Location HQ, USA, CA, San Francisco, 2nd St Description Job Title: Sr. Data Scientist Job Description: Utilize quantitative analysis to understand business drivers and conduct analytics experiments for hypothesis testing. Leverage statistical techniques to provide business recommendations and insights into marketplace dynamics and...
-
Data Engineering Lead
7 days ago
San Francisco, California, United States YO HR CONSULTANCY Full timeWe are seeking a highly skilled Sr Data Engineer to join our team in Glendale, CA. The ideal candidate will have 7+ years of experience in data engineering and proficiency in at least one major programming language (e.g. Python, Java, Scala).The successful candidate will work with technologies such as Airflow, Spark, Databricks, and Snowflake, and contribute...
-
Data Architect and Engineer
2 weeks ago
San Francisco, California, United States Windfall Full timeAbout the JobWe're looking for a talented Sr. Data Engineer to build out the core data asset that everything else at Windfall is built on top of. As a key member of our core data team, you'll design and implement scalable data pipelines, collaborate with cross-functional teams, and leverage cutting-edge technologies like GCP, Java, Python, and Kotlin.Job...
-
Sr Data Entry Specialist
3 weeks ago
San Francisco, United States Robert Half Full timeJob DescriptionJob DescriptionWe are offering a short term contract employment opportunity for a Sr Data Entry Specialist in San Francisco, California. As a Sr Data Entry Specialist, you will be primarily focused on data entry and processing, maintaining both electronic and physical filing systems, troubleshooting data-related issues, and upholding the...
-
San Francisco, United States Amazon Full timeSr. Software Engineer, Data Plane, NPD Forwarding Stack, Data Plane Job ID: 2776438 | Amazon Data Services, Inc. AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, we’re the people who keep the cloud running. We support all AWS data centers and all of the servers, storage,...
-
Senior QA Engineer
2 weeks ago
San Francisco, California, United States Altruist Full time/>We're in the midst of an exceptional growth phase and we're excited to hire a Sr. QA Engineer to join our growing Data team.
-
San Francisco, United States Amazon Full timeSr. Software Engineer, Data Plane, NPD Forwarding Stack, Data PlaneJob ID: 2776438 | Amazon Data Services, Inc.AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, we’re the people who keep the cloud running. We support all AWS data centers and all of the servers, storage,...
-
San Francisco, California, United States Braintrust Data Full timeAbout Braintrust DataWe are a cutting-edge developer platform for building world-class AI products. Our innovative approach combines code and datasets, incrementally refining both using frequent evaluations. We provide a rich set of tools to visualize changes and interrogate failures, empowering developers to integrate our platform into their continuous...
-
Sr Data Entry Specialist
3 weeks ago
San Francisco, United States Robert Half Full timeJob DescriptionJob DescriptionWe are on the lookout for a Sr Data Entry Specialist to become a part of our team. This role is based in San Francisco, California, and offers a short term contract employment opportunity. As a Sr Data Entry Specialist, you will play a crucial role in maintaining and managing data across various database systems, ensuring the...
-
Data Platform Systems Architect
3 weeks ago
San Francisco, California, United States VAST Data Full timeAbout VAST DataVAST Data is a pioneering company in the field of data management, revolutionizing the way enterprises capture, catalog, refine, enrich, and protect massive datasets. As a leader in enterprise software infrastructure, we empower real-time data analysis and AI training and inference.We are driven by innovation, customer-centricity, and a...
-
Sr Data Entry Specialist
3 weeks ago
San Francisco, United States Robert Half Full timeJob DescriptionJob DescriptionWe are in search of a Sr Data Entry Specialist to join our team in San Francisco, California. In this role, you will be tasked with managing data entry tasks, maintaining the integrity of our databases, and resolving any data-related issues. Your key responsibilities will also include maintaining an organized filing system and...
-
Sr Data Entry Specialist
3 weeks ago
San Francisco, United States Robert Half Full timeJob DescriptionJob DescriptionWe are offering a short term contract employment opportunity in San Francisco, California, for a Sr Data Entry Specialist to join our dynamic team. This role primarily involves managing and processing customer data in a variety of database systems.Responsibilities:• Accurately input and upkeep data in several database...
-
Sr. Data Engineer
2 weeks ago
San Francisco Bay Area, United States Flexton Inc. Full timePay Rate: $80/hr to $90/hr on W2Type: ContractMust Haves:JavaPythonShellResponsibilities: -Design, develop, and manage data pipelines and workflows to enable efficient and accurate data processing using Trino SQL/Spark SQL warehoused in HDFS datasets. -Effectively performs code designs and reviews/approves test cases. -Implement data quality checks and...