Platform Data Engineer

2 weeks ago

San Jose, United States Bayone Full time

Location: San Francisco, Seattle, LA or PST
Enterprise Data Platform to enable timely, effective and safe sharing of data to multiple engineering, operations and business teams for building world class data products

Responsibilities
Build data ingestion and processing pipelines to enable data analytics and data science use-cases in areas of digital commerce, service operations, charging, reliability, finance, capex, warranty, customer service and others.
Build modular set of data services using Python, SQL, AWS Glue, lambdas, API Gateway, Kafka, data build tool (dbt), Apache Spark on EMR among others
Build automated unit and integration testing pipelines using frameworks like PySpark
Create and manage CICD pipelines with Gitlab CI and AWS Code Pipeline/CodeDeploy
Automate and schedule jobs using Managed Airflow
Build the ODS and reporting schemas and load the data into AWS Redshift or Snowflake
Design and build data quality management services with Apache Deequ and data observability tools like Splunk, DataDog , CloudWatch
Provide a variety of query services with REST, Athena/Presto, server sent events
Configure and setup the enterprise data lineage and meta data management and data catalog support using tools like Collibra/Alation
Assist the data scientist within the data engineering team as well as other software engineering teams with data cleansing, wrangling and feature engineering
Ensure green builds for deployment and work with program management and senior leads to burn down planned deliverables in a sprint cycle

Qualifications
At least 5+ years building data and analytics platforms using AWS Cloud, Python and SQL
Knowledge of AWS technologies specifically MSK, EMR, Athena, Glue, lambdas, API Gateway as well as Python, SQL is a must
Knowledge of modern data tools like dbt (data build tool) and Airflow orchestration is highly desired
Ability to assist SQL analysts and Tableau developers in business teams in creating the right set of materialized views in a SQL data warehouse like Redshift/Snowflake
Knowledge of automation and CICD best practices
Familiarity with machine learning and data science ecosystems especially AWS Sagemaker and Databricks is highly preferred
Hands-on experience in building and maintaining production data applications, current experience in both relational and distributed columnar date stores.
Deep experience using SQL, Python, and SparkHands-on experience with Big-data technologies (e.g. Redshift, Athena, Glue, EMR, Kinesis, Step Function, or equivalent in other web services)
Familiarity with timeseries database, data streaming applications, Kafka, Flink, and more is a plus

Familiarity with modern data science and product analytics tools and techniques such as R, Machine Learning, and advanced statistics is a plus.

  • Data Platform Engineer

    2 weeks ago

    San Jose, United States Cypress HCM Full time

    Job DescriptionJob DescriptionData Platform EngineerWe have an exciting opportunity for a Data Platform Engineer with the top leading multimedia and creative software company in the world. Responsibilities:Setup and maintain production scale Databricks environment on public cloud such as Microsoft Azure and AWSSetup and maintain production scale data storage...

  • Data Platform Engineer

    2 days ago

    San Jose, United States Cypress Human Capital Management, LLC Full time

    We have an exciting opportunity for a Data Platform Engineer with the top leading multimedia and creative software company in the world. Responsibilities Setup and maintain production scale Databricks environment on public cloud such as Microsoft Azure and AWS Setup and maintain production scale data storage such as ADLS and S3 for multiple tenant teams...

  • Data Platform Engineer

    4 days ago

    San Jose, California, United States Ipro Networks Pte. Ltd. Full time

    Job Title:Data Platform EngineerPosition Type:4-month contractLocation:Remote within the USASalary Range: $60-$62/hr. on w2 (This position does not accept C2C)Job Summary:- Setup and maintain Databricks environment on public clouds like Microsoft Azure and AWS- Configure and manage data storage such as ADLS and S3 for multiple tenant teams on the Data...

  • Senior Data Platform Engineer

    1 month ago

    San Jose, United States Mendel.ai Full time

    About Mendel: At Mendel, we are enabling different stakeholders in healthcare to make better decisions by learning from every patient's journey— from a physician deciding on the best treatment for a patient to a pharma company discovering the next blockbuster drug. In creating depth and breadth for a patient’s health record journey, Mendel enables the...

  • Senior Data Platform Engineer

    3 weeks ago

    San Jose, United States Mendel.ai Full time

    About Mendel: At Mendel, we are enabling different stakeholders in healthcare to make better decisions by learning from every patient's journey— from a physician deciding on the best treatment for a patient to a pharma company discovering the next blockbuster drug. In creating depth and breadth for a patient’s health record journey, Mendel enables the...

  • Senior Data Platform Engineer

    1 month ago

    San Jose, United States Mendel.ai Full time

    About Mendel: At Mendel, we are enabling different stakeholders in healthcare to make better decisions by learning from every patient's journey— from a physician deciding on the best treatment for a patient to a pharma company discovering the next blockbuster drug. In creating depth and breadth for a patient’s health record journey, Mendel enables the...

  • Senior Data Platform Engineer

    3 weeks ago

    San Jose, United States Mendel.ai Full time

    About Mendel: At Mendel, we are enabling different stakeholders in healthcare to make better decisions by learning from every patient's journey— from a physician deciding on the best treatment for a patient to a pharma company discovering the next blockbuster drug. In creating depth and breadth for a patient’s health record journey, Mendel enables the...

  • Data Platform Engineer

    2 weeks ago

    San Jose, United States Intelliswift Software Full time

    Job Title: Data Platform Engineer (Remote) Duration: 4 Months Location: Remote Pay Rate: $62.32. Intelliswift Software Inc. conceptualizes, builds, and supports the world's most amazing technology products and solutions. Our team of rich experts from diverse backgrounds contributes to making Intelliswift one of the most reliable partners in IT and Talent...

  • Data Platform Engineer:

    2 weeks ago

    San Jose, United States Akraya Full time

    Primary Skills: AWS, Azure, Databricks, Kubernetes, Monitoring. Duration: 4+ Months (Possible Extension) Contract Type: W2 Only Location: San Jose, CA Pay Range:$57.00 - $60.00 Per hour on w2 #DP “Work where you’re valued and paid what you’re worth” JOB RESPONSIBILITIES: We are seeking a dedicated professional to establish and manage a...

  • Engineering Manager, Platform

    5 days ago

    San Francisco, California, United States People Data Labs Full time

    Job Description Job Description About UsWe are the provider of high-quality B2B people and company data. At People Data Labs, we're leading the emerging DaaS economy. We empower developers, engineers, and data scientists to create innovative, compliant data products at scale with our clean, easy-to-use datasets of resume, company, location, and education...

  • Tech Lead Software Engineer for Data Platform

    1 month ago

    San Jose, United States HireIO Inc Full time

    Team Introduction: Our mission in experimentation and evaluation team is to build the next-gen A/B testing platform, that empowers the company to make data-driven decision for the products. The supported scenarios include recommendation, push, ads, search, mobile app, UI interaction and service upgrades etc. Our platform's capabilities cover the entire...

  • Tech Lead Software Engineer for Data Platform

    2 months ago

    San Jose, United States Hireio, Inc. Full time

    Job DescriptionJob DescriptionTeam Introduction:Our mission in experimentation and evaluation team is to build the next-gen A/B testing platform, that empowers the company to make data-driven decision for the products. The supported scenarios include recommendation, push, ads, search, mobile app, UI interaction and service upgrades etc. Our platform's...

  • Tech Lead Software Engineer for Data Platform

    3 weeks ago

    San Jose, United States Hireio, Inc. Full time

    Job DescriptionJob DescriptionTeam Introduction:Our mission in experimentation and evaluation team is to build the next-gen A/B testing platform, that empowers the company to make data-driven decision for the products. The supported scenarios include recommendation, push, ads, search, mobile app, UI interaction and service upgrades etc. Our platform's...

  • Data Platform Engineer

    1 week ago

    San Francisco, United States Equilibrium Energy Full time

    About our Company Equilibrium Energy is a well-funded, Series A clean energy startup backed by some of the most prominent institutional investors in climate. We are building a digital native power company operating at the intersection of grid variability, market volatility, economic optimization, commercial structuring, and risk management, across the...

  • Hadoop Platform Engineer

    2 months ago

    San Jose, United States RAMPS International Inc. Full time

    Job DescriptionJob DescriptionHadoop Platform EngineerSan Jose, CA4-12 MonthsPosition summaryThe Data and Analytics Platform team resides within Adobe's Information and Data Services team and we are looking for a Hadoop Platform Engineer who will be responsible for the implementation and ongoing administration of Hadoop infrastructure including...

  • Hadoop Platform Engineer

    3 weeks ago

    San Jose, United States RAMPS International Inc. Full time

    Job DescriptionJob DescriptionHadoop Platform EngineerSan Jose, CA4-12 MonthsPosition summaryThe Data and Analytics Platform team resides within Adobe's Information and Data Services team and we are looking for a Hadoop Platform Engineer who will be responsible for the implementation and ongoing administration of Hadoop infrastructure including...

  • Sr. Platform Engineer

    1 week ago

    San Jose, United States Stellar Cyber, Inc. Full time

    ENGINEERING- Stellar Cyber, Inc. in San Jose, CA seeks Sr. Platform Engineer: Design and implement features to enrich data in data pipeline to make data useful for security detections. Part-time telecom-muting allowed. $184,662/yr.- $194,662/yr. Email res (must reference Job Code # 43543) to tbanerjee@stellarcyber.ai.

  • Engineer, Data Platform

    4 weeks ago

    San Francisco, CA, United States Pinecone Full time

    About Pinecone Pinecone is on a mission to build the search and database technology to power AI applications for the next decade and beyond. Our fully managed vector database makes it easy to add vector search to AI applications. Since creating the “vector database” category, demand has grown incredibly fast and it shows in our user base. We are a...

  • Senior Full-stack Engineer, Data Platform

    4 weeks ago

    San Jose, United States Tik Tok Full time

    Responsibilities TikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok has global offices including Los Angeles, New York, London, Paris, Berlin, Dubai, Singapore, Jakarta, Seoul and Tokyo. Why Join Us At TikTok, our people are humble, intelligent, compassionate and creative. We create to...

  • Senior Full-stack Engineer, Data Platform

    1 week ago

    San Jose, United States Tik Tok Full time

    Responsibilities TikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok has global offices including Los Angeles, New York, London, Paris, Berlin, Dubai, Singapore, Jakarta, Seoul and Tokyo. Why Join Us At TikTok, our people are humble, intelligent, compassionate and creative. We create to...