Data Engineer

1 week ago

San Francisco, United States DynPro Inc. Full time

Job Title: Data Engineer

Location: Downtown San Francisco, CA / Remote

Type: Full-time/Contract

Job Summary:

We are seeking a highly skilled Data Engineer to join our client's team. This role involves designing, building, and maintaining robust data solutions. You will utilize a diverse set of technologies and tools to enhance our data infrastructure and enable high-performance analytics across the organization. The ideal candidate has experience with cloud data warehousing, ETL pipeline development, workflow orchestration, data governance, and master data management.


Key Responsibilities:

  • Develop scalable and cost-effective data warehouse solutions in Snowflake.
  • Leverage AWS cloud infrastructure to ensure optimal performance and scalability.
  • Design, develop, and deploy high-performance data pipelines using tools like dbt™, Fivetran, and Matillion Cloud ETL.
  • Ensure data pipelines are robust, maintainable, and scalable.
  • Utilize Apache Airflow to orchestrate and automate data workflows.
  • Ensure efficient scheduling and execution of data tasks.
  • Develop and implement data privacy policies, including data encryption, anonymization, and access controls using techniques like masking.
  • Establish and enforce data governance frameworks covering data classification, lineage, quality, and retention.
  • Design and build highly redundant and scalable cloud infrastructure for data lakes and data labs using AWS services such as EMR, SageMaker, and S3.
  • Manage and optimize data storage and processing solutions.
  • Develop Python scripts for data enrichment and MDM.
  • Integrate data from various B2B sources to maintain a consistent and accurate master dataset.
  • Build and maintain CI/CD pipelines to streamline deployment processes.
  • Ensure smooth integration of new features and updates into production environments.
  • Work closely with cross-functional teams, including data scientists and analysts, to meet data requirements.
  • Lead projects from conception to production, breaking down tasks and managing implementation effectively.


Preferred Technical Stack:

  • Cloud Data Warehouse: Snowflake
  • ETL Tools: Matillion Cloud ETL, Fivetran, dbt™ Cloud
  • Programming Languages: Python, SQL
  • Workflow Orchestration: Apache Airflow
  • Cloud Platforms: AWS (including EMR, SageMaker, S3, Datalake)
  • Infrastructure as Code: Terraform
  • Big Data Technologies: Apache Spark, Hudi
  • Data Storage: NoSQL databases like MongoDB and DynamoDB
  • Data Visualization: Tableau
  • Data Engineering and Operations: Data Marts, Data Warehousing, Data Governance, Data Privacy, MLOps


Qualifications:

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
  • Proven experience in data engineering, with a strong focus on cloud-based data solutions.
  • Proficiency in Python and SQL.
  • Hands-on experience with Snowflake and AWS services.
  • Strong knowledge of ETL tools like Matillion, Fivetran, and dbt™.
  • Experience with workflow orchestration tools like Apache Airflow.
  • Familiarity with data governance and data privacy best practices.
  • Experience with CI/CD pipeline creation and management.
  • Excellent problem-solving skills and attention to detail.
  • Strong communication and leadership skills.

  • Senior Data Engineer

    2 months ago

    San Francisco, United States People Data Labs Full time

    Job DescriptionJob DescriptionAbout UsAt People Data Labs, we're committed to democratizing access to high-quality B2B data and leading the emerging DaaS economy. We empower developers, engineers, and data scientists to create innovative, compliant data products at scale with our clean, easy-to-use datasets of resume, company, location, and education...

  • Data Engineer

    4 weeks ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! At our company, we're revolutionizing how businesses leverage data to drive insights and make informed decisions. Our mission is to develop scalable and robust data infrastructure that empowers organizations to unlock the full potential of their data. Join...

  • Data Engineer

    3 months ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! At our company, we're revolutionizing how businesses leverage data to drive insights and make informed decisions. Our mission is to develop scalable and robust data infrastructure that empowers organizations to unlock the full potential of their data. Join...

  • Data Engineer

    3 months ago

    San Francisco, United States Legalist Full time

    Job DescriptionJob DescriptionIntro description:Legalist is an institutional alternative asset management firm. Founded in 2016 and incubated at Y Combinator, the firm uses data-driven technology to invest in credit assets at scale. We are always looking for talented people to join our team.As a highly collaborative organization, our data engineers work...

  • Data Engineer

    1 week ago

    San Francisco, United States DynPro Inc. Full time

    Job Title: Data EngineerLocation: Downtown San Francisco, CA / RemoteType: Full-time/Contract Job Summary: We are seeking a highly skilled Data Engineer to join our client's team. This role involves designing, building, and maintaining robust data solutions. You will utilize a diverse set of technologies and tools to enhance our data infrastructure and...

  • Data Engineer

    4 weeks ago

    San Francisco, United States Quantum Leap Healthcare Full time

    Job DescriptionJob DescriptionJob DescriptionPosition: Data EngineerDepartment: Data InformaticsReports to: Chief Data Officer SUMMARY AND DETAILS OF POSITIONQuantum Leap Healthcare Collaborative’s (QLHC) Bioinformatics team provides strategic planning, integrating, execution, build and oversight of clinical trial deliverables. The Bioinformatics group...

  • Data Engineer

    3 days ago

    San Francisco, United States mavinsys Full time

    Hello We are from Mavinsys Talent Acquisition team based on One World Trade Centre New York. We are specializing in IT services and staffing majorly in lateral hiring/contract. Below is one of our requirement to fill immediately if youre interested please share your candidature to Job Title: Data Engineer Location: San Francisco CA Duration: 12months Job...

  • Data Engineer

    3 days ago

    San Francisco, United States Forsyth Barnes Full time

    Permanent Job Alert: Title: Data Engineer Location: San Francisco, California Salary: Up to $140,000 Industry: FinTech Contact: thomas.harris@forsythbarnes.com Brief Overview: As a Data Engineer at a start-up FinTech company in California, you will play a critical role in building and maintaining the data infrastructure that drives our innovative financial...

  • Data Engineer

    2 days ago

    San Francisco, United States Forsyth Barnes Full time

    Permanent Job Alert: Title: Data Engineer Location: San Francisco, California Salary: Up to $140,000 Industry: FinTech Contact: thomas.harris@forsythbarnes.com Brief Overview: As a Data Engineer at a start-up FinTech company in California, you will play a critical role in building and maintaining the data infrastructure that drives our innovative financial...

  • Data Engineer

    4 weeks ago

    San Francisco, United States Ingrain Systems Inc Full time

    Job Tittle : Azure Cloud Data EngineerExperience: 6+ YearsJob DescriptionSenior Azure Cloud Data Engineer with a strong background in Power BI, Azure Data Factory (ADF) and SQL. The candidate will be responsible for designing, developing, and implementing data integration solutions in the Azure cloud environment. Design and implement ETL processes using...

  • Senior Data Engineer

    1 month ago

    San Francisco, United States Unreal Gigs Full time

    Join us and be part of a dynamic team shaping the future of data engineering.If you're a seasoned engineer with expertise in data engineering technologies and a passion for building robust data systems, we want you on our team.Establish and enforce data governance policies and best practices to ensure data quality, security, and compliance with...

  • Senior Data Engineer

    1 month ago

    San Francisco, California, United States Unreal Gigs Full time

    Join us and be part of a dynamic team shaping the future of data engineering.If you're a seasoned engineer with expertise in data engineering technologies and a passion for building robust data systems, we want you on our team.Establish and enforce data governance policies and best practices to ensure data quality, security, and compliance with regulatory...

  • Lead Data Engineer

    4 weeks ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building scalable and reliable data infrastructure that enables advanced analytics, machine learning,...

  • Lead Data Engineer

    3 months ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building scalable and reliable data infrastructure that enables advanced analytics, machine learning,...

  • Data Engineer/Scientist

    6 days ago

    San Francisco, United States Freemind Solutions Full time

    Founding Data Engineer / Scientist Location: SF - Bay Area / Los Angeles Role type: Full time Experience: 5+ years Role We are seeking an exceptional Founding Data Engineer/Scientist to join our core team and shape the data-driven foundation of our company. This unique role will blend the responsibilities of data engineering and data science,...

  • Data Engineer

    5 months ago

    San Francisco, California, United States iTvorks Inc Full time

    Required Skills Data engineer main skills Good knowledge of Data warehousing with expertise in data modeling and ETL. Advanced skills in SQL and HQL Should be good at these core tech AWS/EMR/Spark Hadoop Hive Vertica Qlikview Tableau Should have exp in CI/CD

  • Senior Data Engineer

    4 weeks ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building scalable and reliable data infrastructure that enables advanced analytics, machine learning,...

  • Senior Data Engineer

    3 months ago

    San Francisco, United States Unreal Gigs Full time

    Job DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to harnessing the power of data to drive transformative change and solve complex problems across industries. We're committed to building scalable and reliable data infrastructure that enables advanced analytics, machine learning,...

  • Data Engineer

    4 weeks ago

    San Francisco, United States Infactory Full time

    Job DescriptionJob DescriptionAbout InfactoryWe’re Infactory. Our mission is to further AI innovation through facts and trust. Our fact platform powers AI applications for businesses that depend on accuracy. Whether our customers are building chatbots, search tools, knowledge management systems, or bleeding-edge Gen AI technology, our solution ensures...

  • Data Engineer

    1 week ago

    San Francisco, United States Infactory Full time

    Job DescriptionJob DescriptionAbout InfactoryWe’re Infactory. Our mission is to further AI innovation through facts and trust. Our fact platform powers AI applications for businesses that depend on accuracy. Whether our customers are building chatbots, search tools, knowledge management systems, or bleeding-edge Gen AI technology, our solution ensures...