Data Engineer

2 days ago

San Francisco, California, United States Career Mentors Full time
Job description

Location: San Francisco, CA and Jersey City, NJ
Work Mode: Hybrid (3 Days Onsite / 2 Days Remote)
Employment Type: W2 Only (No C2C or 1099)
Candidate Requirement: Local candidates preferred (must be located near San Francisco or Jersey City)

As an AWS Data Engineer, you will be a key member of our data engineering team, responsible for building and managing scalable data solutions on the AWS platform. This role requires strong expertise in AWS technologies, big data frameworks, and data pipeline orchestration. You will work closely with data scientists, analysts, and business stakeholders to ensure data availability, quality, and governance.

Data Onboarding: Integrate diverse data sources into the AWS-based data lake with consistency and reliability.

  • Pipeline Development: Design and develop scalable data pipelines using services like AWS Lambda, Step Functions, and EMR.

  • Metadata Management: Register data assets and manage metadata to enhance data discoverability.

  • Data Quality: Implement validation checks and data transformation processes to maintain high data integrity.

  • Governance & Compliance: Ensure data practices align with security, privacy, and compliance standards.

  • Infrastructure as Code: Automate and manage cloud infrastructure using Terraform.

  • Big Data Processing: Utilize Apache Spark to process large-scale datasets efficiently.

  • Workflow Orchestration: Manage and automate data workflows using Airflow and AWS Step Functions.

  • Data Modeling: Work with Snowflake and Iceberg table formats to design optimized storage schemas.

  • Team Collaboration: Collaborate across departments to gather requirements and deliver robust data solutions.

Job requirements

  • Proven experience in AWS data engineering using Lake Formation, Lambda, EMR, and related services.

    • Strong proficiency in Python and Terraform scripting.

    • Familiarity with data orchestration tools such as Airflow and dbt.

    • Hands-on experience with Snowflake, RDS, Jupyter, and Iceberg formats.

    • Expertise in ETL design, data pipelines, and data warehousing concepts.

    • Understanding of data governance frameworks and best practices.

    • Strong problem-solving skills and ability to troubleshoot complex data systems.

    • Excellent communication and teamwork skills.

    Must-Have Skills

    • Cloud/Data Engineering: AWS (Lake Formation, Lambda, EMR, Step Functions, EC2, EKS)

    • Big Data & Processing: Spark, Data Lakes, Data Pipelining

    • Programming & Scripting: Python, Terraform

    • Orchestration & Workflow: Airflow, dbt

    • Data Modeling & Warehousing: Snowflake, Iceberg table formats

    • Tools: Jupyter Notebook, RDS

    Preferred Qualifications

    • Certifications such as:

      • AWS Certified Data Analytics – Specialty

      • AWS Certified Solutions Architect

    • Experience working in fast-paced, data-centric environments

    TopTech Talent is proud to be an equal opportunity workplace and is an affirmative action employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, age, national origin, citizenship status, disability, protected veteran status, gender identity or any other factor protected by applicable federal, state, or local laws.

    Third-party recruiters, please do not reach out for this role.

All done

Your application has been successfully submitted

Other jobs

  • Data Engineer

    1 week ago

    San Francisco, California, United States Zapier Full time

    About ZapierWe're humans who simply think computers should do more work.At Zapier, we're not just making software—we're building a platform to help millions of businesses globally scale with automation and AI. Our mission is to make automation work for everyone by delivering products that delight our customers. You'll collaborate with brilliant people, use...

  • Data Engineer

    4 days ago

    San Francisco, California, United States InterSources Inc Full time

    Job Title:AI Data EngineerDuration:3-6 Months CTHExperience:Minimum 12+ YearsJob Description:Design, build, and maintain large-scale data pipelines and architectures to support AI and machine learning initiatives.Collaborate with data scientists, AI engineers, and analytics teams to ensure seamless data flow for model development and deployment.Develop and...

  • Data Engineer

    4 days ago

    San Francisco, California, United States Air Apps Full time

    About Air AppsAt Air Apps, we believe in thinking bigger—and moving faster. We're a family-founded company on a mission to create the world's first AI-powered Personal & Entrepreneurial Resource Planner (PRP), and we need your passion and ambition to help us change how people plan, work, and live. Born in Lisbon, Portugal in 2018—and now with offices in...

  • Data Engineer

    2 weeks ago

    San Francisco, California, United States Sigma Full time $140,000 - $155,000 per year

    About The RoleWe're hiring our first Data Engineer within Tech Operations at Sigma. In this role, you'll build the data foundation that powers critical insights across Engineering and Tech Operations. You will architect, scale, and optimize data models and pipelines across Snowflake and Databricks, fueling everything from internal decision-making to...

  • Data Engineer

    4 days ago

    San Francisco, California, United States Stefanini Group Full time

      Job DescriptionStefanini Group is hiringStefanini is looking for a Data Engineer for various location across USA (Hybrid).For quick apply, please connect with Prakhar Goel: / W2 Candidates OnlyPosition SummaryAs a Data Engineer, this CW will be responsible for collecting, parsing, managing, analyzing, and visualizing large sets of data to turn...

  • Data Engineer

    2 weeks ago

    San Francisco, California, United States Kikoff Full time

    ResponsibilitiesWe are looking for a Data Engineer or Analytics Engineer to join our Data team. You will collaborate with the data scientist and engineers to design, build, and scale high-leverage data models, foundational datasets and scalable infrastructure that enables analytics, modeling, and experimentation. Your responsibilities includes:Build and...

  • Data Engineer

    1 week ago

    San Francisco, California, United States Haystack Full time

    We're working with Cognizant on this role.Azure Data EngineerSan Francisco, HybridAbout the RoleCognizant is looking for an experienced Azure Data Engineer with strong skills across Azure Databricks, Azure Synapse Analytics, PySpark, and Azure Data Factory. You will design and implement scalable data solutions that support advanced analytics and enterprise...

  • Data Engineer

    1 week ago

    San Francisco, California, United States Mercor Full time

    About MercorMercor is at the intersection of labor markets and AI research. We partner with leading AI labs and enterprises to provide the human intelligence essential to AI development.Our vast talent network trains frontier AI models in the same way teachers teach students: by sharing knowledge, experience, and context that can't be captured in code alone....

  • Platform Data Engineer

    2 days ago

    San Francisco, California, United States Neon Redwood Full time

    About Neon RedwoodNeon Redwood is a data services consulting company, working on cutting-edge AI and data-driven solutions. We are a team of passionate engineers and data experts, and we are currently looking for a Data Engineer to join our team and help us develop and expand our data infrastructure and analytics capabilities.The RoleWe are seeking an...

  • Sr. Data Engineer

    2 days ago

    San Francisco, California, United States Neon Redwood Full time

    About Neon RedwoodNeon Redwood is a data services consulting company, working on cutting-edge AI and data-driven solutions. We are a team of passionate engineers and data experts, and we are currently looking for a Senior Data Engineer to join our team and help us develop and expand our data infrastructure and analytics capabilities.The RoleWe are seeking an...