Data Engineer

4 weeks ago


United, United States eTeam Full time

Databricks, Spark

Snowflake Cloud Data Platform

Data Vault 2.0 model

Enterprise Data Integrations

WS Cloud architecture

Event and messaging patterns, streaming data, AWS, Kafk

Role & Responsibilities:

Data Engineers will be responsible for design, build and maintain data pipelines ensuring data quality, efficient processing, and timely delivery of accurate and trusted data.

The ability to design, implement and optimize large-scale data and analytics solutions on Databricks, Spark, Snowflake Cloud Data Warehouse is essential.

Ensure performance, security, and availability of the data warehouse.

Establish ongoing end-to-end monitoring for the data pipelines.

Strong understanding of full CI/CD lifecycle.

Must Haves:

  • 2+ years of recent experience with Databricks / Spark / Snowflake and a total of 6+ years in data engineering role.
  • Designing and implementing highly performant data ingestion pipelines from multiple sources using spark and databricks.
  • Extensive working knowledge of Spark and Databricks
  • Demonstrable experience designing and implementing modern data warehouse/data lake solutions with an understanding of best practices.
  • Hands-on development experience with Snowflake data platform including Snowpipes, SnowSQL,tasks, stored procedures, streams, resource monitors, RBAC controls, virtual warehouse sizing, query performance tuning, cloning, time travel, data sharing and understanding how to use these features.
  • Advanced proficiency in writing complex SQL statements and manipulating large structured and semi-structured datasets.
  • Data loading/Unloading and Data sharing
  • Strog hands-on Experience on SNOWSQL queries, script preparation and stored procedures and performance tunning
  • Knowledge of SnowPipe implementation
  • Create Spark jobs for data transformation and aggregation
  • Produce unit tests for Spark transformations and helper methods
  • Security design and implementation on Databricks
  • Build processes supporting data transformation, data structures, metadata, dependency and workload management.
  • successful history of manipulating, processing and extracting value from large disconnected datasets.
  • Working knowledge of message queuing, stream processing, and scalable 'big data' data stores.
Good to Have:
  • Valid professional certification
  • Experience in Python/Pyspark/Scala/Hive Programming.
  • Excellent verbal and written communication and interpersonal skills
  • Confidence and agility in challenging times
Ability to work collaboratively with cross-functional teams in a fast-paced, team environment
  • Data Engineer

    2 weeks ago


    United, United States eTeam Full time

    Databricks, Spark Snowflake Cloud Data Platform Data Vault 2.0 model Enterprise Data Integrations WS Cloud architecture Event and messaging patterns, streaming data, AWS, Kafk Role & Responsibilities: Data Engineers will be responsible for design, build and maintain data pipelines ensuring data quality, efficient processing, and timely delivery...

  • Data Engineer

    2 months ago


    United States Rakirs Systems LLC Full time

    Job Title Data EngineerLocation Alpharetta GADuration 12 months (extendable)Experience 2 - 8+yrspay rate $47.26/hourJob DescriptionThe Senior/ Junior Data Admin Analyst will work in the Brokerage platform that develops and supports applications that is responsible for compliance and reporting. The title/role is synonomous with that of Senior &...

  • Data Engineer

    2 months ago


    United States Rakirs Systems LLC Full time

    Job Title Data EngineerLocation Alpharetta GADuration 12 months (extendable)Experience - 8+yrspay rate $47.26/hourJob DescriptionThe Senior/ Junior Data Admin Analyst will work in the Brokerage platform that develops and supports applications that is responsible for compliance and reporting. The title/role is synonomous with that of Senior & Junior Data...

  • DATA ENGINEER III

    3 weeks ago


    , RI, United States FM Global Full time

    FM Global is a leading property insurer of the world's largest businesses, providing more than one-third of FORTUNE 1000-size companies with engineering-based risk management and property insurance solutions. FM Global helps clients maintain continuity in their business operations by drawing upon state-of-the-art loss-prevention engineering and research;...

  • Data Engineer

    2 weeks ago


    Charlotte, NC, United States Synechron Inc. Full time

    We are At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron's progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud &...

  • Data Engineer III

    2 weeks ago


    Johnston, RI, United States FM Global Full time

    FM Global is a leading property insurer of the world's largest businesses, providing more than one-third of FORTUNE 1000-size companies with engineering-based risk management and property insurance solutions. FM Global helps clients maintain continuity in their business operations by drawing upon state-of-the-art loss-prevention engineering and research;...

  • Data Engineer

    2 weeks ago


    New York, NY, United States Synechron Inc. Full time

    We are At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron's progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud &...

  • Senior Data Engineer

    4 hours ago


    Rocky Hill, CT, United States ebbo Full time

    We're looking for this person to come into our Rocky Hill, CT office twice a week:Position Overview of Senior Data Engineer Ebbo is looking for a Senior Data Engineer to help build and support data infrastructure for the virtual data warehouse. This Senior Data Engineer will develop jobs to ingest, validate and verify incoming data from internal and external...


  • United States of America Venture Search Full time

    Software Engineer- Data Python New York 150k-175k USD base + bonus Our client, a leading quantitative investment company focused on computer-driven trading in global financial markets, is seeking a Software Engineer to join their Core Technology team.As a key player, the Software Engineer will contribute to building and scaling the company's central...

  • Staff Data Engineer

    2 weeks ago


    Boston, MA, United States State Street Corporation Full time

    Who we are looking for The State Street Security Architecture, Analytics & Fusion Engineering (SA 2 FE) team is looking for a Staff Data Engineer, Integrations Lead . The Fusion Analytics and Data Engineering team delivers models, insights, and tooling to help Cybersecurity teams make faster, more informed decisions as we work to secure State Street's...


  • Chicago, IL, United States CME Group Full time

    Description This role is hybrid requires to be 2 days on site in our Chicago office. This role does not allow to work outside of Illinois state. Position Overview: Data System Reliability Engineer (dSRE) CME Group: Where Futures Are Made CME Group is the world's leading and most diverse derivatives marketplace. But who we are goes deeper than...


  • United States of America Venture Search Full time

    Software Engineer- Data  Python  New York  150k-175k USD base + bonus  Our client, a leading quantitative investment company focused on computer-driven trading in global financial markets, is seeking a Software Engineer to join their Core Technology team. As a key player, the Software Engineer will contribute to building and scaling the...


  • United States Magellan Health Full time

    Interested in a fully remote opportunity as a Lead Data and Integration Engineer while working an innovative healthcare company where you can have an impact on people's lives? Apply today and come join us at Magellan Health as we empower people to lead healthier, more vibrant lives.we're looking for people with the following skill set for this role:In-depth...


  • United, United States ECRI Full time

    Job DetailsJob Location Remote - , PA Remote Type Fully Remote Position Type Full Time Education Level Bachelor's Degree Required Travel Percentage 0% Job Shift Day Job Category Healthcare DescriptionWHY CHOOSE ECRI? ECRI is an inspiring place to work. We share a common mission to help healthcare organizations make smart, compassionate, and ethical decisions...


  • Norristown, PA, United States Comcast Corporation Full time

    FreeWheel, a Comcast company, provides comprehensive ad platforms for publishers, advertisers, and media buyers. Powered by premium video content, robust data, and advanced technology, we’re making it easier for buyers and sellers to transact across all screens, data types, and sales channels. As a global company, we have offices in nine countries and can...


  • Austin, TX, United States State Street Corporation Full time

    State Street's Bionics team has the mission to explore, enable and exploit artificial intelligence, machine learning, natural language processing, computer vision and cognitive computing at scale for multiple use cases across Global Delivery, Global Advisors, Global Markets, and Global Exchange business lines. The team brings together a good mix of...

  • ML Engineer

    4 days ago


    Reading, PA, United States Penske Truck Leasing Full time

    Position Summary: As the Machine Learning Engineer working in Penske’s Advanced Analytics team, you will be in a high impact role. You will be supporting multiple businesses and functions within Penske with their data science initiatives. You will play a key role in maturing the AI/ML Ops at Penske organization. This is a great opportunity for...


  • Chantilly, VA, United States Amazon Data Services, Inc. Full time

    At Amazon, we are working to be the most customer-centric company on earth. To get there, we need exceptionally talented, bright, and driven people. As part of the AWS Controls Team, you will work with highly motivated experts and innovators in the data center industry. You will be responsible for innovating, deploying, optimizing, and maintaining the...

  • Data Analyst III

    2 weeks ago


    Pensacola, FL, United States Navy Federal Credit Union Full time

    OverviewPerform complex data analysis and data engineering solutions using Python, SQL, and Power BI, following a mature development methodology. Define requirements, perform data collection, processing, cleaning, analysis, modeling, and visualization. Deliver solutions which are accurate, reliable, and scalable, aligning with industry best practice and...


  • Boston, MA, United States Boston Consulting Group Full time

    WHAT YOU'LL DO The BCG Data Team is an integral part of BCG’s global operations. The team is responsible for developing and maintaining the BCG internal data platform. Our team consists of members from Boston, London, Munich, and New Delhi working together in a highly collaborative environment. We are responsible for setting the standards and guardrails...