Data Engineer
2 weeks ago
Databricks, Spark
Snowflake Cloud Data Platform
Data Vault 2.0 model
Enterprise Data Integrations
WS Cloud architecture
Event and messaging patterns, streaming data, AWS, Kafk
Role & Responsibilities:
Data Engineers will be responsible for design, build and maintain data pipelines ensuring data quality, efficient processing, and timely delivery of accurate and trusted data.
The ability to design, implement and optimize large-scale data and analytics solutions on Databricks, Spark, Snowflake Cloud Data Warehouse is essential.
Ensure performance, security, and availability of the data warehouse.
Establish ongoing end-to-end monitoring for the data pipelines.
Strong understanding of full CI/CD lifecycle.
Must Haves:
- 2+ years of recent experience with Databricks / Spark / Snowflake and a total of 6+ years in data engineering role.
- Designing and implementing highly performant data ingestion pipelines from multiple sources using spark and databricks.
- Extensive working knowledge of Spark and Databricks
- Demonstrable experience designing and implementing modern data warehouse/data lake solutions with an understanding of best practices.
- Hands-on development experience with Snowflake data platform including Snowpipes, SnowSQL,tasks, stored procedures, streams, resource monitors, RBAC controls, virtual warehouse sizing, query performance tuning, cloning, time travel, data sharing and understanding how to use these features.
- Advanced proficiency in writing complex SQL statements and manipulating large structured and semi-structured datasets.
- Data loading/Unloading and Data sharing
- Strog hands-on Experience on SNOWSQL queries, script preparation and stored procedures and performance tunning
- Knowledge of SnowPipe implementation
- Create Spark jobs for data transformation and aggregation
- Produce unit tests for Spark transformations and helper methods
- Security design and implementation on Databricks
- Build processes supporting data transformation, data structures, metadata, dependency and workload management.
- successful history of manipulating, processing and extracting value from large disconnected datasets.
- Working knowledge of message queuing, stream processing, and scalable 'big data' data stores.
- Valid professional certification
- Experience in Python/Pyspark/Scala/Hive Programming.
- Excellent verbal and written communication and interpersonal skills
- Confidence and agility in challenging times
-
Data Engineer
2 months ago
United States Rakirs Systems LLC Full timeJob Title Data EngineerLocation Alpharetta GADuration 12 months (extendable)Experience 2 - 8+yrspay rate $47.26/hourJob DescriptionThe Senior/ Junior Data Admin Analyst will work in the Brokerage platform that develops and supports applications that is responsible for compliance and reporting. The title/role is synonomous with that of Senior &...
-
Data Engineer
2 months ago
United States Rakirs Systems LLC Full timeJob Title Data EngineerLocation Alpharetta GADuration 12 months (extendable)Experience - 8+yrspay rate $47.26/hourJob DescriptionThe Senior/ Junior Data Admin Analyst will work in the Brokerage platform that develops and supports applications that is responsible for compliance and reporting. The title/role is synonomous with that of Senior & Junior Data...
-
DATA ENGINEER III
4 weeks ago
, RI, United States FM Global Full timeFM Global is a leading property insurer of the world's largest businesses, providing more than one-third of FORTUNE 1000-size companies with engineering-based risk management and property insurance solutions. FM Global helps clients maintain continuity in their business operations by drawing upon state-of-the-art loss-prevention engineering and research;...
-
Senior Data Engineer
1 day ago
Seattle, WA, United States GTT, LLC Full timeSenior Data EngineerContract Duration 6+ MonthsPay range - $70 to $85/hrJob Summary:Our mission is to leverage cutting-edge technology to design, build, and maintain rockets that are not only safe and reliable but also environmentally friendly.Our client is currently seeking a Senior Data Engineer to join our team and play a pivotal role in leveraging the...
-
Data Engineer
2 weeks ago
Charlotte, NC, United States Synechron Inc. Full timeWe are At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron's progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud &...
-
Data Engineer III
2 weeks ago
Johnston, RI, United States FM Global Full timeFM Global is a leading property insurer of the world's largest businesses, providing more than one-third of FORTUNE 1000-size companies with engineering-based risk management and property insurance solutions. FM Global helps clients maintain continuity in their business operations by drawing upon state-of-the-art loss-prevention engineering and research;...
-
Data Engineer
2 weeks ago
New York, NY, United States Synechron Inc. Full timeWe are At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron's progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud &...
-
Senior Data Engineer
2 days ago
Rocky Hill, CT, United States ebbo Full timeWe're looking for this person to come into our Rocky Hill, CT office twice a week:Position Overview of Senior Data Engineer Ebbo is looking for a Senior Data Engineer to help build and support data infrastructure for the virtual data warehouse. This Senior Data Engineer will develop jobs to ingest, validate and verify incoming data from internal and external...
-
Software Engineer- Data
1 month ago
United States of America Venture Search Full timeSoftware Engineer- Data Python New York 150k-175k USD base + bonus Our client, a leading quantitative investment company focused on computer-driven trading in global financial markets, is seeking a Software Engineer to join their Core Technology team.As a key player, the Software Engineer will contribute to building and scaling the company's central...
-
Staff Data Engineer
2 weeks ago
Boston, MA, United States State Street Corporation Full timeWho we are looking for The State Street Security Architecture, Analytics & Fusion Engineering (SA 2 FE) team is looking for a Staff Data Engineer, Integrations Lead . The Fusion Analytics and Data Engineering team delivers models, insights, and tooling to help Cybersecurity teams make faster, more informed decisions as we work to secure State Street's...
-
Senior Data Reliability Engineer
2 weeks ago
Chicago, IL, United States CME Group Full timeDescription This role is hybrid requires to be 2 days on site in our Chicago office. This role does not allow to work outside of Illinois state. Position Overview: Data System Reliability Engineer (dSRE) CME Group: Where Futures Are Made CME Group is the world's leading and most diverse derivatives marketplace. But who we are goes deeper than...
-
Software Engineer- Data
4 weeks ago
United States of America Venture Search Full timeSoftware Engineer- Data Python New York 150k-175k USD base + bonus Our client, a leading quantitative investment company focused on computer-driven trading in global financial markets, is seeking a Software Engineer to join their Core Technology team. As a key player, the Software Engineer will contribute to building and scaling the...
-
Lead Data and Integration Engineer
2 weeks ago
United States Magellan Health Full timeInterested in a fully remote opportunity as a Lead Data and Integration Engineer while working an innovative healthcare company where you can have an impact on people's lives? Apply today and come join us at Magellan Health as we empower people to lead healthier, more vibrant lives.we're looking for people with the following skill set for this role:In-depth...
-
United, United States ECRI Full timeJob DetailsJob Location Remote - , PA Remote Type Fully Remote Position Type Full Time Education Level Bachelor's Degree Required Travel Percentage 0% Job Shift Day Job Category Healthcare DescriptionWHY CHOOSE ECRI? ECRI is an inspiring place to work. We share a common mission to help healthcare organizations make smart, compassionate, and ethical decisions...
-
Principal Software Engineer
5 days ago
Norristown, PA, United States Comcast Corporation Full timeFreeWheel, a Comcast company, provides comprehensive ad platforms for publishers, advertisers, and media buyers. Powered by premium video content, robust data, and advanced technology, we’re making it easier for buyers and sellers to transact across all screens, data types, and sales channels. As a global company, we have offices in nine countries and can...
-
Data Science Software Engineer
2 weeks ago
Austin, TX, United States State Street Corporation Full timeState Street's Bionics team has the mission to explore, enable and exploit artificial intelligence, machine learning, natural language processing, computer vision and cognitive computing at scale for multiple use cases across Global Delivery, Global Advisors, Global Markets, and Global Exchange business lines. The team brings together a good mix of...
-
ML Engineer
5 days ago
Reading, PA, United States Penske Truck Leasing Full timePosition Summary: As the Machine Learning Engineer working in Penske’s Advanced Analytics team, you will be in a high impact role. You will be supporting multiple businesses and functions within Penske with their data science initiatives. You will play a key role in maturing the AI/ML Ops at Penske organization. This is a great opportunity for...
-
Sr. Controls Engineer, AMER Controls
1 month ago
Chantilly, VA, United States Amazon Data Services, Inc. Full timeAt Amazon, we are working to be the most customer-centric company on earth. To get there, we need exceptionally talented, bright, and driven people. As part of the AWS Controls Team, you will work with highly motivated experts and innovators in the data center industry. You will be responsible for innovating, deploying, optimizing, and maintaining the...
-
Data Analyst III
2 weeks ago
Pensacola, FL, United States Navy Federal Credit Union Full timeOverviewPerform complex data analysis and data engineering solutions using Python, SQL, and Power BI, following a mature development methodology. Define requirements, perform data collection, processing, cleaning, analysis, modeling, and visualization. Deliver solutions which are accurate, reliable, and scalable, aligning with industry best practice and...
-
Global IT Data Engineer Senior Specialist
2 weeks ago
Boston, MA, United States Boston Consulting Group Full timeWHAT YOU'LL DO The BCG Data Team is an integral part of BCG’s global operations. The team is responsible for developing and maintaining the BCG internal data platform. Our team consists of members from Boston, London, Munich, and New Delhi working together in a highly collaborative environment. We are responsible for setting the standards and guardrails...