Data Engineering Expert
2 days ago
The Custom Group of Companies is seeking a highly skilled Data Engineering Expert to join our team. In this role, you will design, develop, and maintain data pipelines in Databricks on an AWS 6 ecosystem with Delta Lake, Python, SQL, and Starburst as the technology stack.
As a Senior Data Engineer, you will be responsible for establishing data quality checks, ensuring data integrity and accuracy throughout the data lifecycle, automating testing of the data pipelines, optimizing data processing and query performance for large-scale datasets within AWS and Databricks environments, documenting data engineering processes, architecture, and configurations, troubleshooting and debugging data-related issues on the AWS Databricks platform, and integrating Databricks with other AWS products such as SNS, SQS, and MSK.
Responsibilities:
- Design, develop, and maintain data pipelines in Databricks on an AWS 6 ecosystem.
- Establish data quality checks and ensure data integrity and accuracy throughout the data lifecycle.
- Automate testing of the data pipelines and configure as part of CICD.
- Optimize data processing and query performance for large-scale datasets within AWS and Databricks environments.
- Document data engineering processes, architecture, and configurations.
- Troubleshoot and debug data-related issues on the AWS Databricks platform.
- Integrate Databricks with other AWS products such as SNS, SQS, and MSK.
-
Cloud Data Engineer Expert
2 days ago
New York, New York, United States Hudson Data Full timeHudson Data is seeking a highly skilled Cloud Data Engineer Expert to join our team.We are looking for an experienced professional with hands-on knowledge of various Azure services, including but not limited to Azure Data Factory, Databricks, Data Lake, and Power BI. The ideal candidate should be able to design, build, and maintain ETL pipelines, manage data...
-
Azure Data Engineering Lead
7 days ago
New York, New York, United States Hudson Data Full timeHudson Data is a cutting-edge technology company that is seeking a highly skilled Azure Data Engineering Lead to join our team.We are looking for an experienced professional with hands-on knowledge of various Azure services, including but not limited to Azure Data Factory, Databricks, Data Lake, and Power BI. The ideal candidate should be able to design,...
-
Data Engineering Expert
15 hours ago
New York, New York, United States Inizio Partners Full timeAbout Inizio PartnersWe provide data-driven solutions to business problems through statistical data mining and cutting-edge analytics techniques. Our team of experts includes 2,000+ data scientists and analysts who assist client organizations with complex risk minimization methods and advanced marketing strategies.Job OverviewWe are seeking a talented Data...
-
Data Engineering Expert
2 days ago
New York, New York, United States Glow Networks Full timeJob Title: Data Engineering ExpertWe are seeking a seasoned Data Engineering Expert to join our team at Glow Networks in New York City. This is a 6-month contract position with possible extensions, offering a salary of $140,000 per year.Job Description:The successful candidate will have expertise in designing and implementing big data architecture in cloud...
-
Data Engineer Expert
2 days ago
New York, New York, United States Capital One Services, LLC Full timeCompany OverviewAt Capital One Services, LLC, we are seeking a talented Data Engineer Expert to join our team. As a leading financial institution, we are committed to innovation and excellence in data engineering.
-
Data Engineer Expert
1 day ago
New York, New York, United States Lubin Talent Solutions Full timeJob Title: Data Engineer ExpertAbout Us: At Lubin Talent Solutions, we are a dynamic team of professionals dedicated to delivering innovative solutions in the field of data engineering. Our expertise spans across designing and implementing efficient ETL workflows, ensuring data governance principles, and providing financial services experience.Role Summary:...
-
Data Engineering Expert
18 hours ago
New York, New York, United States Tech Tammina Full timeJob Title: Data Engineering ExpertLocation: New YorkSalary: $180,000 - $220,000 per yearAbout the Role:The ideal candidate should have extensive experience in IT and data engineering, particularly with GCP in the financial sector. They should be proficient in managing and optimizing data storage, ensuring data security, and using Python and SQL for data...
-
Data Engineering Expert
16 hours ago
New York, New York, United States Bloomberg Full timeJob SummaryWe are seeking a highly motivated and experienced Data Engineering Expert to join our team at Bloomberg. As a key member of our data management team, you will be responsible for developing strategies to optimize the value of Equity Corporate Actions data for our clients and improve data operations.In this role, you will be diving deep into complex...
-
Data Engineering Expert
3 weeks ago
New York, New York, United States Infojini Full time**Job Description:**We are seeking a highly skilled Data Engineering Expert to join our team at Infojini. As a key member of our data engineering team, you will be responsible for designing and implementing scalable data pipelines that meet the needs of our business.**Key Responsibilities:**Create optimal data pipeline architecture using best practices and...
-
Data Engineering Expert
2 weeks ago
New York, New York, United States Prophecy Full timeAbout ProphecyWe're a data copilot company that helps Fortune 500 enterprises accelerate AI and analytics by delivering clean, trusted, and timely data.Our customers rely on us to develop, deploy, and observe data pipelines on cloud data platforms. We make all data users productive by helping them create, manage, and monitor massive volumes of data for AI...
-
Data Engineer Cloud Expert
2 days ago
New York, New York, United States Futran Tech Solutions Pvt. Ltd. Full timeJob Title: Data Engineer Cloud ExpertAbout Us:Futran Tech Solutions Pvt. Ltd. is a leading provider of data engineering services, helping organizations unlock the full potential of their data.Salary: $120,000 - $180,000 per year based on experience and qualifications.Location: New York City, NY (day one onsite)Job Description:We are seeking an experienced...
-
Data Engineering Expert for Databricks and AWS
3 weeks ago
New York, New York, United States The Custom Group of Companies Full timeWe are seeking a seasoned Data Engineering expert to join our team at The Custom Group of Companies. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines in Databricks on an AWS 6 ecosystem with Delta Lake, Python, SQL, and Starburst as the technology stack.In this role, you will collaborate with...
-
Enterprise Engineering Senior DB2 Expert
20 hours ago
New York, New York, United States Enterprise Engineering Full timeJoin us at Enterprise Engineering as a Senior DB2 DBA and take on a challenging role that requires expertise, creativity, and a drive to deliver exceptional results.About the PositionThis position involves providing expert-level support for our clients' DB2 environments, ensuring they meet their business objectives while maintaining optimal performance and...
-
Speed and Scalability Expert
18 hours ago
New York, New York, United States Amazon Full timeAt Amazon, we're committed to delivering exceptional customer experiences. To achieve this, we're seeking a skilled data engineer to join our Worldwide Delivery Experience team. As a key contributor, you'll play a vital role in designing, developing, and enhancing systems and pipelines that optimize processes in reporting, planning, and controlling...
-
Data Engineering Expert
1 day ago
New York, New York, United States Gen II Fund Services Full timeGen II Fund ServicesWe are seeking an experienced Data Engineer to join our team in New York. This is a challenging role that requires strong technical skills and excellent communication abilities.The ideal candidate will have a strong background in data engineering and experience working with cloud-based data platforms, such as Snowflake and dbt. The...
-
Data Engineer Expert
3 weeks ago
New York, New York, United States Capital One Full timeJoin Capital One as a Sr. Data EngineerWe are seeking an experienced Data Engineer to join our team at Capital One in Plano, Texas. As a Sr. Data Engineer, you will be responsible for designing, developing, testing, implementing, and supporting technical solutions in full-stack development tools and technologies.Responsibilities:Collaborate with Agile teams...
-
Big Data Analytics Expert
2 days ago
New York, New York, United States Early Warning Services Full timeAbout Our TeamWe are Early Warning Services, a leading provider of payment solutions. We are seeking a highly skilled Big Data Analytics Expert to join our team.The successful candidate will be responsible for developing, documenting, testing, debugging, and maintaining data engineering solutions for Paze.The ideal candidate will have expertise in software...
-
Data Architecture and Engineering Expert
7 days ago
New York, New York, United States Quanta Search Full time**Job Description**Quanta Search is looking for an experienced Data Architecture and Engineering Expert to design and implement scalable database infrastructure for their Algo Trading Strategies and Technology department. The ideal candidate will have a strong background in database administration and software development.**Key Responsibilities:**Design,...
-
GCP Data Science Engineering Lead
7 days ago
New York, New York, United States The Data Sherpas Full timeJob Summary:We are seeking an experienced GCP Data Science Engineering Lead to join our team at The Data Sherpas. As a GCP Data Science Engineering Lead, you will be responsible for driving technology innovation by delivering world-class machine learning solutions on Google Cloud Platform (GCP).Key Responsibilities:Lead the design and deployment of...
-
Cloud Data Engineer
17 hours ago
New York, New York, United States Diverse Lynx Full timeWe are seeking a highly skilled Cloud Data Engineer - Pyspark and Databricks Expert to join our team at Diverse Lynx LLC. This is an excellent opportunity to work on challenging projects and develop your skills in data engineering.About the RoleWe are looking for someone with at least 5 years of experience working as a Databricks developer and 3 years of...