Data Engineer

14 hours ago


New York, United States Resiliency Full time

We are seeking a highly skilled Data Engineer with previous hands-on experience in the Financial or Banking industry, capable of hitting the ground running and quickly building scalable data solutions. The ideal candidate will have a strong background in Azure Cloud and will work within a cross-functional Agile team of 15 people. This role requires someone who can collaborate closely with stakeholders to understand business requirements, particularly for capturing revenue data for brokers. The engineer will also work with compliance and security teams to ensure all systems meet regulatory standards. They will be responsible for building a flexible data capture system while maintaining BAU (Business as Usual) functionality. This is a dynamic, fast-paced environment where the ability to deliver high-quality solutions quickly is critical.

Job Details

Here is the job requirement for a Data Engineer position with expertise in Python ETL and Azure experience. We are seeking a highly skilled and motivated individual who can contribute to our team and help drive our data engineering initiatives.

Job Requirements

1. Python ETL: The ideal candidate should have strong proficiency in Python programming language and experience in Extract, Transform, Load (ETL) processes. They should be able to design and develop efficient ETL workflows to extract data from various sources, transform it as per business requirements, and load it into target systems.

2. Azure Experience: The candidate should have hands-on experience working with Microsoft Azure cloud platform. They should be familiar with Azure services and tools, and have a good understanding of Azure architecture and best practices.

3. Azure Data Factory, DataBricks, Azure Storage, and Azure VM: The candidate should have practical experience with Azure Data Factory, DataBricks, Azure Storage, and Azure Virtual Machines. They should be able to design and implement data pipelines using Azure Data Factory, perform data transformations and analytics using DataBricks, and manage data storage and virtual machines in Azure.

4. Data Governance, Data Quality, and Controls: The candidate should have a strong understanding of data governance principles, data quality management, and data controls. They should be able to implement data governance frameworks, establish data quality standards, and ensure compliance with data regulations and policies.

5. Implementing alerts and notifications for batch jobs: The candidate should have experience in setting up alerts and notifications for batch jobs. They should be able to configure monitoring and alerting mechanisms to ensure timely identification and resolution of issues in batch job execution.

  • New York, United States Hudson Data Full time

    Job Title: Sr. Azure Data Engineer Location: NYC, NY Duration: Long Term W2 Role Job Description: The candidate should have hands-on experience with various Azure services including but not limited to Azure Data Factory, Databricks, Data Lake, and Power BI. They should be able to design, build, and maintain ETL pipelines, manage data lakes, and...


  • New York, United States VAST Data Full time

    VAST Data is looking for a Senior Systems Engineer to join our growing team! This is a great opportunity to be part of one of the fastest-growing infrastructure companies in history, an organization that is in the center of the hurricane being created by the revolution in artificial intelligence. "VAST's data management vision is the future of the...

  • Data Engineer

    4 weeks ago


    New York, United States Brooksource Full time

    Data EngineerLocation: New York, NY (Hybrid)Contract-to-HireWe’re looking for an experienced Data Engineer to join a Fortune 50 Broadcast Media & Entertainment leader. This role involves designing, building, and optimizing data pipelines, integrating multiple datasets, and creating applications that drive strategic business decisions. You’ll collaborate...

  • Data Engineer

    6 months ago


    New York, United States Silver Search, Inc Full time

    Data Engineer New York, NY Senior Data Engineer--Redshift/AWS/Python/Scripting---**Will be hybrid onsite in January 2022 onwards** Contract-to-hire Our client is a cutting edge E-Learning company based in NYC and is looking to hire a Senior Data Engineer to join their data and analytics center of excellence. He/She will be building...

  • Data Engineer

    3 weeks ago


    New York, United States Walsh Employment Full time

    Our client is a leading quantitative investment company with a focus on computer-driven trading for global financial markets. The company harnesses the power of data and advanced technologies to drive innovative strategies and superior investment returns through a platform that is driven by cutting-edge, cloud-based data solutions. We are now looking for a...

  • Data Engineer

    2 weeks ago


    New York, United States SimplifyVMS Full time

    As a Data Engineer, you will be responsible for building and optimizing data pipelines, architectures, and datasets for effective data analysis and machine learning applications. Responsibilities: Developing, constructing, testing, and maintaining architectures for data ingestion, processing, and extraction. Designing and implementing data pipelines to...

  • Data Engineer

    1 day ago


    New York, United States TechnoGen Full time

    Data Engineer Remote Snowflake Expertise: Design, develop, and optimize data pipelines within the Snowflake data platform. Implement best practices for Snowflake data modeling, warehouse architecture, security and performance optimization. Proficient in using Snowflake Snowpipe streaming for real-time data streaming and ingestion and different batch load...

  • Data Engineer

    14 hours ago


    New York, United States TechnoGen Full time

    Data Engineer Remote Snowflake Expertise: Design, develop, and optimize data pipelines within the Snowflake data platform. Implement best practices for Snowflake data modeling, warehouse architecture, security and performance optimization. Proficient in using Snowflake Snowpipe streaming for real-time data streaming and ingestion and different batch load...

  • Data Engineer

    12 hours ago


    New York, United States TechnoGen Full time

    Data Engineer Remote Snowflake Expertise: Design, develop, and optimize data pipelines within the Snowflake data platform. Implement best practices for Snowflake data modeling, warehouse architecture, security and performance optimization. Proficient in using Snowflake Snowpipe streaming for real-time data streaming and ingestion and different batch load...

  • Data Engineer

    2 days ago


    New York, United States Glow Networks Full time

    Role: Data EngineerLocation: New York City, NY (Hybrid-3 days a week onsite)Duration: 6 Months with possible extensionsContract: C2C/ W2Job Requirements: Deep hands-on knowledge in big data engineering in a cloud environment, including: Excellent knowledge of SQL in a large scale data warehouse or data lakehouse environment such as Spark, Databricks,...

  • Data Engineer

    4 weeks ago


    New York, United States hackajob Full time

    Exciting Opportunity Alert! Join our Team as a Data Engineer!hackajob has partnered with an innovative company seeking a skilled Data Engineer to leverage data for impactful solutions. If you have expertise in big data tools like Spark and Kafka, and experience with cloud services like AWS, we want to hear from YOU!Role: Data EngineerLocation: Hybrid (New...

  • Data Engineer

    6 months ago


    New York, United States Capstone Investment Advisors Full time

    RESPONSIBILITIES: Be part of Data Engineering team responsible for all market data in the firmDesign and implement data-ingestion and data-processing pipelines and tools for a diverse range of datasets using Python, Linux and AWS Cloud platform.Work with variety of backend database storage systems including MS SQL Server, AWS Redshift, Cassandra,...

  • Data Engineer

    2 days ago


    New York, United States BEHAVIORALLY, Inc. Full time

    Title: Data Engineer MUST: Hybrid in New York City, NYNo Sponsorship provided.Full-Time position Minimum Qualifications: The role is ideal for someone who thrives in a fast-paced, cutting-edge technology environment. Proven Experience with data modeling and developing Enterprise Data Warehouse solutions - OLTP, OLAP, Dimensions, Facts, and Data modeling....

  • Data Engineer

    2 weeks ago


    New York, United States Atlas Search Full time

    ResponsibilitiesCloud Data Engineer is responsible for building and maintaining data pipelines within an AWS cloud environment. Work heavily with MongoDB and Snowflake as well as Cypress, Selenium and Kafka. Requirements3+ years of experience in the Data Engineering space working heavily with Public cloud solutions.Experience with MongoDB or...


  • New York, United States Tek Leaders Inc Full time

    Title: Healthcare Data Engineer Duration: 12 Months Location: 100% Remote (NYC NY) Essential skills for the role: Minimum of 35 years of experience in Healthcare Domain Minimum of 10 years of experience in Data Engineering. Required technical skills: Python PySpark SQL AWS Required Domain: Healthcare/Insurance Candidate is expected to have...

  • Data Engineer

    6 days ago


    New York, United States Robert Half Full time

    Job DescriptionJob DescriptionWe are offering an exciting opportunity for a Data Engineer in NEW YORK, New York. This role involves working in the industry, specifically focusing on AWS Technologies, Python, NoSQL, and Snowflake. The workplace is dynamic and requires someone who can efficiently handle data-related tasks. Responsibilities:• Build efficient...

  • Data Engineer

    6 months ago


    New York, United States Jane Street Full time

    About the PositionWe are looking for a Data Engineer who can help us understand, clean, manage, and share the data that guides our trading. At Jane Street, having a thorough and accurate understanding of data is at the core of the work we do.Using our mix of in-house and open-source software, you will analyze datasets gathered from a variety of sources,...

  • Data Engineer

    2 days ago


    New York, United States City of New York Full time

    Company DescriptionJob Description The New York City Taxi and Limousine Commission (TLC) is the City agency responsible for regulating for-hire transportation in New York City, including taxis, street hail liveries, high-volume for-hire services such as Uber and Lyft, black cars, luxury limousines, livery vehicles, commuter vans, and paratransit vehicles....

  • Data Engineer

    1 day ago


    New York, United States Open Systems Technologies Full time

    A financial firm is looking for a Data Engineer to join their team in New York, NY. Pay: $120-125/hr Hybrid; 2 days per week onsite Top Skills: Databricks, Python, SQL, Spark, AWS, Trino Must be a US Citizen Responsibilities: • Design, develop, monitor, and maintain data pipelines in an AWS ecosystem with Databricks, Delta Lake, Python, SQL and Starburst...


  • New York, United States The Data Sherpas Full time

    J ob Summary : As a GCP Machine Learning Engineer, you will drive technology innovation by delivering world-class machine learning solutions. You will integrate GCP-native and third-party technologies to build scalable data infrastructures, focusing on modernizing data pipelines and machine learning platforms. You will collaborate with implementation teams...