Data Engineer

1 month ago


New York, United States TechnoGen Full time
Data Engineer

Remote

Snowflake Expertise:

Design, develop, and optimize data pipelines within the Snowflake data platform. Implement best practices for Snowflake data modeling, warehouse architecture, security and performance optimization.
Proficient in using Snowflake Snowpipe streaming for real-time data streaming and ingestion and different batch load ingestion.

AWS Proficiency:

Utilize AWS services for data integration, storage, and processing. Hands-on experience with AWS Data Migration Service (DMS) for seamless and secure data migration.
Implement and manage streaming data solutions using AWS Kinesis. Design and develop ETL processes using AWS Glue, MSK for efficient data transformation.

Airflow Workflow Orchestration:

Implement and maintain data workflows using Apache Airflow. Ensure the reliability and scalability of data workflows for efficient data processing. Data Aggregation and Transformation:
Develop and implement data aggregation strategies for large-scale datasets. Collaborate with data scientists and analysts to understand data requirements and ensure data quality.

Architecture Design:

Collaborate with cross-functional teams to design and implement scalable and reliable data architecture. Ensure adherence to best practices for data security, privacy, and compliance.

Qualifications:
•o Minimum - 7 to 10 years of Data Engineering experience

o Strong Experience with Snowflake, AWS and ETL

o Hands-on experience with snowflake security, roles, encryption, masking, replication, snowpipe streaming.

o Hands-on experience with AWS Data Migration Service (DMS), Apache AirFlow, MSK, Kinesis Firehose for seamless and secure data migration.

o Hands-on experience with RESTapi, java script, Python, Scala, SQL and Java.

o Computer skills - a thorough knowledge of computer programming, coding, and various operating and database systems is a must

o Time management - developers should have the ability to quickly develop data warehousing systems and solve any issues to ensure the continued accuracy of business data

o Creativity - the ability to create mappings from scratch often calls for strong creative skills on the part of a developer

o Analytical thinking -Developers should be able to analyze data needs and options and understand the needs of various clients

o Troubleshooting - when data warehousing systems are down, Data engineer should be quickly assess the problem and provide a solution

o Team collaboration - Data engineer rarely work alone; they typically interact closely with database managers and other IT specialists when maintaining, storing, and retrieving da

  • New York, United States Hudson Data Full time

    Job Title: Sr. Azure Data Engineer Location: NYC, NY Duration: Long Term W2 Role Job Description: The candidate should have hands-on experience with various Azure services including but not limited to Azure Data Factory, Databricks, Data Lake, and Power BI. They should be able to design, build, and maintain ETL pipelines, manage data lakes, and...


  • New York, New York, United States Hudson Data Full time

    Hudson Data is seeking a highly skilled Cloud Data Engineer Expert to join our team.We are looking for an experienced professional with hands-on knowledge of various Azure services, including but not limited to Azure Data Factory, Databricks, Data Lake, and Power BI. The ideal candidate should be able to design, build, and maintain ETL pipelines, manage data...


  • New York, New York, United States Hudson Data Full time

    Hudson Data is a cutting-edge technology company that is seeking a highly skilled Azure Data Engineering Lead to join our team.We are looking for an experienced professional with hands-on knowledge of various Azure services, including but not limited to Azure Data Factory, Databricks, Data Lake, and Power BI. The ideal candidate should be able to design,...

  • Kdb+ Data Engineer

    5 days ago


    New York, New York, United States Data Intellect Full time

    Job OverviewData Intellect is a data and technology consultancy firm that specializes in financial and capital markets technology solutions. We're seeking an experienced Kdb+ Developer to join our team.Salary:$80,000 - $110,000 per annum (dependent on experience)Key ResponsibilitiesWork with business users to define requirements for new application...


  • New York, New York, United States The Data Sherpas Full time

    Job Summary:We are seeking an experienced GCP Data Science Engineering Lead to join our team at The Data Sherpas. As a GCP Data Science Engineering Lead, you will be responsible for driving technology innovation by delivering world-class machine learning solutions on Google Cloud Platform (GCP).Key Responsibilities:Lead the design and deployment of...


  • New York, New York, United States Simon Data Full time

    About UsSIMON DATA is a forward-thinking customer experience orchestration platform designed to disrupt the marketing technology and marketing cloud category. Our unique approach empowers businesses to develop one-to-one relationships with their customers without building a bespoke in-house data infrastructure.At SIMON, we believe that business success...


  • New York, New York, United States Simon Data Full time

    About UsSIMON Data is an enterprise customer data platform empowering marketers to create personalized data-driven experiences for their customers. Founded in 2015 by a team of successful serial entrepreneurs, we're a scrappy problem-solving company that believes in tackling big challenges with disruptive thinking and giving our customers the support they...


  • New York, New York, United States PitchBook Data Full time

    OverviewPitchBook Data is a leading provider of data and analytics solutions. We're seeking an experienced Technical Product Lead to join our team.Job DescriptionWe're looking for a highly skilled Technical Product Lead to oversee the integration of AI/ML technologies into our data collection processes. The ideal candidate will have a strong background in...


  • New York, New York, United States Data Intellect Full time

    About Data IntellectWe're dedicated to harnessing the power of human intellect and collaboration to drive innovative solutions for complex challenges.Our approach is guided by the principles of Humility, Can-do Attitude, and Togetherness, which form the foundation of our [DI] code.Compensation Package: $150,000 - $200,000 per annum, commensurate with...


  • New York, New York, United States People Data Labs Full time

    Company OverviewPeople Data Labs simplifies data collection and standardization so our customers can focus on building innovative data solutions. We integrate thousands of compliant datasets into a single source of truth.This role involves taking ownership of person data products and driving innovation in new data solutions. The ideal candidate will have...


  • New York, New York, United States People Data Labs Full time

    Job DescriptionAbout Us: At People Data Labs, we do the heavy lifting of data collection and standardization, allowing our customers to focus on building innovative, compliant data solutions.You will own our person data, which represents a significant portion of our customer base, and work cross-functionally in a fast-paced environment to champion a shared...


  • New York, United States VAST Data Full time

    VAST Data is looking for a Senior Systems Engineer to join our growing team! This is a great opportunity to be part of one of the fastest-growing infrastructure companies in history, an organization that is in the center of the hurricane being created by the revolution in artificial intelligence. "VAST's data management vision is the future of the...

  • Senior Data Analyst

    5 days ago


    New York, New York, United States Data Intellect Full time

    Job OverviewData Intellect is a dynamic data and technology consultancy firm, specializing in financial and capital markets technology solutions. We are currently seeking an experienced Senior Data Analyst to join our team.About the RoleWe are looking for a highly skilled Senior Data Analyst with extensive experience in kdb+ development and analysis of...

  • Data Engineer

    2 weeks ago


    New York, United States Mitchell Martin Full time

    Title: Data Engineer II Location: Remote Description: The Data Engineer II will play a vital role in designing and building scalable, efficient, and robust data pipelines to support reporting and analytics needs. The role involves working with modern data engineering technologies to create solutions that feed various analytics platforms, ensuring alignment...


  • New York, New York, United States People Data Labs Full time

    About UsWe do the heavy lifting of data collection and standardization at People Data Labs, allowing our customers to focus on building innovative data solutions. Our sole focus is on integrating thousands of compliantly sourced datasets into a single developer-friendly source of truth.This is your chance to join us at a pivotal moment as we bring on a new...

  • Data Engineer

    2 weeks ago


    New York, United States Vorkis LLC Full time

    About the job Data Engineer About Company provides cutting-edge trade surveillance technology that protects investors, enhances transparency, and ensures regulatory compliance across traditional financial assets and crypto markets. The Role Were looking for a strong Senior Software Engineer with Data Engineering experience. Someone who is proficient in...

  • Data Engineer

    4 weeks ago


    New York, United States Glow Networks Full time

    Role: Data Engineer Location: New York City, NY (Hybrid-3 days a week onsite) Duration: 6 Months with possible extensions Contract: C2C/ W2 Job Requirements: Deep hands-on knowledge in big data engineering in a cloud environment, including: Excellent knowledge of SQL in a large scale data warehouse or data lakehouse environment such as Spark, Databricks,...

  • Data Engineer

    1 month ago


    New York, United States TechnoGen Full time

    Data Engineer Remote Snowflake Expertise: Design, develop, and optimize data pipelines within the Snowflake data platform. Implement best practices for Snowflake data modeling, warehouse architecture, security and performance optimization. Proficient in using Snowflake Snowpipe streaming for real-time data streaming and ingestion and different batch load...

  • Data Engineer

    1 month ago


    New York, United States TechnoGen Full time

    Data Engineer Remote Snowflake Expertise: Design, develop, and optimize data pipelines within the Snowflake data platform. Implement best practices for Snowflake data modeling, warehouse architecture, security and performance optimization. Proficient in using Snowflake Snowpipe streaming for real-time data streaming and ingestion and different batch load...

  • Data Engineer

    3 weeks ago


    New York, United States WeWork Full time

    About UsAt WeWork, we provide flexible workspace solutions to help businesses of all sizes thrive in more than 37 countries worldwide. The future of work is here, and we‘re not just a part of that movement-we‘re leading it. United by a desire to build a global community, our purpose is to give members the support to do their best work, and the space to...