Current jobs related to Data Engineer - New York - Randstad Digital

  • Data Engineer for Cloud Data Platforms

    1 month ago

    New York, New York, United States The Data Sherpas Full time

    About The Data SherpasWe are a team of highly skilled and motivated engineers who help our clients at every phase of their cloud journey. Our expertise spans cloud infrastructure, data engineering, and application development.Job DescriptionWe are seeking a self-motivated and data-driven engineer to join our team. As a Data Engineer for Cloud Data Platforms,...

  • Data Engineer for Cloud Data Platform

    1 month ago

    New York, New York, United States The Data Sherpas Full time

    Job Title: Data Engineer for Cloud Data PlatformThe Data Sherpas are seeking a highly skilled Data Engineer to join our team and help our clients at every phase of their cloud journey. As a Data Engineer, you will be responsible for designing, building, and maintaining scalable data platforms on cloud-based infrastructure.Responsibilities:Design and...

  • Data Engineer for Cloud Data Platforms

    3 weeks ago

    New York, New York, United States The Data Sherpas Full time

    About The Data SherpasWe are a team of highly skilled and motivated engineers who help our clients at every phase of their cloud journey.Our expertise spans cloud infrastructure, data engineering, and analytics, with a focus on Snowflake, FiveTran, and DBT.Job DescriptionWe are seeking a self-motivated and data-driven engineer to join our team and help our...

  • Senior Azure Data Engineer

    6 days ago

    New York, New York, United States Hudson Data Full time

    Job Title: Senior Azure Data EngineerLocation: NYC, NYDuration: Long Term W2 RoleJob Description:The ideal candidate should have hands-on experience with various Azure services, including Azure Data Factory, Databricks, Data Lake, and Power BI. They should be able to design, build, and maintain ETL pipelines, manage data lakes, and create insightful reports...

  • Data Engineer for Cloud Data Platforms

    2 weeks ago

    New York, New York, United States The Data Sherpas Full time

    Who We Are:The Data Sherpas are a team of highly skilled and motivated engineers who help our clients at every phase of their cloud journey. We have expertise in cloud, data, and applications, and we're passionate about delivering innovative solutions that drive business value.What We're Looking For:We're seeking a self-motivated and data-driven engineer to...

  • Azure Data Engineer

    1 week ago

    New York, New York, United States Hudson Data Full time

    Job Title: Azure Data Engineer - Design and Implement Data SolutionsLocation: NYC, NYDuration: Long Term W2 RoleJob Description:The ideal candidate will have hands-on experience with various Azure services, including Azure Data Factory, Databricks, Data Lake, and Power BI. They should be able to design, build, and maintain ETL pipelines, manage data lakes,...

  • Data Engineer for Cloud Data Platforms

    2 weeks ago

    New York, New York, United States The Data Sherpas Full time

    Job Description:The Data Sherpas are a team of skilled engineers who help clients with their cloud journey. We are looking for a self-motivated and data-driven engineer to join our team.Responsibilities:Leverage DBT on Snowflake to build scalable transformations for landed dataBuild business logic and transform raw data for analytic, ML, and reporting use...

  • Data Engineering Specialist

    5 days ago

    New York, New York, United States The Data Sherpas Full time

    About The Data SherpasWe are a team of highly skilled and motivated engineers who help our clients at every phase of their cloud journey.Our team has extensive experience in data and analytics engineering, with a strong foundation in Snowflake, FiveTran, and DBT.We are looking for a self-motivated and data-driven engineer to join our team and collaborate...

  • Senior Data Engineer

    1 week ago

    New York, New York, United States The Data Sherpas Full time

    Job SummaryWe are seeking a highly skilled Senior Data Engineer to lead our team in designing, developing, and deploying high-performance data analytics solutions on Google Cloud Platform (GCP). As a key member of our team, you will be responsible for architecting transformation and modernization of enterprise data solutions on GCP, integrating native GCP...

  • Project Engineer

    2 weeks ago

    New York, New York, United States Engineer Rec Full time

    Project Engineer Job DescriptionWe are seeking a highly skilled Project Engineer to join our team at Engineer Rec. The successful candidate will be responsible for performing engineering work, project preparation, and providing operational support for multiple projects along the East Coast, USA.Key Responsibilities:Develop and maintain project documentation,...

  • Principal Consultant

    2 weeks ago

    New York, United States CTI DATA Full time

    We are looking for Data Architects and Senior Consulting Leaders with a strong background building modern data & analytics solutions. You must have a solid understanding of cloud data platforms and the data & analytics ecosystem, with hands-on experience across data engineering, data science and data visualization.You will be part of a highly collaborative...

  • Data Architect Lead

    1 month ago

    New York, New York, United States CTI DATA Full time

    Join Our Team of Data and Analytics ExpertsWe are seeking a highly skilled and experienced Data Architect Lead to join our team at CTI Data. As a key member of our team, you will be responsible for crafting modern data and analytics solutions for our Global 2000 customers.Key ResponsibilitiesLead the development of cloud data strategies and implementation...

  • Senior Director of Engineering, AI

    2 weeks ago

    New York, New York, United States PitchBook Data Full time

    About the RolePitchBook Data is seeking a highly experienced Senior Director of Engineering, AI & ML to lead our global AI and machine learning initiatives. As a key member of our Product and Engineering team, you will be responsible for shaping the strategic vision and roadmap for AI and ML initiatives, ensuring global alignment with business goals and...

  • Data Engineer

    5 months ago

    New York, United States Silver Search, Inc Full time

    Data Engineer New York, NY Senior Data Engineer--Redshift/AWS/Python/Scripting---**Will be hybrid onsite in January 2022 onwards** Contract-to-hire Our client is a cutting edge E-Learning company based in NYC and is looking to hire a Senior Data Engineer to join their data and analytics center of excellence. He/She will be building...

  • Data Engineer

    4 weeks ago

    New York, New York, United States Oxford Knight Full time

    Data Engineer Wanted for Oxford KnightOxford Knight is seeking a highly skilled Data Engineer to join our growing Data Engineering team. As a Data Engineer, you will play a crucial role in designing, implementing, and evolving our data platform, including pipeline, repository, data warehouse, and data access APIs.About the RoleYou will work closely with our...

  • Data Engineer

    5 days ago

    New York, New York, United States CCG Business Solutions, LLC Full time

    Job DescriptionCCG Business Solutions, LLC is seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems and architectures.The ideal candidate will have a strong background in data engineering, with experience in designing and implementing data...

  • Data Engineer

    1 month ago

    New York, New York, United States ONEOK Full time

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at ONEOK. As a Data Engineer, you will play a critical role in designing, developing, and maintaining our enterprise data pipelines.Key Responsibilities:Design and develop data pipelines using AWS servicesCollaborate with cross-functional teams to ensure data quality and...

  • Customer Support Automation Engineer

    2 weeks ago

    New York, New York, United States VAST Data Full time

    VAST Data is Revolutionizing Customer SupportWe are seeking a highly skilled Customer Support Automation Engineer to join our team at VAST Data. As a key member of our customer success team, you will be responsible for designing, developing, and deploying automated workflows to optimize customer success operations.Key Responsibilities:Develop and implement...

  • Senior Director of Engineering, AI and Machine Learning

    1 week ago

    New York, New York, United States PitchBook Data Full time

    About the RoleWe are seeking a highly experienced Senior Director of Engineering, AI and Machine Learning to lead our global AI and machine learning initiatives. As a key member of our Product and Engineering team, you will be responsible for guiding distributed teams across multiple regions to build robust, scalable AI/ML solutions that power our products...

  • Data Engineer

    2 weeks ago

    New York, New York, United States Center for Justice Innovation Full time

    Job OpportunityThe Center for Justice Innovation is seeking a highly skilled Data Engineer to join its team. As a Data Engineer, you will be responsible for designing, developing, and maintaining the organization's data systems and infrastructure.Key Responsibilities:Develop and maintain data pipelines and infrastructure to support the organization's data...

Data Engineer

2 months ago

New York, United States Randstad Digital Full time

Required: Strong OOP Python skills for Data Engineering

Desirable: MongoDB and Redis


Everything else below is a bonus


W2 only - sponsorship available

No c2c

Job Description Summary:

Generative AI (GenAI) presents an exciting opportunity to derive valuable insights from data and drive revenue growth, efficiencies, and improved business processes. Technology will collaborate with Global Markets Sales & Trading, Quantitative Strategies & Data Group (QSDG) & Platform teams to the design and buildout its global GenAI platform.

The platform will cater to a rapidly growing number of use cases that harness the power of GenAI. Both proprietary and open-source Large Language Models, and large structured and un-structured data sets will be leveraged to produce insights for Global Markets and its clients.

  • We are seeking a Data Engineer to build out data pipelines to source large volumes of structured (ex: KDB) & unstructured data (ex: Research documents, Term Sheets), classify, and store data to meet GenAI requirements.
  • The Data Engineer will design, develop, and engineer platform for high performance and scalability.

Key Responsibilities:

  • Collaborate with data scientists and software engineers to understand data requirements.
  • Develop and maintain scalable data ingestion pipelines from various data sources, including large databases (Hadoop, etc.), web services, and APIs.
  • Analyze and organize raw data from different sources: internal or external.
  • Implement data cleaning, transformation, and normalization processes to ensure data quality and consistency.
  • Implement automated unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance.
  • Automate reports and processes to run with varying frequencies.
  • Monitor and troubleshoot data pipelines to ensure data accuracy and pipeline performance.
  • Explore ways to enhance data quality, reliability, and efficiency of data pipelines.

Experience required:

MUST HAVE Technical:

  • Proficient in data engineering practices and using design and architectural patterns. o
  • At least 4+ years' experience as a Data Engineer or in a similar role, in Extract, Transform, Load (ETL) and/or Extract, Load, Transform (ELT) processes. o
  • Proficiency in working with unstructured and structured data. o
  • Experience with data processing frameworks such as Hadoop, Spark, or similar technologies. o
  • Experience with data platforms like SQL, HDFS, and NoSQL (MongoDB) databases. o
  • Extensive experience with object-oriented programming (OOP)/Scripting languages (Python preferred). o
  • Technical expertise with data models, data mining, and segmentation techniques. o
  • Experience working in multiple technology deployment lanes (development through production). o
  • DevOps processes and CICD tooling (Jira, Git/Bitbucket, Jenkins, Datical, Artifactory, Ansible), orchestration & automation.
  • Containerization technologies such as Docker and Kubernetes
  • Must have minimum 5 years similar experience

Non-Technical

  • Ability to communicate effectively to a wide range of audience (business stakeholders, developer & support teams). o
  • Detail oriented & highly organized. o
  • Adaptable to shifting & competing priorities. o
  • Problem solving skills to diagnose & resolve complex issues. o
  • Committed and pro-active in ensuring high quality of service

Experience desired:

  • Familiarity in AI & Deep learning, modeling techniques, Generative AI application stack. o
  • Familiarity with containerization and orchestration technologies such as Docker, Kubernetes and OpenShift. o
  • Experience with creating visualization dashboards (Tableau). o
  • Experience with vector databases such as Redis.