Senior Data Engineer

2 months ago


New York, United States Harnham Full time

In this opportunity, you will work with a large-scale Media company that partners with some of the biggest firms across the globe. They are looking for a Senior Data Engineer to join their Engineering team.


Responsibilities:

  • Design, develop, and maintain scalable data pipelines and ETL processes using Apache Spark.
  • Collaborate closely with data scientists, analysts, and other stakeholders to understand data requirements and implement solutions.
  • Optimize Spark jobs for performance and scalability, ensuring efficient processing of large datasets.
  • Troubleshoot and debug Spark job failures, ensuring data integrity and reliability.
  • Implement best practices for data governance, security, and compliance within Spark-based environments.
  • Stay updated with the latest advancements in Apache Spark and Big Data technologies, and propose improvements to existing frameworks.

Requirements:

  • Bachelor’s degree in Computer Science, Engineering, or a related field (Master’s preferred).
  • Proven experience working with Apache Spark for large-scale data processing and analytics.
  • Proficiency in Scala or Python programming languages.
  • Solid understanding of distributed computing principles and experience with Spark's RDD, DataFrame, and Dataset APIs.
  • Experience with Spark SQL, Spark Streaming, and other Spark modules.
  • Strong knowledge of SQL and NoSQL databases.
  • Familiarity with data warehousing concepts and cloud platforms (AWS, Azure, Google Cloud).
  • Excellent problem-solving skills and ability to work independently or in a team environment.

Preferred Qualifications:

  • Certification in Apache Spark or related Big Data technologies.
  • Experience with containerization technologies (Docker, Kubernetes).
  • Familiarity with streaming technologies (Kafka, Flink, etc.).
  • Understanding of machine learning frameworks (TensorFlow, PyTorch) and their integration with Spark.


Please apply directly or email me: benbochner@harnham.com


  • Senior Data Engineer

    4 weeks ago


    New York, United States Forsyth Barnes Full time

    Title: Senior Data EngineerLocation: Remote - HQ in NYCIndustry: RetailSalary: $150,000 + benefits*Please note - my client is not offering any form sponsorship at this time*Overview:We have partnered with a multi-brand retailer looking to add a Senior Data Engineer to their growing team. This is a senior role reporting to the Director of Data Engineering...

  • Senior Data Engineer

    4 weeks ago


    New York, United States Forsyth Barnes Full time

    Title: Senior Data EngineerLocation: Remote - HQ in NYCIndustry: RetailSalary: $150,000 + benefits*Please note - my client is not offering any form sponsorship at this time*Overview:We have partnered with a multi-brand retailer looking to add a Senior Data Engineer to their growing team. This is a senior role reporting to the Director of Data Engineering...

  • Senior Data Engineer

    3 months ago


    New York, United States ASCENDING Full time

    Job DescriptionJob DescriptionFull-time, Remote - Must work standard EST hoursAvailable for W-2 or 1099 Individual. Senior Data Engineer Our client is seeking a Senior Data Engineer to accelerate data sharing between applications, engineers, and data scientists. The successful candidate will demonstrate mature programming and design ability. The Data...


  • New York, New York, United States Data Intelligence Full time

    Company Overview:Data Intelligence, LLC (DI) is a reputable small business dedicated to supporting the vital missions of government clients since its inception in 2005. We specialize in full life cycle system development, systems engineering, cybersecurity, and comprehensive analytical and logistics support for C4ISR and other intricate systems.Position...

  • Senior Data Engineer

    2 weeks ago


    New York, New York, United States Tickets Full time

    , an MLB company, delivers innovative, cutting-edge technologies to enable frictionless and unforgettable fan experiences in venues across the globe. Together with MLB, is changing the landscape of the live sports and entertainment industry, delivering new digital venue and ticketing experiences to millions of fans. Our Technology team builds platforms and...


  • New York, United States Harnham Full time

    Senior Data EngineerHybrid (1 day in office in NYC)6-Month Contract-to-Hire$60-$75/hrUp to $140,000 AnnualOur client seeking a skilled Senior Data Engineer to join the Data & AI Platforms team, responsible for maintaining and advancing a comprehensive data ecosystem that supports B2B SaaS products. Their focus is on developing a robust platform that...


  • New York, United States Harnham Full time

    Senior Data EngineerHybrid (1 day in office in NYC)6-Month Contract-to-Hire$60-$75/hrUp to $140,000 AnnualOur client seeking a skilled Senior Data Engineer to join the Data & AI Platforms team, responsible for maintaining and advancing a comprehensive data ecosystem that supports B2B SaaS products. Their focus is on developing a robust platform that...

  • Senior Data Engineer

    2 weeks ago


    New York, United States Tech Tammina LLC Full time

    Job DescriptionJob DescriptionJob Title: Senior Data Engineer (Mid-Level) - AnalyticsLocation: NYC (Hybrid - 3 days onsite)Position: Full-Time PermanentJob Description: Join a leading organization as a Senior Data Engineer in the Analytics team. In this hybrid role, you will design, develop, and maintain data pipelines to support data-driven initiatives....

  • Senior Data Engineer

    3 weeks ago


    New York, United States Epik Solutions Full time

    Job DescriptionJob DescriptionSenior Data EngineerNew York, NY (Remote)About the PositionAs a Senior Data Engineer you will be a major contributor to the tech future for Formation Bio as we expand into an AI enabled world. You would be responsible for contributing to our fundamental Data Infrastructure by building data pipelines, integrations, and tools that...

  • Data Engineer

    1 week ago


    New York, United States NTT DATA Full time

    Req ID: 270954 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in New York, New York (US-NY), United States (US).Responsibilities:- Expand and optimize...

  • Senior Data Engineer

    3 months ago


    New York, United States Quilt Software LLC Full time

    Job DescriptionJob DescriptionAbout Quilt Software: Quilt LLC is PSG Equity's latest vertical software and integrated payments platform, similar to their successful EverCommerce investment which recently went public. Quilt provides specialty inventory and POS software through a family of vertically focused, independently managed subsidiaries. See...


  • New York, United States Roots Automation Full time

    Senior Data Engineer Company Overview: At Roots, our mission is to make work more human. We are developing fully autonomous, AI-powered Digital Coworkers that streamline tedious and repetitive tasks for the enterprise. By tackling core challenges in natural language understanding and computer vision, we are building an automation product that embodies the...


  • New York, United States Roots Automation Full time

    Senior Data Engineer Company Overview: At Roots, our mission is to make work more human. We are developing fully autonomous, AI-powered Digital Coworkers that streamline tedious and repetitive tasks for the enterprise. By tackling core challenges in natural language understanding and computer vision, we are building an automation product that embodies the...

  • Senior Data Engineer

    2 months ago


    New York, United States iO Associates - US Full time

    We are currently looking for a Senior Data Engineer to join us on a contract basis, collaborating with teams within a dynamic technology organization. This role offers a hybrid work setup based at our Customer's office in New York City.Responsibilities:Lead the complete development lifecycle of intricate data projects involving integrations and...

  • Senior Data Engineer

    2 months ago


    New York, United States iO Associates - US Full time

    We are currently looking for a Senior Data Engineer to join us on a contract basis, collaborating with teams within a dynamic technology organization. This role offers a hybrid work setup based at our Customer's office in New York City.Responsibilities:Lead the complete development lifecycle of intricate data projects involving integrations and...

  • Senior Data Engineer

    3 months ago


    New York, United States VeracityID Full time

    Job DescriptionJob DescriptionSalary: The roleVeracityID’s development team is looking for Senior Data Engineer with deep experience in building analytical solutions for distributed data environments. VeracityID is a cloud services firm specialising in near real-time fraud detection in the insurance industry.Its application suite employs multiple large...

  • Senior Data Engineer

    2 months ago


    New York, New York, United States Hulu Full time

    Job Posting Title:Senior Data Engineer - Identity DataReq ID:Job Description:Disney Entertainment & ESPN TechnologyOn any given day at Disney Entertainment & ESPN Technology, we're reimagining ways to create magical viewing experiences for the world's most beloved stories while also transforming Disney's media business for the future.Whether that's evolving...


  • New York, United States Publicis Sapient Full time

    Job Description Your Impact: The purpose of the Senior Associate, Data Science is to work with clients undergoing a data-driven transformation (DDT) across the globe. The Senior Associate will accelerate and drive the DDT strategy for clients, working in partnership with digital directors, client teams, and practice capability. Furthermore, the...

  • Data Engineer

    2 weeks ago


    New York, New York, United States NTT DATA Services Full time

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in New York, New York (US-NY), United States (US). Responsibilities:- Expand and optimize the...

  • Senior Data Engineer

    3 weeks ago


    New York, United States Capital One Full time

    11 West 19th Street (22008), United States of America, New York, New YorkSenior Data EngineerJob DescriptionDo you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative,inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers,...