Data Pipeline Development Director

4 weeks ago


New York, New York, United States Unreal Gigs Full time
Job Title: Lead Data Pipeline Developer

Company Overview:

Welcome to Unreal Gigs, a pioneering company that harnesses the power of data to drive innovation and solve complex problems. Our mission is to create cutting-edge data pipeline solutions that enable efficient data ingestion, processing, and delivery. We're seeking a seasoned professional to lead our data pipeline development initiatives and drive the design, implementation, and optimization of our data pipeline solutions.

Key Responsibilities:

• Provide technical leadership and guidance to a team of data pipeline developers, fostering a culture of excellence and innovation.

• Lead the design and architecture of data pipeline solutions, including data ingestion, transformation, and delivery, to meet business requirements and ensure efficient data processing and analytics.

• Develop and maintain data ingestion processes to collect data from diverse sources, including databases, APIs, files, and streaming sources, ensuring seamless data flow and interoperability.

• Lead data transformation efforts to cleanse, enrich, and transform data as it moves through the pipeline, applying business rules, data validation, and enrichment to meet business needs.

• Implement workflow orchestration and automation solutions to schedule and manage data pipeline workflows, reducing manual intervention and improving operational efficiency.

• Optimize data pipeline performance through parallel processing, partitioning, and other techniques, ensuring scalability and responsiveness for large-scale data processing needs.

• Implement monitoring and alerting systems to track data pipeline performance and health, proactively identifying and resolving issues to minimize downtime and data loss.

• Implement error handling and retry mechanisms to handle data processing failures and ensure data reliability and consistency.

Requirements:

• Bachelor's degree or higher in Computer Science, Engineering, Mathematics, or related field.

• 8+ years of experience in data engineering, with significant experience in designing, building, and optimizing data pipeline solutions.

• Proven leadership experience, with a track record of successfully leading data pipeline development teams and delivering complex projects.

• Proficiency in data pipeline technologies such as Apache Kafka, Apache NiFi, Apache Airflow, or similar.

• Strong programming skills in languages such as Python, Java, or Scala, with experience in data processing frameworks like Apache Spark or Apache Beam.

• Experience with cloud platforms such as AWS, Azure, or Google Cloud Platform, and services like AWS Glue, Azure Data Factory, or Google Dataflow.

Benefits:

• Competitive salary: The industry standard salary for Lead Data Pipeline Developers typically ranges from $190,000 to $270,000 per year, depending on experience and qualifications.

• Comprehensive benefits package, including health insurance, retirement plans, and wellness programs.

• Flexible work arrangements, including remote work options and flexible hours.

• Generous vacation and paid time off.

• Professional development opportunities, including access to training programs, conferences, and workshops.

• State-of-the-art technology environment with access to cutting-edge tools and resources.

• Vibrant and inclusive company culture with opportunities for growth and advancement.

• Exciting projects with real-world impact at the forefront of data-driven innovation.

  • New York, New York, United States MongoDB Full time

    About MongoDBMongoDB empowers innovators to transform industries by unleashing the power of software and data. We enable organizations to build, scale, and run modern applications by helping them modernize legacy workloads, innovate with emerging technologies, and unlock AI-driven insights.We offer a globally distributed, multi-cloud database in over 115...


  • New York, New York, United States Garner Health Test Full time

    Job OverviewGarner Health Test is revolutionizing the healthcare economy by delivering high-quality and affordable care for all. Our mission is to empower employers to restructure their healthcare benefits, providing clear incentives and data-driven insights that direct employees to higher quality and lower cost healthcare providers.We are seeking a skilled...


  • New York, New York, United States United Nations Development Programme Full time

    The United Nations Development Programme (UNDP) is seeking a highly skilled and motivated Business Intelligence Developer to join its team in New York. As a Business Intelligence Developer, you will play a key role in the development and implementation of business intelligence solutions that drive data-driven decision-making across the organization.Your...


  • New York, New York, United States AI4ALL Full time

    Job DescriptionAI4ALL is a national nonprofit organization dedicated to transforming the pipeline of AI practitioners and creating a more inclusive, human-centered discipline. We empower students to be AI Changemakers by cultivating an environment where they develop skills in critical thinking and relationship-building, and expertise in responsible AI.To...


  • New York, New York, United States Extreme Reach Full time

    We are seeking a seasoned Sales Director to join our team at Extreme Reach. This is an exciting opportunity to expand our customer base, elevate client utilization of our platform, and drive growth in existing accounts.Key Responsibilities:Maintain strong relationships with XR Brand Direct sellers, XR Agency Sellers, Customer Success, and Subject Matter...


  • New York, New York, United States American Academy in Rome Full time

    Job Title: Associate Director of DevelopmentThe American Academy in Rome seeks a highly skilled and experienced Associate Director of Development to lead its Development Department. This key role will oversee alumni affairs, major gifts, and institutional giving, building and executing fundraising appeals, creating and managing financial reports, and...


  • New York, New York, United States Mediabrands Full time

    Job SummaryWe are seeking a seasoned Data Analytics Director to join our team at Mediabrands.The ideal candidate will have a strong background in data analysis, strategic planning, and team leadership. This role will be responsible for developing and implementing data-driven strategies that drive business growth and optimize media investments.The Data...

  • Data Science Director

    1 month ago


    New York, New York, United States Scholastic Full time

    Job Title: Director, Data ScienceAt Scholastic, we are seeking a visionary leader to spearhead our Data Science initiatives. As the Director of Data Science, you will be responsible for leading a team of data scientists to drive business growth, mitigate risks, and identify opportunities for innovation.Key Responsibilities:Develop and execute a data science...

  • Engineering Director

    1 month ago


    New York, New York, United States Elysium Health Full time

    About the RoleElysium Health is seeking an experienced Engineering Director to lead our platform and data engineering team. This role involves overseeing sprint planning and management, data engineering and architecture, platform engineering and DevOps, and team leadership and development. The ideal candidate will have a strong background in software...


  • New York, New York, United States GLG Full time

    Job SummaryGLG is seeking a seasoned Talent Development Director, Leadership Specialist to drive the creation and implementation of comprehensive strategies for cultivating leadership skills and engaging, developing, and retaining talent. Reporting to the Global Head of Talent Development, this role will oversee the design and delivery of leadership and...

  • Senior Data Engineer

    1 month ago


    New York, New York, United States NYU Langone Full time

    Job Summary:NYU Langone Health is seeking a highly skilled Senior Data Engineer to join our Enterprise Data and Analytics department. As a key member of our team, you will play a critical role in designing and developing robust ETL pipelines to integrate diverse data sources. This role requires expertise in creating efficient data pipelines, handling both...

  • Cloud Data Architect

    3 weeks ago


    New York, New York, United States Unreal Gigs Full time

    At Unreal Gigs, we're seeking a highly skilled Cloud Data Engineer to join our team. As a Cloud Data Engineer at Unreal Gigs, you'll play a vital role in designing, developing, and managing cloud-based data infrastructures that support analytical and operational needs across the organization. You'll collaborate with data scientists, analysts, and software...

  • Senior Data Engineer

    4 weeks ago


    New York, New York, United States VALiNTRY Full time

    Job SummaryVALiNTRY is seeking a highly skilled Senior Analytics Engineer to support our Business Intelligence lead in managing the enterprise's data pipeline and ecosystem from end to end. As a key member of our team, you will collaborate with cross-functional business partners to understand their needs and transform raw data into actionable insights.Key...


  • New York, New York, United States Fitch Group Full time

    Join Our Team as an Associate Director of Data StrategyFitch Group is seeking a highly skilled Associate Director of Data Strategy to join our team in New York or London. As a key member of our Data Strategy team, you will partner with stakeholders across Fitch Group to source and deliver fit-for-purpose datasets to power growth and operational...

  • Data Engineer

    4 weeks ago


    New York, New York, United States CCG Business Solutions, LLC Full time

    Job DescriptionCCG Business Solutions, LLC is seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems and architectures.The ideal candidate will have a strong background in data engineering, with experience in designing and implementing data...


  • New York, New York, United States Click Therapeutics, Inc. Full time

    Role OverviewWe are seeking a proactive and dynamic Associate Business Development Director to join our Business Development team at Click Therapeutics, Inc.This role is responsible for generating requests for proposal (RFPs) with pharma companies to create software-enhanced dosage forms of their drugs.The position can be based out of Click's headquarters in...


  • New York, New York, United States First Mile Group Full time

    Unlock Growth with Our Senior Director, Demand Generation & Digital Marketing RoleAt Alloy, we're seeking a seasoned leader to drive our demand generation and digital marketing strategy, fueling our next phase of growth. As our Senior Director, Demand Generation & Digital Marketing, you'll own full-stack pipeline generation, rallying sales, product, and...


  • New York, New York, United States Conde Nast Full time

    Conde Nast is a global media company renowned for producing high-quality content for influential audiences, attracting over 100 million consumers across its industry-leading print, digital, and video brands.The company's portfolio includes respected media properties like Vogue, Vanity Fair, Glamour, and Wired, among others.We are seeking a VP of Data...


  • New York, New York, United States invisible-technologies Full time

    Director of AI Data QualityAbout Invisible:We are Invisible Technologies, a company dedicated to merging human creativity and technology to create a future rich in impact and meaning. Our unique approach ensures AI is developed with greater accuracy, quality, safety, reliability, and fairness. This people-first approach underpins our AI training and...


  • New York, New York, United States KPMG Full time

    Key Responsibilities:Lead the customer acquisition life cycle, from pipeline development to account management and revenue attainment.Key Qualifications:Minimum 10 years of solution sales experience, preferably in major national or global accounts, with a track record of exceeding expectations. Experience selling to state and local governments is highly...