Lead Data Engineer

4 weeks ago

New York, New York, United States NYU Langone Full time

About the Role:

We are seeking a highly skilled Lead Data Engineer to join our team at NYU Langone Health. As a key member of our Enterprise Data and Analytics department, you will play a critical role in designing and developing robust ETL pipelines to integrate diverse data sources.

Key Responsibilities:

  • Design and develop ETL code to integrate various data sources
  • Create and maintain efficient data pipelines on the Databricks platform
  • Develop, maintain, and optimize ETL pipelines for both streaming and batch data processing
  • Ensure data integrity and consistency throughout the ETL lifecycle
  • Provide comprehensive support for ETL processes from data ingestion to final output
  • Implement monitoring solutions for ETL jobs to ensure timely and successful data processing
  • Optimize ETL jobs for maximum performance and efficiency through performance tuning and troubleshooting
  • Mentor other data engineers in the team, cross-train, and provide guidance

Requirements:

  • Minimum of Bachelor's degree in Computer Science, Information Systems, Engineering, or Data Science
  • Minimum 10 years of experience in designing, developing, and optimizing ETL processes
  • 5 years of experience in developing/supporting a data platform in Azure Databricks
  • Proficiency in creating and maintaining efficient data pipelines on the Databricks platform
  • Well-rounded in working in a DEV Ops environment, supporting processes in data platform, and supporting business units and their use cases
  • Strong verbal and written communication skills with the ability to explain complex technical concepts to non-technical stakeholders
  • Ability to troubleshoot and resolve issues in a timely manner
  • Experience working in an Agile/Scrum environment
  • Team player with the ability to work collaboratively in a cross-functional team
  • Strong analytical and problem-solving skills
  • Ability to work independently, handle multiple tasks simultaneously, and adapt quickly to change with a variety of people and work styles
  • Willingness to learn new technologies and continuously improve skills

About NYU Langone Health:

NYU Langone Health is a world-class, patient-centered, integrated academic medical center, known for its excellence in clinical care, research, and education. We strive to be a place where our exceptionally talented faculty, staff, and students of all identities can thrive. We embrace diversity, inclusion, and individual skills, ideas, and knowledge.

Equal Employment Opportunity:

NYU Langone Health is an equal opportunity and affirmative action employer committed to diversity and inclusion in all aspects of recruiting and employment. All qualified individuals are encouraged to apply and will receive consideration without regard to race, color, gender, gender identity or expression, sex, sexual orientation, transgender status, gender dysphoria, national origin, age, religion, disability, military and veteran status, marital or parental status, citizenship status, genetic information, or any other factor which cannot lawfully be used as a basis for an employment decision.

  • Senior Azure Data Engineer

    4 weeks ago

    New York, New York, United States Hudson Data Full time

    Job Title: Senior Azure Data EngineerLocation: NYC, NYDuration: Long Term W2 RoleJob Description:The ideal candidate should have hands-on experience with various Azure services, including Azure Data Factory, Databricks, Data Lake, and Power BI. They should be able to design, build, and maintain ETL pipelines, manage data lakes, and create insightful reports...

  • Data Engineering Lead

    4 weeks ago

    New York, New York, United States Blood Bank of Delmarva Full time

    Job Title: Data Engineering LeadDescription:We are seeking a highly skilled Data Engineering Lead to join our team at the Blood Bank of Delmarva. As a key member of our data engineering team, you will be responsible for designing, implementing, and leading initiatives for optimizing databases and data pipelines.Responsibilities:* Lead the architectural...

  • Data Engineering Specialist

    4 weeks ago

    New York, New York, United States The Data Sherpas Full time

    About The Data SherpasWe are a team of highly skilled and motivated engineers who help our clients at every phase of their cloud journey.Our team has extensive experience in data and analytics engineering, with a strong foundation in Snowflake, FiveTran, and DBT.We are looking for a self-motivated and data-driven engineer to join our team and collaborate...

  • Data Engineer Lead

    4 weeks ago

    New York, New York, United States Pfizer Full time

    Pfizer is seeking a highly skilled Data Engineer to lead the development of its cloud-based data lake and drive digital transformation across the enterprise.The successful candidate will design, architect, and build data products that span the areas of supply chain execution, planning, manufacturing, quality, logistics, and more.As a Data Engineer Lead, you...

  • Data Engineering Lead

    1 month ago

    New York, New York, United States Pfizer Full time

    Job SummaryPfizer is seeking a highly skilled Data Engineering Lead to join our team. As a key member of our Enterprise Data and Analytics team, you will be responsible for designing and delivering data products that drive business outcomes. You will work closely with our Digital Partners, Enterprise Architecture, and Business Unit stakeholders to develop...

  • Lead Data Engineer

    1 month ago

    New York, New York, United States Capital One Services, LLC Full time

    Job Title: Lead Data EngineerCapital One is seeking a Lead Data Engineer to join our team. As a Lead Data Engineer, you will be responsible for designing, developing, and implementing technical solutions in full-stack development tools and technologies.You will work with a team of developers with deep experience in machine learning, distributed...

  • Lead Data Engineer

    4 weeks ago

    New York, New York, United States Rhode Island Blood Center Full time

    Job Title: Data Infrastructure ArchitectJob Summary:We are seeking a highly skilled Data Infrastructure Architect to lead the design and implementation of our data infrastructure. As a key member of our team, you will be responsible for architecting and implementing scalable, secure, and efficient data pipelines using SQL, Python, and cloud-based ETL tools...

  • Data Engineering Team Lead

    4 weeks ago

    New York, New York, United States City of New York Full time

    Job Title: Data Engineering Team LeadJob Summary:The City of New York is seeking a highly skilled Data Engineering Team Lead to join the Information Technology Division (ITD). As a key member of the Data Engineering team, you will be responsible for leading a team of data engineers, planning and executing projects, and ensuring the delivery of high-quality...

  • Lead Data Engineer

    1 month ago

    New York, New York, United States Capital One Full time

    Job DescriptionCapital One is seeking a Lead Data Engineer to join our team. As a Lead Data Engineer, you will be responsible for designing, developing, testing, implementing, and supporting technical solutions in full-stack development tools and technologies.You will work with a team of developers with deep experience in machine learning, distributed...

  • Lead Data Engineer

    4 weeks ago

    New York, New York, United States Capital One Full time

    About the Role:Capital One is seeking a highly skilled Data Engineer to join our team. As a Lead Data Engineer, you will be responsible for designing, developing, and implementing technical solutions in full-stack development tools and technologies.Key Responsibilities:Collaborate with Agile teams to design and develop technical solutionsWork with a team of...

  • Data Engineer Lead

    4 weeks ago

    New York, New York, United States GroupM Choreograph Full time

    Job SummaryGroupM Choreograph is seeking a highly skilled Data Engineer to lead the design and implementation of cloud-based data solutions. The ideal candidate will have a strong background in data engineering, cloud platforms, and business intelligence tools.Key ResponsibilitiesDesign and build cloud-based data solutions using data engineering platforms...

  • Lead Data Engineer

    4 weeks ago

    New York, New York, United States Capital One Full time

    About the Role:Capital One is seeking a skilled Data Engineer to join our team. As a Lead Data Engineer, you will be responsible for designing, developing, and implementing technical solutions in full-stack development tools and technologies. You will work closely with Agile teams to deliver robust cloud-based solutions that drive powerful experiences for...

  • Data Engineer Lead

    4 weeks ago

    New York, New York, United States GroupM Choreograph Full time

    Job Summary:A highly skilled Data Engineer is sought to lead the design and build of cloud-based data solutions at GroupM Choreograph. The ideal candidate will have a minimum of four to five years' experience in data governance processes in a data warehousing, ETL, and business intelligence environment.Key Responsibilities:Work closely with a team of data...

  • Big Data Engineering Lead

    4 weeks ago

    New York, New York, United States InterSources Full time

    Job Title: Big Data EngineerType: ContractLocation: New York, NY ( Day 1 onsite)Job Description:A Big Data Engineering lead handles the ETL jobs, Optimization, Automation of data processing for complex scenarios. This role requires expertise in designing and building ETL pipelines in ADF or ADLS Gen2, as well as experience with Spark, Synapse, and Azure...

  • Data Engineering and BI Development Team Lead

    4 weeks ago

    New York, New York, United States City of New York Full time

    About the RoleThe City of New York is seeking a highly skilled Data Engineering Team Lead to join the Information Technology Division (ITD). As a key member of the Data Engineering team, you will be responsible for leading a team of data engineers, planning and executing projects, and designing, deploying, and scaling current infrastructure.Key...

  • Engineering Director

    1 month ago

    New York, New York, United States Elysium Health Full time

    About the RoleElysium Health is seeking an experienced Engineering Director to lead our platform and data engineering team. This role involves overseeing sprint planning and management, data engineering and architecture, platform engineering and DevOps, and team leadership and development. The ideal candidate will have a strong background in software...

  • Senior Lead Data Engineer

    4 weeks ago

    New York, New York, United States Capital One Services, LLC Full time

    About the Role:We are seeking a Senior Lead Data Engineer to join our team at Capital One Services, LLC. As a Senior Lead Data Engineer, you will be responsible for leading a team of developers in designing, developing, and implementing technical solutions in full-stack development tools and technologies.Key Responsibilities:Collaborate with Agile teams to...

  • Senior Data Engineering Lead

    4 weeks ago

    New York, New York, United States Delphi-US Full time

    Job Title: Senior Data Engineer - MartechLocation : RemoteWe are seeking a Senior Data Engineer to collaborate with our team and contribute to the transition of our legacy Martech pipelines to a modern ELT architecture.You will work with BASH, Perl, dbt, Fivetran, Snowflake, GitHub, and modern CI/CD & IAC tools to design, develop, test, deploy, and support...

  • Lead Data Engineer

    4 weeks ago

    New York, New York, United States Harnham Full time

    Job Title: Lead Data Engineer - Machine Learning SpecialistAbout the Role:We are seeking a highly skilled Lead Data Engineer to join our team at Harnham. As a key member of our data engineering team, you will be responsible for designing and developing scalable data pipelines that integrate with machine learning models to enhance automation tools for...

  • Staff Software Engineer

    4 weeks ago

    New York, New York, United States Robin Hood Full time

    About the RoleWe are seeking a highly skilled Staff Software Engineer to lead the development of our data ingestion pipelines. As a key member of our data engineering team, you will design, build, and maintain efficient and reliable batch and streaming data pipelines that drive key data insights across our products.Key ResponsibilitiesLead initiatives to...