Senior Data Engineer

1 week ago

New York, New York, United States CDW Full time

The Senior Data Engineer will play a pivotal role in building and operationalizing the minimally inclusive data necessary for the enterprise data and analytics initiatives following best practices. The bulk of the data engineer's work would be in building, managing and optimizing data pipelines and then moving these data pipelines effectively into production for key data and analytics consumers like business/data analysts, data scientists or any persona that needs curated data for data and analytics use cases across the enterprise.

Key Responsibilities:

  • Develops and maintains scalable data pipelines to support continuing increases in data volume and complexity.
  • Interfaces with other technology teams to extract, transform, and load data from a wide variety of data sources using big data technologies and SQL.
  • Collaborates with business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organization.
  • Collaborates with other technology teams to help engineer data sets that data science teams uses to implement advanced analytics algorithms that exploit our rich datasets for statistical analysis, prediction, clustering and machine learning.
  • Responsible for using innovative and modern tools, techniques and architectures to partially or completely automate the most-common, repeatable and tedious data preparation and integration tasks in order to minimize manual and error-prone processes and improve productivity.
  • Trains counterparts such as data scientists, data analysts, data consumers in data pipelines and preparation techniques, which make it easier for them to integrate and consume the data they need for their own use cases.
  • Ensures compliance and governance during use of data.
  • Remains curious and knowledgeable about new data management techniques and how to apply them to solve business problems.

Qualifications:

  • BS or MS degree in Computer Science or a related technical field.
  • 7+ years data application development experience.

Requirements:

  • Strong experience with various Data Management architectures like Data Warehouse, Data Lake, Data Hub and the supporting processes like Data Integration, Governance, Metadata Management.
  • Extensive experience with popular data processing languages including SQL, PL/SQL, Python, others for relational databases and on NoSQL/Hadoop oriented databases like MongoDB, Cassandra, others for nonrelational databases.
  • Demonstrated ability to build rapport and maintain productive working relationships cross-departmentally and cross-functionally.
  • Demonstrated ability to coach and mentor others.
  • Excellent written and verbal communication skills with the ability to effectively interact with and present to all stakeholders including senior leadership.
  • Strong organizational, planning and creative problem solving-skills with critical attention to detail.
  • Demonstrated success of facilitation and solutions implementation.
  • History of balancing competing priorities with the ability to adapt to the changing needs of the business while meeting deadlines.

Preferred Qualifications:

  • Extensive experience with Azure Data Factory.
  • Extensive experience working with cloud platform (at least one of Azure, AWS, GCP).
  • Experience with ETL Tools (SSIS, Informatica, Ab Initio, Talend), Python, Databricks, Microsoft SQL Server Platform (version 2012 or later), and/or working in an Agile environment.

  • Senior Azure Data Engineer

    5 days ago

    New York, New York, United States Hudson Data Full time

    Job Title: Senior Azure Data EngineerLocation: NYC, NYDuration: Long Term W2 RoleJob Description:The ideal candidate should have hands-on experience with various Azure services, including Azure Data Factory, Databricks, Data Lake, and Power BI. They should be able to design, build, and maintain ETL pipelines, manage data lakes, and create insightful reports...

  • Senior Data Engineer

    1 week ago

    New York, New York, United States The Data Sherpas Full time

    Job SummaryWe are seeking a highly skilled Senior Data Engineer to lead our team in designing, developing, and deploying high-performance data analytics solutions on Google Cloud Platform (GCP). As a key member of our team, you will be responsible for architecting transformation and modernization of enterprise data solutions on GCP, integrating native GCP...

  • Senior Data Engineer

    4 weeks ago

    New York, New York, United States Net2Source Inc. Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Net2Source Inc. As a Senior Data Engineer, you will be responsible for designing and managing data ecosystems, particularly within the finance sector.Key Responsibilities:Building and maintaining data pipelines to ensure data quality and integrityDriving...

  • Senior Data Engineer

    2 weeks ago

    New York, New York, United States Capital One Full time

    Job Title: Senior Data EngineerCapital One is seeking a Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and implementing data engineering solutions that drive business value.Key Responsibilities:Collaborate with cross-functional teams to design and develop data engineering solutionsDevelop...

  • Senior Data Engineer

    1 month ago

    New York, New York, United States Harnham Full time

    Senior Data EngineerAbout the RoleWe are seeking a highly skilled Senior Data Engineer to join our team at Harnham. As a Senior Data Engineer, you will be responsible for designing and developing comprehensive technical designs for both streaming and batch processing systems.Key ResponsibilitiesDevelop technical designs for streaming and batch processing...

  • Senior Data Engineer

    3 weeks ago

    New York, New York, United States Capital One Full time

    Job Title: Senior Data EngineerCapital One is seeking a Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and implementing data engineering solutions that drive business value.Key Responsibilities:Collaborate with cross-functional teams to design and develop data engineering solutions that...

  • Senior Data Engineer

    2 weeks ago

    New York, New York, United States Capital One Full time

    Job Title: Senior Data EngineerCapital One is seeking a Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and implementing data engineering solutions that drive business value.Key Responsibilities:Collaborate with cross-functional teams to design and develop data engineering solutionsDevelop...

  • Senior Data Engineer

    2 weeks ago

    New York, New York, United States Capital One Full time

    Job Title: Senior Data EngineerCapital One is seeking a Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and implementing data pipelines and architectures to support our business needs.About the Role:Collaborate with cross-functional teams to design and develop data pipelines and...

  • Senior Data Engineer

    1 month ago

    New York, New York, United States A Place For Mom Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at A Place for Mom. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining our data warehouse on Databricks within AWS, ensuring optimal performance and reliability.Key Responsibilities:Design and develop ETL processes...

  • Senior Data Engineer

    2 weeks ago

    New York, New York, United States Silver Search, Inc Full time

    Senior Data EngineerSilver Search, Inc is seeking a highly skilled Senior Data Engineer to join our data and analytics team. As a key member of our team, you will be responsible for designing and building data processing frameworks, working closely with data science and visualization teams, and ensuring the integrity and scalability of our data...

  • Senior Engineer

    2 weeks ago

    New York, New York, United States Acquired Data Solutions Full time

    Job SummaryWe are seeking a highly skilled Senior Level Engineer to join our Transportation Security Laboratory (TSL) Team in Egg Harbor, NJ as Key Personnel to support Development Test & Evaluation (DT&E) of existing and emerging technologies for airport security equipment.Key ResponsibilitiesPerform in-depth engineering analysis of security related systems...

  • Senior Data Engineer

    3 weeks ago

    New York, New York, United States Coinbase Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Coinbase. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining robust data pipelines, trusted data sources, and analytics/data products that inject automation into data science processes.Key Responsibilities:Design...

  • Senior Data Engineer

    4 weeks ago

    New York, New York, United States Coinbase Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Coinbase. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining robust data pipelines, trusted data sources, and analytics/data products that inject automation into data science processes.Key Responsibilities:Design...

  • Senior Data Engineer

    1 month ago

    New York, New York, United States Tech Tammina Full time

    Senior PySpark/Python EngineerWe are seeking a highly skilled Senior PySpark/Python Engineer to join our team at Tech Tammina. The ideal candidate will have a strong background in data engineering, with expertise in PySpark, Python, and related technologies.Key Responsibilities:Design and implement data pipelines using PySpark and PythonWork with data lakes,...

  • Senior Data Engineer

    2 weeks ago

    New York, New York, United States MidStream PA Full time

    Join Our Team as a Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to lead the development of our core data processing platform. As a key member of our engineering team, you will be responsible for designing and implementing scalable, reliable, and high-performance data systems.Your expertise in data modeling, warehousing, and...

  • Senior Data Engineer

    2 weeks ago

    New York, New York, United States A Place For Mom Full time

    Job Title: Senior Data EngineerA Place for Mom is seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining our data warehouse on Databricks within AWS, ensuring optimal performance and reliability.Key Responsibilities:Design and develop ETL processes to...

  • Senior Data Engineer

    1 month ago

    New York, New York, United States TENTH MOUNTAIN LLC Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at TENTH MOUNTAIN LLC. As a Senior Data Engineer, you will play a crucial role in migrating our existing data platform to an on-premises Hadoop environment.Key Responsibilities:Data Pipeline Development: Design and develop scalable, efficient ETL pipelines...

  • Senior Data Engineer

    2 weeks ago

    New York, New York, United States Capital One Full time

    Job Title: Senior Data EngineerCapital One is seeking a Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and implementing data engineering solutions that drive business value.Key Responsibilities:Collaborate with cross-functional teams to design and develop data engineering solutionsDevelop...

  • Senior Data Engineer

    5 days ago

    New York, New York, United States TechnoGen Full time

    Job Title: Senior Data EngineerJob Summary:We are seeking a Senior Data Engineer to join our Data Infrastructure team at TechnoGen. As a Senior Data Engineer, you will be responsible for building and maintaining tools and infrastructure to write, test, and schedule batch data pipelines. Your work will enable other developers, data scientists, and analysts to...

  • Senior Engineer

    4 weeks ago

    New York, New York, United States Acquired Data Solutions Full time

    Job OverviewWe are seeking a highly skilled Senior Level Engineer to join our Transportation Security Laboratory (TSL) Team in Egg Harbor, NJ as Key Personnel to support Development Test & Evaluation (DT&E) of existing and emerging technologies for airport security equipment.Key ResponsibilitiesPerform in-depth engineering analysis of security related...