Lead Data Engineer

Found in: Talent US C2 - 2 weeks ago

Newark, United States JSR Tech Consulting Full time
Long term contract to hire. Lead data engineer role with a major Financial Services company.

Hybrid work environment. 2 - 3 days onsite in Newark, NJ.

- Engineering lead to help the design/build of the enterprise data platforms
Design, Develop and Implement
  • End-to-end data solutions (ingest, storage, integration, processing, access) on AWS.
  • Data intake/ request/ onboarding services and service documentation.
  • Data ingestion services for batch/ real time data ingest and service documentation.
  • Data processing services for batch/ real time (Glue/ Kinesis/ EMR) and service documentation.
  • Data storage services for data lake (S3)/ data warehouses (RDS/ Redshift)/ data marts and service documentation.
  • Data services layer including Athena, Redshift, RDS, microservices and APIs.
  • Pipeline orchestration services including lambda, step functions, MWAA (optional).
  • Data security services (IAM/ KMS/ SM/ encryption/ anonymization/ RBAC) and service documentation.
  • Data access provisioning services (Accounts, IAM Roles RBAC), processes, documentation and education.
  • Data provisioning services for data consumption patterns including microservices, APIs and extracts.
  • Metadata capture and catalog services for data lake (S3/ Athena), data warehouses (RDS/ Redshift), Microservices/ APIs.
  • Metadata capture and catalog services for pipeline/ log data for monitoring / support.
  • Architect and implement CI/ CD strategy for EDP .
  • Implement high velocity streaming solutions using Amazon Kinesis, SQS, and SMS.
  • Migrate data from traditional relational database systems, file systems, NAS shares to AWS relational databases such as Amazon RDS, Aurora, and Redshift.
  • Migrate data from APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift.
  • Implement cost/ spend monitoring for AWS based data platform.
  • Implement audit reporting for access of AWS based data platform.
  • Contribute to the implementation of a data strategy to enable LOBs and Corporate Functions with a robust, holistic view of data - driven decision making.
  • Partner with immediate engineering team, product owner, IT, partners on EDP agenda.
  • Leverage and continuously develop best practices, standards, and frameworks.
  • Provide technology thought leadership, consulting, and coaching/ mentoring.
  • Work with scrum master to develop and own backlog, stories, epics, sprints.
  • System design and Architecture for products/ applications for EDP.
  • Work closely with various stakeholders (BI, AIML, MDM and other teams) to understand their use-cases and design optimal solution for them.
  • Execute technical design and infrastructure strategy for EDP .
  • Implement POCs on any new technology or tools to be implemented on EDP and onboard for real use-case.
Qualifications
  • Bachelor's degree in computer science, Software Engineering, MIS or equivalent combination of education and experience.
  • Experience implementing, supporting data platforms on AWS for large enterprises.
  • Full stack development experience building secure internal facing applications.
  • Programming experience with Java, Python/ Scala, Shell scripting.
  • Solid experience of AWS services such as CloudFormation, S3, Glue, EMR/ Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, SM etc.
  • Solid experience implementing solutions on AWS based data lakes preferred.
  • Experience implementing metadata solutions leveraging AWS non-relational data solutions such as ElastiCache and DynamoDB.
  • AWS Solutions Architect or AWS Big Data Certification preferred.
  • Experience in AWS data lake, data warehouse and business analytics.
  • Experience and understanding of various core AWS services such as IAM, Cloud Formation, EC2, S3, EMR/Spark, Glue, Datasync, CloudHealth, CloudWatch, Lambda, Athena, and Redshift.
  • Experience in system analysis, design, development, and implementation of data ingestion pipeline in AWS.
  • Experience with DevOps and Continuous Integration/ Delivery (CI/ CD) concepts and tools.
  • Experience with business intelligence tools such as Tableau, Power BI or equivalent.
  • Knowledge of ETL/ ELT.
  • Awareness of Data Management & Governance tools.
  • Working experience with Hadoop, HDFS, SQOOP, Hive, Python, and Spark is desired.
  • Experience working on Agile projects.

  • Lead Data Engineer

    Found in: Appcast US C2 - 2 days ago

    Newark, United States HireTalent - Diversity Staffing & Recruiting Firm Full time

    NOTE: NO C2CJob Description:Implement and support end-to-end data lake/warehousing/mart/business intelligence/ analytics/ services solutions (ingest, storage, integration, processing, services, access) in AWS.Data lake, data intake/request/onboarding services and service documentation, data lake ingestion services for batch/real time data ingest and service...

  • Lead Data Engineer

    Found in: Appcast Linkedin GBL C2 - 2 days ago

    Newark, United States HireTalent - Diversity Staffing & Recruiting Firm Full time

    NOTE: NO C2CJob Description:Implement and support end-to-end data lake/warehousing/mart/business intelligence/ analytics/ services solutions (ingest, storage, integration, processing, services, access) in AWS.Data lake, data intake/request/onboarding services and service documentation, data lake ingestion services for batch/real time data ingest and service...

  • AWS Data Engineer

    4 days ago

    Newark, United States Abidi Solutions Full time

    Job DescriptionJob DescriptionJob Title: Lead AWS Data EngineerLocation: Newark, NJ On-sitePosition Type: ContractOnly LocalsJob Description:Our client, a global financial company, is seeking a skilled AWS Data Engineer to join their team. As an AWS Data Engineer, you will play a key role in designing, building, and maintaining efficient and reliable data...

  • Data Engineer

    2 weeks ago

    Newark, United States HireTalent - Diversity Staffing & Recruiting Firm Full time

    High level requirements for the role:-Data ingestion from various on-prem and cloud sources-Ability to build ETL pipelines to transform raw data into format suitable for analysis (hands on experience working with ETL tools like AWS Glue, Informatica etc.) and orchestrate transformation (using Airflow, other DAG tools)-Experience working with big data...

  • Data Engineer

    Found in: Appcast Linkedin GBL C2 - 3 weeks ago

    Newark, United States HireTalent - Diversity Staffing & Recruiting Firm Full time

    High level requirements for the role:-Data ingestion from various on-prem and cloud sources-Ability to build ETL pipelines to transform raw data into format suitable for analysis (hands on experience working with ETL tools like AWS Glue, Informatica etc.) and orchestrate transformation (using Airflow, other DAG tools)-Experience working with big data...

  • Lead Cloud Engineer

    1 week ago

    Newark, United States Diverse Lynx Full time

    Job Tittle: Lead Cloud Engineer Location: Hybrid - Newark, NJ Duration: Contract Job Summary: Lead Cloud Engineer on the enterprise Cloud Business Office (CBO), you will focuses on Cloud Cost Management and evangelizing our FinOps principles within Prudential. You will also be collaborating with the broader CBO team on Governance and Demand Management...

  • Lead Data Engineer

    Found in: Appcast Linkedin GBL C2 - 6 days ago

    Newark, United States FYI Solutions Full time

    ResponsibilitiesImplement and support· end-to-end data lake/warehousing/mart/business intelligence/ analytics/ services solutions (ingest, storage, integration, processing, services, access) in AWS· data lake data intake/request/onboarding services and service documentation· data lake ingestion services for batch/real time data ingest and service...

  • Lead Data Engineer

    Found in: Appcast US C2 - 6 days ago

    Newark, United States FYI Solutions Full time

    ResponsibilitiesImplement and support· end-to-end data lake/warehousing/mart/business intelligence/ analytics/ services solutions (ingest, storage, integration, processing, services, access) in AWS· data lake data intake/request/onboarding services and service documentation· data lake ingestion services for batch/real time data ingest and service...

  • Lead Software Engineer

    1 week ago

    Newark, United States Atlantic Partners Full time

    Our direct client is seeking 2 Lead Software Engineers for a 12-month contract-to-hire opportunity with their Newark, NJ location. Job Description: As a Lead Software Engineer in the US Business Technology Organization, you will partner with product owners, tech leads, designers, engineers and delivery professionals to improve the Global Commissions and...

  • AWS Data Engineer

    4 days ago

    Newark, United States Iris Software Inc. Full time

    Iris's client, a large insurance company, is currently searching for a strong for AWS Data Engineer. Our client has offices all over the world and has the global reach, advisory services, and distribution power to meet the needs of issuers and investors worldwide. Job title: AWS Data EngineerLocation: Hybrid 2 days onsite (Newark, NJ)Duration: 24 monthsOnly...

  • AWS Data Engineer

    Found in: Appcast Linkedin GBL C2 - 5 days ago

    Newark, United States Iris Software Inc. Full time

    Iris's client, a large insurance company, is currently searching for a strong for AWS Data Engineer. Our client has offices all over the world and has the global reach, advisory services, and distribution power to meet the needs of issuers and investors worldwide. Job title: AWS Data EngineerLocation: Hybrid 2 days onsite (Newark, NJ)Duration: 24 monthsOnly...

  • AWS Data Engineer

    Found in: Appcast US C2 - 6 days ago

    Newark, United States Iris Software Inc. Full time

    Iris's client, a large insurance company, is currently searching for a strong for AWS Data Engineer. Our client has offices all over the world and has the global reach, advisory services, and distribution power to meet the needs of issuers and investors worldwide. Job title: AWS Data EngineerLocation: Hybrid 2 days onsite (Newark, NJ)Duration: 24 monthsOnly...

  • Sr. Data Engineer

    23 hours ago

    Newark, United States MDMS Recruiting LLC Full time

    Job DescriptionJob Description***W2 or Self Inc. Only | this is not open to subcontracting (no C2C)***Senior Data Engineer to help the design/build of the enterprise data platformsDesign, Develop and ImplementEnd-to-end data solutions (ingest, storage, integration, processing, access) on AWS.Data intake/ request/ onboarding services and service...

  • Data Scientist

    1 week ago

    Newark, United States Avispa Full time

    Data Scientist - Product and Insights 1282622 A leading podcast and audiobook company is seeking a Data Scientist - Product and Insights. The successful candidate will operationalize existing Multimedia Mix Model (MMM) framework to cover all current company's marketplaces. The ideal candidate is a motivated, results-oriented Data Scientist with strong rigor...

  • ADAS / AD Systems Engineering Technical Lead

    1 day ago

    Newark, United States Lucid Motors Full time

    In this role, you are embedded within the ADAS department as a Sr. ADAS and AD Systems Engineering Technical Lead to collaborate with a wide range of cross-functional teams at Lucid Motors on tracking and bringing to completion of all phases of ADAS/AD feature development and release process. The Sr. ADAS and AD Systems Engineering Technical Lead will help...

  • Tech Apprentice, Data Services

    1 day ago

    Newark, United States WebMD Full time

    Position at WebMD WebMD is the most recognized and trusted brand of health information and the leading provider of health information services, serving consumers, physicians, healthcare professionals, employers and health plans through our public and Data Services, Apprentice, Tech, Health, Data Engineer, Technology

  • Intern, Experimental Battery Cell Engineer, Summer

    Found in: beBee S US - 1 week ago

    Newark, United States Abs Data Full time

    Company: Lucid MotorsThese positions benefit from an intimate knowledge of lithium-ion batteries as well as a strong background in electrochemical testing and data analysis.Strong knowledge of lithium-ion cell chemistry and general electrochemistryVisa Sponsorship: Lucid may but is not obligated to sponsor foreign national employees requiring company...

  • Technical Lead

    1 week ago

    Newark, United States VSG Business Solutions LLC Full time

    Job DescriptionJob DescriptionAt-a-Glance:Are you ready to build your career by joining a global financial company? If so, our client is hiring a Technical Lead!What You'll Do:As a Lead Platform Engineer, will partner with product owners, tech leads, designers, engineers, and delivery professionals to improve next generation customer service systems....

  • Lead Supplier Industrialization Engineer, Interiors Trim

    1 week ago

    Newark, United States CareerBuilder Full time

    Due to significant growth, we are currently seeking a Lead Supplier Industrialization Engineer. Come work alongside some of the most accomplished minds in the industry. Beyond providing competitive salaries, were providing a community for innovators who want to make an immediate and significant impact. If you are driven to create a better, more sustainable...

  • Senior Data Engineer

    6 hours ago

    Newark, United States Epsilon Full time

    Amazon TA posted yesterday More... front runner Software Development Engineer Amazon TA Leipzig, SN 04109 posted 04/28/2024 More... front runner Field Service Engineer Danaher Lutherstadt Eisleben, Sachsen-Anhalt 06295 posted 04/29/2024 More... front runner Field Service Engineer Danaher Merseburg, Sachsen-Anhalt 06217 posted 04/29/2024 More... front runner...