Lead Data Engineer

3 weeks ago

Newark, United States FYI Solutions Full time

Responsibilities

Implement and support

· end-to-end data lake/warehousing/mart/business intelligence/ analytics/ services solutions (ingest, storage, integration, processing, services, access) in AWS

· data lake data intake/request/onboarding services and service documentation

· data lake ingestion services for batch/real time data ingest and service documentation

· data processing services (ETL/ELT) for batch/real time (Glue/Kinesis/EMR) and service documentation

· data storage services for data lake (S3)/ data warehouses (RDS/Redshift)/ data marts and service documentation

· data services layer including Athena, Redshift, RDS, microservices and APIs

· pipeline orchestration services including lambda, step functions, MWAA (optional)

· data security services (IAM/KMS/SM/encryption/anonymization/RBAC) and service documentation

· data access provisioning services (Accounts, IAM Roles RBAC), processes, documentation and education

· data provisioning services for data consumption patterns including microservices, APIs and extracts

· metadata capture and catalog services for data lake(S3/Athena), data warehouses (RDS/Redshift), Microservices/APIs

· metadata capture and catalog services for pipeline/log data for monitoring /support

· Implement CI/CD pipelines

· Prepare documentation for data projects utilizing AWS based enterprise data platform

· Implement high velocity streaming solutions using Amazon Kinesis, SQS, and SMS

· Migrate data from traditional relational database systems to AWS relational databases such as Amazon RDS, Aurora, and Redshift

· Migrate data from traditional file systems and NAS shares to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift

· Migrate data from APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift

· Provide cost/spend monitoring reporting for AWS based data platform data initiatives

· Provide governance/audit reporting for access of AWS based data platform

· Lead the implementation of a data lake strategy to enable LOBs and Corporate Functions with a robust, holistic view of data - driven decision making

· Serve as delivery lead for EDP data initiatives product owner

· Partner with immediate engineering team, product owner, IT, partners on EDP agenda

· Provide technology thought leadership, consulting, and coaching/mentoring

· Establish development, qa, stage and production migration/support processes

· Establish best practices for development and support teams

· Deliver end-end data initiatives from ingest-consume via microservices/ apis, jdbc/ odbc, file extracts etc.

· Work with scrum master to develop and own backlog, stories, epics, sprints

Qualifications

  • Bachelor's degree in Computer Science, Software Engineering, MIS or equivalent combination of education and experience
  • Experience implementing, supporting data lakes, data warehouses and data applications on AWS for large enterprises
  • Programming experience with Java, Python/ Scala, Shell scripting
  • Solid experience of AWS services such as CloudFormation, S3, Glue, EMR/ Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, SM etc.
  • Solid experience implementing solutions on AWS based data lakes
  • Experience implementing metadata solutions leveraging AWS non-relational data solutions such as ElastiCache and DynamoDB
  • AWS Solutions Architect or AWS Big Data Certification preferred

· Experience in AWS data lake/data warehouse/business analytics

· Experience and understanding of various core AWS services such as IAM, Cloud Formation, EC2, S3, EMR/Spark, Glue, Datasync, CloudHealth, CloudWatch, Lambda, Athena, and Redshift

· Experience in system analysis, design, development, and implementation of data ingestion pipeline in AWS

· Experience with DevOps and Continuous Integration/ Delivery (CI/ CD) concepts and tools

· Experience with business intelligence tools such as Tableau, Power BI or equivalent

· Knowledge of ETL/ ELT

· Experience in production support from Level 1 to Level 3

· Awareness of Data Management & Governance tools

· Working experience with Hadoop, HDFS, SQOOP, Hive, Python, and Spark is desired

· Experience working on Agile projects

  • Lead Data Engineer

    1 week ago

    Newark, United States JSR Tech Consulting Full time

    Long term contract to hire. Lead data engineer role with a major Financial Services company. Hybrid work environment. 2 - 3 days onsite in Newark, NJ. - Engineering lead to help the design/build of the enterprise data platforms Design, Develop and Implement End-to-end data solutions (ingest, storage, integration, processing, access) on AWS. Data intake/...

  • Sr. Data Engineer

    1 week ago

    Newark, United States JSR Tech Consulting Full time

    Job Description : 8p - Engineering lead to help the design/build of the CDO enterprise data platforms Design, Develop and Implement End-to-end data solutions (ingest, storage, integration, processing, access) on AWS. Data intake/ request/ onboarding services and service documentation. Data ingestion services for batch/ real time data ingest and service...

  • Lead Cloud Engineer

    1 week ago

    Newark, United States Diverse Lynx Full time

    Job Tittle: Lead Cloud Engineer Location: Hybrid - Newark, NJ Duration: Contract Job Summary: Lead Cloud Engineer on the enterprise Cloud Business Office (CBO), you will focuses on Cloud Cost Management and evangelizing our FinOps principles within Prudential. You will also be collaborating with the broader CBO team on Governance and Demand Management...

  • Data Engineer

    1 week ago

    Newark, United States Diverse Lynx Full time

    Snowflake Data Engineer- snowflake , python Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the...

  • Data Engineer

    1 week ago

    Newark, United States 4 Staffing Corp Full time

    About the job Data Engineer (Financial Services) Data Engineer (Fixed Income Capital Markets) Full-time, direct employment Hybrid onsite/remote - Newark, NJ No visa sponsorship or C2C at this time Job Description: Our client is seeking a skilled and experienced Data Engineer with expertise in Fixed Income Capital Markets to join their dynamic team. The...

  • Data/Business Analyst

    1 week ago

    Newark, United States Diverse Lynx Full time

    Data/Business Analyst (Data Engineering) Newark - NJ onsite 2.Job summary Utilize knowledge of business analysis concepts and Software Development Life Cycle methodologies to support business initiatives and system enhancements within our Data Engineering Enablement team. Perform business process reviews and workflow changes by working with business users,...

  • AWS Data Engineer

    3 weeks ago

    Newark, United States Iris Software Inc. Full time

    Iris's client, a large insurance company, is currently searching for a strong for AWS Data Engineer. Our client has offices all over the world and has the global reach, advisory services, and distribution power to meet the needs of issuers and investors worldwide. Job title: AWS Data EngineerLocation: Hybrid 2 days onsite (Newark, NJ)Duration: 24 monthsOnly...

  • AWS Data Engineer

    3 weeks ago

    Newark, United States Iris Software Inc. Full time

    Iris's client, a large insurance company, is currently searching for a strong for AWS Data Engineer. Our client has offices all over the world and has the global reach, advisory services, and distribution power to meet the needs of issuers and investors worldwide. Job title: AWS Data EngineerLocation: Hybrid 2 days onsite (Newark, NJ)Duration: 24 monthsOnly...

  • Cloud Data Engineer

    2 days ago

    Newark, United States Lucid Motors Full time

    As part of this position, you will work closely with experienced engineers to address intriguing and complex challenges within the ADAS/AD domain. Our team is actively gathering extensive data, amounting to Terabytes, to support autonomous driving operations. you will play a crucial role in developing and maintaining software solutions to process, manage,...

  • Lead Desktop Engineer

    2 days ago

    Newark, United States Tandym Group Full time

    A recognized services organization in New Jersey is actively seeking an experienced Desktop Engineer for a great Lead role with their team. In this position, the Lead Desktop Engineer will be responsible for overseeing the implementation to new technology within digital workspace. ***This is a Hybrid opportunity requiring the qualified professional to work...

  • Lead Desktop Engineer

    1 week ago

    Newark, United States Tandym Group Full time

    A recognized services organization in New Jersey is actively seeking an experienced Desktop Engineer for a great Lead role with their team. In this position, the Lead Desktop Engineer will be responsible for overseeing the implementation to new technology within digital workspace. ***This is a Hybrid opportunity requiring the qualified professional to work...

  • Lead Customer Resident Engineer Electrification

    1 week ago

    Newark, United States Dana Corporation Full time

    Job Purpose The Lead Resident Engineer-Electrification (RE)/ is a highly visible technical, product & systems expert responsible for providing engineering support to one or more of Dana's main customers. Interfacing directly with external customers and internal teams to identify and implement Customer electrification solutions based on Dana's portfolio of...

  • Lead Supplier Industrialization Engineer, Interiors Cabin

    1 week ago

    Newark, United States Lucid Motors Full time

    Due to significant growth, we are currently seeking a Lead Supplier Industrialization Engineer. Come work alongside some of the most accomplished minds in the industry. Beyond providing competitive salaries, we’re providing a community for innovators who want to make an immediate and significant impact. If you are driven to create a better, more...

  • Senior Data Scientist

    2 weeks ago

    Newark, United States ZAR IT Solutions Full time

    Job DescriptionJob DescriptionSenior Data Scientist, Product and Insights Candidate must have the breadth and depth in various domains like Marketing Measurement, Causal Inference. Experience with graph neural networks, reinforcement learning. Experience with MCMC would be best.Description:Good storytelling starts with great listening. At Client, that means...

  • Technical Lead

    1 week ago

    Newark, United States MDMS Recruiting Full time

    Job Description W2 or Self Inc consultants only No corp to corp (no subcontracting) Job description : As a Lead Salesforce Engineer on the Technology team, you will partner with product owners, tech leads, designers, engineers and delivery professionals to improve our next generation customer service systems. You will code, test and debug new and existing...

  • Technical Lead

    2 weeks ago

    Newark, United States MDMS Recruiting LLC Full time

    Job DescriptionJob DescriptionW2 or Self Inc consultants only | No corp to corp (no subcontracting)Job description :As a Lead Salesforce Engineer on the Technology team, you will partner with product owners, tech leads, designers, engineers and delivery professionals to improve our next generation customer service systems. You will code, test and debug new...

  • Lead DevSecOps Engineer

    1 week ago

    Newark, United States VTal Technology Solutions Full time

    Seeking for Lead DevSecOps Engineer for a long-term contact, Need 2 days onsite every week. Selected candidate will get a chance to work with a large Insurance firmResponsibilities Provide security services and support for Client's business groups. Application vulnerability assessments automation. Support secure application development practices and a...

  • Data Analyst

    3 weeks ago

    Newark, United States Ecclesiastes Inc Full time

    Job DescriptionJob DescriptionPOSITION DUTIES:Analyze user’s data processing needs, manipulate or process large amounts of data using software applications, and convey the results of the data analysis to stakeholders. Position is based out of headquarters in Newark, DE but may be assigned to various unanticipated worksites throughout the US as determined...

  • Lead, Solution Architect

    1 week ago

    Newark, United States Prudential Ins Co of America Full time

    Job Classification: Technology - Architecture Are you interested in building capabilities that enable the organization with innovation, speed, agility, scalability, and efficiency? The Global Technology team takes great pride in our culture where digital transformation is built into our DNA! When you join our organization at Prudential, youll unlock an...

  • Senior Data Architect

    1 week ago

    Newark, United States 4 Staffing Corp Full time

    About the job Senior Data Architect - VP Senior Data Architect (VP Level) - Financial Services Full-time, direct employment Hybrid onsite/remote - Newark, NJ No visa sponsorship or C2C at this time Job Description: Our client is seeking a highly skilled Senior Data Architect at the VP level with extensive experience in designing and implementing...