Lead Data Engineer

3 weeks ago

Newark, United States FYI Solutions Full time

Responsibilities

Implement and support

· end-to-end data lake/warehousing/mart/business intelligence/ analytics/ services solutions (ingest, storage, integration, processing, services, access) in AWS

· data lake data intake/request/onboarding services and service documentation

· data lake ingestion services for batch/real time data ingest and service documentation

· data processing services (ETL/ELT) for batch/real time (Glue/Kinesis/EMR) and service documentation

· data storage services for data lake (S3)/ data warehouses (RDS/Redshift)/ data marts and service documentation

· data services layer including Athena, Redshift, RDS, microservices and APIs

· pipeline orchestration services including lambda, step functions, MWAA (optional)

· data security services (IAM/KMS/SM/encryption/anonymization/RBAC) and service documentation

· data access provisioning services (Accounts, IAM Roles RBAC), processes, documentation and education

· data provisioning services for data consumption patterns including microservices, APIs and extracts

· metadata capture and catalog services for data lake(S3/Athena), data warehouses (RDS/Redshift), Microservices/APIs

· metadata capture and catalog services for pipeline/log data for monitoring /support

· Implement CI/CD pipelines

· Prepare documentation for data projects utilizing AWS based enterprise data platform

· Implement high velocity streaming solutions using Amazon Kinesis, SQS, and SMS

· Migrate data from traditional relational database systems to AWS relational databases such as Amazon RDS, Aurora, and Redshift

· Migrate data from traditional file systems and NAS shares to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift

· Migrate data from APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift

· Provide cost/spend monitoring reporting for AWS based data platform data initiatives

· Provide governance/audit reporting for access of AWS based data platform

· Lead the implementation of a data lake strategy to enable LOBs and Corporate Functions with a robust, holistic view of data - driven decision making

· Serve as delivery lead for EDP data initiatives product owner

· Partner with immediate engineering team, product owner, IT, partners on EDP agenda

· Provide technology thought leadership, consulting, and coaching/mentoring

· Establish development, qa, stage and production migration/support processes

· Establish best practices for development and support teams

· Deliver end-end data initiatives from ingest-consume via microservices/ apis, jdbc/ odbc, file extracts etc.

· Work with scrum master to develop and own backlog, stories, epics, sprints

Qualifications

  • Bachelor's degree in Computer Science, Software Engineering, MIS or equivalent combination of education and experience
  • Experience implementing, supporting data lakes, data warehouses and data applications on AWS for large enterprises
  • Programming experience with Java, Python/ Scala, Shell scripting
  • Solid experience of AWS services such as CloudFormation, S3, Glue, EMR/ Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, SM etc.
  • Solid experience implementing solutions on AWS based data lakes
  • Experience implementing metadata solutions leveraging AWS non-relational data solutions such as ElastiCache and DynamoDB
  • AWS Solutions Architect or AWS Big Data Certification preferred

· Experience in AWS data lake/data warehouse/business analytics

· Experience and understanding of various core AWS services such as IAM, Cloud Formation, EC2, S3, EMR/Spark, Glue, Datasync, CloudHealth, CloudWatch, Lambda, Athena, and Redshift

· Experience in system analysis, design, development, and implementation of data ingestion pipeline in AWS

· Experience with DevOps and Continuous Integration/ Delivery (CI/ CD) concepts and tools

· Experience with business intelligence tools such as Tableau, Power BI or equivalent

· Knowledge of ETL/ ELT

· Experience in production support from Level 1 to Level 3

· Awareness of Data Management & Governance tools

· Working experience with Hadoop, HDFS, SQOOP, Hive, Python, and Spark is desired

· Experience working on Agile projects

  • Lead Salesforce Engineer

    4 hours ago

    Newark, United States MDMS Recruiting Full time

    Lead Salesforce Engineer ***W2 or Self Inc Only , no 3rd partyLocation: Newark, NJ (Hybrid)Duration: 12 Months +As a Lead Salesforce Platform Engineer, you will partner with product owners, tech leads, designers, engineers and delivery professionals to improve our next generation customer service systems. You will code, test and debug new and existing...

  • Data Engineer

    15 hours ago

    Newark, United States CognITek Group Full time

    We have a challenging opportunity for a Data Engineer for a high-growth and highly acquisitive $1B+ residential services company in Newark DE (this position can work remotely with periodic visits to corporate).The Data Engineer will design and maintain our client’s data Infrastructure and ETL processes and will collaborate with an amazing Data Analytics...

  • Data Engineer

    5 hours ago

    Newark, United States CognITek Group Full time

    We have a challenging opportunity for a Data Engineer for a high-growth and highly acquisitive $1B+ residential services company in Newark DE (this position can work remotely with periodic visits to corporate).The Data Engineer will design and maintain our client’s data Infrastructure and ETL processes and will collaborate with an amazing Data Analytics...

  • Data Engineer

    21 hours ago

    Newark, United States CognITek Group Full time

    We have a challenging opportunity for a Data Engineer for a high-growth and highly acquisitive $1B+ residential services company in Newark DE (this position can work remotely with periodic visits to corporate).The Data Engineer will design and maintain our client’s data Infrastructure and ETL processes and will collaborate with an amazing Data Analytics...

  • AWS Data Engineer

    3 weeks ago

    Newark, United States Iris Software Inc. Full time

    Iris's client, a large insurance company, is currently searching for a strong for AWS Data Engineer. Our client has offices all over the world and has the global reach, advisory services, and distribution power to meet the needs of issuers and investors worldwide. Job title: AWS Data EngineerLocation: Hybrid 2 days onsite (Newark, NJ)Duration: 24 monthsOnly...

  • AWS Data Engineer

    3 weeks ago

    Newark, United States Iris Software Inc. Full time

    Iris's client, a large insurance company, is currently searching for a strong for AWS Data Engineer. Our client has offices all over the world and has the global reach, advisory services, and distribution power to meet the needs of issuers and investors worldwide. Job title: AWS Data EngineerLocation: Hybrid 2 days onsite (Newark, NJ)Duration: 24 monthsOnly...

  • Cloud Data Engineer

    6 days ago

    Newark, United States Lucid Motors Full time

    As part of this position, you will work closely with experienced engineers to address intriguing and complex challenges within the ADAS/AD domain. Our team is actively gathering extensive data, amounting to Terabytes, to support autonomous driving operations. you will play a crucial role in developing and maintaining software solutions to process, manage,...

  • Lead Desktop Engineer

    6 days ago

    Newark, United States Tandym Group Full time

    A recognized services organization in New Jersey is actively seeking an experienced Desktop Engineer for a great Lead role with their team. In this position, the Lead Desktop Engineer will be responsible for overseeing the implementation to new technology within digital workspace. ***This is a Hybrid opportunity requiring the qualified professional to work...

  • Technical Lead(Salesforce)

    3 hours ago

    Newark, United States MDMS Recruiting Full time

    Our client is seeking a Salesforce Technidal Lead with . This is a Hybrid role (2-3 days in office in Newark, NJ) As a Lead Salesforce Engineer on the Technology team, you will partner with product owners, tech leads, designers, engineers and delivery professionals to improve our next generation customer service systems.  You will code, test and debug new...

  • Lead Customer Resident Engineer Electrification

    2 weeks ago

    Newark, United States Dana Corporation Full time

    Job Purpose The Lead Resident Engineer-Electrification (RE)/ is a highly visible technical, product & systems expert responsible for providing engineering support to one or more of Dana's main customers. Interfacing directly with external customers and internal teams to identify and implement Customer electrification solutions based on Dana's portfolio of...

  • Fixed Income Data Governance Engineer

    3 hours ago

    Newark, United States Combined Computer Resources Full time

    . We are in middle of a technology transformation to build a new platform on Cloud/AWS from ground up. We are seeking a highly skilled and experienced hands-on Informatica Data Governance Engineer to join our team. As an Informatica Data Governance Developer, your primary responsibility is to leverage the Informatica suite to implement and maintain...

  • Supervisor of Student Support Services

    21 hours ago

    Newark, United States Lead Charter School Full time

    **LEAD Charter School** - Job Description: Supervisor of Student Support Services_ **LEAD Charter School**, NJ's first alternative charter high school, seeks an exceptional educational leader to join its community of educators. The mission of LEAD is to enable students ages 16-21 in grades 9-12 in Newark, NJ to master the essential skills crucial for...

  • Technical Lead

    11 hours ago

    Newark, United States MDMS Recruiting Full time

    Job Description W2 or Self Inc consultants only No corp to corp (no subcontracting) Job description : As a Lead Salesforce Engineer on the Technology team, you will partner with product owners, tech leads, designers, engineers and delivery professionals to improve our next generation customer service systems. You will code, test and debug new and existing...

  • Senior Data Scientist

    3 weeks ago

    Newark, United States ZAR IT Solutions Full time

    Job DescriptionJob DescriptionSenior Data Scientist, Product and Insights Candidate must have the breadth and depth in various domains like Marketing Measurement, Causal Inference. Experience with graph neural networks, reinforcement learning. Experience with MCMC would be best.Description:Good storytelling starts with great listening. At Client, that means...

  • Technical Lead

    2 weeks ago

    Newark, United States MDMS Recruiting Full time

    Job Description W2 or Self Inc consultants only No corp to corp (no subcontracting) Job description : As a Lead Salesforce Engineer on the Technology team, you will partner with product owners, tech leads, designers, engineers and delivery professionals to improve our next generation customer service systems. You will code, test and debug new and existing...

  • Technical Lead

    3 weeks ago

    Newark, United States MDMS Recruiting LLC Full time

    Job DescriptionJob DescriptionW2 or Self Inc consultants only | No corp to corp (no subcontracting)Job description :As a Lead Salesforce Engineer on the Technology team, you will partner with product owners, tech leads, designers, engineers and delivery professionals to improve our next generation customer service systems. You will code, test and debug new...

  • Data Analyst

    4 minutes ago

    Newark, United States Toll International LLC Full time

    Toll International LLC seeks a Data Analyst to support the development and maintenance of its asset management program. This position requires a skilled individual to ensure data integrity and alignment with the business requirements and enterprise data standards. **Responsibilities**: - Conduct general data validation and cleanup to enhance the asset...

  • Data Analyst

    4 weeks ago

    Newark, United States Ecclesiastes Inc Full time

    Job DescriptionJob DescriptionPOSITION DUTIES:Analyze user’s data processing needs, manipulate or process large amounts of data using software applications, and convey the results of the data analysis to stakeholders. Position is based out of headquarters in Newark, DE but may be assigned to various unanticipated worksites throughout the US as determined...

  • Data Analyst

    2 days ago

    Newark, United States Macon Pace Inc Full time

    Job DescriptionJob DescriptionSECTION IA – GENERAL SKILL/KNOWLEDGE REQUIREMENTS The Data Analyst will complete general data validation and clean up in support of the Port Department’s development of its asset management program. The analyst will be responsible for preparing, managing, and administering data confirming its integrity and accuracy. Ensuring...

  • Lead Supplier Industrialization Engineer, Body Stampings

    11 hours ago

    Newark, United States Lucid Motors Full time

    Lead Supplier Industrialization Engineer, Body Stampings Leading the future in luxury electric and mobility At Lucid, we set out to introduce the most captivating, luxury electric vehicles that elevate the human experience and transcend the perceived limitations of space, performance, and intelligence. Vehicles that are intuitive, liberating, and designed...