Lead Big Data Engineer

4 weeks ago


Jersey City, United States Petadata Full time
Job DescriptionJob Description

Position: Lead Big Data Engineer

Location: Jersy City, NJ (Hybrid)

Experience: 10+ Years

Work type: Fulltime(W2) only

PETADATA is currently looking to hire for the position of Lead Big Data Engineer for one of their clients.

Roles & Responsibilities:

  • The Ideal candidate should be able to define the overall strategy and roadmap for Data-driven decision-making and work towards Big Data Technology solutions implementation.
  • Provide technical leadership and guidance towards creating cutting-edge technology solutions for the Bank.
  • Must be able to research, evaluate and utilize new technologies/tools/frameworks centered around high-volume data processing.
  • Develop a set process for Data mining, Data modeling, and Data production.
  • Should lead and guide your teams towards developing optimized high quality code deliverables, continual knowledge management and adherence to the organizational guidelines and processes.
  • Participate in project estimation, provide inputs for solution delivery, conduct technical risk planning, perform code reviews, and unit test plan reviews.
  • Understand requirements, create and review designs, validate the architecture and ensure a high level of service offerings to clients in the technology domain.
  • Define guidelines and best practices to drive consistency across client implementation and make sure consistency is followed across the board.
  • Work with the architecture engineering team to ensure quality solutions are implemented, and engineering best practices are adhered to.
  • Work closely with cross-functional leaders (development/product management/business stakeholders/ sales etc.) to ensure a clear understanding of the solution that will be built.

Required Skills:

  • The Ideal candidate should have 10+ years of experience in Data warehouse, SQL, and Cloud Platform (Azure) for the development of applications.
  • Must have in-depth and strong technical knowledge on Hive, Spark, Kafka, Streaming, SQL, Hadoop platform, and Python.
  • Required experience in data architecture, data quality, metadata management, ETL, analytics, reporting, and database administration
  • Should be an expert in ETL/Pipeline Development using tools such as Azure Databricks and Azure Data Factory with development expertise on batch and real-time data integration
  • Hands-on experience on major components of the Hadoop ecosystem such as Spark, HDFS, Hive, Sqoop, Kafka, and Streaming is a must.
  • Experience working with structured and unstructured data, Orchestration tools
  • Should be Expert in relational data processing technology like MS SQL, Delta Lake, Spark SQL, SQL Server, and GitHub.
  • The idle candidate should be Capable of handling more than one project in parallel.
  • Must have strong data analysis skills - ability to identify, analyze, and integrate various large complex data sources (Internal and external data providers) into a readily consumable data product.
  • Should have extensive knowledge in cloud platforms like Azure, AWS
  • Good oral and written communication skills required.

Educational Qualification:

Bachelor's/ Master's degree in Computer Science, Engineering, or a related field.

We offer a professional work environment and are given every opportunity to grow in the Information technology world.

Note:

Candidates required to attend Phone/Video Call / In person interviews and after Selection of candidate (He/She) should go through all background checks on Education and Experience.

Please email your resume to:swaroopb@petadata.co

After carefully review on your experience and skills one of our HR team members will contact you on the next steps.


  • Lead Data BSA

    6 days ago


    Jersey City, United States Creative Data Resources Full time

    Come Work with Us! At client, our culture is deeply supportive and rich in opportunity and reward. You will help our clients thrive and our communities prosper, empowered by a spirit of shared purpose. Whether you're helping clients find new opportunities, developing new technology, or providing expert advice to internal partners, you will be doing work that...


  • Jersey City, New Jersey, United States Tiger Analytics, LLC Full time

    As a Principal Data Engineer (Azure), you would have hands on experience working on Azure as cloud, Databricks and some exposure/experience on Data Modelling.using different Open Source, Big Data, and Cloud technologies on Microsoft Azure.Experience in implementing Data Lake with technologies like Azure Data Factory (ADF), PySpark, Databricks,...

  • Big Data Developer

    1 month ago


    Texas City, United States Involgix Full time

    Role Big Data Developer Duration 18+ Months Contract Location Austin TX Roles and Responsibilities Must have * Prior experience in developing application with big data technologies(hadoop spark kafka amazon aws) * Excellent in coding large scale application in java (at least 5+ years of experience in java) * Good understanding of SQL & relational...

  • Senior Big Data

    6 days ago


    Jersey City, United States Diverse Lynx Full time

    Role: Senior Big Data /Scala Developer Location: Jersey city NJ and Tampa , FL (Onsite) Contract JD:- Big Dat Hadoop Spark Scal Cluster Kafk Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated...


  • Jersey City, United States Tiger Analytics Full time

    Job DescriptionJob DescriptionTiger Analytics is a fast-growing advanced analytics consulting firm. Our consultants bring deep expertise in Data Science, Machine Learning and AI. We are the trusted analytics partner for several Fortune 100 companies, enabling them to generate business value from data. Our business value and leadership has been recognized by...


  • Jersey City, United States JPMorgan Chase & Co. Full time

    If you are excited about shaping the future of technology and driving significant business impact in financial services, we are looking for people just like you. Join our team and help us develop game-changing, high-quality solutions. As a Senior Lead Architect at JPMorgan Chase within the Consumer and Community Banking Data Platform team, you are an...

  • Lead Data Analyst

    2 weeks ago


    Jersey City, United States hackajob Full time

    The Analytical Data Operations (ADO) division is Verisk/ISO's data refinery, ensuring that ISO's analytical engines are properly fueled. As a member of ADO you will be part of a team of data analysts, BI developers, data engineers, data architects, and actuarial/analytical-engineers that are dedicated to improve our data analytics platform to support a host...


  • Jersey City, United States Tiger Analytics Full time

    Job DescriptionJob DescriptionTiger Analytics is a global AI and analytics consulting firm. With data and technology at the core of our solutions, we are solving problems that eventually impact the lives of millions globally. Our culture is modeled around expertise and respect with a team-first mindset. Headquartered in Silicon Valley, you’ll find our...


  • Jersey City, United States Tiger Analytics Full time

    Tiger Analytics is a global AI and analytics consulting firm. With data and technology at the core of our solutions, we are solving problems that eventually impact the lives of millions globally. Our culture is modeled around expertise and respect with a team-first mindset. Headquartered in Silicon Valley, you'll find our delivery centers across the globe...


  • Jersey City, United States Tiger Analytics Full time

    Tiger Analytics is a global AI and analytics consulting firm. With data and technology at the core of our solutions, we are solving problems that eventually impact the lives of millions globally. Our culture is modeled around expertise and respect with a team-first mindset. Headquartered in Silicon Valley, you'll find our delivery centers across the globe...


  • Jersey City, United States Verisk Analytics Full time

    As the Data Engineering Director, you will lead a team of data engineers responsible for building efficient, maintainable production-grade data pipelines. This role can be based in our Jersey City, NJ location (hybrid work model) or remote. This is a Data Engineer, Director, Engineer, Product Engineer, Project Management, Technology, Data

  • Data Analyst

    6 days ago


    Jersey City, United States Underwriting Solutions Full time

    **Company Description** We help the world see new possibilities and inspire change for better tomorrows. Our analytic solutions bridge content, data, and analytics to help business, people, and society become stronger, more resilient, and sustainable. The Analytical Data Operations (ADO) division of Verisk’s ISO business is the information refinery for...


  • Crystal City, United States ALTA IT Services Full time

    LEAD DATA ENGINEER – ACTIVE SECRET CLEARANCE – HYBRID ALTA IT Services has a direct hire opening for a Lead Data Engineer with an Active Secret clearance to support a leading DOD program. This is a hybrid position, 2-3 days a week onsite in Crystal City, VA. As a Lead Data Engineer, you will lead a team of data engineers and collaborate closely with data...


  • Jersey City, New Jersey, United States Verisk Full time

    Job Description As the Data Engineering Director, you will lead a team of data engineers responsible for building efficient, maintainable production-grade data pipelines. This role can be based in our Jersey City, NJ location (hybrid work model) or remote. This is a hands-on role with a small team, where you will have the opportunity to provide technical...

  • Cloud Data Engineer

    6 days ago


    Jersey City, United States ShiftCode Analytics Full time

    Title: Cloud Data Engineer Location: Charlotte, NC This role is 100% remote But Local Candidate is highly Preferred Sector: Banking Duration: 8 month (C2H) (USC or GC candidates Only***) Job Summary: Project Details: Job Description: Be the subject matter expert for data, pipeline design, architecture, and other data engineering technologies for the...

  • Senior QA Engineer

    2 weeks ago


    Charles City, United States binance Full time

    Binance is the leading global blockchain ecosystem and cryptocurrency infrastructure provider whose suite of financial products includes the world’s largest digital-asset exchange. Our mission is to accelerate cryptocurrency adoption and increase the freedom of money. If you’re looking for a fast-paced, mission-driven organization where opportunities to...


  • Jersey City, New Jersey, United States Royal Bank of Canada Full time

    Come Work with UsAt RBC, our culture is deeply supportive and rich in opportunity and reward. You will help our clients thrive and our communities prosper, empowered by a spirit of shared purpose.Whether you're helping clients find new opportunities, developing new technology, or providing expert advice to internal partners, you will be doing work that...

  • Big Data developer

    1 month ago


    Ohio City, United States Infoprompt Inc Full time

    Responsibilities Participate in Team activities Design discussions Stand up meetings and planning Review with team. Perform data analysis data profiling data quality and data ingestion in various layers using big data/Hadoop/Hive/Impala queries PySpark programs and UNIX shell scripts. Follow the organization coding standard document Create mappings ...


  • Jersey City, United States Delta System & Software, Inc. Full time

    Position: LEAD Level Azure Databricks Data Engineer with Finance Experience Duration: 12+ months Location: Hybrid In Jersey City, NJ (Wednesday & Thursday) from DAY 1 Rate: $75/hr – W2 Job DescriptionsAZURE experience: Azure Data Factory (ADF), ADLS Gen 2, Databricksyears database/warehouse design, and development, prior experience with SQL Server SSIS...


  • Jersey City, United States Delta System & Software, Inc. Full time

    Position: LEAD Level Azure Databricks Data Engineer with Finance Experience Duration: 12+ months Location: Hybrid In Jersey City, NJ (Wednesday & Thursday) from DAY 1 Rate: $75/hr – W2 Job DescriptionsAZURE experience: Azure Data Factory (ADF), ADLS Gen 2, Databricksyears database/warehouse design, and development, prior experience with SQL Server SSIS...