Azure Data Engineer

2 weeks ago

Allendale, United States Diverse Lynx Full time
Job Title Azure Data Engineer Relevant Experience
(in Yrs) 5 - 6 yrs Technical/Functional Skills Azure data factory, data bricks, data lake, automation and performance optimization of ETL Experience Required
  • Good Knowledge of Data Brick lakehouse and Azure DataLake concept
  • Knowledge of Data Bricks delta concept- Delta live tables (DLT)
  • Strong hands-on experience in ELT- pipeline development using Azure Data factory and Databricks Autoloader, Notebook scripting and Azure Synapse Activity Copy, Data Flow Task
  • Strong knowledge of metadata-driven data pipeline, metadata management, dynamic logic
  • In-depth knowledge of data storage solutions, including Azure Data Lake Storage (ADLS), and Azure Serverless SQL Pool.
  • Experience with data transformation using Spark, and SQL technologies.
  • Solid understanding of design patterns, and best practices of the cloud stack.
  • Experience with code management and version control using Git or similar tools.
  • Strong problem-solving and debugging skills in ETL workflows and data pipelines.
  • Strong understanding of Azure Data bricks and Azure Synapse internals - features and capabilities.
  • Knowledge of Azure DevOps and continuous integration and deployment (CI/CD) process.
  • Knowledge of data quality and data profiling techniques, with experience in data validation and data cleansing.
Roles & Responsibilities
  • Conducting technical sessions, design reviews, code reviews, and demos of pipelines and their functionality
  • Developing technical specification for Data pipelines and workflow and getting sign-off from Architect and Stryker leads.
  • Developing, deploying, and maintaining workflows and data pipelines using Azure Data bricks and Azure Synapse.
  • Developing pipeline/notebook for Delta live tables ( DLT) Databricks Auto Loader, Notebook scripting and Azure Synapse Activity Copy, Data Flow Task.
  • Collaborating with data architects, data analysts, and other stakeholders to design and implement ETL solutions that meet business requirements.
  • Writing efficient and high-performing ETL code using PySpark, and SQL technologies.
  • Building and testing data pipelines using Azure Data bricks and Azure Synapse.
  • Ensuring the accuracy, completeness, and timeliness of data being processed and integrated.
  • Troubleshooting and resolving issues related to data pipelines and notebooks.
  • Performance benchmarking of data ingestion and Data flow pipeline/notebook and ensuring consistency
Generic Managerial Skills Project Planning , Team management, Stakeholders Management

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.

  • Azure Data Engineer

    3 weeks ago

    Allendale, United States Tata Consultancy Services Full time

    Good Knowledge of Data Brick lakehouse and Azure DataLake concept Knowledge of Data Bricks delta conceptDelta live tables (DLT) Strong hands-on experience in ELTpipeline development using Azure Data factory and Databricks Autoloader, Notebook scripting and Azure Synapse Activity Copy, Data Flow Task Strong knowledge of metadata-driven data pipeline,...

  • Integration Specialist

    4 weeks ago

    Allendale, United States CareerBuilder Full time

    Stark Techis seeking aIntegration Specialistwith a strong background in Temperature Control building controls systems to join our team. If you have the ability to configure, verify and operate these systems and other low voltage control sub-systems to meet project requirements, we want to talk to you! How are you successful in this role? You should be able...

  • Integration Specialist

    4 weeks ago

    Allendale, United States Stark Tech Full time

    Job DescriptionJob DescriptionStark Tech is on the lookout for an Integration Specialist proficient in Temperature Control building controls systems to become a part of our dynamic team. If you possess the skills to configure, validate, and manage these systems along with other low voltage control sub-systems to fulfill project specifications, we are eager...

  • Integration Specialist

    2 weeks ago

    Allendale, United States Stark Tech Group Full time

    Allendale, NJ, USA | Integration | Full Time Stark Tech is on the lookout for an Integration Specialist proficient in Temperature Control building controls systems to become a part of our dynamic team. If you possess the skills to configure, validate, and manage these systems along with other low voltage control sub-systems to fulfill project specifications,...

  • Automotive Technician

    3 weeks ago

    Allendale, United States Abacus Serivice Corporation Full time

    Job DescriptionJob DescriptionResponsibilities:Perform detailed and thorough inspections and repair of all test vehiclesPerform software and hardware updates including reporting systems with guidance from Sr. TechniciansPerform scheduled and unscheduled maintenance of test vehicles as definedProvide full vehicle support for entire concern test vehicles as...