Azure Data Engineer

2 weeks ago

Minneapolis, United States Compunnel Full time
Project Summary:

This project involves building AC360 data pipelines in Azure using Spark technologies. As part of this initiative, the development of Zafin data pipelines in Azure is also included.

Key Responsibilities:

1. Identify Requirements and Develop Solutions

· Identify requirements for RMS replacement and develop solutions in Azure.

· Re-engineer solutions to retrofit the rules with AC360 in Azure.

2. Data Pipeline Management

· Register data pipelines in Azure, onboard datasets, and enable data processing.

· Build Airflow DAGs and ensure scheduling is integrated.

3. DevOps and Production Management

· Apply a DevOps mindset, take ownership of production success, and optimize operations via automation, active alerting, and self-healing.

· Lead the resolution of production issues to ensure high availability and performance.

4. Code Development and Review

· Develop high-quality code and define best engineering practices.

· Perform peer code reviews to ensure successful deliverables with engineering excellence.

5. Documentation

· Document engineering artifacts such as technical design documents, flowcharts, system context diagrams, and code comments.

6. Collaboration

· Collaborate cross-functionally with product owners, data scientists, business users, project managers, and other engineers to achieve elegant solutions.

· Work with other engineers on the team to elevate technology and consistently apply best practices.

7. End-to-End Development and Testing

· Execute true end-to-end development and testing of software products in an agile lifecycle.

· Partner with key technical and business stakeholders to ensure the quality of the final product through manual and automated tests.

· Support the new software as it is moved to production.

Required Skills/Technologies:

· Mainframe COBOL: Good knowledge to understand the rules in Strata rules engine.

· Spark-Scala or Spark-SQL, Spark-Py: Minimum 5 years of experience.

· Airflow, Elastic, Azure: 3 years or higher.

· Big Data Management: 10 years or higher.

· Microservices-Based Applications: 5+ years.

· Azure, Kubernetes, PostGres: Experience required.

· Maven, Docker, Kubernetes, Kibana, ELK, Jenkins, Spring Boot: 4+ years.

· CI/CD Processes (Maven, Git, Jenkins): 4+ years.

· Cloud Technologies: 4+ years.

· Server-Side Languages (Python, Java, GoLang): 3+ years.

· Communication Skills: Excellent verbal and written communication skills, 10 years or higher.

· Technical, Logical, Analytical, and Problem-Solving Skills: 10 years or higher.

Preferred Skills/Attributes:

· Experience with Spark, Scala, Python, Azure stack.

Interview Process:

· Rounds: 1

· Format: Video

· Technical Level: Real-time coding session

Education: Bachelors Degree

  • Azure Data Engineer

    1 month ago

    Minneapolis, United States Diverse Lynx Full time

    Role: Azure Data Engineer Location: Downtown Minneapolis,MN (Day 1 Onsite) Contract Position Overview We are seeking a Lead Developer with advanced skills in Python, Apache Spark, Azure Synapse, and Azure Data Engineering services to develop and manage ETL pipelines and data processing solutions that support AI/ML initiatives. The ideal candidate will have...

  • Azure Data Engineer Specialist

    2 weeks ago

    Minneapolis, Minnesota, United States Diverse Lynx Full time

    Job OverviewDiverse Lynx LLC is a leading company in the field of data engineering, and we are currently seeking a highly skilled Azure Data Engineer Specialist to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and optimizing scalable ETL processes using Python, Apache Spark, and Azure Synapse....

  • Lead Azure Data Engineer

    1 month ago

    Minneapolis, United States Aloden, Inc. Full time

    Job Details: Lead Azure Data Engineer Duration: 12+ Month (s) Contract - 12m (possibility for extension or conversion) Location: - Hybrid Roles (3 Days Onsite/2 Days WFH) Note: ENTIRE TEAM IS OOO LAST 2 WEEKS OF THE YEAR INCLUDING CONTRACTORS - PLEASE MAKE SURE THEY ARE AWARE PRIOR TO SUBMISSION Job Descriptions: In this contingent resource assignment,...

  • Azure Data Scientist

    2 weeks ago

    Minneapolis, Minnesota, United States STIAOS Technologies Full time

    Job Summary: We are seeking an Azure Data Scientist to design, develop, and implement data engineering solutions using Azure cloud services. The estimated salary for this role is $160,000 per year.Key Responsibilities:• Develop and maintain data pipelines using Databricks (PySpark) for data ingestion, processing, and analysis.• Configure and manage Azure...

  • Azure Data Engineer

    4 weeks ago

    Minneapolis, United States Compunnel Inc. Full time

    Role: Azure Data Engineer Location: Downtown Minneapolis (Day-1 Onsite)Duration: W2 Contract (12+Months) Job responsibilities: Position OverviewWe are seeking a Lead Developer with advanced skills in Python, Apache Spark, Azure Synapse, and Azure Data Engineering services to develop and manage ETL pipelines and data processing solutions that support AI/ML...

  • Azure Data Engineer

    1 month ago

    Minneapolis, United States Futran Tech Solutions Pvt. Ltd. Full time

    Azure Data Engineer Minneapolis, MN ADF CICD Airflow SQL Cloud Databases Understanding of Agile methodologies 4-6 years of working experience in primary skills Overall 8-10 years in ETL/Data Engineering Understand business requirements. Understand source systems, source data and source data formats that are available on-prem / cloud. Design and build data...

  • Senior Azure Data Engineer Lead

    2 weeks ago

    Minneapolis, Minnesota, United States Aloden, Inc. Full time

    Job DescriptionWe are seeking an experienced Azure Data Engineer Lead to join our team at Aloden, Inc. As a key member of our engineering team, you will be responsible for leading the design, development, and implementation of large-scale data solutions on Microsoft Azure.The ideal candidate will have a strong understanding of relational database concepts,...

  • Azure Data Architect

    2 weeks ago

    Minneapolis, Minnesota, United States Omni Inclusive Full time

    We are looking for a skilled Senior Data Engineer to join our team at Omni Inclusive. This role offers a competitive salary range of $220,000 - $280,000 per year and comes with various benefits.About the RoleThe successful candidate will have extensive experience in designing and implementing large-scale data engineering solutions using Azure services.Main...

  • Senior Azure Data Engineer Position

    2 weeks ago

    Minneapolis, Minnesota, United States Eateam Full time

    Job Description: We are looking for an experienced Data Engineer to join our team at Eateam. As a Sr. Data Engineer, you will play a key role in designing and implementing data pipelines that drive business growth. Your primary responsibilities will include developing scalable data pipelines using Azure Data Factory and Azure Databricks, implementing data...

  • Azure Engineer

    4 weeks ago

    Minneapolis, United States Saxon Global Full time

    This is a 12 month contract to hire role with CHS. 100% remote. USC/GC. Linkedin Must. Salary on conversion is 110-125K. A growing industry is looking for a contract or likely contract to hire Azure Engineer. The primary responsibilities will be to build and enable an Azure cloud environment including include responsibility for the design/planning,...

  • Senior Data Engineer

    4 weeks ago

    Minneapolis, United States Allianz Full time

    At Allianz Life, we are guided by our mission - we secure your future - and by our common goal of caring for tomorrow. We care for our employees by promoting an inclusive culture where everyone has the opportunity to grow and be rewarded for their success. We care for our customers by creating new products that help them protect their families and pursue...

  • Azure Data Solutions Director

    2 weeks ago

    Minneapolis, Minnesota, United States Aloden, Inc. Full time

    Key ResponsibilitiesThe Senior Azure Data Engineer Lead will be responsible for leading the design, development, and implementation of large-scale data solutions on Microsoft Azure, as well as collaborating with cross-functional teams to ensure smooth project delivery.The successful candidate will have a strong understanding of relational database concepts,...

  • Senior Azure Data Scientist

    6 days ago

    Minneapolis, Minnesota, United States Compunnel Full time

    About the RoleWe're looking for an exceptional Senior Azure Data Scientist to join our team at Compunnel. As a key member of our data science team, you'll be responsible for designing, developing, and deploying cutting-edge data analytics solutions using Azure services and machine learning algorithms.The ideal candidate will have a Ph.D. in Computer Science,...

  • Data Engineer

    2 weeks ago

    Minneapolis, United States STIAOS Technologies Full time

    Required : 8+ Years of hands on experience on below skillset 1. Snowflake 2. Azure Databricks 3. Azure Data Factory 4. Pyspark 5. Python 6. SQL Role and Responsibilities: Design, implement, and manage Azure cloud solutions for various projects. • Develop and maintain data pipelines using Databricks (PySpark) for data ingestion, processing, and analysis....

  • Azure Spark Engineering Specialist

    2 weeks ago

    Minneapolis, Minnesota, United States Futran Tech Solutions Pvt. Ltd. Full time

    We are hiring a seasoned Azure Spark Engineering Specialist to join our team at Futran Tech Solutions Pvt. Ltd.The successful candidate will have extensive experience in designing, developing, and deploying scalable cloud-based data engineering solutions using Pyspark and Spark SQL programming.This role involves working closely with the development team to...

  • Compunnel Inc. | Azure Data Engineer

    4 weeks ago

    minneapolis, United States Compunnel Inc. Full time

    Role: Azure Data Engineer Location: Downtown Minneapolis (Day-1 Onsite)Duration: W2 Contract (12+Months) Job responsibilities: Position OverviewWe are seeking a Lead Developer with advanced skills in Python, Apache Spark, Azure Synapse, and Azure Data Engineering services to develop and manage ETL pipelines and data processing solutions that support AI/ML...

  • Data Engineer

    1 month ago

    Minneapolis, United States Eateam Full time

    Primary : SQL, Azure, Azure Data Factory, Azure DevOps, Azure DataBricks Snowflake, SQL Server, GitHub Secondary: Python, Pyspark Sr Data Engineer with expertise to design and create Data Pipelines in Azure Cloudusing Azure Data factory, ADLS, Azure Databricks, Log Analytics , Application Insight, Azure MonitorExpertise in writing python and SQL...

  • Data Engineering Leader

    2 weeks ago

    Minneapolis, Minnesota, United States Omni Inclusive Full time

    Omni Inclusive is offering a highly competitive salary of $250,000 - $300,000 per year for this Senior Data Engineer role. The position requires a strong background in developing and delivering scalable data solutions using Azure Databricks, Azure Data Lake, and Azure Cosmos DB.Job DescriptionWe are seeking an experienced Senior Data Engineer to join our...

  • Senior Data Engineer

    1 month ago

    Minneapolis, United States Omni Inclusive Full time

    Job Summary: We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. The ideal candidate will have a strong background in developing, and delivering scalable data solutions using Azure Databricks, Azure Data Lake, and Azure Cosmos DB. This role will involve coaching data engineers technically on topics like Azure...

  • Data Engineer Expert

    2 weeks ago

    Minneapolis, Minnesota, United States SysMind Tech Full time

    At SysMind Tech, we are seeking a skilled Data Engineer Expert to join our team. As a key member of our engineering department, you will be responsible for designing and implementing scalable data systems that meet the needs of our organization.The ideal candidate will have a strong background in data engineering with experience in structured databases...