Azure Data Engineer

12 hours ago

Irving, United States Siri InfoSolutions Inc Full time
Job DescriptionJob Description

Role: Azure Data Engineer

Location: Irving, TX or Downtown Minneapolis (Day 1 Onsite)

Position Overview

We are seeking a Lead Developer with advanced skills in Python, Apache Spark, Azure Synapse, and Azure Data Engineering services to develop and

manage ETL pipelines and data processing solutions that support AI/ML initiatives. The ideal candidate will have strong expertise in Azure Cloud,

including SQL, Data Factory, SQL Pools, Spark Pools, and Data Warehousing. A background in CI/CD processes, cross-functional collaboration, and

methodologies is essential. In addition, knowledge of Data Science, Java, Kubernetes, Azure Data Lake Storage (ADLS) Gen2, and API development is required.

Key Responsibilities

1. ETL and Data Pipeline Development

- Design, develop, and optimize scalable ETL processes using Python, Apache Spark, and Azure Synapse.

- Build and manage Azure Data Factory pipelines to orchestrate complex data workflows.

- Use SQL Pools and Spark Pools within Synapse to manage and process large datasets efficiently.

- Implement Data Warehousing solutions using Azure Synapse Analytics to provide structured and queryable data layers.

- Ensure the data platform supports real-time and batch AI/ML data requirements.

2. Azure Cloud Development & CI/CD Deployment

- Build, configure, and manage CI/CD pipelines on Azure DevOps for ETL and data processing tasks.

- Automate infrastructure provisioning, testing, and deployment using Infrastructure-as-Code (IaC) tools like ARM templates or Terraform.

- Optimize Azure Data Lake Storage (ADLS Gen2) to store and manage raw and processed data efficiently, ensuring proper access control and data security.

3. Cross-Functional Collaboration

- Collaborate with Data Scientists, Data Engineers, ML Engineers, and Business Analysts to translate business requirements into data solutions.

- Work with the DevOps and Security teams to ensure smooth and secure deployment of applications and pipelines.

- Act as the technical lead in designing, developing, and implementing data solutions, mentoring junior team members.

4. Data Engineering and API Development

- Develop and integrate with external and internal APIs for data ingestion and data exchange.

- Build, test, and deploy RESTful APIs for secure data access.

- Use Kubernetes for containerizing and deploying data processing applications.

- Manage data storage and transformation to support advanced Data Science and AI/ML models.

5. Agile Project Management

- Participate in and lead Agile ceremonies, such as sprint planning, daily stand-ups, and retrospectives.

- Collaborate with cross-functional teams in iterative development to ensure high-quality and timely feature delivery.

- Adapt to changing project priorities and business needs in an Agile environment.

Required Skills and Qualifications

1. Technical Skills:

- Expertise in Python and Apache Spark for large-scale data processing.

- Strong experience in Azure Synapse Analytics, including SQL Pools and Spark Pools.

- Advanced proficiency in Azure Data Factory for ETL pipeline orchestration and management.

- Knowledge of Data Warehousing principles, with hands-on experience building solutions on Azure.

- Experience with SQL, including complex queries, optimization, and performance tuning.

- Familiarity with CI/CD tools like Azure DevOps and managing infrastructure in Azure Cloud.

- Experience in Java for API integration and microservices architecture.

- Hands-on knowledge of Kubernetes for containerized data processing environments.

- Proficiency in working with Azure Data Lake Storage (ADLS) Gen2 for data storage and management.

- Experience working with APIs (REST, SOAP) and building API-based data integrations.

2. Agile and Cross-Functional Skills:

- Experience working in an Agile environment, using Scrum or Kanban.

- Ability to lead, mentor, and coach junior developers in the team.

- Strong collaboration skills to work with data scientists, analysts, and cross-functional teams to deliver end-to-end data solutions.

3. Behavioral Skills:

- Strong analytical and problem-solving skills with a passion for data-driven solutions.

- Excellent communication and presentation skills, able to explain complex technical concepts to non-technical stakeholders.

- Ability to work in a fast-paced, dynamic environment with changing priorities.

- Self-motivated and results-oriented with attention to detail.

  • Azure Data Engineer

    3 weeks ago

    Irving, United States Amtex Systems Inc. Full time

    They are looking someone with ADF experience, who knows how to build ADF brick by brick. AZURE Devops , unstructured data into ADF , Azure datafactory experience must have.Unstructured data Json API into ADF.Job Description:Position: Data EngineerDuration: 6 month contract to hireLocation: Irving, TX (Onsite)Required: LinkedIn Job Responsibilities: Must...

  • Data Engineering Expert

    1 week ago

    Irving, Texas, United States Siri InfoSolutions Inc Full time

    Job SummaryWe are seeking a highly skilled Data Engineering Expert to join our team at Siri InfoSolutions Inc. The ideal candidate will have expertise in designing, developing, and optimizing scalable data pipelines using Python, Apache Spark, and Azure Synapse Analytics.About the RoleThis is an exciting opportunity for a seasoned professional to take on a...

  • Azure Cloud Data Engineer

    1 month ago

    Irving, United States Motion Recruitment Full time

    We are looking for a Junior - Mid level Data Engineer with healthcare experience to join our team in Irving, Tx. This is a full-time hybrid position and will require 4 days a week in-office. Do you have a healthcare background in clinical data sets and a passion for enhancing patient and provider experience? In this role, you will be tasked with...

  • Sr Data Engineer

    1 month ago

    Irving, United States Staff Perm Full time

    Sr Data Engineer - Azure The Senior Data Engineer is responsible for the design, implementation and maintenance of all operational and reporting database systems. The Senior Data Engineer constructs conceptual, logical, and physical data models and develops and maintains business logic within stored procedures and ETL transformations. ***Local DFW, TX...

  • Data Engineer

    10 hours ago

    Irving, United States Talent Groups Full time

    Role: Cloud Data Engineer (Python, Spark, Synapse, Azure storage) for ETL and Data Processing in AI/ML Location: Minneapolis, MN or Irving, TX (3+ days in office/Week) Yrs. of experience: 10+ Yrs.Job Description:Position OverviewWe are seeking a Lead Developer with advanced skills in Python, Apache Spark, Azure Synapse, and Azure Data Engineering services to...

  • Data Engineer

    2 hours ago

    Irving, United States Talent Groups Full time

    Role: Cloud Data Engineer (Python, Spark, Synapse, Azure storage) for ETL and Data Processing in AI/ML Location: Minneapolis, MN or Irving, TX (3+ days in office/Week) Yrs. of experience: 10+ Yrs.Job Description:Position OverviewWe are seeking a Lead Developer with advanced skills in Python, Apache Spark, Azure Synapse, and Azure Data Engineering services to...

  • Senior Azure Data Factory Specialist

    2 months ago

    Irving, United States Motion Recruitment Full time

    We are partnered with a leading automotive financial organization to find a data engineer specializing in building ETL pipelines in Azure Data Factory. In this position you will be building & maintaining data pipelines from internal & external data sources supporting financial & other analytical/predictive teams. This position will require strong prior...

  • Senior Azure Data Factory Specialist

    15 hours ago

    Irving, United States Motion Recruitment Partners, LLC Full time

    We are partnered with a leading automotive financial organization to find a data engineer specializing in building ETL pipelines in Azure Data Factory. In this position you will be building & maintaining data pipelines from internal & external data sources supporting financial & other analytical/predictive teams. This position will require strong prior...

  • Cloud Data Engineer

    1 month ago

    Irving, United States Diverse Lynx Full time

    Role: Azure Data Engineer Location: Downtown Minneapolis (Day 1 Onsite) Full-Time Key Responsibilities: Position Overview We are seeking a Lead Developer with advanced skills in Python, Apache Spark, Azure Synapse, and Azure Data Engineering services to develop and manage ETL pipelines and data processing solutions that support AI/ML initiatives. The ideal...

  • Azure Software Engineer

    3 weeks ago

    Irving, Texas, United States Resource Informatics Group Full time

    Job Title: Azure DeveloperDescription:We are seeking a highly skilled Azure Developer to join our team at Resource Informatics Group. The ideal candidate will have a strong background in software development, cloud computing, and Azure services.Responsibilities:* Design and develop cloud solutions using Microsoft Azure services, including Azure Data Lake,...

  • Azure Developer

    4 weeks ago

    Irving, United States Resource Informatics Group Full time

    Client - WAB (Western Alliance Bank) Rate - Best Rate Please ( Need on our W2 ) Banking Experience not Mandatory. Please share resumes with .NET Development Background. Strong on Azure. Do not share people with multiple clouds, We need Strong Azure focused resource. Below is the Job description for the same - Job Title: Azure Developer / Engineer Location:...

  • Cloud Data Engineer

    7 days ago

    Irving, United States Programmers.io Full time

    One of our leading client is looking for Cloud Data Engineer in Irving TX/ Minneapolis MNRequired Skills and Qualifications 1. Technical Skills: - Expertise in Python and Apache Spark for large-scale data processing. - Strong experience in Azure Synapse Analytics, including SQL Pools and Spark Pools. - Advanced proficiency in Azure Data Factory for ETL...

  • Cloud Data Engineer

    1 month ago

    Irving, United States Programmers.io Full time

    One of our leading client is looking for Cloud Data Engineer in Irving TX/ Minneapolis MNRequired Skills and Qualifications 1. Technical Skills: - Expertise in Python and Apache Spark for large-scale data processing. - Strong experience in Azure Synapse Analytics, including SQL Pools and Spark Pools. - Advanced proficiency in Azure Data Factory for ETL...

  • Data Engineering Lead

    3 weeks ago

    Irving, Texas, United States ShiftCode Analytics Full time

    Job Description:At ShiftCode Analytics, we're looking for a seasoned Senior Data Engineer to join our team. As a key member of our data engineering team, you'll be responsible for designing, developing, and delivering large-scale data solutions.Key Responsibilities:Design and implement complex data pipelines using various technologies.Develop and maintain...

  • Azure DevOps engineer

    10 hours ago

    Irving, United States Resolve Tech Solutions Full time

    About the Company - [Provide a brief introduction to the company, its mission, and culture.]About the Role - [A short paragraph summarizing the key role responsibilities.]Responsibilities -One of our clients is looking for a Senior Azure DevOps engineerRole: Azure DevOps EngineerLocation: Irving, TXOnsite jobFull-time job Qualifications - [Education...

  • Data Engineer

    5 months ago

    Irving, United States FusionHealth Full time

    Job Description Job Description The Data Engineer will work cross-functionally with application development, data science, and product management to support growth, strategy, and development of innovative solutions internally and for our external clients. This position will work with operational and business leaders to deliver business intelligence and...

  • Data engineer

    7 months ago

    Irving, United States CyberTec Full time

    Client: Client, project implemented for Sonepar (Reminder- Do not post client info online) Data Engineer US Citizen ONLY Rate options: $75/hr w2 + benefits + $7/hr referral fee to you OR $ 8 0/hr c2c AIC Expected Start Date ASAP End Date (if applicable): 12/31/24 Location: Irving, on-site 3x per week - locals to DFW only at this time. Interview Process:...

  • Senior Data Architect with Azure Cloud

    7 months ago

    Irving, United States Resource Informatics Group Full time

    JOB TITLE: Senior Data Architect (Azure Cloud) SHIFT: 2 days in office (Mon and Tues Onsite!! Candidate needs to be local to the DMV area or relocate 1 week prior) LOCATION: Pennsylvania Ave, Washington DC DURATION: 6+ MonthsManager Notes: Looking for a senior data architect- someone who can define structures of data using data world for a new subject area...

  • Lead Data Engineer

    7 months ago

    Irving, United States IVID TEK INC Remote Work Freelance Full time $65 - $70

    Position: Lead Data Engineer (GCP,Azure)Location: Dallas, TXJob Type: Contract C2CRate: 65-70/ HrRoles and Responsibilities:Key Skills: Vertex AI , GCP , Kubeflow , CI/CD , Python , BigDataJob Description:Identify new opportunities to improve business processes and improve consumer experiences, and prototype solutions to demonstrate value with a crawl, walk,...

  • Senior Azure Devops Engineer

    5 months ago

    Irving, United States Resource Informatics Group Full time

    JOB TITLE: Senior Azure Devops Engineer SHIFT: 2 days onsite (Candidate needs to be local to the DMV area or relocate 1 week prior) (This is Not Negotiable) LOCATION: Pennsylvania Ave, Washington DC • Azure DevOps certification required Job Description Job Description • Minimum 8 years of experience • Prior experience of working with Azure DevOps...