Senior Azure Data Engineer with Databricks

19 hours ago
Chicago, Illinois, United States Confiar Services Full-time
Job Summary We are seeking a highly experienced Senior Azure Data Engineer with strong Databricks expertise to design, develop, optimize, and maintain scalable cloud-based data solutions. The ideal candidate will have strong hands-on experience with SQL Server, Azure SQL, T-SQL, Databricks, Apache Spark, PySpark, and data warehousing. The candidate will work closely with data engineers, application developers, BI teams, and business stakeholders to build reliable enterprise data platforms and support analytics workloads.

Key Responsibilities
Database Development & Optimization
• Design, develop, and maintain complex SQL Server and Azure SQL databases.
• Develop advanced T-SQL queries, stored procedures, functions, and views.
• Perform query tuning, indexing, and database performance optimization.
• Implement database security, data integrity, availability, and reliability best practices.
• Troubleshoot database and production issues and perform root cause analysis. Azure & Databricks
• Develop and manage data solutions using Azure SQL Database.
• Design and implement data processing solutions using Databricks and Apache Spark.
• Develop large-scale data transformations using PySpark, Spark SQL, and SQL.
• Integrate Azure SQL with Databricks and support Delta Lake implementations.
• Work with cloud-based and hybrid data environments.
• Optimize Databricks/Spark workloads for performance and scalability. Data Modeling & Data Warehousing
• Design and maintain relational and dimensional data models.
• Implement data warehousing best practices and patterns.
• Develop and maintain Slowly Changing Dimensions (SCD).
• Support ETL/ELT pipelines and enterprise data integration processes.
• Ensure data quality, consistency, and integrity across data platforms. Collaboration & Leadership
• Collaborate with application developers, data engineers, BI teams, and business stakeholders.
• Translate business and technical requirements into scalable data solutions.
• Conduct code reviews and enforce database development standards.
• Provide technical guidance and mentoring to junior developers.
• Participate in troubleshooting, production support, and performance improvement initiatives. Required Skills
• 8+ years of experience in Database Development, Data Engineering, or a similar role.
• Strong expertise in SQL Server and Azure SQL.
• Advanced knowledge of T-SQL.
• Strong experience in query optimization, performance tuning, and indexing.
• Hands-on experience with Databricks and Apache Spark.
• Experience with PySpark and Spark SQL.
• Strong understanding of Delta Lake.
• Experience working with Microsoft Azure cloud environments.
• Strong knowledge of data warehousing and dimensional modeling.
• Experience implementing Slowly Changing Dimensions (SCD).
• Experience with ETL/ELT processes.
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and collaboration skills.

  • Data Engineer

    3 days ago

    Chicago, Illinois, United States Collectiv Remote Full-time

    Role Overview The Data Engineer is responsible for designing, implementing, and supporting modern data platforms using Microsoft Fabric and Databricks, with a strong emphasis on lakehouse architecture, analytics engineering, and semantic modeling. This role is client-facing and requires translating business requirements into scalable technical solutions...

  • Senior Data Engineer

    4 days ago

    Chicago, Illinois, United States Capgemini Full-time $133,000 - $153,000 Contract

    Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you’d like, where you’ll be supported and inspired by a collaborative community of colleagues around the world, and where you’ll be able to reimagine what’s possible. Join us and help the world’s leading organizations unlock the value of...

  • Senior Manager Data Engineer

    3 days ago

    Chicago, Illinois, United States Capital One Full-time $209,000 - $238,500 Contract

    Senior Manager Data Engineer (Databricks, Pyspark, Snowflake)Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors who solve real problems...

  • Data Engineer – Databricks

    3 days ago

    Chicago, Illinois, United States Artech Full-time $60 - $63 Permanent

    Requisition ID: 107458 -1 Title: Data Engineer – Databricks / Snowflake Duration: 6-12 months with possible extension Location: Chicago IL (Hybrid – 3 days a week onsite) Salary Range: $60- $63 an hour on W2/C2C Job Description: Pipeline Development Design| implement| and optimize scalable ETL/ELT pipelines using Databricks (PySpark| SQL|...

  • Senior Azure Engineer

    3 days ago

    Chicago, Illinois, United States 1872 CONSULTING Full-time

    Senior Azure Engineer Chicago, IL – hybrid WFH (2 days WFH, 3 days onsite in the loop) Summary: The Sr. Azure Engineer sits on a team of 10, and will focus on Cloud Engineering, with some DevOps, in an Azure environment. This person will specifically support data, software and AI engineering teams. You will be one of 2 Azure SMEs, and given the other...

  • Data Engineer Snowflake and Databricks

    3 days ago

    Chicago, Illinois, United States FinTrust Connect Full-time $85 - $165 Permanent

    Data Engineer Snowflake and Databricks -Chicago, IL -Hybrid FinTrust Connect Chicago IL Hybrid Share Your Resume and Build Your Future! Join our Talent Community for Chicago. Winning posts here stress Azure aligned stacks with Databricks plus Snowflake and Azure Data Factory and Synapse or SQL Server integration, and clear impact on risk and finance...

  • Sr. Azure Data Engineer with AI Experience

    3 days ago

    Chicago, Illinois, United States Citiustech Full-time $60 - $65 Permanent

    Skills Azure DataBricks ADF, DataBricks, Python, Pyspark, ETL, SQL, LLM, RAG, AI/ML.   Mandatory skills 10+ years in data engineering, with recent hands-on delivery ownership. Strong Databricks and PySpark jobs, workflows, Delta Lake, performance tuning Expert SQL including the ability to read unfamiliar, poorly documented SQL at volume and reason about...

  • Senior Databricks Engineer

    3 days ago

    Chicago, Illinois, United States UST Full-time $106,000 - $159,000 Contract

    Senior Databricks Engineer Associate Data Architect I Who We Are: Born digital, UST transforms lives through the power of technology. We walk alongside our clients and partners, embedding innovation and agility into everything they do. We help them create transformative experiences and human-centered solutions for a better world. UST is a...

  • Sr Data Quality Engineer – Azure Focus

    3 days ago

    Chicago, Illinois, United States 1872 CONSULTING Full-time

    Sr Data Quality Engineer – Azure Focus Chicago, IL (loop/downtown) – 2 days work from home (WFH), 3 days onsite Summary: This position focuses on building automated tesing frameworks to ensure data quality and reliability across data pipelines in our Azure, Databricks, SQL Server/Azure SQL and Medallion Architecture (Bronze, Silver, Gold layers)...

  • Senior Azure DevOps Engineer

    3 days ago

    Chicago, Illinois, United States 1872 CONSULTING Full-time

    Senior Azure DevOps Engineer – 100% Remote Essential Job Functions The Senior Azure DevOps Engineer will be the first Azure/cloud focused DevOps Engineer that we bring onboard. We have traditionally been on-prem, and are just starting our migration to the cloud. We are a global enterprise, so this will be an exciting project for the right person to help...

Similar jobs in Chicago

Top cities