Data Engineering Expert for Azure Databricks Lakehouse

6 days ago


Chicago, Illinois, United States Lorven Technologies Full time

Lorven Technologies is seeking a skilled Data Engineering Expert to join our team in Chicago, IL. As an integral part of our organization, you will design and develop scalable data platforms using Azure Databricks.

Estimated Salary: $140,000 - $170,000 per year, depending on experience.

Job Description

We are looking for a seasoned professional with 6+ years of experience in data development, including 3+ years of experience working with Azure Databricks or Hadoop/HDFS. You will be responsible for designing and supporting business databases and table schemas, creating ETL processes, and ensuring the data platform's scalability and performance.

In this role, you will collaborate closely with Product Owners, application engineers, and other data consumers to gather and deliver high-quality data. Your analytical skills will help you quickly understand business requirements and translate them into effective technical data designs.

You will also play a crucial role in implementing Chaos Engineering practices and measures to ensure the end-to-end infrastructure functions as expected even if individual components fail.

Required Skills and Qualifications
  • 6+ years of professional data development experience
  • 3+ years of experience with Azure Databricks or Hadoop/HDFS
  • 3+ years of experience with PySpark/Spark
  • 3+ years of experience with SQL
  • 3+ years of experience developing with Python
  • Familiarity with ETL concepts and data warehousing concepts
  • Data modeling and query optimization skills
  • Experience with CI/CD and version control software
  • Strong understanding of Agile Principles (Scrum)
  • Experience with Azure and Databricks Delta Tables, Delta Lake, and Delta Live Tables
Bonus Points For Experience In The Following
  • Proficient with relational data modeling
  • Experience with Python library development
  • Experience with structured streaming (Spark or otherwise)
  • Experience with Kafka and/or Azure Event Hub
  • Experience with GitHub SaaS/GitHub Actions
  • Experience with Snowflake/exposure to BI tooling (Tableau, Power BI, Cognos, etc.)


  • Chicago, Illinois, United States Sogeti Full time

    Sogeti – Expert Data EngineerAt Sogeti, we strive to deliver top-notch professional technology services, leveraging our expertise in methodologies and global delivery models. Our mission is to consistently exceed the expectations of both our employees and clients.What You'll DoAs a skilled Data Engineer, you will be responsible for:Supporting a dynamic...


  • Chicago, Illinois, United States CapB InfoteK Full time

    **Job Title:** Senior Data Engineer - Databricks DeveloperAt CapB InfoteK, we are seeking an experienced Data Engineer to join our team as a Senior Data Engineer - Databricks Developer. This is a full-time or contract position.**Key Responsibilities:


  • Chicago, Illinois, United States Cyclotron, Inc. Full time

    Cyclotron, Inc. is a company that values innovation and excellence in the field of data engineering.We are seeking a highly skilled Cloud Data Engineering Leader to join our team in Anywhere in US or Canada (fully remote).The estimated salary for this position is $180,000 - $250,000 per year.Job Description:We are looking for a seasoned data engineer with...


  • Chicago, Illinois, United States Lorven technologies Full time

    Job DescriptionLorven Technologies is seeking a highly skilled Cloud-Based Data Science Expert to join our team.Estimated Salary: $150,000 - $200,000 per year, depending on experience.Location: Chicago, IL (Remote or Onsite)Duration: 6+ monthsContract Position.Key Responsibilities:Develop and implement data science solutions using Azure DataFactory, ADLS,...


  • Chicago, Illinois, United States PamTen Full time

    PamTen is seeking an experienced Cloud Solutions Architect to join our Agile team and contribute to the development of healthcare applications. This is a challenging role that requires expertise in designing and implementing scalable, high-performance, and cost-effective cloud-based solutions using Azure services and Databricks.About the RoleThe ideal...


  • Chicago, Illinois, United States Databricks Full time

    Databricks is seeking a highly motivated Business Development Representative to represent the company to prospective customers. This role is ideal for sales professionals early in their career who are looking to help scale a game-changing enterprise software company and develop their sales career.About the RoleYou will be part of a dynamic team of...


  • Chicago, Illinois, United States Databricks Full time

    Job SummaryDatabricks is seeking a seasoned industry expert to lead our efforts in developing and implementing effective enablement strategies across key verticals. This role will be responsible for driving the creation of industry-specific content, training programs, and sales playbooks to empower our sales teams with the knowledge and tools needed to...


  • Chicago, Illinois, United States Wavicle Data Solutions Full time

    About the RoleWe are seeking a seasoned Data Solutions Architect to join our Emerging Technologies practice. The ideal candidate will share our passion for reinventing the data and analytics landscape and thrive in dynamic environments.The successful candidate will lead client-facing projects, helping clients build and deploy solutions using our portfolio of...


  • Chicago, Illinois, United States Databricks Inc. Full time

    Databricks Inc. is seeking a visionary Global Enablement Strategist to spearhead the development and implementation of industry-specific enablement strategies across our major verticals.This pivotal role will own the strategy and creation of content for Financial Services, Retail, Manufacturing, Healthcare and Life Sciences, Public Sector, and Media and...


  • Chicago, Illinois, United States Tiger Analytics Full time

    At Tiger Analytics, we are on a mission to unlock business value from data. We are seeking an exceptional Data Architect to join our team of experts in designing and implementing advanced analytics capabilities.The role involves architecting, designing, and implementing batch and streaming analytics, machine learning models, natural language generation, and...


  • Chicago, Illinois, United States WEX, Inc. Full time

    About WEX, Inc.WEX, Inc. is a leading provider of business solutions for the automotive repair industry and other markets. Our sophisticated business solutions empower a diverse range of customers, generating rich and complex data that holds immense potential to drive value for our customers and the business.The Data TeamThe Data team at WEX, Inc. builds big...


  • Chicago, Illinois, United States WTW inc. Full time

    Job Title: Senior Cloud Data EngineerJob Summary:We are seeking a skilled Senior Cloud Data Engineer to join our team at WTW inc. As a Senior Cloud Data Engineer, you will play a pivotal role in designing, building, and optimizing complex data pipelines and ETL processes while leveraging your expertise in Azure Synapse and PySpark to build advanced data...


  • Chicago, Illinois, United States Diverse Lynx Full time

    Job Title: Azure DevOps EngineerAbout the Role:We are seeking a skilled Azure DevOps Engineer to join our team at Diverse Lynx LLC. As a key member of our infrastructure team, you will be responsible for designing, deploying, and managing infrastructure solutions on Microsoft Azure. Your expertise in Azure DevOps, Azure Infrastructure, CI/CD, and Azure VMs...


  • Chicago, Illinois, United States eTek IT Services, Inc. Full time

    Job SummaryeTek IT Services, Inc. is seeking a seasoned Senior DevOps Engineer to lead the development of our cloud-native infrastructure on Azure. This is a multi-year effort that requires strong expertise in Azure services, Terraform, Github Actions, Kubernetes administration, and Helm charts.About the RoleThis position involves working closely with our...


  • Chicago, Illinois, United States eTek IT Services, Inc. Full time

    Job DescriptionJob SummaryAzure DevOps/SRE position available at eTek IT Services, Inc. in Downtown Chicago. The role requires 3 days on-site per week. Contract duration is 12 months with possible extension. Functional RequirementsThe successful candidate will work in the Financial Services Domain, primarily in Wealth management. Key responsibilities...


  • Chicago, Illinois, United States NielsenIQ Full time

    Job OverviewNielsenIQ is seeking a Senior Data Engineer with hands-on experience in Python and Pyspark to support software development activities in an Agile environment. The ideal candidate will possess strong technical, analytical, and interpersonal skills.About NielsenIQNielsenIQ is the world's leading consumer intelligence company, delivering...

  • Lead Data Engineer

    3 days ago


    Chicago, Illinois, United States Capital One Full time

    Job OverviewWe are seeking a highly skilled Lead Data Engineer to join our team at Capital One. As a key member of our engineering community, you will be responsible for designing, developing, testing, implementing, and supporting technical solutions in full-stack development tools and technologies.In this role, you will work closely with a team of...

  • Data Engineer

    6 days ago


    Chicago, Illinois, United States Sinch Full time

    {"Roles and Responsibilities": "Data Mediation DevelopmentDesign and implement mediation systems to collect, transform, and process raw usage data into standardized formats for billing and analytics platforms.ETL Pipeline CreationBuild and maintain efficient ETL processes to process high volumes of usage data, ensuring data accuracy, consistency, and...


  • Chicago, Illinois, United States Saxon Global Full time

    Data Engineering Expertise SoughtWe are in search of a skilled data engineer to contribute to our high data volume application. This critical system demands ongoing production maintenance and enhancements through integration with Machine Learning (ML) and Master Data applications. Future plans include migrating the existing AWS EMR application to AWS...


  • Chicago, Illinois, United States Inficare Full time

    At Inficare, we're looking for a seasoned Cloud Solutions Expert with expertise in Azure. As an experienced Azure Developer, you'll play a key role in designing and implementing cloud-based solutions that drive business growth.Job Highlights:Estimated Salary: $80/hourLocation: Chicago, IL9-13 years of experience requiredAbout the Role:We're seeking a highly...