Sr. Databricks Engineer

2 weeks ago

Dallas, TX, United States Diverse Lynx Full time

Sr. Databricks Engineer || Dallas , TX , Charlotte , NC ,Atlanta , GA , Richmond , VA || Fulltime

Job Description
Must Have Technical/Functional Skills
• 10+ years overall experience in data engineering or related fields.
• 3-5 years hands-on experience with Databricks and Spark.
• Strong proficiency in SQL and data analysis techniques.
• Experience with ETL processes, data modeling, and performance tuning.
• Familiarity with Python or Scala for data engineering tasks.
• Excellent problem-solving and communication skills.
Roles & Responsibilities
We are seeking a hands-on Sr. Databricks Data Engineer to design, develop, and optimize data pipelines and analytics solutions. The ideal candidate will have strong experience in data engineering, ETL development, and production support, ensuring reliable, scalable, and high-performing data operations within Azure environment and can work in a fast-paced environment. Knowledge of insurance domain and Power BI is a plus but not mandatory.
Development
• Design, develop, and deploy scalable ETL/ELT data pipelines using Apache Spark, PySpark, and Databricks.
• Develop and optimize SQL queries for data transformation and analysis.
• Collaborate with product owners, data architects and analysts to build data models, delta lake structures, and data workflows.
• Collaborate with data analysts and business teams to deliver actionable insights.
• Build job orchestration and monitoring solutions
• Ensure data quality, performance, and reliability across workflows.
• Develop and maintain CI/CD pipelines for Databricks notebooks, jobs, and workflows.
• Work with cloud-based data platforms (Azure preferred).

Production Support
• Provide L2/L3 support for Databricks production jobs, resolving incidents and performance issues.
• Monitor and troubleshoot Spark jobs, cluster performance, job failures, and resource utilization.
• Ensure data pipelines meet SLAs and data accuracy standards.
• Coordinate with infrastructure and DevOps teams to optimize cluster management and costs.
• Perform root cause analysis (RCA) and implement long-term fixes for recurring issues.
• Maintain documentation for production jobs, data flows, and incident management procedures.

Required Skills & Experience:
• 10+ years overall experience in data engineering or related fields.
• 3-5 years hands-on experience with Databricks and Spark.
• Strong proficiency in SQL and data analysis techniques.
• Experience with ETL processes, data modeling, and performance tuning.
• Familiarity with Python or Scala for data engineering tasks.
• Excellent problem-solving and communication skills.

Nice-to-Have:
• Knowledge of insurance industry data.
• Experience with Power BI or other BI tools.

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.

  • Sr. Databricks Engineer

    6 days ago

    Dallas, TX, United States Insight Global Full time

    Job Description We're looking for a seasoned Senior Data Engineer with hands-on expertise in Databricks to help architect, build, and refine data pipelines and analytics solutions. This role requires a strong background in data engineering, ETL development, and production support, with a focus on delivering scalable, high-performance data systems within an...

  • Sr. Databricks Engineer

    2 weeks ago

    Dallas, TX, United States Insight Global Full time

    We're looking for a seasoned Senior Data Engineer with hands-on expertise in Databricks to help architect, build, and refine data pipelines and analytics solutions. This role requires a strong background in data engineering, ETL development, and production support, with a focus on delivering scalable, high-performance data systems within an Azure cloud...

  • Sr. Databricks Engineer

    2 weeks ago

    Dallas, TX, United States Insight Global Full time

    We're looking for a seasoned Senior Data Engineer with hands-on expertise in Databricks to help architect, build, and refine data pipelines and analytics solutions. This role requires a strong background in data engineering, ETL development, and production support, with a focus on delivering scalable, high-performance data systems within an Azure cloud...

  • Sr. Databricks Engineer

    6 days ago

    Dallas, TX, United States Insight Global Full time

    We're looking for a seasoned Senior Data Engineer with hands-on expertise in Databricks to help architect, build, and refine data pipelines and analytics solutions. This role requires a strong background in data engineering, ETL development, and production support, with a focus on delivering scalable, high-performance data systems within an Azure cloud...

  • Senior Technical Solutions Engineer, Lakebase

    2 weeks ago

    Dallas, TX, United States Databricks Full time

    RDQ127R37 While candidates in the listed location(s) are encouraged for this role, candidates in other locations will be considered. As a Senior Technical Solutions Engineer, you will provide technical support for Databricks Lakebase related issues and resolve any challenges involving the Neon.com cloud-native Postgres service. You will assist customers in...

  • Senior Technical Solutions Engineer, Lakebase

    2 hours ago

    Dallas, TX, United States Databricks Full time

    RDQ127R37 While candidates in the listed location(s) are encouraged for this role, candidates in other locations will be considered. As a Senior Technical Solutions Engineer, you will provide technical support for Databricks Lakebase related issues and resolve any challenges involving the Neon.com cloud-native Postgres service. You will assist customers in...

  • Azure Databricks Engineer

    7 days ago

    Dallas, TX, United States Cedent Full time

    Key Responsibilities: Azure Databricks Architecture & Design: Design, build, and optimize scalable, high-performance data pipelines and data lakes using Azure Databricks. Architect and implement end-to-end analytics solutions leveraging Databricks and other Azure services (e.g., Azure Data Lake, Azure SQL, Azure Blob Storage). Lead the design of...

  • Azure Databricks Engineer

    2 weeks ago

    Dallas, TX, United States Cedent Full time

    Key Responsibilities: Azure Databricks Architecture & Design: Design, build, and optimize scalable, high-performance data pipelines and data lakes using Azure Databricks. Architect and implement end-to-end analytics solutions leveraging Databricks and other Azure services (e.g., Azure Data Lake, Azure SQL, Azure Blob Storage). Lead the design of...

  • Azure Databricks Engineer

    1 week ago

    Dallas, TX, United States Cedent Full time

    Key Responsibilities: Azure Databricks Architecture & Design: Design, build, and optimize scalable, high-performance data pipelines and data lakes using Azure Databricks. Architect and implement end-to-end analytics solutions leveraging Databricks and other Azure services (e.g., Azure Data Lake, Azure SQL, Azure Blob Storage). Lead the design of...

  • Sr. Data Engineer

    1 week ago

    Dallas, TX, United States TrinityRail Full time

    Trinity Industries is searching for a Sr. Data Engineer to join our Data Analytics team in Dallas, TX! The successful candidate will work with the Trinity Rail teams to develop and maintain data pipelines in Azure utilizing Databricks, Python and SQL. Join our team today and be a part of Delivering Goods for the Good of All! What you'll do: Facilitate...