Databricks Platform Engineer- Unity Catalog and DevOps

2 days ago

Philadelphia, PA, United States eSolutionsFirst, LLC Full-time

Databricks Platform Engineer- Unity Catalog and DevOps

Job Type: Contract (6-12 Months) possible extension

Location: Philadelphia, PA  - (Hybrid – Flexible Work Model)

Job description:

Job summary:

  • Program: Consolidation of five legacy data and BI platforms onto Azure Databricks and Power BI
  • We are seeking a platform engineer to build and automate an Azure Databricks environment governed by Unity Catalog.
  • Working under the platform lead, you will configure catalogs and permissions as code, build deployment pipelines, and support data engineering teams as they migrate workloads onto the platform.

Key responsibilities:

  • Configure Unity Catalog objects as code: catalogs, schemas, external locations, storage credentials and grants.
  • Implement access controls, including row filters and column masks, to the lead's design.
  • Build and maintain CI/CD pipelines for platform configuration, pipelines and jobs.
  • Manage repositories, branching and release promotion across environments.
  • Automate workspace, cluster policy and job configuration.
  • Monitor usage, cost and access using Databricks system tables.
  • Troubleshoot access, deployment and integration issues for engineering teams.