DIRECTOR, DATA

About AbarVa

AbarVa is an AI-native enterprise transformation company that helps organizations move from strategy to execution. Our Nexus platform connects business, technology, data, vendors, programs, risk and value into one shared enterprise context so leaders can prioritize the right initiatives and deliver measurable outcomes. We build the company and the product alongside real enterprise transformations, currently in healthcare.

The role

We are hiring a Director, Data & AI Engineering to lead our US Databricks practice — a player-coach role, not architecture from a distance and not slideware. You will build and lead the engineering team and still own the hardest engineering yourself: architect a modern Databricks environment, set the engineering patterns, review and write code, and personally lead the implementations that matter.

You will operate at Databricks Solutions-Architect / Champion caliber, and you’ll be judged on whether the architecture becomes real — and on whether the engineers around you get better and ship faster because of you.

The work is a mix of greenfield build and forensic recovery. You will stand up ingestion from core operational, transactional and financial systems; assess and migrate existing SQL / ETL / data-warehouse estates — often undocumented — using Databricks Lakebridge to inventory code, score complexity and reconcile parity; and build the governance that makes the result trustworthy enough to act on.

This is an AI-native engineering role in both directions. You build with AI — Claude Code and Codex are part of the normal development loop, and you set the team standard for where to trust them and where to review hard. And you build for AI — everything you model must be consumable by Databricks Genie and Claude on AWS Bedrock, so semantics, metric definitions and column descriptions are architecture decisions, not documentation chores.

What you’ll do

• Lead and build: build and lead the US Databricks engineering team — hire, set standards, review code, mentor and grow engineers — while staying hands-on in the hardest problems yourself.
• Ingestion framework: design and build a metadata-driven ingestion framework so a new source is onboarded by configuration, not another hand-written pipeline — and hand it to the client’s team as a working asset.
• Estate assessment & migration: assess existing SQL, ETL, stored-procedure and warehouse estates with Databricks Lakebridge — inventory code, score complexity and dependencies, accelerate conversion, reconcile legacy-vs-Databricks parity — then turn the assessment into a sized, wave-based build plan.
• Lineage & recovery: trace lineage end to end and reconstruct undocumented business logic from analytics estates being decommissioned; validate against source code, job metadata and owners; record the unknowns.
• Medallion & data products: design scalable Bronze/Silver/Gold architectures and data products — named owner, contract, quality thresholds, lineage, documented consumers — run as a backlog and release cadence, not project phases.
• Governance: design and implement Unity Catalog governance — lineage, security, access control, tenant isolation, data-product patterns.
• Engineering velocity: stand up CI/CD and the toolchain for speed — Databricks Asset Bundles, Git workflows, automated testing, environment promotion, IaC, and observability that catches a broken pipeline before the business does.
• Data for AI: model and certify Gold data for natural-language consumption through Databricks Genie and Claude on Bedrock — clear semantics, defined metrics, described columns, certified tables.
• Partner & scale: partner directly with client CTO, CDAO, architecture, security and engineering leaders, and help build AbarVa’s Data & AI engineering capability as we scale.

What you bring

• Certification (required): an active Databricks certification — Data Engineer Professional, Solutions Architect, or an equivalent current credential. MVP or Champion status is a strong plus.
• Team leadership: proven experience building and leading data engineering teams — hiring, standards, code review, mentoring — while remaining personally hands-on.
• Hands-on depth: 12–18+ years of hands-on data engineering across modern data architecture, modeling and delivery, with exceptional depth in Databricks, Spark, SQL and Python.
• Ingestion engineering: demonstrated experience building metadata-driven ingestion frameworks — Auto Loader / Lakeflow, CDC, schema evolution, data-quality expectations and reconciliation.
• Lakebridge / migration: hands-on Databricks Lakebridge experie