Managing Consultant, Databricks Data Engineering Lead

3 days ago

San Antonio, TX, United States Guidehouse Full-time $130,000 - $216,000/year

Job Family:

Data Science Consulting

Travel Required:

Up to 25%

Clearance Required:

Ability to Obtain Public Trust

What You Will Do

  • Lead Databricks data engineering and DataOps delivery across ingestion, transformation, orchestration, migration, integration, serving, and consumption patterns.
  • Translate architecture, data models, mappings, data contracts, and migration requirements into scalable technical designs and delivery plans.
  • Design and develop reusable pipelines, migration utilities, transformation frameworks, data-quality controls, reconciliation routines, and governed data products.
  • Establish standards for coding, testing, CI/CD, deployment, observability, logging, error handling, performance, security, and operational support.
  • Lead engineering reviews and troubleshoot Databricks job failures, pipeline bottlenecks, data-quality issues, and performance problems.
  • Support batch and, where applicable, streaming data-processing patterns using Databricks, Apache Spark, and Delta Lake.
  • Coordinate technical delivery with data architects, source-system teams, integration engineers, security specialists, governance leads, testing teams, and business stakeholders.
  • Lead technical estimates, capacity planning, backlog refinement, sprint execution, release readiness, and stabilization activities.
  • Mentor data engineers and ensure reusable code, design patterns, runbooks, and lessons learned are adopted across later modernization waves.
  • Communicate technical decisions, risks, dependencies, and delivery impacts to client and program leadership.

What You Will Need

  • Bachelor’s degree from an accredited universtiy.
  • Minimum SIX (6)+ years of experience.
  • Experience in data engineering, ETL/ELT, cloud data platforms, software development, or analytics engineering.
  • Hands-on experience designing and developing production Databricks solutions using Apache Spark, PySpark, Python, SQL, Delta Lake, notebooks, workflows, and jobs.
  • Experience building scalable batch data pipelines and reusable ingestion, transformation, and orchestration frameworks.
  • Experience with enterprise data migration, source-to-target mapping, data conversion, reconciliation, validation, and defect remediation.
  • Experience implementing data-quality controls, metadata capture, lineage, logging, monitoring, and technical auditability.
  • Experience with Git-based development, automated testing, CI/CD, release management, and production stabilization.
  • Experience leading data engineers and delivering complex data solutions in a client-facing environment.
  • Ability to obtain and maintain a Federal Public Trust.

What Would Be Nice to Have

  • Databricks Certified Data Engineer Professional or comparable Databricks certification.
  • Experience with Unity Catalog, Delta Live Tables, Databricks Workflows, Auto Loader, or related platform capabilities.
  • Experience with cloud storage, identity and access management, secrets management, networking, and platform security.
  • Experience with Azure DevOps, GitHub Actions, Jenkins, Terraform, or other infrastructure-as-code and deployment tools.
  • Experience developing APIs, data services, integration patterns, or event-driven pipelines.
  • Experience modernizing HR, ERP, workforce, financial, or other sensitive enterprise data.
  • Experience supporting federal agencies, regulated organizations, or public-sector modernization programs.
  • Experience with Agile Scrum or Kanban delivery.

What We Offer:

Guidehouse offers a comprehensive, total rewards package that includes competitive compensation and a flexible benefits package that reflects our commitment to creating a diverse and supportive workplace.

Benefits include:

  • Medical, Rx, Dental & Vision Insurance
  • Personal and Family Sick Time & Company Paid Holidays
  • Position may be eligible for a discretionary variable incentive bonus
  • Parental Leave and Adoption Assistance
  • 401(k) Retirement Plan
  • Basic Life & Supplemental Life
  • Health Savings Account, Dental/Vision & Dependent Care Flexible Spending Accounts
  • Short-Term & Long-Term Disability
  • Student Loan PayDown
  • Tuition Reimbursement, Personal Development & Learning Opportunities
  • Skills Development & Certifications
  • Employee Referral Program
  • Corporate Sponsored Events & Community Outreach
  • Emergency Back-Up Childcare Program
  • Mobility Stipend

The annual salary range for this position is $130,000.00-$216,000.00. Compensation decisions depend on a wide range of factors, including but not limited to skill sets, experience and training, securit