Senior Databricks Engineer

1 week ago


Kansas City, Missouri, United States Diverse Lynx Full time
Job Title: Senior Databricks Engineer

We are seeking a highly skilled Senior Databricks Engineer to join our team at Diverse Lynx LLC. As a Senior Databricks Engineer, you will be responsible for designing and implementing complex data solutions aligned with business objectives. You will work closely with our team to develop and maintain our data infrastructure, ensuring it is scalable, secure, and efficient.

Key Responsibilities:

* Design and implement data governance solutions like Unity Catalog and StarBust
* Build strong orchestration layers in Databricks/ADF Workflows
* Develop CICD pipelines for Databricks in Azure DevOps
* Process near real-time data through Auto Loader and DLT Pipelines
* Implement security layers in Delta Lake
* Implement massive parallel processing layers in Spark SQL and PySpark
* Implement cost-effective infrastructure in Databricks
* Extract logic from on-prem layers like SSIS, stored procedures, Informatica, Vertica, Apache Hudi, and filesystems into PySpark
* Build solutions as cloud-agnostic

Requirements:

* Strong hands-on experience with Pyspark and Apache Spark
* Experience in native Spark migration to Databricks
* Experience in Hadoop migration to Databricks
* Experience in building data governance solutions
* Experience in building strong orchestration layers
* Experience in developing CICD pipelines
* Experience in processing near real-time data
* Experience in implementing security layers
* Experience in implementing massive parallel processing layers
* Experience in implementing cost-effective infrastructure
* Experience in extracting logic from on-prem layers
* Ability to build solutions as cloud-agnostic
* Proven experience in designing and implementing complex data solutions
* Expertise in data modeling, integration, security, and governance
* Hands-on experience with guiding the virtual data model definition, defining data virtualization architecture, and deployment with focus on Azure, Databricks, and PySpark technologies
* Prior experience with establishing best practices for business optimizations
* Experience with relational and non-relational data stores, ETL or ELT tools, DevOps, data lake, and data fabric concepts
* In-depth experience with data governance, data integration, and related technologies
* Proficiency in a variety of database technologies, both relational and non-relational
* Knowledge of cloud-based data solutions
* Excellent collaboration and communication skills

About Us:

Diverse Lynx LLC is an Equal Employment Opportunity employer. We promote and support a diverse workforce across all levels in the company. All qualified applicants will receive due consideration for employment without any discrimination. We evaluate all applicants solely on the basis of their ability, competence, and proven capability to perform the functions outlined in the corresponding role.

  • Kansas City, Missouri, United States Diverse Lynx Full time

    Job Title: Senior Databricks ArchitectJob Summary: We are seeking a highly skilled Senior Databricks Architect to join our team at Diverse Lynx LLC. As a key member of our data engineering team, you will be responsible for designing and implementing complex data solutions aligned with business objectives. Key Responsibilities: Design and implement data...


  • Kansas City, Missouri, United States C4 Technical Services Full time

    About the Role:C4 Technical Services is seeking a skilled Azure Databricks Engineer to design, implement, and optimize data processing workflows and analytics solutions within the Azure Databricks environment.Key Responsibilities:Data Engineering: Develop scalable data pipelines using Azure Databricks to handle large volumes of data from various sources.Data...

  • Databricks Architect

    2 weeks ago


    Kansas City, Missouri, United States Sogeti Full time

    Job DescriptionSogeti is a leading provider of professional technology services, specializing in Application Management, Infrastructure Management, and High-Tech Engineering. We offer cutting-edge solutions around Testing, Business Intelligence, Mobility, Cloud, and Security, combining world-class methodologies and the global delivery model, Rightshore.As a...


  • Kansas City, Missouri, United States Databricks Full time

    Job Description:As a Specialist Solutions Architect at Databricks, you will play a key role in guiding customers to successful implementations of big data projects. Your expertise in data engineering, cloud data technologies, and technical leadership will be instrumental in architecting production-level data pipelines, including end-to-end pipeline load...


  • Kansas City, Missouri, United States CyberTec Full time

    CyberTec is seeking a highly skilled Senior Data Solutions Architect to lead the design, development, and implementation of scalable and efficient data solutions on the Azure platform.As the Lead Azure Data Engineer, you will be responsible for architecting data pipelines, ensuring data quality and reliability, and leading a team of data engineers to deliver...


  • Kansas City, Missouri, United States The Friedkin Group Full time

    About the RoleWe are seeking a highly skilled Lead Machine Learning Operations Engineer to join our Corporate Data & Analytics Team. As a key member of our team, you will play a pivotal role in implementing DevOps and ML Ops practices to support AI/ML application enablement across The Friedkin Group of companies.Key ResponsibilitiesDrive the adoption of best...


  • Kansas City, Missouri, United States CyberTec Full time

    Job Summary:CyberTec is seeking a highly skilled Senior Data Architect to lead the design, development, and implementation of scalable and efficient data solutions on the Azure platform.Key Responsibilities:Design and implement data models, schemas, and structures to support efficient data storage and retrieval.Collaborate with data scientists, analysts, and...


  • Kansas City, Missouri, United States Technogen, Inc. Full time

    Job Title: Data Quality EngineerJob Summary:We are seeking a highly skilled Data Quality Engineer to join our team at Technogen, Inc. The successful candidate will work closely with our P&C Analytics group to design and develop tools and enhancements for data quality monitoring across systems.Key Responsibilities:* Analyze business scenarios, rules, and...


  • Jefferson City, Missouri, United States Centene Corporation Full time

    Job Title: Senior Infrastructure Security EngineerThis is a critical role that will support the Threat Infrastructure Security tools in the Centene environment. The focus will be on Data engineering/science for a Security Data Lake and automation projects.Key Responsibilities:Develop strategy, execute design, and implement efficient data pipelines to ensure...


  • Kansas City, Missouri, United States IMEG Corp Full time

    Job Title: Senior Mechanical EngineerWe are seeking a highly skilled Senior Mechanical Engineer to join our team at IMEG Corp. As a Senior Mechanical Engineer, you will be responsible for leading projects as a project manager or lead engineer for the delivery of large and highly complex projects with minimal support required from senior staff.Key...


  • Kansas City, Missouri, United States Marvin Full time

    Job Title: Senior Manufacturing EngineerMarvin, a premier window and door manufacturer, is seeking a Senior Manufacturing Engineer to join our team at our new location in Kansas City, KS.Job Summary:We are looking for a highly skilled and experienced Senior Manufacturing Engineer to lead our engineering department and drive process improvement initiatives....


  • Kansas City, Missouri, United States Hoefer Wysocki Architecture Full time

    Job Title: Senior Mechanical EngineerWe are seeking a highly skilled Senior Mechanical Engineer to lead project teams in creating mechanical system construction documents for healthcare building type projects.Key Responsibilities:Design and quality control of mechanical systems, including calculations and layouts.Maintain best practice engineering methods...


  • Kansas City, Missouri, United States Burns & McDonnell Full time

    Job Title: Senior Project EngineerBurns & McDonnell is a global engineering, architecture, and construction firm that delivers innovative solutions to complex challenges. We're seeking a Senior Project Engineer to join our team in Kansas City, MO.Job Summary:The Senior Project Engineer will be responsible for managing overall engineering design efforts for...


  • Kansas City, Missouri, United States Harrison Consulting Solutions Full time

    Senior Substation Engineer Job Description **About the Role** Harrison Consulting Solutions is seeking a highly skilled Senior Substation Engineer to join our Kansas City team. As a key member of our engineering team, you will be responsible for leading the design and engineering of substation portions of power delivery projects. **Responsibilities** *...


  • Kansas City, Missouri, United States Actalent Full time

    Job Title:Senior Electrical EngineerJob Description:This is a full-time on-site Senior Electrical Engineer role located in Kansas City, MO.The Senior Electrical Engineer will be responsible for designing electrical systems, analyzing electrical data, ensuring power distribution systems are efficient and reliable, and overseeing power...


  • Kansas City, Missouri, United States IMEG Corp Full time

    Job Title: Senior Mechanical EngineerWe are seeking a highly skilled Senior Mechanical Engineer to join our team at IMEG Corp. As a Senior Mechanical Engineer, you will be responsible for leading projects as a project manager or lead engineer for the delivery of large and highly complex projects with minimal support required from senior staff.Key...


  • Kansas City, Missouri, United States Burns & McDonnell Full time

    About Burns & McDonnellBurns & McDonnell is a leading engineering, architecture, construction, and environmental services company. Our Transportation Practice serves departments of transportation, cities, counties, and private industry across the country.Job SummaryWe are seeking a Senior Roadway Engineer to join our rapidly growing Transportation Practice...


  • Kansas City, Missouri, United States Burns & McDonnell Full time

    About Burns & McDonnellBurns & McDonnell is a leading engineering, architecture, construction, and environmental services company. Our Transportation Practice serves departments of transportation, cities, counties, and private industry across the country.Job SummaryWe are seeking a Senior Roadway Engineer to join our rapidly growing Transportation Practice...

  • Electrical Engineer

    3 weeks ago


    Kansas City, Missouri, United States U.S. Army Corps of Engineers Full time

    About the PositionThis is a senior electrical engineer position in the Kansas City District Design Branch, supporting Military Construction and Civil Works projects. The successful candidate will serve as the senior electrical engineer, responsible for developing detailed drawings, specifications, design analysis, computations, and calculations for assigned...


  • Kansas City, Missouri, United States Actalent Full time

    Senior Electrical EngineerActalent is seeking a highly skilled Senior Electrical Engineer to join our team in Kansas City, MO. As a key member of our engineering team, you will be responsible for designing and analyzing electrical systems, ensuring power distribution systems are efficient and reliable.Key Responsibilities:Design and develop electrical...