Lead Databricks Engineer

2 months ago


O'Fallon, United States Programmers.io Full time

Role – Lead Databricks Engineer

Location: O’Fallon, MO (Onsite from Day 1 – No Remote)


Yrs. of experience: 10+ Yrs.


Mode of employment: Full-Time



Job Description:


Candidates are required to have these mandatory skills:


• Strong hands on in Pyspark and Apache Spark.


• Experience in Native Spark Migration to Databricks.


• Experience in Hadoop Migration to Databricks.


• Experience in Building Data Governance Solutions like Unity Catalog, StarBust …etc


• Build Very Strong Orchestration Layer in Databricks/ADF…. Workflows.


• Build CICD for Databricks in Azure Devops.


• Process near Real time Data thru Auto Loader, DLT Pipelines.


• Implement Security Layer in Delta Lake.


• Implement Massive Parallel Processing Layers in Spark SQL and PySpark.


• Implement Cost effective Infrastructure in Databricks.


• Experience In extracting logic and from on prem layers like SSIS, Stored procedures, Informatica, Vertica, Apache Hudi, Filesystems. etc into Pyspark.


• Ability to Build Solutions as Cloud Agnostic.


• Proven experience in designing and implementing complex data solutions aligned with business objectives.


• Expertise in data modelling, integration, security, and governance


• Hands-on experience with guiding the virtual data model definition, defining Data Virtualization architecture and deployment with focus on Azure, Databricks, PySpark technologies.


• Prior experience with establishing best practices for business optimizations.


• Experience with relational and non-relational data stores (Hadoop, SQL, Mongo DB), ETL or ELT tools (SSIS, Informatica, Matillion, DBT), DevOps, Data Lake and Data Fabric concepts


• In-depth experience with data governance, data integration and related technologies.


• Proficiency in a variety of database technologies, both relational and non-relational.


• Knowledge of cloud-based data solutions (e.g., AWS, Azure).


• Excellent collaboration and communication skills



  • O'Fallon, United States Programmers.io Full time

    Role – Lead Databricks EngineerLocation: O’Fallon, MO (Onsite from Day 1 – No Remote)Yrs. of experience: 10+ Yrs.Mode of employment: Full-TimeJob Description: Candidates are required to have these mandatory skills:• Strong hands on in Pyspark and Apache Spark.• Experience in Native Spark Migration to Databricks.• Experience in Hadoop Migration to...


  • O'Fallon, United States Mastercard, Inc. Full time

    About the RoleWe are seeking a highly skilled Cloud Data Engineer Lead to join our team at Mastercard, Inc. This is a critical role that will drive the development and implementation of our cloud data platforms, ensuring the secure and scalable storage and processing of our vast amounts of data.Key ResponsibilitiesDesign and develop high-quality, secure, and...


  • O'Fallon, MO , USA, United States Mastercard, Inc. Full time

    About the RoleWe are seeking a highly skilled Cloud Data Engineer Lead to join our team at Mastercard, Inc. This is a critical role that will drive the development and implementation of our cloud data platforms, ensuring the secure and scalable storage and processing of our vast amounts of data.Key ResponsibilitiesDesign and develop high-quality, secure, and...


  • O'Fallon, United States Mastercard, Inc. Full time

    About the RoleWe are seeking a highly skilled Cloud Data Engineer Lead to join our team at Mastercard, Inc. This is a critical role that will drive the development and implementation of our cloud data platforms, ensuring the secure and scalable storage and processing of our vast amounts of data.Key ResponsibilitiesDesign and develop high-quality, secure, and...


  • O'Fallon, Missouri, United States BOEING Full time

    Job Overview At Boeing, we are dedicated to innovation and collaboration to enhance the world we live in. Our commitment to diversity, equity, and inclusion creates a welcoming and respectful environment for all teammates, fostering professional growth opportunities. Boeing Defense, Space, and Security (BDS) is seeking skilled and driven Structural...

  • Lead Systems Engineer

    2 weeks ago


    O'Fallon, United States EPITEC Full time

    Position Summary: Epitec is looking for an experienced Systems Engineer specializing in Network and Monitoring Solutions to oversee and improve our enterprise-level monitoring and network management frameworks. The successful candidate will possess practical experience with a variety of monitoring solutions, scripting capabilities, and a deep understanding...


  • O'Fallon, Missouri, United States PRI Global Full time

    Position OverviewPRI Global is seeking a Senior Quality Assurance Engineer to join our dynamic team. This role requires a dedicated professional who is committed to ensuring the highest quality of software products.Key Responsibilities• Formulate and implement testing strategies that align with project objectives.• Enhance testing efficiency and...


  • O'Fallon, Missouri, United States Quadrant Inc Full time

    Job ID:Solutions Architect Location:O'Fallon, IL REQUIREMENTSProven experience as a Solutions ArchitectActive Secret clearance requiredPossession of Security+ certification or the ability to secure one within 90 daysA minimum of 12 years in systems architecture, showcasing expertise in Java and C#TECHNICAL EXPERTISE:Deep knowledge of AWS, GovCloud, ArcGIS,...


  • O'Fallon, United States City of O'Fallon Full time

    Job DescriptionJob DescriptionJoin our team at City of O'Fallon as a Assistant Director of Engineering and be part of a dynamic environment where your problem-solving skills and innovative thinking can truly shine. As a key player in our forward-thinking culture, you will have the opportunity to contribute to the excellence of our engineering projects in...


  • O'Fallon, United States Rose International Full time

    Position OverviewRose International is seeking a highly skilled Lead Technical Program Manager to enhance our customer experience strategy through innovative problem-solving and strategic execution. The ideal candidate will possess a strong passion for improving customer journeys and will be driven, analytical, and possess an entrepreneurial spirit.Key...

  • Data Center Engineer

    2 months ago


    O'Fallon, United States PRI Global Full time

    I humble request you to dont share Data Analyst resumes please.... This is a Data Center Engineer role.The requirement as followsJob title: Data Center Engineer/TechnicianDuration: 12+ monthsLocation: O'Fallon, MOOverview• Our mission is to provide the highest quality network and data center engineering services possible• Deliver comprehensive, reliable...

  • DevOps Engineer

    3 weeks ago


    O'Fallon, United States EPITEC Full time

    *This is a 2-year contract with a possibility of getting direct hired with a well-known financial corporation!We are seeking an engineer with experience in industry standard CI/CD tools like Git/BitBucket, Jenkins, Maven, Artifactory, and Chef. Experience designing and implementing an effective and efficient CI/CD flow that gets code from dev to prod with...

  • DevOps Engineer

    3 weeks ago


    O'Fallon, United States EPITEC Full time

    *This is a 2-year contract with a possibility of getting direct hired with a well-known financial corporation!We are seeking an engineer with experience in industry standard CI/CD tools like Git/BitBucket, Jenkins, Maven, Artifactory, and Chef. Experience designing and implementing an effective and efficient CI/CD flow that gets code from dev to prod with...

  • Electrical Engineer

    2 months ago


    O'Fallon, United States Motion Recruitment Full time

    Our client, a French multinational company that specializes in digital automation and energy management, is looking to hire an Electrical Engineer to join their team on a HYBRID model in O'Fallon, MO!Alternative Title: Customer Engagement Power Systems Engineer*** This begins as a 12-month contract with any opportunity to extend/convert ****** HYBRID: 80%...

  • Electrical Engineer

    2 months ago


    O'Fallon, United States Motion Recruitment Full time

    Our client, a French multinational company that specializes in digital automation and energy management, is looking to hire an Electrical Engineer to join their team on a HYBRID model in O'Fallon, MO!Alternative Title: Customer Engagement Power Systems Engineer*** This begins as a 12-month contract with any opportunity to extend/convert ****** HYBRID: 80%...

  • Electrical Engineer

    2 days ago


    O'Fallon, United States JobsRUs.com Full time

    JobsRUs.com is seeking to hire a Electrical Engineer for our client in O Fallon, MO! Benefits Available! Weekly Pay! $57.00/Hour Shift Details:8:00AM to 5:00PM DESCRIPTION: The goal of the EBC is to accelerate the sales process and enhance sales growth by creating a customized customer experience. Your role will be as a member of a larger EBC team...


  • O'Fallon, United States Fulcrum Digital Full time

    Job Description Who are we ​ FulcrumDigital is an agile and next-generation digital accelerating company providingdigital transformation and technology services right from ideation toimplementation. These services have applicability across a variety ofindustries, including banking & financial services, insurance, retail,higher education, food,...


  • O'Fallon, Missouri, United States Tekshapers Inc Full time

    Position OverviewSenior Java Backend Engineer (API)We are seeking a highly skilled Senior Java Backend Engineer with a strong focus on API development. The ideal candidate will possess extensive experience in Java (J2EE), Golang, and RESTful services.Key Responsibilities:Demonstrate substantial hands-on expertise in Java (J2EE), Golang, and RESTful API...

  • Electrical Engineer

    4 weeks ago


    O'Fallon, United States Rose International Full time

    •**Only qualified Electrical Engineer candidates located near the O'Fallon, MO area to be considered due to the position requiring an onsite presence***Qualifications: Bachelor’s (B.S.) Engineering or equivalent in certifications and experience. 5-10+ years Power Applications Engineering experience is ideal. Data center experience is...


  • O'Fallon, Missouri, United States Artmac Soft LLC Full time

    Job OverviewCompany Profile Artmac Soft LLC is a forward-thinking technology consulting firm committed to delivering cutting-edge IT solutions and services to its clientele.Position Summary Job Title: Lead Java Automation Tester (SDET - Jbehave/Cucumber frameworks / TDD and BDD)Employment Type: W2 Experience Required: 6-15 years Work Location: RemoteKey...