Lead Databricks Engineer

7 days ago


O'Fallon, United States Programmers.io Full time

One of our leading technology company is looking for Lead Databricks Engineer in O'Fallon, Missouri.


Job Description:

Candidates are required to have these mandatory skills:

• Strong hands on in Pyspark and Apache Spark.

• Experience in Native Spark Migration to Databricks.

• Experience in Hadoop Migration to Databricks.

• Experience in Building Data Governance Solutions like Unity Catalog, StarBust …etc

• Build Very Strong Orchestration Layer in Databricks/ADF…. Workflows.

• Build CICD for Databricks in Azure Devops.

• Process near Real time Data thru Auto Loader, DLT Pipelines.

• Implement Security Layer in Delta Lake.

• Implement Massive Parallel Processing Layers in Spark SQL and PySpark.

• Implement Cost effective Infrastructure in Databricks.

• Experience In extracting logic and from on prem layers like SSIS, Stored procedures, Informatica, Vertica, Apache Hudi, Filesystems. etc into Pyspark.

• Ability to Build Solutions as Cloud Agnostic.

• Proven experience in designing and implementing complex data solutions aligned with business objectives.

• Expertise in data modelling, integration, security, and governance

• Hands-on experience with guiding the virtual data model definition, defining Data Virtualization architecture and deployment with focus on Azure, Databricks, PySpark technologies.

• Prior experience with establishing best practices for business optimizations.

• Experience with relational and non-relational data stores (Hadoop, SQL, Mongo DB), ETL or ELT tools (SSIS, Informatica, Matillion, DBT), DevOps, Data Lake and Data Fabric concepts

• In-depth experience with data governance, data integration and related technologies.

• Proficiency in a variety of database technologies, both relational and non-relational.

• Knowledge of cloud-based data solutions (e.g., AWS, Azure).

• Excellent collaboration and communication skills



  • O'Fallon, United States Programmers Full time

    Job Title: Lead Databricks EngineerWe are seeking a highly skilled Lead Databricks Engineer to join our team at Programmers. As a key member of our engineering team, you will be responsible for designing and implementing complex data solutions aligned with business objectives.Key Responsibilities:Design and implement data governance solutions, including...


  • O'Fallon, United States Programmers.io Full time

    One of our leading technology company is looking for Lead Databricks Engineer in O'Fallon, Missouri.Job Description: Candidates are required to have these mandatory skills:• Strong hands on in Pyspark and Apache Spark.• Experience in Native Spark Migration to Databricks.• Experience in Hadoop Migration to Databricks.• Experience in Building Data...


  • O'Fallon, United States Programmers.io Full time

    Role – Lead Databricks EngineerLocation: O’Fallon, MO (Onsite from Day 1 – No Remote)Yrs. of experience: 10+ Yrs.Mode of employment: Full-TimeJob Description: Candidates are required to have these mandatory skills:• Strong hands on in Pyspark and Apache Spark.• Experience in Native Spark Migration to Databricks.• Experience in Hadoop Migration to...


  • O'Fallon, United States Mastercard, Inc. Full time

    About the RoleWe are seeking a highly skilled Cloud Data Engineer Lead to join our team at Mastercard, Inc. This is a critical role that will drive the development and implementation of our cloud data platforms, ensuring the secure and scalable storage and processing of our vast amounts of data.Key ResponsibilitiesDesign and develop high-quality, secure, and...


  • O'Fallon, MO, United States MasterCard Full time

    Job Title: Cloud Data Engineer LeadWe are seeking a highly skilled Cloud Data Engineer Lead to join our team at Mastercard. As a key member of our Data Warehouse team, you will be responsible for driving our mission to unlock the potential of data assets by consistently innovating and eliminating friction in how users access data from our Big Data...


  • O'Fallon, MO , USA, United States Mastercard, Inc. Full time

    About the RoleWe are seeking a highly skilled Cloud Data Engineer Lead to join our team at Mastercard, Inc. This is a critical role that will drive the development and implementation of our cloud data platforms, ensuring the secure and scalable storage and processing of our vast amounts of data.Key ResponsibilitiesDesign and develop high-quality, secure, and...


  • O'Fallon, United States Mastercard, Inc. Full time

    About the RoleWe are seeking a highly skilled Cloud Data Engineer Lead to join our team at Mastercard, Inc. This is a critical role that will drive the development and implementation of our cloud data platforms, ensuring the secure and scalable storage and processing of our vast amounts of data.Key ResponsibilitiesDesign and develop high-quality, secure, and...


  • O'Fallon, MO, United States MasterCard Full time

    About the RoleWe are seeking a highly skilled Cloud Data Engineer Lead to join our team at Mastercard. As a key member of our Data Warehouse team, you will be responsible for designing and developing high-quality, secure, scalable software solutions based on technical requirements specifications and design artifacts.Key ResponsibilitiesDesign and develop...


  • O'Fallon, Missouri, United States BOEING Full time

    Job Overview At Boeing, we are dedicated to innovation and collaboration to enhance the world we live in. Our commitment to diversity, equity, and inclusion creates a welcoming and respectful environment for all teammates, fostering professional growth opportunities. Boeing Defense, Space, and Security (BDS) is seeking skilled and driven Structural...

  • Lead Systems Engineer

    3 weeks ago


    O'Fallon, United States EPITEC Full time

    Position Summary: Epitec is looking for an experienced Systems Engineer specializing in Network and Monitoring Solutions to oversee and improve our enterprise-level monitoring and network management frameworks. The successful candidate will possess practical experience with a variety of monitoring solutions, scripting capabilities, and a deep understanding...


  • O'Fallon, Missouri, United States PRI Global Full time

    Position OverviewPRI Global is seeking a Senior Quality Assurance Engineer to join our dynamic team. This role requires a dedicated professional who is committed to ensuring the highest quality of software products.Key Responsibilities• Formulate and implement testing strategies that align with project objectives.• Enhance testing efficiency and...


  • O'Fallon, Missouri, United States Quadrant Inc Full time

    Job ID:Solutions Architect Location:O'Fallon, IL REQUIREMENTSProven experience as a Solutions ArchitectActive Secret clearance requiredPossession of Security+ certification or the ability to secure one within 90 daysA minimum of 12 years in systems architecture, showcasing expertise in Java and C#TECHNICAL EXPERTISE:Deep knowledge of AWS, GovCloud, ArcGIS,...


  • O'Fallon, United States City of O'Fallon Full time

    Job DescriptionJob DescriptionJoin our team at City of O'Fallon as a Assistant Director of Engineering and be part of a dynamic environment where your problem-solving skills and innovative thinking can truly shine. As a key player in our forward-thinking culture, you will have the opportunity to contribute to the excellence of our engineering projects in...


  • O'Fallon, United States Rose International Full time

    Position OverviewRose International is seeking a highly skilled Lead Technical Program Manager to enhance our customer experience strategy through innovative problem-solving and strategic execution. The ideal candidate will possess a strong passion for improving customer journeys and will be driven, analytical, and possess an entrepreneurial spirit.Key...

  • Lead Software Engineer

    18 hours ago


    O'Fallon, MO, United States MasterCard Full time

    About the RoleWe are seeking a highly skilled Lead Software Engineer to join our Corporate Security Engineering team. As a key member of our team, you will be responsible for designing, building, and supporting service platforms that provide cryptographic, key management, and certificate management functions utilized by global Mastercard application teams.As...


  • O'Fallon, United States Insight Global Full time

    JOB DESCRIPTION:Insight Global is looking for a Mechanical Operating Engineer to join a leading commercial real estate firm in the Ofallon, MO area! This person will be responsible for ensuring the efficient operation and maintenance of the electrical, and mechanical systems in the facilities. Some responsibilities include, doing daily rounds, conducting...


  • O'Fallon, United States Insight Global Full time

    JOB DESCRIPTION:Insight Global is looking for a Mechanical Operating Engineer to join a leading commercial real estate firm in the Ofallon, MO area! This person will be responsible for ensuring the efficient operation and maintenance of the electrical, and mechanical systems in the facilities. Some responsibilities include, doing daily rounds, conducting...


  • O'Fallon, Missouri, United States MasterCard Full time

    About the RoleWe are seeking a highly skilled Principal BizOps Engineer to join our team at Mastercard. As a key member of our Business Operations team, you will play a critical role in leading the DevOps transformation and driving the adoption of Artificial Intelligence (AI) across the organization.Key ResponsibilitiesLead the development and implementation...

  • DevOps Engineer

    1 month ago


    O'Fallon, United States EPITEC Full time

    *This is a 2-year contract with a possibility of getting direct hired with a well-known financial corporation!We are seeking an engineer with experience in industry standard CI/CD tools like Git/BitBucket, Jenkins, Maven, Artifactory, and Chef. Experience designing and implementing an effective and efficient CI/CD flow that gets code from dev to prod with...


  • O'Fallon, MO, United States MasterCard Full time

    About the RoleWe are seeking a highly skilled Lead Software Developer Engineer - AWS to join our Global Customer Care IVR team. As a key member of our team, you will play a critical role in driving our customer experience strategy forward by consistently innovating and problem-solving.Key ResponsibilitiesCollaborate with other engineers to design, develop,...