Lead Data Platform Engineer

3 days ago

Houston, Texas, United States INSPYR Solutions Full time $120,000 - $240,000 per year

Title:
 Lead Data Platform Engineer

Location:
Remote

Duration:
 9+ months contract

Compensation:
$ /hr

Work Requirements:
US Citizen, GC Holders or Authorized to Work in the U.S.

Lead Data Platform Engineer
Summary:
Are you passionate about building and supporting modern, cloud-based data platforms? We're seeking a Sr. Data Platform Engineer to scale our DataOps infrastructure in a hybrid role (60% administration, 40% development/support). You'll work with cutting-edge technologies such as Databricks, Apache Spark, Delta Lake, and AWS, while supporting mission-critical data pipelines and integrations. If you are a hands-on engineer with strong Python skills, deep AWS expertise, and a talent for solving complex data challenges, we want to hear from you.

Key Responsibilities:

  • Design, develop, and maintain scalable ETL pipelines and integration frameworks.
  • Administer and optimize Databricks and Apache Spark environments for data workloads.
  • Build and manage data workflows using AWS services: Lambda, Glue, Redshift, SageMaker, and S3.
  • Support and troubleshoot DataOps pipelines, ensuring reliability and performance.
  • Automate platform operations with Python, PySpark, and infrastructure-as-code tools.
  • Collaborate with cross-functional teams on data ingestion, transformation, and deployment.
  • Provide technical leadership and mentor junior developers and third-party teams.
  • Create and maintain technical documentation and training materials.
  • Troubleshoot recurring issues and implement long-term solutions.

Minimum Qualifications:

  • Bachelor's or Master's degree in Computer Science or related field.
  • 5+ years of experience in data engineering or platform administration.
  • 3+ years of experience in integration framework development, with strong emphasis on Databricks, AWS, and ETL.

Required Technical Skills:

  • Strong programming in Python and PySpark.
  • Expertise in Databricks, Apache Spark, and Delta Lake.
  • Proficiency in AWS CloudOps and Cloud Security, including configuration, deployment, and monitoring.
  • Experience with: Kafka, Pandas, Airflow, Neo4j, GraphDB, MongoDB, PostgreSQL, OWL, Python functions, New Relic, Grafana, OpenLineage, Apache Atlas, Databricks Unity Catalog, DLT, Great Expectations, Databricks Delta-sharing.
  • Familiarity with distributed data processing, real-time streaming pipelines, unstructured/semi-structured/structured data, flexible data modeling, semantic engineering, knowledge graphs, data-as-a-service, data observability, quality frameworks, data syndication, data fabric and marketplace development, cognitive search engine development, master data management, data governance, and data migration.

Information collected and processed through your application with INSPYR Solutions (including any job applications you choose to submit) is subject to INSPYR Solutions' Privacy Policy and INSPYR Solutions' AI and Automated Employment Decision Tool Policy: By submitting an application, you are consenting to being contacted by INSPYR Solutions through phone, email, or text.

  • Director, Data Platform

    2 weeks ago

    Houston, Texas, United States Crane Worldwide Logistics Full time $150,000 - $250,000 per year

    ESSENTIAL JOB FUNCTIONSDirect the Data Engineering function:Deploy and operate key infrastructure, includingCompute & storage in AWS, Azure, and SnowflakeStreaming platform, including Kafka as an internal serviceDeployment pipelinesObservability toolingIncident management tooling & processImplementation & maintenance of data ingestion pipelinesReliable...

  • Platform Engineer

    3 days ago

    Houston, Texas, United States Leidos Full time $104,000 - $189,175 per year

    DescriptionJob DescriptionLaunch Your Next Mission: Platform Engineer – San Antonio, TXStep into the future of national defense technology with Leidos We're searching for a driven Platform Engineer ready to take on high-stakes migration projects in the DAF CLOUDworks environment. Located in San Antonio, TX, this hybrid role offers flexibility, challenge,...

  • Data Engineer

    3 days ago

    Houston, Texas, United States Templeton & Partners - Innovative & Inclusive Hiring Solutions Full time $120,000 - $180,000 per year

    About the RoleWe are seeking aSenior Data Engineerto design and build robust, scalable data foundations and end-to-end solutions that unlock business value through data-driven insights. In this strategic role, you'll help shape and embed a data-driven culture across both IT and business functions, supporting global energy and trading operations.As a...

  • Data Engineer- Houston Based

    3 days ago

    Houston, Texas, United States CG Infinity Full time $130,000 - $170,000 per year

    Data Engineer – Full-TimeLocation: Sugar Land, TX Salary Range: $130,000 – $170,000 (DOE)About CG InfinityCG Infinity is a technology consulting firm founded in 1998. We deliver tailored solutions that solve real business challenges—no templates, no shortcuts. Our teams work closely with clients to build scalable, impactful systems that drive...

  • GCP Data Engineer

    3 days ago

    Houston, Texas, United States eNcloud Services LLC Full time $120,000 - $180,000 per year

    eNcloud Services Immediate RequirementTitle: GCP Data EngineerLocation: Houston, Texas (Must be willing to work 4 days a week onsite in downtown, Houston)Type of Hire: ContractWork Authorization: AnyGCP Data Engineer:The client's Data Analytics team operates under a product platform operating model and functions as a product team composed of multiple...

  • Senior Lead Data Architect

    1 day ago

    Houston, Texas, United States JPMorgan Chase Full time $200,000 - $250,000 per year

    You strive to be an essential member of a diverse team of visionaries dedicated to making a lasting impact. Don't pass up this opportunity to collaborate with some of the brightest minds in the field and deliver best-in-class solutions to the industry.As a Senior Lead Data Architect at JPMorgan Chase within the Corporate Technology in Corporate Data &...

  • Sr Data Engineer _Azure AWS SAP

    20 hours ago

    Houston, Texas, United States Engineers and Constructors International Inc. Full time $150,000 - $250,000 per year

    Join the ECI Group's Professional Technical Services Network as a contract Senior Data Engineer.  This position supports our clients' Crude Data Exchange project, part of a broader effort to modernize the WONA Data Warehouse using Azure cloud technology. This role focuses on aligning and transforming data across the T&S Crude business, with a long-term...

  • GCP Data Engineer

    3 days ago

    Houston, Texas, United States Zeus Solutions Inc Full time $120,000 - $180,000 per year

    Job descriptionWe are seeking a skilled GCP Data Engineer based in Houston (onsite), with 8+ years of overall data engineering experience and at least 4 years of hands-on expertise in Google Cloud Platform (GCP).ResponsibilitiesDevelop, construct, test, and maintain data acquisition pipelines for large volumes of structured and unstructured data. This...

  • AWS Data Migration Engineer/Architect

    2 weeks ago

    Houston, Texas, United States Slalom Full time $125,000 - $170,000 per year

    Who You'll Work WithAs a modern technology company, we've never met a technical challenge we didn't like. We enable our clients to learn from their data, create incredible digital experiences, and make the most of new technologies. We blend design, engineering, and analytics expertise to build the future. We surround our technologists with interesting...

  • Senior Lead Networking Test Engineer

    3 days ago

    Houston, Texas, United States Celestica Full time $120,000 - $200,000 per year

    Req ID: 129192Region: AmericasCountry: USAState/Province: New HampshireCity: MerrimackGeneral OverviewFunctional Area: EngineeringCareer Stream: Design - Software EngineeringSAP Short Name: SLE-ENG-DSEJob Level: Level 09IC/MGR: Individual ContributorDirect/Indirect Indicator: IndirectSummaryThe Senior Lead Networking Test Engineer will be responsible for...