Lead Data Engineer

2 months ago

Houston, United States Harnham Full time

As a Lead Data Engineer you will:

  • Design and implement scalable and reliable data pipelines to ingest, process, and store diverse data at scale, using technologies such as Apache Spark, Hadoop, and Kafka.
  • Work within cloud environments like AWS or Azure to leverage services including but not limited to EC2, RDS, S3, Lambda, and Azure Data Lake for efficient data handling and processing.
  • Develop and optimize data models and storage solutions (SQL, NoSQL, Data Lakes) to support operational and analytical applications, ensuring data quality and accessibility.
  • Utilize ETL tools and frameworks (e.g., Apache Airflow, Talend) to automate data workflows, ensuring efficient data integration and timely availability of data for analytics.
  • Collaborate closely with data scientists, providing the data infrastructure and tools needed for complex analytical models, leveraging Python or R for data processing scripts.
  • Ensure compliance with data governance and security policies, implementing best practices in data encryption, masking, and access controls within a cloud environment.
  • Monitor and troubleshoot data pipelines and databases for performance issues, applying tuning techniques to optimize data access and throughput.
  • Stay abreast of emerging technologies and methodologies in data engineering, advocating for and implementing improvements to the data ecosystem.

What We Need From You

  • Bachelor's Degree computer science, MIS, or other business discipline and 10+ years of experience in data engineering, with a proven track record in designing and operating large-scale data pipelines and architectures Req or
  • Master's Degree computer science, MIS, or other business discipline and 5+ years of experience in data engineering, with a proven track record in designing and operating large-scale data pipelines and architectures Req
  • Expertise in developing ETL/ELT workflows
  • Comprehensive knowledge of platforms and services like Databricks, Dataiku, and AWS native data offerings
  • Solid experience with big data technologies (Apache Spark, Hadoop, Kafka) and cloud services (AWS, Azure) related to data processing and storage
  • Strong experience in AWS and Azure cloud services, with hands-on experience in integrating cloud storage and compute services with Databricks
  • Proficient in SQL and programming languages relevant to data engineering (Python, Java, Scala)
  • Hands on RDBMS experience (data modeling, analysis, programming, stored procedures)
  • Familiarity with machine learning model deployment and management practices is a plus
  • Strong communication skills, capable of collaborating effectively across technical and non-technical teams
  • AWS Certified Solution Architect Preferred
  • Databricks Certified Associate Developer for Apache Spark Preferred
  • Azure Data Engineer Associate Preferred
  • or other relevant certifications. Preferred

  • Lead Data Engineer

    2 weeks ago

    Houston, Texas, United States The University of Texas at Austin Full time

    Job Title: Lead Data EngineerJoin The University of Texas at Austin as a Lead Data Engineer and play a key role in building a trusted, integrated, and scalable information infrastructure that transforms complex UT data into valued insights for data-informed decisions.About the RoleWe are seeking a highly skilled and experienced Lead Data Engineer to lead our...

  • Data Engineering Lead

    2 weeks ago

    Houston, Texas, United States Harnham Full time

    Lead Data EngineerJob Summary:We are seeking a highly skilled Lead Data Engineer to join our team at Harnham. As a key member of our data engineering team, you will be responsible for designing, implementing, and architecting new AI/ML initiatives at the enterprise level.About the Role:This is a unique opportunity to join a brand new team as our client's...

  • Lead Data Engineer

    2 weeks ago

    Houston, Texas, United States CenterPoint Energy Full time

    Job SummaryAs a Lead Data Engineer at CenterPoint Energy, you will be responsible for developing and maintaining data pipelines that support analytical analysis. You will partner with various business clients, data scientists, and data analysts to ensure that the company has access to the data it needs to make informed decisions. You will also be responsible...

  • Lead Data Engineer

    2 weeks ago

    Houston, Texas, United States The Friedkin Group Full time

    Job SummaryWe are seeking a highly skilled Lead Data Engineer to join our team at The Friedkin Group. As a key member of our data engineering team, you will be responsible for designing, implementing, and managing robust, scalable data infrastructure to support advanced analytics, machine learning projects, and real-time data processing needs.As a Lead Data...

  • Data Engineering Lead

    4 weeks ago

    Houston, Texas, United States Harnham Full time

    Lead Data EngineerJob Summary:We are seeking a highly skilled Lead Data Engineer to join our team at Harnham. As a key member of our data engineering team, you will be responsible for designing, implementing, and architecting new AI/ML initiatives at the enterprise level.About the Role:This is a unique opportunity to join a brand new team as our client's...

  • Data Engineering Lead

    2 weeks ago

    Houston, Texas, United States Harnham Full time

    Job Title: Lead Data EngineerJob Summary:We are seeking a highly skilled Lead Data Engineer to join our team at Harnham. As a key member of our data engineering team, you will be responsible for designing and implementing scalable data pipelines, leveraging cloud services, and collaborating with data scientists to deliver high-quality data solutions.Key...

  • Lead Data Engineer

    1 month ago

    Houston, United States Harnham Full time

    Title: Lead Data EngineerLocation: Houston, TX 77077 - Onsite 4 days per week.Overview of Role:As Lead Data Engineer, you will be responsible for helping to scale, implement, and architect new AI/ML initiatives at the enterprise level. This role is part of a brand new team as our client's business grows. This role is a combination of systems engineering,...

  • Lead Data Engineer

    2 weeks ago

    houston, United States Harnham Full time

    Title: Lead Data EngineerLocation: Houston, TX 77077 - Onsite 4 days per week.Overview of Role:As Lead Data Engineer, you will be responsible for helping to scale, implement, and architect new AI/ML initiatives at the enterprise level. This role is part of a brand new team as our client's business grows. This role is a combination of systems engineering,...

  • Lead Data Engineer

    4 weeks ago

    Houston, United States Harnham Full time

    Title: Lead Data EngineerLocation: Houston, TX 77077 - Onsite 4 days per week.Overview of Role:As Lead Data Engineer, you will be responsible for helping to scale, implement, and architect new AI/ML initiatives at the enterprise level. This role is part of a brand new team as our client's business grows. This role is a combination of systems engineering,...

  • Lead Data Engineer

    1 month ago

    Houston, United States Harnham Full time

    Title: Lead Data EngineerLocation: Houston, TX 77077 - Onsite 4 days per week.Overview of Role:As Lead Data Engineer, you will be responsible for helping to scale, implement, and architect new AI/ML initiatives at the enterprise level. This role is part of a brand new team as our client's business grows. This role is a combination of systems engineering,...

  • Data Engineering and Science Lead

    2 weeks ago

    Houston, Texas, United States Asset Living Full time

    Data Engineering and Science LeadAsset Living is a third-party management firm and a proven partner in fostering thriving communities nationwide. With a strong emphasis on innovation and results, we empower our team members to lead by example, collaborate, and evolve.We are seeking a highly skilled Data Engineering and Science Lead to join our dynamic team....

  • Data Solutions Engineer

    2 weeks ago

    Houston, Texas, United States DATAGRATION - Connecting Data | Powering Decisions Full time

    Job Summary:We are seeking a highly skilled Data Solutions Engineer to join our team at Datagration. As a Data Solutions Engineer, you will be responsible for implementing E&P and ESG related data solutions in the Oil and Gas industry, addressing customer challenges in a customer-facing environment.Key Responsibilities:Analyze customer requirements and...

  • Data Engineering and Science Lead

    6 days ago

    Houston, Texas, United States Asset Living Full time

    About Asset LivingAsset Living is a third-party management firm and a proven partner in fostering thriving communities nationwide. Founded in 1986, Asset Living has decades of experience delivering exceptional value to our partners.We empower each other to lead by example, collaborate, and evolve - inspired by our belief that we can continually improve as...

  • Senior Manager Data Engineering Lead

    2 weeks ago

    Houston, Texas, United States Publicis Sapient Full time

    Job Title: Senior Manager Data EngineeringPublicis Sapient is a global digital transformation partner that empowers organizations to embrace digital innovation, enhance customer experiences, and accelerate business growth through a blend of strategy, consulting, agile engineering, and problem-solving creativity.Job DescriptionWe are seeking a Senior Manager...

  • Data Engineer

    2 weeks ago

    Houston, Texas, United States VC5 Consulting Full time

    Job Title: Senior Data EngineerWe are seeking a highly motivated and experienced Senior Data Engineer to join our team at VC5 Consulting. As a key member of our Enterprise Data Engineering team, you will play a critical role in designing and developing scalable and efficient data solutions on the Azure platform.Key Responsibilities:Define and drive the data...

  • Data Engineer

    1 month ago

    Houston, United States VC5 Consulting Full time

    We are looking for highly-motivated individuals who enjoy working as part of a team in a dynamic and entrepreneurial work environment. Position Summary: The company is initiating new initiatives within the Enterprise Data Engineering team where an experienced Azure Data Engineer.  We are seeking someone who can demonstrate their qualifications and...

  • Lead Software Engineer

    1 month ago

    Houston, United States JPMorganChase Full time

    Job DescriptionJOB DESCRIPTIONWe have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible.As a Lead Software Engineer at JPMorgan Chase within Corporate Technology, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a...

  • Lead Data Architect

    2 weeks ago

    Houston, Texas, United States Harnham Full time

    Lead Data EngineerAt Harnham, we are seeking a highly skilled Lead Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing and implementing scalable data pipelines, utilizing cloud environments and services for efficient data handling and processing.Key Responsibilities:Design and implement data...

  • Data Engineer

    2 weeks ago

    Houston, Texas, United States Eateam Full time

    Job Title: ETL DeveloperJob Description:We are seeking an experienced ETL Developer to join our team at Eateam Inc. in Plano, TX. The successful candidate will have a strong background in data engineering, with expertise in architecting, developing, and implementing end-to-end scalable data applications and data pipelines.Key Responsibilities:* Design,...

  • Data Engineer

    4 weeks ago

    Houston, Texas, United States Eateam Full time

    Job Title: ETL DeveloperWe are seeking an experienced ETL Developer to join our team at Eateam Inc. in Plano, TX. As an ETL Developer, you will be responsible for designing, implementing, and maintaining automated data pipelines using Informatica Cloud (IICS) and other technologies.Key Responsibilities:Design and develop end-to-end scalable data applications...