Senior Data Engineer

5 days ago

houston, United States Harnham Full time

JOB SUMMARY

As a Sr. Data Engineer you will play a crucial role in architecting, implementing, and managing robust, scalable data infrastructure. This position demands a blend of systems engineering, data integration, and data analytics skills to enhance our data capabilities, supporting advanced analytics, machine learning projects, and real-time data processing needs.


ESSENTIAL FUNCTIONS

  • Design and implement scalable and reliable data pipelines to ingest, process, and store diverse data at scale, using technologies such as Apache Spark, Hadoop, and Kafka.
  • Work within cloud environments like AWS or Azure to leverage services including but not limited to EC2, RDS, S3, Lambda, and Azure Data Lake for efficient data handling and processing.
  • Develop and optimize data models and storage solutions (SQL, NoSQL, Data Lakes) to support operational and analytical applications, ensuring data quality and accessibility.
  • Utilize ETL tools and frameworks (e.g., Apache Airflow, Talend) to automate data workflows, ensuring efficient data integration and timely availability of data for analytics.
  • Collaborate closely with data scientists, providing the data infrastructure and tools needed for complex analytical models, leveraging Python or R for data processing scripts.
  • Ensure compliance with data governance and security policies, implementing best practices in data encryption, masking, and access controls within a cloud environment.
  • Monitor and troubleshoot data pipelines and databases for performance issues, applying tuning techniques to optimize data access and throughput.
  • Stay abreast of emerging technologies and methodologies in data engineering, advocating for and implementing improvements to the data ecosystem.


QUALIFICATIONS

  • Bachelor's degree in computer science, MIS, or other business discipline or equivalent combination of education and/or experience.
  • 8+ years of experience in data engineering, with a proven track record in designing and operating large-scale data pipelines and architectures.
  • Expertise in developing ETL/ELT workflows.
  • Comprehensive knowledge of platforms and services like Databricks, Dataiku, and AWS native
  • data offerings
  • Solid experience with big data technologies (Apache Spark, Hadoop, Kafka) and cloud services (AWS, Azure) related to data processing and storage.
  • Strong experience in AWS and Azure cloud services, with hands-on experience in integrating cloud storage and compute services with Databricks.
  • Proficient in SQL and programming languages relevant to data engineering (Python, Java, Scala).
  • Hands on RDBMS experience (data modeling, analysis, programming, stored procedures)
  • Familiarity with machine learning model deployment and management practices is a plus.
  • Strong communication skills, capable of collaborating effectively across technical and non-technical teams.


CERTIFICATES, LICENSES, REGISTRATIONS*

  • Preferred: Databricks Certified Associate Developer for Apache Spark, AWS Certified Solutions Architect, Azure Data Engineer Associate, or other relevant certifications.

  • Senior Data Engineer

    2 weeks ago

    Houston, Texas, United States IntagHire Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at IntagHire. As a Senior Data Engineer, you will be responsible for designing and building pipelines to ingest and transport data, owning data quality and reliability for app-related data pipelines, and collaborating with data and product teams to deliver...

  • Senior Data Engineer

    4 days ago

    Houston, Texas, United States IntagHire Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing and building pipelines to ingest and transport data, owning data quality and reliability for app-related data pipelines, and collaborating with data and product teams to deliver robust,...

  • Senior Data Engineer

    2 days ago

    Houston, Texas, United States INSPYR Solutions Full time

    Job Title: Senior Data Engineer (Azure/D365)Location: Houston, TXDuration: Long Term ContractWork Requirements: US Citizen, GC Holders or Authorized to Work in the USJob Description:We are seeking a highly skilled Data Engineer to join our team at INSPYR Solutions. As a Senior Data Engineer, you will be responsible for designing, implementing, and...

  • Senior Data Engineer

    1 month ago

    Houston, United States Walker Elliott Full time

    Our client is looking for a Senior Data Engineer to lead the development and execution of the data engineering strategy, collaborating with senior leadership to ensure data initiatives align with business objectives. This role requires expertise in data integration, data modeling, and the implementation of Azure services.Key Responsibilities:Define and drive...

  • Senior Data Engineer

    6 hours ago

    Houston, Texas, United States Plains Full time

    Job Title: Senior Data EngineerPlains All American is seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and implementing data solutions on the Azure platform.Key Responsibilities:Design and develop scalable and efficient data solutions on the Azure platform.Develop...

  • Snowflake Data Engineer

    4 weeks ago

    Houston, United States Data Piper Full time

    Develop and implement scalable and efficient data architectures using Snowflake.5+ years of experience in data engineering, with a focus on Snowflake.Proven experience with Snowflake design, development, and optimization.Strong proficiency in SQL and query optimization techniques.Experience with ETL/ELT processes and tools (e.g., Talend, Informatica, Apache...

  • Senior Software Engineer

    2 days ago

    Houston, Texas, United States Resource Data Full time

    Job OverviewWe are seeking a highly skilled Senior Software Engineer to join our team at Resource Data. As a key member of our software development team, you will play a crucial role in designing and developing innovative software solutions that drive business value and efficiency.Key ResponsibilitiesDesign, develop, and test software modules to support...

  • Senior Data Engineer

    3 months ago

    Houston, United States ManhattanLife Insurance & Annuity Company Full time

    Job DescriptionJob DescriptionWho We Are:ManhattanLife Insurance and Annuity Company was founded in 1850, the Company’s longevity makes it one of the oldest and most reliable health and life insurance companies in the country. Operating successfully for 174 years is a testimony to ManhattanLife’s enduring history, and an indicator of the reliability of...

  • Senior Data Engineer

    2 months ago

    Houston, United States Mondo Full time

    Job Summary:We are seeking a highly skilled Senior Data Engineer to design and implement scalable, reliable data pipelines that drive our analytics and operational applications. As a key member of our data engineering team, you will work with cutting-edge technologies such as Apache Spark, Hadoop, Kafka, and cloud services from AWS and Azure. Your expertise...

  • Senior Data Engineer

    2 months ago

    Houston, United States Mondo Full time

    Job Summary:We are seeking a highly skilled Senior Data Engineer to design and implement scalable, reliable data pipelines that drive our analytics and operational applications. As a key member of our data engineering team, you will work with cutting-edge technologies such as Apache Spark, Hadoop, Kafka, and cloud services from AWS and Azure. Your expertise...

  • Senior Software Engineer

    4 days ago

    Houston, Texas, United States Resource Data Inc Full time

    About the RoleWe are seeking a highly skilled Senior Software Engineer to join our team at Resource Data Inc. As a key member of our software development team, you will play a crucial role in designing and developing innovative software solutions that drive business value and efficiency for our clients.Key ResponsibilitiesAnalyze and design software modules...

  • Senior Data Engineer

    1 month ago

    North Houston, United States Infomatics Corp Full time

    We are looking for a Data engineer for our project. Please use the following job description for the role to look for a candidate with good leadership and communication skills. Senior Data EngineerLeads the design, development, and operation of robust, scalable, and secure data platforms on the Azure cloud. Drives the adoption of cloud-native technologies...

  • Senior Software Engineer

    7 days ago

    Houston, Texas, United States Resource Data Inc Full time

    About the RoleWe're seeking a highly skilled Senior Software Engineer to join our team at Resource Data Inc. As a key member of our software development team, you'll play a crucial role in designing and developing innovative software solutions that drive business value and efficiency.Key ResponsibilitiesAnalyze and design software modules to support clients'...

  • Senior Data Engineer

    4 days ago

    Houston, Texas, United States Klanik Full time

    Job DescriptionKlanik is seeking a highly skilled Senior Data Engineer to join our team.Job SummaryWe are looking for a talented individual to lead the development of our data engineering applications using third-party and in-house frameworks. The successful candidate will have a strong background in data engineering, data accessibility, and cloud...

  • Senior Data Engineer

    1 day ago

    Houston, United States Plains Full time

    Job Type:RegularPlains All American is an industry-leading transportation and midstream provider specializing in transportation, storage, processing and marketing solutions for crude oil and natural gas liquids (NGLs). We own an extensive network of pipeline transportation, terminalling, storage and gathering assets in key crude oil and NGL producing basins...

  • Senior Battery Data Engineer

    7 days ago

    Houston, Texas, United States Fluence Corporation Full time

    Senior Battery Data EngineerFluence Corporation is a leader in energy storage solutions, committed to innovation and excellence in battery technology. We are dedicated to advancing the integration of renewable energy and ensuring the reliability and efficiency of energy storage systems.Key ResponsibilitiesDevelop, implement, and optimize data pipelines and...

  • Senior Software Engineer

    6 days ago

    Houston, Texas, United States Resource Data Inc Full time

    About the RoleWe are seeking a highly skilled Senior Software Engineer to join our team at Resource Data Inc. As a key member of our software development team, you will play a crucial role in designing and developing innovative software solutions that drive business efficiency and performance.Key ResponsibilitiesAnalyze and design software modules to support...

  • Senior Data Scientist

    1 week ago

    Houston, Texas, United States Jobot Full time

    Senior Data EngineerWe are a dynamic team at Jobot, seeking a highly skilled Senior Data Engineer to join our ranks. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining databases, data systems, and analytic solutions that drive business insights.Data Engineering: Design and develop scalable data...

  • Senior Frontend Developer

    7 days ago

    Houston, Texas, United States Engineer Up Full time

    Job DescriptionWe are seeking a skilled Senior Frontend Developer to join our team at Engineer Up. As a key member of our development team, you will be responsible for designing and implementing high-quality user interface components using React.Key Responsibilities:Developing reusable components and front-end libraries for future useTranslating designs and...

  • Senior Data Scientist

    1 week ago

    Houston, Texas, United States Jobot Full time

    Senior Data EngineerWe are a dynamic team at Jobot, seeking a highly skilled Senior Data Engineer to join our ranks. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining databases, data systems, and analytic solutions that drive business insights.Data Engineering: Design and develop scalable data...