Senior Data Engineer

6 months ago

Columbus, United States g2o Full time

Your future starts here

Imagine being part of a team that helps clients build better relationships with customers. When you join us, you'll help top-notch clients to execute the digital strategies of the future. Every day, we collaborate with clients and each other to provide technology expertise, human-centered design and industry experience to deliver real business results.

Position description

Work collaboratively with technical team members to develop, deploy and production support data engineering jobs and pipelines - spark structured stream processing jobs across one or more streams, persisted to a data lake. Work with leads and architects to prototype designs and technologies. Design and develop data engineering jobs with an eye on quality attributes (and trade-offs) such as simplicity, reliability, scalability, extensibility, performance. Build or reuse common frameworks where possible. Set and adhere to coding and style guides to ensure consistency. Ensure code quality through mechanisms including code coverage, peer reviews, pair programming. Create and deploy streaming data quality checks with remediation runbooks. Adhere to data governance and compliance requirements and standards. Provide recommendations for and help curate technical backlogs for technical debt management and continuous improvement.

Required qualifications

5+ years of total experience working with data and data pipeline technologies at scale. 3+ years of solid Python development experience – including: Robust unit and integration testing and Git driven CI/CD to production 2+ years of proven experience developing streaming data processing tasks using pySpark and SparkSQL Solid understanding of Spark, preferably Databricks Spark and Apache Delta Read streaming data from Kafka, Apache Delta or similar, merge /enrich /qualify data inline, capture operations metrics, persist to target, debug and optimize At scale and in production Experience leveraging public cloud data and analytics services. Azure preferred, but AWS/GCP also works. Nice to have: Super user experience with reporting tools such a PowerBI.

Required education and experience

College/University Degree or equivalent.

  • Senior Data Engineer

    7 days ago

    Columbus, United States Women Impact Tech Full time

    Overview: Data Engineer should have proficiency in data engineering for Azure Data Engineers. Mastery in designing robust, scalable, and secure data solutions on Azure is essential. This includes understanding Azure's data services ecosystem, such as Azure Data Lake, Azure Synapse Analytics, and Azure Data bricks. Data Engineers must architect solutions...

  • Senior Data Engineer

    6 months ago

    Columbus, United States Resource Informatics Group Full time

    The client is looking for a solid Senior Data Engineer with strong zure Data Factory (ADF) , zure Data Lake (ADLS), Azure Databricks, SQL Server, Data Warehouse and REST API call and SFTP. Please let me know if you have anyone else that has at least the 10 years of required experience? Role: Senior Data Engineer Pay Rate: $Market Duration: 6+ Months:...

  • Senior Data Engineer

    1 month ago

    columbus, United States PTR Global Full time

    Senior Data EngineerColumbus Ohio (hybrid, 3 days onsite)6/12 Month Contract (Possible Extension or Hire)No C2C / W2 OnlyUSC/GCQualificationsMinimum10+ years of experience required IT.3+ years of experience required in related field (AWS/bigdata).3+ years of experience working with any cloud platforms (design and development).Strong data analysis skillsGood...

  • Senior Data Engineer

    1 month ago

    columbus, United States PTR Global Full time

    Senior Data EngineerColumbus Ohio (hybrid, 3 days onsite)6/12 Month Contract (Possible Extension or Hire)No C2C / W2 OnlyUSC/GCQualificationsMinimum10+ years of experience required IT.3+ years of experience required in related field (AWS/bigdata).3+ years of experience working with any cloud platforms (design and development).Strong data analysis skillsGood...

  • Data Engineer

    1 month ago

    columbus, United States Revel IT Full time

    This is a contract to hire with our direct client in Columbus, Ohio.Hybrid!NO THIRD PARTIES!Data Engineer should have proficiency in data engineering for Azure Data Engineers. Mastery in designing robust, scalable, and secure data solutions on Azure is essential. This includes understanding Azure's data services ecosystem, such as Azure Data Lake, Azure...

  • Data Engineer

    1 month ago

    columbus, United States Revel IT Full time

    This is a contract to hire with our direct client in Columbus, Ohio.Hybrid!NO THIRD PARTIES!Data Engineer should have proficiency in data engineering for Azure Data Engineers. Mastery in designing robust, scalable, and secure data solutions on Azure is essential. This includes understanding Azure's data services ecosystem, such as Azure Data Lake, Azure...

  • Data Engineer

    1 month ago

    Columbus, United States Revel IT Full time

    This is a contract to hire with our direct client in Columbus, Ohio.Hybrid!NO THIRD PARTIES!Data Engineer should have proficiency in data engineering for Azure Data Engineers. Mastery in designing robust, scalable, and secure data solutions on Azure is essential. This includes understanding Azure's data services ecosystem, such as Azure Data Lake, Azure...

  • Senior Data Quality Engineer for Healthcare

    5 days ago

    Columbus, Ohio, United States Beacon Hill Staffing Group Full time

    Beacon Hill Staffing Group is seeking a Senior Data Quality Engineer to support our clients in the healthcare industry.About the RoleThe Senior Data Quality Engineer will be responsible for ensuring the accuracy, completeness, and integrity of data sets for use in internal data systems and for delivery from internal systems. This includes designing,...

  • Senior Data Engineer

    1 month ago

    Columbus, United States PTR Global Full time

    Senior Data EngineerColumbus Ohio (hybrid, 3 days onsite)6/12 Month Contract (Possible Extension or Hire)No C2C / W2 OnlyUSC/GCQualificationsMinimum10+ years of experience required IT.3+ years of experience required in related field (AWS/bigdata).3+ years of experience working with any cloud platforms (design and development).Strong data analysis skillsGood...

  • Senior Data Analytics Engineer

    3 weeks ago

    Columbus, United States Vaco Remote Work Freelance Full time

    Senior Data EngineerColumbus, OH. Candidates local to Columbus preferred, 2nd choice would be anywhere in Ohio.6 month contract-to-hirePay range: $60 - $80 / hr.  *Candidates MUST live in the state of Ohio to be considered* PURPOSE OF POSITION:The Senior Data Analytics Engineer is a Technical Analyst responsible for building datasets, reports, analytics,...

  • Senior Data Engineer

    6 days ago

    Columbus, Ohio, United States Diverse Lynx Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our team in Columbus, OH. As a key member of our data engineering team, you will play a critical role in building and maintaining our data platforms using Python Spark AWS.ResponsibilitiesCollaborate with cross-functional teams to design and implement data pipelines that drive...

  • Senior Data Systems Engineer for Manufacturing Operations

    6 days ago

    Columbus, Ohio, United States Abbott Laboratories Full time

    Job SummaryWe are seeking a skilled Senior OT Systems Engineer to join our team at Abbott Laboratories. As a key member of our operations team, you will be responsible for developing, deploying, and maintaining manufacturing operation data systems.About UsAbbott is a global healthcare leader that helps people live more fully at all stages of life. Our...

  • Senior Data Engineering Specialist

    6 days ago

    Columbus, Ohio, United States Maintec Technologies Full time

    Job Title: Data EngineerExperience: 11 to 13 YearsAt Maintec Technologies, we are seeking a highly skilled Data Engineer with 6-8 years of experience in PySpark and AWS. The ideal candidate will have a strong background in data acquisition, processing, and integration.Key Responsibilities:Data Acquisition and Processing: Develop and implement robust data...

  • Data Engineer

    4 days ago

    Columbus, Ohio, United States Resource Informatics Group Full time

    Job Description:We are seeking a highly skilled Senior Data Engineer to join our team at Resource Informatics Group. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data pipelines using Azure Data Factory (ADF), Azure Data Lake (ADLS), Azure Databricks, SQL Server, and Data...

  • Data Engineering Expert for Inclusive Data Ecosystem

    6 days ago

    Columbus, Ohio, United States Jobs for Humanity Full time

    Job DescriptionAbout UsJobs for Humanity is a collaborative platform that aims to build an inclusive and just employment ecosystem. We strive to provide opportunities for individuals from diverse backgrounds.About the RoleWe are seeking a highly skilled Data Engineer to join our team at Safelite, a leading auto glass company. As a Senior Data Analytics...

  • Data Engineer

    7 days ago

    Columbus, United States Maintec Technologies Full time

    Job Title: Data Engineer Experience: 11 to 13 Years Job Summary: Seeking a highly skilled Data Engineer with 6-8 years of experience in PySpark and AWS. Roles & Responsibilities: 1. Collaborate with cross-functional teams to understand data requirements and establish seamless connections with external vendors. 2. Develop and implement robust data...

  • Senior Data Scientist

    7 days ago

    Columbus, United States BJSS Full time

    About UsWe're an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we've been finding better, more sustainable ways to solve complex technology problems for some of the world's leading organizations and delivered solutions that millions of people use every day.In the last 30 years we won several awards, including our...

  • Senior Data Scientist

    6 days ago

    Columbus, United States BJSS Full time

    About Us We're an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we've been finding better, more sustainable ways to solve complex technology problems for some of the world's leading organizations and delivered solutions that millions of people use every day. In the last 30 years we won several awards, including...

  • Senior Data Scientist

    4 months ago

    Columbus, United States BJSS Full time

    About UsWe’re an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we’ve been finding better, more sustainable ways to solve complex technology problems for some of the world’s leading organizations and delivered solutions that millions of people use every day.In the last 30 years we won several awards,...

  • Senior Data Scientist

    2 months ago

    columbus, United States BJSS Full time

    About UsWe’re an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we’ve been finding better, more sustainable ways to solve complex technology problems for some of the world’s leading organizations and delivered solutions that millions of people use every day.In the last 30 years we won several awards,...