Senior Data Engineer

1 month ago


Columbus, United States g2o Full time

Your future starts here

Imagine being part of a team that helps clients build better relationships with customers. When you join us, you'll help top-notch clients to execute the digital strategies of the future. Every day, we collaborate with clients and each other to provide technology expertise, human-centered design and industry experience to deliver real business results.

Position description

Work collaboratively with technical team members to develop, deploy and production support data engineering jobs and pipelines - spark structured stream processing jobs across one or more streams, persisted to a data lake. Work with leads and architects to prototype designs and technologies. Design and develop data engineering jobs with an eye on quality attributes (and trade-offs) such as simplicity, reliability, scalability, extensibility, performance. Build or reuse common frameworks where possible. Set and adhere to coding and style guides to ensure consistency. Ensure code quality through mechanisms including code coverage, peer reviews, pair programming. Create and deploy streaming data quality checks with remediation runbooks. Adhere to data governance and compliance requirements and standards. Provide recommendations for and help curate technical backlogs for technical debt management and continuous improvement.

Required qualifications

5+ years of total experience working with data and data pipeline technologies at scale. 3+ years of solid Python development experience – including: Robust unit and integration testing and Git driven CI/CD to production 2+ years of proven experience developing streaming data processing tasks using pySpark and SparkSQL Solid understanding of Spark, preferably Databricks Spark and Apache Delta Read streaming data from Kafka, Apache Delta or similar, merge /enrich /qualify data inline, capture operations metrics, persist to target, debug and optimize At scale and in production Experience leveraging public cloud data and analytics services. Azure preferred, but AWS/GCP also works. Nice to have: Super user experience with reporting tools such a PowerBI.

Required education and experience

College/University Degree or equivalent.

  • Columbus, Ohio, United States Amazon Data Services, Inc. Full time

    Amazon Web Services (AWS) is looking for highly technical and critical individual to solve the most challenging problems in our industry. AWS Data Center Planning, Deliver, & Operations is a global team responsible for the design and operation of industry-leading, large-scale, business critical facilities. We are currently seeking highly motivated...


  • Columbus, United States ComResource Full time

    Job DescriptionJob DescriptionComResource is looking for a Senior Data Center Engineer.This person will be responsible for effectively provisioning, installing/configuring, operating, and maintaining data center hardware, software, and related systems. They will research, recommend, and implement processes and tools to continuously improve the performance...

  • Network Engineer

    7 days ago


    Columbus, United States Columbus Regional Health Full time

    What you need to know about this position: Under the direction of I.S. Manager of Data Center Services, the Network Engineer – Senior has direct responsibility for research development and technical assistance in production support, data security, disaster recovery, network engineering, business continuity planning, problem determination and...

  • Data Engineer

    5 days ago


    Columbus, United States Lambdanets Inc Full time

    Detailed Job Description – Data Engineer Position Title: Data Engineer Location: Columbus, OH Job Description: Data application design and development, Installation and Configuration, Customization of application, functional testing of software applications, perform business analysis for applications in Systems. The Data Engineer or Data Engineer...


  • Columbus, United States LHP Analytics & IoT Full time

    Job Title:            Senior Data Analytics DeveloperJob Location:  305 Franklin Street, Columbus, Indiana 47201, and various unanticipated worksites throughout the U.S.DutiesLHP Data Analytic Solutions LLC seeks a Senior Data Analytics Developer in Columbus, Indiana, and various unanticipated worksites throughout the U.S. Telecommuting...


  • Columbus, United States LHP Analytics & IoT Full time

    Job Title : Senior Data Analytics Developer Job Location : 305 Franklin Street, Columbus, Indiana 47201, and various unanticipated worksites throughout the U.S. Duties LHP Data Analytic Solutions LLC seeks a Senior Data Analytics Developer in Columbus, Indiana, and various unanticipated worksites throughout the U.S. Telecommuting permitted 100% of time....

  • Data Engineer

    2 days ago


    Columbus, United States Revel IT Full time

    Our Columbus, Ohio client is looking for a Data Engineer (Python focused) contractor to help support the IT organization’s data strategy in support of business customers, as well as internal initiatives. If you are a “Data Person,” who knows how to build data pipelines, aggregate data from multiple sources into a common data store and have experience...

  • Data Engineer

    2 days ago


    Columbus, United States Revel IT Full time

    Our Columbus, Ohio client is looking for a Data Engineer (Python focused) contractor to help support the IT organization’s data strategy in support of business customers, as well as internal initiatives. If you are a “Data Person,” who knows how to build data pipelines, aggregate data from multiple sources into a common data store and have experience...


  • Columbus, United States Ohiox Full time

    Beam is looking for a Senior Software Engineer with a passion for building user-focused projects in an agile environment. Our engineers get the opportunity to work throughout our broad product giving them an opportunity to work with a diverse set of technologies, shipping code at a regular cadence. We also look to our senior developers to be good mentors for...


  • Columbus, United States AECOM Full time

    Job Description Considering applying for this job Do not delay, scroll down and make your application as soon as possible to avoid missing out.AECOM is seeking a Sr Mechanical Engineer to be based in Columbus, Ohio.The Senior Mechanical Engineer will be responsible for working on various designs of high-performance buildings across multiple sectors,...


  • Columbus, United States Russ Hadick & Associates Full time

    Are you a skilled Construction Senior Project Manager with a background in Hyperscale Data Centers? We are looking for someone just like you! Our company is seeking a highly experienced Senior Project Manager to join our team. As a leading provider of Hyperscale Data Center solutions, we are looking for someone who has a proven track record of delivering...

  • Security Engineer

    16 hours ago


    Columbus, United States Data Resource Technologies Full time

    Job Description: Data ModelerTeam: Cyber Data ManagementAs a Data Modeler in our Cyber Data Management team, you will play a critical role in designing and implementing data models that aid the understanding, use, and protection of critical data streams across the corporate environment. You will work closely with cybersecurity experts and data scientists to...


  • Columbus, United States MM International Full time

    Job DescriptionJob DescriptionPython Data Engineer IN Columbus, OH (All new fresh roles) (USC, GC, GC EAD only), 4 openings 1 mid-level min 3-5 years exp. , 2 senior level min. 5-7+ years exp.Columbus, OHPython or Java is required Must be good with Microservices and API's THE PREFERENCE TO HIRE IS PYTHON WITH DATA BACKGROUND, the java is secondaryThere...

  • Senior MDM Engineer

    2 weeks ago


    Columbus, United States Ringside Talent Full time

    We are working on a remote opportunity for a Senior MDM Engineer. The ideal candidate will align with the responsibilities and qualifications outlined below.Responsibilities:Design and architect MDM solutions that meet business requirements and align with industry best practices.Lead the implementation of MDM solutions, including data modeling, data...

  • Senior MDM Engineer

    2 weeks ago


    Columbus, United States Ringside Talent Full time

    We are working on a remote opportunity for a Senior MDM Engineer. The ideal candidate will align with the responsibilities and qualifications outlined below.Responsibilities:Design and architect MDM solutions that meet business requirements and align with industry best practices.Lead the implementation of MDM solutions, including data modeling, data...


  • Columbus, United States LVI Associates Full time

    Having been in the industry for 50 years this Ohio based firm has built up a reputation through their history, their vision and their constant open mind and willingness to adapt to new idea. With a national reach, they are the perfect match for someone who loves a challenge and wants to get out there and meet their clients and engage with buildings. No more...


  • Columbus, United States LVI Associates Full time

    Having been in the industry for 50 years this Ohio based firm has built up a reputation through their history, their vision and their constant open mind and willingness to adapt to new idea. With a national reach, they are the perfect match for someone who loves a challenge and wants to get out there and meet their clients and engage with buildings. No more...


  • Columbus, Ohio, United States Burns & McDonnell Full time

    Burns & McDonnell is a company comprised of more than 14,000 engineers, architects, construction professionals, scientists, consultants, and entrepreneurs with offices across the country and throughout the world. Burns & McDonnell's Mission Critical team was ranked #2 by the ENR in 2023. We have exciting opportunities for those interested in growing their...

  • Senior Engineer

    7 days ago


    Columbus, United States Pennoni Full time

    Job DescriptionPennoni is a multidisciplinary consulting engineering firm that was founded over five decades ago! We approach engineering challenges from a wider spectrum of angles than most, from land development to energy management. Our goal is to help communities and private sector clients alike navigate the ever-changing technological advancements...


  • Columbus, United States BHI Energy Full time

    Job Description: Seeking experienced Senior Geotechnical Engineer is critical for the success of the team. Our Sr. Geotechnical Engineers will provide engineering and consulting services for various types of projects and clients. This may include performing and/or leading field investigations, engineering analyses, calculations, and recommendations, design...