Data Engineering Leader for Large Data Set Processing

3 days ago


Columbus, Ohio, United States Seamless Full time
Job Description

At Seamless, we're seeking a highly skilled Principal Data Engineer with expertise in Python, Spark, AWS Glue, and other ETL technologies. This role involves designing, developing, and maintaining robust and scalable ETL pipelines to acquire, transform, and load data from various sources into our data ecosystem.

Responsibilities:
  • Develop efficient data acquisition and integration strategies in collaboration with cross-functional teams.
  • Implement data transformation logic using Python and other relevant programming languages and frameworks.
  • Create and manage ETL jobs, workflows, and data catalogs using AWS Glue or similar tools.
  • Optimize and tune ETL processes for improved performance and scalability, particularly with large data sets.
  • Apply methodologies and techniques for data matching, deduplication, and aggregation to ensure data accuracy and quality.
  • Implement and maintain data governance practices to ensure compliance, data security, and privacy.

Skillset:
  • Strong proficiency in Python and experience with related libraries and frameworks (e.g., pandas, NumPy, PySpark).
  • Hands-on experience with AWS Glue or similar ETL tools and technologies.
  • Solid understanding of data modeling, data warehousing, and data architecture principles.
  • Expertise in working with large data sets, data lakes, and distributed computing frameworks.
  • Experience developing and training machine learning models.
  • Strong proficiency in SQL.
  • Familiarity with data matching, deduplication, and aggregation methodologies.
  • Experience with data governance, data security, and privacy practices.
  • Strong problem-solving and analytical skills, with the ability to identify and resolve data-related issues.

Compensation: The estimated salary for this position is $160,000 - $200,000 per year, depending on location and experience.

  • Columbus, Ohio, United States Seamless Full time

    Job Title: Principal Data EngineerThe Opportunity:We are seeking a highly skilled and experienced Principal Data Engineer with expertise in Python, Spark, AWS Glue, and other ETL technologies to join our team at Seamless.AI.The ideal candidate will have a proven track record in data acquisition and transformation, as well as experience working with large...


  • Columbus, Ohio, United States Jobs for Humanity Full time

    Job DescriptionAbout UsJobs for Humanity is a collaborative platform that aims to build an inclusive and just employment ecosystem. We strive to provide opportunities for individuals from diverse backgrounds.About the RoleWe are seeking a highly skilled Data Engineer to join our team at Safelite, a leading auto glass company. As a Senior Data Analytics...

  • Data Science Leader

    3 days ago


    Columbus, Ohio, United States BJSS Full time

    Senior Data Scientist Role at BJSSInnovative Tech Consultancy Seeks Experienced ExpertiseWe are an award-winning tech consultancy dedicated to innovative problem-solving. With a 30-year history of delivering sustainable solutions for leading organizations, our teams bring together diverse expertise and collaborative cultures. Our latest recognition in the...

  • Data Engineer

    7 days ago


    Columbus, Ohio, United States Resource Informatics Group Full time

    Job Description:We are seeking a highly skilled Senior Data Engineer to join our team at Resource Informatics Group. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data pipelines using Azure Data Factory (ADF), Azure Data Lake (ADLS), Azure Databricks, SQL Server, and Data...


  • Columbus, Ohio, United States Abercrombie and Fitch Co. Full time

    Abercrombie & Fitch Co. is a global leader in the retail industry, operating five iconic lifestyle brands. We are seeking an experienced Data Engineering Leader to join our team and drive the development of cutting-edge data solutions.Job Description:The successful candidate will be responsible for leading the development of data engineering infrastructure,...


  • Columbus, Ohio, United States SysMind Tech Full time

    About the RoleWe are seeking an experienced Data Scientist and Engineering Leader to join our team at SysMind Tech. As a key member of our organization, you will be responsible for developing and productionizing Python and Java applications using machine learning frameworks like Keras or PyTorch.Key ResponsibilitiesDevelop high-quality software solutions...


  • Columbus, Ohio, United States Seamless Full time

    Job OverviewAt Seamless.AI, we're seeking a highly skilled Principal Data Engineer to lead our data engineering efforts.Key ResponsibilitiesDesign and develop scalable ETL pipelines using Python and AWS Glue.Collaborate with cross-functional teams to understand data requirements and develop efficient data acquisition and integration strategies.Implement data...


  • Columbus, Ohio, United States BJSS Full time

    About UsAt BJSS, we're an award-winning tech consultancy with a 30-year track record of delivering innovative solutions to global clients. Our collaborative culture and diverse teams of experts are driven by a passion for creative problem-solving.We've established a strong presence in the US for over a decade, with offices in Columbus, Ohio, and Houston,...


  • Columbus, Ohio, United States Manpower Group Inc. Full time

    Job Title: Enterprise Data Modeling Leader">About the Role:">We are seeking a seasoned Data Engineer - Data Modeler to play a critical leadership role in defining and refining enterprise data models at Manpower Group Inc. As an integral member of our team, you will guide the selection and development of data models for various applications across the...


  • Columbus, Ohio, United States Cyborgwave Full time

    Job OverviewCyborgwave seeks a skilled Data Engineer to join our team. This role requires expertise in building and maintaining large-scale data warehouses using cloud-based systems.

  • Data Scientist II

    7 days ago


    Columbus, Ohio, United States Nationwide Children's Hospital Full time

    Welcome to Nationwide Children's Hospital, where we're committed to improving the lives of children and families through innovative data-driven solutions. As a Data Scientist II in our Research Information Solutions and Innovation division, you'll have the opportunity to work on cutting-edge clinical informatics research projects that impact clinical care...

  • Data Scientist

    4 weeks ago


    Columbus, Ohio, United States Aerotek Full time

    As a Data Scientist (Machine Learning Engineer) at {company}, you will be working on developing and implementing machine learning models to solve complex business problems. Your primary responsibility will be to design, train, and deploy models using various algorithms and tools. In this role, you will collaborate with cross-functional teams to identify key...


  • Columbus, Ohio, United States Chemical Abstracts Service Full time

    About the RoleCAS is a leading provider of scientific information solutions, and we are seeking a highly skilled Data Engineer to join our team. As a key member of our infrastructure team, you will be responsible for designing, developing, and maintaining our data infrastructure.We are looking for an expert in AWS services, with strong programming skills in...


  • Columbus, Ohio, United States PTR Global Full time

    About the RolePTR Global is seeking a highly skilled Principal Data Architect to join our team. As a key member of our data engineering team, you will be responsible for designing and implementing large-scale data systems that drive business growth.Key ResponsibilitiesDesign and develop data models that meet business requirementsDevelop and implement data...


  • Columbus, Ohio, United States Maintec Technologies Full time

    Job Title: Data EngineerExperience: 11 to 13 YearsAt Maintec Technologies, we are seeking a highly skilled Data Engineer with 6-8 years of experience in PySpark and AWS. The ideal candidate will have a strong background in data acquisition, processing, and integration.Key Responsibilities:Data Acquisition and Processing: Develop and implement robust data...


  • Columbus, Ohio, United States Beacon Hill Staffing Group Full time

    Beacon Hill Staffing Group is seeking a Senior Data Quality Engineer to support our clients in the healthcare industry.About the RoleThe Senior Data Quality Engineer will be responsible for ensuring the accuracy, completeness, and integrity of data sets for use in internal data systems and for delivery from internal systems. This includes designing,...


  • Columbus, Ohio, United States Syntricate Technologies Full time

    We are seeking a highly skilled Data Engineer to join our team at Syntricate Technologies. In this role, you will be responsible for designing and implementing scalable data pipelines using Apache Spark and Python.The ideal candidate will have strong expertise in both Java and PySpark, with a focus on developing efficient and reliable data processing...


  • Columbus, Ohio, United States Diverse Lynx Full time

    Job DescriptionWe are seeking a highly skilled Distributed Data Engineer to join our team at Diverse Lynx LLC. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data platforms using Python, Spark, and PySpark.Key Responsibilities:Develop and maintain data platforms using Python,...


  • Columbus, Ohio, United States Diverse Lynx Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our team in Columbus, OH. As a key member of our data engineering team, you will play a critical role in building and maintaining our data platforms using Python Spark AWS.ResponsibilitiesCollaborate with cross-functional teams to design and implement data pipelines that drive...


  • Columbus, Ohio, United States Experis Full time

    Experis, a leading workforce solutions company, is seeking a seasoned Data Engineer to join its team in Columbus, Ohio or remotely. This role offers a competitive pay rate of $90-100/hour.About the RoleThe successful candidate will work closely with business stakeholders to design and implement data analytics solutions using Power BI Service and cloud &...