Advanced Data Engineer

4 days ago

Cincinnati, United States Compunnel Inc. Full time

Job Title:- Advanced Data Engineer

Duration:- 12+ Months Contract

Location:- Hybrid (3 days/week onsite) at Cincinnati, OH


Job Description:


***Note from the hiring manager on the technical skillset here: This is very important to have GCP, BigQuery/BigTable and some understanding of development in Python. Kafka, Java, NoSQL are plusses***


As an Advanced Data Engineer, you will have the opportunity to lead the development of innovative data solutions, enabling the effective use of data across the organization. You will be responsible for designing, building, and maintaining robust data pipelines and platforms to meet business objectives, focusing on data as a strategic asset. Your role will involve collaboration with cross-functional teams, leveraging cutting-edge technologies, and ensuring scalable, efficient, and secure data engineering practices. A strong emphasis will be placed on expertise in GCP, Vertex AI, and advanced feature engineering techniques.



Requirements:

  • 4+ years of professional Data Development experience.
  • 4+ years of experience with SQL and NoSQL technologies.
  • 3+ years of experience building and maintaining data pipelines and workflows.
  • 5+ years of experience developing with Java.
  • 2+ years of experience developing with Python.
  • 3+ years of experience developing Kafka solutions.
  • 2+ years of experience in feature engineering for machine learning pipelines.
  • Experience with GCP services such as BigQuery, Vertex AI Platform, Cloud Storage, AutoMLOps, and Dataflow.
  • Experience with CI/CD pipelines and processes.
  • Experience with automated unit, integration, and performance testing.
  • Experience with version control software such as Git.
  • Full understanding of ETL and Data Warehousing concepts.
  • Strong understanding of Agile principles (Scrum).



Additional Qualifications:

  • Knowledge of Structured Streaming (Spark, Kafka, EventHub, or similar technologies).
  • Experience with GitHub SaaS/GitHub Actions.
  • Experience understanding Databricks concepts.
  • Experience with PySpark and Spark development.


Key Responsibilities:

  • Provide Technical Leadership: Offer technical leadership to ensure clarity between ongoing projects and facilitate collaboration across teams to solve complex data engineering challenges.
  • Build and Maintain Data Pipelines: Design, build, and maintain scalable, efficient, and reliable data pipelines to support data ingestion, transformation, and integration across diverse sources and destinations, using tools such as Kafka, Databricks, and similar toolsets.
  • Drive Digital Innovation: Leverage innovative technologies and approaches to modernize and extend core data assets, including SQL-based, NoSQL-based, cloud-based, and real-time streaming data platforms.
  • Implement Feature Engineering: Develop and manage feature engineering pipelines for machine learning workflows, utilizing tools like Vertex AI, BigQuery ML, and custom Python libraries.
  • Implement Automated Testing: Design and implement automated unit, integration, and performance testing frameworks to ensure data quality, reliability, and compliance with organizational standards.
  • Optimize Data Workflows: Optimize data workflows for performance, cost efficiency, and scalability across large datasets and complex environments.
  • Mentor Team Members: Mentor team members in data principles, patterns, processes, and practices to promote best practices and improve team capabilities.
  • Draft and Review Documentation: Draft and review architectural diagrams, interface specifications, and other design documents to ensure clear communication of data solutions and technical requirements.
  • Cost/Benefit Analysis: Present opportunities with cost/benefit analysis to leadership, guiding sound architectural decisions for scalable and efficient data solutions.

  • Data Engineer | Data Engineer

    2 weeks ago

    cincinnati, United States Data Engineer Full time

    • 6+ years of software product development and delivery experience.• 5+ years of ETL testing/SQL experience.• 3+ years of Informatica Power Center, IDQ, IICS/IDMC experience.• 3+ years’ experience overseeing the development of data test plans, test scenarios, test cases, execution of test case and documentation of test results• 2+ years of...

  • Data Engineer

    2 weeks ago

    Cincinnati, United States Data Engineer Full time

    • 6+ years of software product development and delivery experience.• 5+ years of ETL testing/SQL experience.• 3+ years of Informatica Power Center, IDQ, IICS/IDMC experience.• 3+ years’ experience overseeing the development of data test plans, test scenarios, test cases, execution of test case and documentation of test results• 2+ years of...

  • Compunnel Inc. | Advanced Data Engineer

    5 days ago

    cincinnati, United States Compunnel Inc. Full time

    Job Title:- Advanced Data EngineerDuration:- 12+ Months ContractLocation:- Hybrid (3 days/week onsite) at Cincinnati, OHJob Description:***Note from the hiring manager on the technical skillset here: This is very important to have GCP, BigQuery/BigTable and some understanding of development in Python. Kafka, Java, NoSQL are plusses***As an Advanced Data...

  • Compunnel Inc. | Advanced Data Engineer

    5 days ago

    cincinnati, United States Compunnel Inc. Full time

    Job Title:- Advanced Data EngineerDuration:- 12+ Months ContractLocation:- Hybrid (3 days/week onsite) at Cincinnati, OHJob Description:***Note from the hiring manager on the technical skillset here: This is very important to have GCP, BigQuery/BigTable and some understanding of development in Python. Kafka, Java, NoSQL are plusses***As an Advanced Data...

  • Data Engineer with Advanced Data Architecture

    6 days ago

    Cincinnati, Ohio, United States eTek IT Services, Inc. Full time

    Job DescriptionWe are seeking a skilled Data Engineer with strong expertise in mortgage banking to join our IT team. As a key member of our data engineering team, you will be responsible for designing, constructing, and maintaining highly scalable data management systems that support data analysis and decision-making processes.Key Responsibilities:Design and...

  • Data Engineer

    3 weeks ago

    Cincinnati, United States Apex Systems Full time

    Job#: 2055521 Job Description: Apex Systems is looking for a Data Engineer for one of our large clients in Cincinnati, OH. Job Title: Data Engineer Role Overview: We are looking for a Data Engineer with a focus on processing unstructured data to enhance our enterprise data governance framework. You will design and implement scalable workflows that...

  • Senior Data Engineer

    4 days ago

    Cincinnati, United States Flexton Inc. Full time

    As an Advanced Data Engineer, you will have the opportunity to lead the development of innovative data solutions, enabling the effective use of data across the organization. You will be responsible for designing, building, and maintaining robust data pipelines and platforms to meet business objectives, focusing on data as a strategic asset. Your role will...

  • Senior Data Engineer

    5 days ago

    Cincinnati, United States Flexton Inc. Full time

    As an Advanced Data Engineer, you will have the opportunity to lead the development of innovative data solutions, enabling the effective use of data across the organization. You will be responsible for designing, building, and maintaining robust data pipelines and platforms to meet business objectives, focusing on data as a strategic asset. Your role will...

  • Data Engineer

    4 weeks ago

    Cincinnati, United States JobRialto Full time

    Job Summary: We are seeking a highly skilled Data Engineer specializing in unstructured data processing to enhance our enterprise data governance framework. This role involves designing and implementing scalable workflows to transform unstructured data into valuable assets while integrating their metadata into cataloging systems like Informatica CDGC. The...

  • Data Engineer

    3 weeks ago

    Cincinnati, United States w3r Consulting Full time

    Job Title: Data Engineer GENERAL FUNCTION: The data engineer designs and builds platforms, tools, and solutions that help the bank manage, secure, and generate value from its data. The person in this role creates scalable and reusable solutions for gathering, collecting, storing, processing, and serving data on both small and very large (i.e. Big Data)...

  • Data Engineering Specialist

    2 weeks ago

    Cincinnati, Ohio, United States InterSources Full time

    Job Description:Azure Data Solutions Architect is required for a 6-month project. The successful candidate will have experience in data modeling, advanced SQL techniques, and cloud migration methodologies. They will also be proficient in tools like Databricks, Azure Data Factory, and Azure Functions.Responsibilities:- Develop data models and implement...

  • Data Engineer:

    1 month ago

    Cincinnati, United States Saxon Global Full time

    Data Engineer: Must be in Cincinnati OH from day 1. Expectation is 2-3x/week in office and remote the other 2 days. Location: Cincy OH (they're going to convert this person and will be hybrid in office) C2C: $70/HR. Visa: USC or GC Perm - must be able to convert after 6 months Adv Data Engineer: Databricks, Azure, GCP. Streaming Interfaces: Kafka, Azure...

  • Data Engineer

    24 hours ago

    Cincinnati, United States Encore Talent Solutions Full time

    Encore Talent has partnered with a local client in staffing Entry Level Data Engineers.These roles are contract roles with the right-to-hire joining our clients cadet program in their training environment. This program is a one year training program with our Entry Level Engineers working with other recent hires.JOB DESCRIPTIONHandle the design and...

  • Data Engineer

    2 months ago

    Cincinnati, United States Encore Talent Solutions Full time

    Encore Talent has partnered with a local client in staffing Entry Level Data Engineers.These roles are contract roles with the right-to-hire joining our clients cadet program in their training environment. This program is a one year training program with our Entry Level Engineers working with other recent hires.JOB DESCRIPTIONHandle the design and...

  • Data Engineer II

    1 month ago

    Cincinnati, United States w3r Consulting Full time

    Job Summary: Handle the design and construction of scalable management systems, ensure that all data systems meet company requirements, and also research new uses for data acquisition. Required to know and understand the ins and outs of the industry such as data mining practices, algorithms, and how data can be used. Primary Responsibilities: Design,...

  • Flexton Inc. | Senior Data Engineer

    5 days ago

    cincinnati, United States Flexton Inc. Full time

    As an Advanced Data Engineer, you will have the opportunity to lead the development of innovative data solutions, enabling the effective use of data across the organization. You will be responsible for designing, building, and maintaining robust data pipelines and platforms to meet business objectives, focusing on data as a strategic asset. Your role will...

  • Flexton Inc. | Senior Data Engineer

    5 days ago

    cincinnati, United States Flexton Inc. Full time

    As an Advanced Data Engineer, you will have the opportunity to lead the development of innovative data solutions, enabling the effective use of data across the organization. You will be responsible for designing, building, and maintaining robust data pipelines and platforms to meet business objectives, focusing on data as a strategic asset. Your role will...

  • Data Engineer

    1 month ago

    Cincinnati, United States PeopleCaddie Full time

    Job Description: DATA ENGINEER | $55-$65/HR | DURATION: 6+ MONTHS | REMOTE! Our client, a developer-led custom software development consulting firm, is seeking an experienced data engineer to work on high value projects across diverse industries. This is an excellent opportunity for an experienced data engineer to contribute to the overall team goals and...

  • Senior Data Science Engineer

    2 weeks ago

    Cincinnati, Ohio, United States w3r Consulting Full time

    Job Description:As a Senior Data Science Engineer at w3r Consulting, you will utilize your expertise in machine learning and MLOps to establish a secure and automated deployment pipeline for our existing batch inference model. This role involves a combination of engineering and change management tasks, with a focus on educating data scientists and other team...

  • Senior Cloud Data Engineer

    2 weeks ago

    Cincinnati, Ohio, United States Worldpay Full time

    Job Title: Senior Cloud Data EngineerLocation: Remote (or onsite)Estimated Salary: $160,000 - $200,000 per yearAbout the Job:We are seeking a highly skilled Senior Cloud Data Engineer to join our ever-evolving Cloud Data Platform team. As a key member of this team, you will be responsible for designing, implementing, and operationalizing large-scale data and...