Airflow Data Engineer

8 hours ago


Hattiesburg, Mississippi, United States HonorVet Technologies Full time
Position Overview

We are seeking a skilled Data Engineer to enhance and automate our data workflows using Astronomer Airflow. The successful candidate will be responsible for improving data processing capabilities, optimizing task scheduling, and reducing our dependency on Databricks for orchestration.

Key Responsibilities
  • Install and configure Astronomer Airflow, establishing necessary connections and ensuring the Airflow web server and scheduler are operational.
  • Migrate existing workflows to the Astronomer Airflow platform, ensuring they run error-free and meet performance benchmarks.
  • Conduct training sessions for the data engineering team, providing supporting materials and documentation to ensure team proficiency in creating and managing workflows and DAGs.
  • Set up monitoring tools to track workflow performance, configure alerts for critical issues, and establish regular reporting procedures for ongoing monitoring.
  • Review and optimize workflows for performance, create detailed documentation covering workflows, configurations, and best practices, and establish a feedback loop for continuous improvement.
  • Integrate Airflow with Azure functions, REST endpoints, and event-driven architectures, implementing support for external services to extend orchestration beyond Databricks.
  • Develop a system to handle job dependencies and implement checks to ensure jobs run only if the underlying data has changed.
Required Qualifications
  • Proven experience with Astronomer Airflow and data workflow orchestration.
  • Strong understanding of CI/CD pipelines and version control systems (e.g., GitHub).
  • Experience with cloud platforms (e.g., Azure) and integrating external services.
  • Proficiency in Python and SQL.
  • Excellent problem-solving skills and attention to detail.
Preferred Qualifications
  • Experience with Databricks and Snowflake.
  • Familiarity with machine learning and artificial intelligence workflows.
  • Strong communication and training skills.
  • Ability to work collaboratively in a team environment.

  • Engineering Manager

    4 weeks ago


    Hattiesburg, Mississippi, United States Selig Group Full time

    About the PositionSelig Group is seeking a seasoned operations leader to drive process improvement and champion a strong safety culture within our production facility. As a key member of our Operations team, you will be responsible for orienting processes, methods, and production technologies to meet current and future business needs.Key...


  • Hattiesburg, Mississippi, United States Raytheon Technologies Full time

    About the RoleWe are seeking a highly skilled Mission Assurance Engineer to join our team at Raytheon Technologies. As a key member of our Next Generation Jammer (NGJ) program, you will play a critical role in ensuring the quality and mission assurance of our products.Key ResponsibilitiesImplement quality and mission assurance responsibilities as a member of...


  • Hattiesburg, Mississippi, United States L3Harris Full time

    Job Title:Senior Associate, Systems EngineeringJob Summary:The Factory to Field Systems Engineer implements complex IT systems that satisfy specific customer requirements. This role involves designing and implementing IP networks, RF Base Station & Repeater equipment, and other custom subsystems. The successful candidate will have the ability to implement...


  • Hattiesburg, Mississippi, United States Jones Companies Full time

    Job SummaryJones Capital's Strategic Resource Group (SRG) is seeking a highly skilled and motivated Cybersecurity Engineer to join our dynamic team. The ideal candidate will be responsible for protecting the organization's computer systems, networks, and data from cyber threats.Key ResponsibilitiesDesign, implement, and maintain security infrastructure,...


  • Hattiesburg, Mississippi, United States RTX Full time

    Job SummaryWe are seeking a highly skilled Senior Linux Administrator to join our team at RTX. As a key member of our IT department, you will be responsible for troubleshooting and resolving technical problems of a moderate to high scope and complexity, including the integration of hardware and operating systems.Key ResponsibilitiesTroubleshoot and resolve...


  • Hattiesburg, Mississippi, United States Kohler Energy Full time

    Job SummaryWe are seeking a highly skilled Quality Assurance Specialist to join our team at Kohler Energy. As a Quality Assurance Specialist, you will be responsible for ensuring the effectiveness of our quality improvement program by conducting audits, evaluating processes, and implementing improvements.Key ResponsibilitiesConduct systems, procedure,...

  • Network Architect

    4 weeks ago


    Hattiesburg, Mississippi, United States HANWHA AZDEL, Inc. Full time

    {"h1": "Network Engineer Position at HANWHA AZDEL, Inc.", "p": "We are seeking a highly skilled Network Engineer to join our team at HANWHA AZDEL, Inc. The ideal candidate should have a strong combination of technical skills and relevant experience in network security and management.", "ul": [{"li": "Implementing and testing new security features to ensure...


  • Hattiesburg, Mississippi, United States University of Southern Mississippi Full time

    Job SummaryThis position provides administrative support to the School Director, faculty, staff, and students in the School of Child and Family Sciences. The successful candidate will serve as the first point of contact for the school, handling correspondence, coordinating student visits, and maintaining student files.Key ResponsibilitiesProvide academic...


  • Hattiesburg, Mississippi, United States Army National Guard Full time

    Geospatial Intelligence Imagery AnalystAs a Geospatial Intelligence Imagery Analyst for the Army National Guard, you will play a critical role in providing commanders with accurate and timely information about the battlefield.Your primary responsibility will be to analyze and interpret imagery information about enemy forces, identifying key targets and...


  • Hattiesburg, Mississippi, United States Army National Guard Full time

    Geospatial Intelligence Imagery AnalystAs a Geospatial Intelligence Imagery Analyst for the Army National Guard, you will play a critical role in providing commanders with accurate and timely information about the battlefield.Your primary responsibility will be to analyze and interpret imagery information about enemy forces, identifying key targets and...


  • Hattiesburg, Mississippi, United States University of Southern Mississippi Full time

    Postdoctoral Fellow OpportunityWe are seeking a highly motivated and experienced Postdoctoral Fellow to join our research team in the Karim Lab at the University of Southern Mississippi. The successful candidate will contribute to our ongoing research projects focused on the arthropod vector-transmitted pathogen Rickettsia parkeri.Key...


  • Hattiesburg, Mississippi, United States Army National Guard Full time

    Geospatial Intelligence Imagery AnalystAs a Geospatial Intelligence Imagery Analyst with the Army National Guard, you will play a critical role in providing commanders with accurate and timely information about the battlefield. Your expertise in analyzing imagery and geospatial data will help inform tactical and operational plans, ultimately contributing to...


  • Hattiesburg, Mississippi, United States Army National Guard Full time

    Geospatial Intelligence Imagery AnalystAs a Geospatial Intelligence Imagery Analyst with the Army National Guard, you will play a critical role in providing commanders with accurate and timely information about the battlefield. Your expertise in analyzing imagery and geospatial data will help inform tactical and operational plans, ultimately contributing to...


  • Hattiesburg, Mississippi, United States Army National Guard Full time

    Geospatial Intelligence Imagery AnalystAs a Geospatial Intelligence Imagery Analyst with the Army National Guard, you will play a critical role in providing commanders with accurate and timely information about the battlefield. Your expertise in analyzing imagery and geospatial data will help inform tactical and operational plans, ultimately contributing to...


  • Hattiesburg, Mississippi, United States University of Southern Mississippi Full time

    Job SummaryThe University of Southern Mississippi is seeking a highly skilled Associate Director of Mechanical, Electrical and Plumbing Systems to join our team. As a key member of our Physical Plant division, you will be responsible for providing technical expertise and oversight to ensure the efficient operation of our mechanical and electrical systems.Key...


  • Hattiesburg, Mississippi, United States Army National Guard Full time

    Geospatial Intelligence Imagery AnalystAs a Geospatial Intelligence Imagery Analyst for the Army National Guard, you will play a critical role in providing commanders with accurate and timely information about the battlefield.Your primary responsibility will be to analyze and interpret imagery information about enemy forces, identifying key targets and...


  • Hattiesburg, Mississippi, United States Army National Guard Full time

    Geospatial Intelligence Imagery AnalystAs a Geospatial Intelligence Imagery Analyst for the Army National Guard, you will play a critical role in providing commanders with accurate and timely information about the battlefield.Your primary responsibility will be to analyze and interpret imagery information about enemy forces, identifying key targets and...


  • Hattiesburg, Mississippi, United States University of Southern Mississippi Full time

    Job SummaryThis position provides administrative support to the Dean's Office, faculty, staff, and students in the College of Education and Human Sciences. The successful candidate will serve as the first point of contact for the Dean's Office, the public, and the university community.Key ResponsibilitiesProvide academic support to the Dean's Office,...


  • Hattiesburg, Mississippi, United States Trace3 Full time

    Job SummaryThe Senior Regional Solutions Architect will be the consultative technology leader aligned with a solutions-focused capability who creates strong client intimacy with their technical expertise. The role will provide education and thought leadership while translating client business requirements into implementable technology solutions. With the...


  • Hattiesburg, Mississippi, United States University of Southern Mississippi Full time

    Job SummaryWe are seeking a highly skilled PeopleSoft Administrator II to join our team at the University of Southern Mississippi. As a key member of our IT department, you will be responsible for the management, implementation, and monitoring of PeopleSoft Enterprise systems and the infrastructure necessary to support them.Key ResponsibilitiesInstall,...