Data Engineer Sr

5 days ago

Augusta, Maine, United States Zapata Technology Full time
Job Summary:

The DevOps and Data Engineer will be part of a skilled IT team that works independently and/or collectively with other engineers to identify and test potential enhancements and resolve technical issues with the rapid scaling of an intelligence application.

This position is contingent upon award. The expected award is in October.

Key Responsibilities:

Data Engineering:
Collaboratively develop and implement custom-developed ETL scripts and support software products as needed. Ensure data quality, consistency, and integrity throughout the pipeline. Create, monitor, and adjust data ingestion flows. Process raw data by transforming it into a usable form (ETL).

Rapidly create data pipelines to support training exercises with little prior notification to connect fielded systems to cloud-based infrastructure as well as other fielded systems. Develop for and work with the ZIngest system, an ingestion system developed by Zapata Technology.

Systems Administration:
Database management, data management, and fixing issues that arise. Perform basic SysAdmin tasks, including the installation of peripheral software and software updates.

Software Engineering:
Provide full life cycle database design and software development systems analysis to include requirements gathering; development; testing; documentation; supporting technical writers and testers; configuration; maintenance of developed software projects. Manage lifecycle development of software, gaining and understanding requirements, executing actual development, documentation, maintenance, and expansion of deliverables as the need arises. Implement rapid features and API capabilities to meet evolving customer needs for data access and dissemination. Interface with the customer(s) to create ingestion feeds for unique systems with the ability to implement solutions with few prior known details/documentations about said systems. Use source-code control to track and protect changes to the code baseline.

Infrastructure Management:
Design, architect, and implement scalable infrastructure on multiple IC networks and application "nodes" using DevSecOps practices. Manage containerization efforts (Docker, Podman) and orchestration. Upgrade and migrate applications to containers to allow the required scalability and move legacy monolith applications to a microservices type environment. Document Infrastructure administration, tear down, and creation.

Automation and Security:
Implement Infrastructure as Code (IaC) using Puppet, Ansible or similar technology to meet security mandated change management guidelines. Ensure application and database compliance with DISA and GISA standards and regulations. Design and implement API security to protect unauthorized access to data and reduce exposed endpoints. Review API code and designs to identify potential security risks. Implement Role Based Access Control (RBAC) for APIs to meet basic security requirements. Document API endpoints using OpenAPI documentation standards. Work with Cyber-Security personnel to document security vulnerabilities and any mitigations to reduce risks that cannot be removed.

Requirements:
Knowledge of JavaScript frameworks and API development. Knowledge of Agile programming and development concepts including Jira or similar. Knowledge of DevSecOps processes including Infrastructure as Code using Puppet, Ansible, etc. Knowledge of containerization and container orchestration including networking using Docker, Podman, or Kubernetes. Experience with SIGINT and All Source Intelligence reports standards such as USMTF and lifecycle requirements management development. Experience with the full DataOps lifecycle including ingestion and ETL processes into databases, data lakes, or data warehouses. Ability to look at the big picture while evaluating software requests in order to determine where existing requirements intersect or can be used in conjunction with new products as needed. Ability to estimate the technical debt of the creation of new features or systems. Interpret specifications, troubleshoot, and define needed software solutions. Detail-oriented and organized; able to understand information systems and ensure accuracy of work. Grasp of Software Engineering concepts, especially scripting in Python and Groovy. Knowledge of Linux System administration including the Linux command line, service management, software configuration concepts in lieu of virtual machine constraints. Ability to work with the customer(s) to find out specifications and formats for the data. Ability to convert raw data into a human-readable and easily viewable format. Ability to convert data and fix malformed data. This requires an understanding of XML, JSON, and regular expressions. *Requires an IAT Level II certification.

Preferred Qualifications and Skills:
An understanding of pub-sub messaging systems including RabbitMQ, Pulsar, and Kafka. A solid understanding of and experience in NiFi data flow, from configuration to creating new flows and making new processors. Experience with troubleshooting the NiFi software. Knowledge of NiFi, with the ability to keep the cluster running. Ability to write scripts in Groovy and Python within NiFi and Linux. Ability to write new processors in NiFi using Java.

  • Data Engineer Sr

    1 week ago

    Augusta, Maine, United States Zapata Technology Full time

    Job Summary:The DevOps and Data Engineer will be part of a skilled IT team that works independently and/or collectively with other engineers to identify and test potential enhancements and resolve technical issues with the rapid scaling of an intelligence application.This position is contingent upon award, with an expected award in October.Key...

  • Data Engineer

    1 month ago

    Augusta, Maine, United States Peraton Full time

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at Peraton. As a Data Engineer, you will play a critical role in designing, building, and maintaining large-scale data processing systems.Key Responsibilities:Design and implement data pipelines using Apache NiFi, Apache Kafka, and Apache SparkDevelop and maintain data...

  • Data Engineer

    2 weeks ago

    Augusta, Maine, United States Peraton Full time

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at Peraton. As a Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems and pipelines.Responsibilities:Design and implement data pipelines using Apache NiFi, Apache Kafka, and Apache SparkDevelop and maintain...

  • Data Engineer

    1 month ago

    Augusta, Maine, United States Peraton Full time

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at Peraton. As a Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems and pipelines.Responsibilities:Design and implement data pipelines using Apache NiFi, Apache Kafka, and Apache SparkDevelop and maintain...

  • Data Engineer

    4 weeks ago

    Augusta, Maine, United States Beacon Hill Staffing Group Full time

    Job SummaryThe Data Engineer is responsible for designing, developing, and maintaining data pipelines and infrastructure to support data-driven decision-making within the organization. This role requires strong proficiency in Python, SQL, cloud technologies, and Kubernetes to ensure the efficient processing, storage, and retrieval of data.Key...

  • Data Engineer

    2 weeks ago

    Augusta, Maine, United States Huntington Ingalls Industries Full time

    Job DescriptionEnlighten, a leader in big data solution development and deployment, seeks a Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, building, and optimizing Big Data Platform (BDP) systems for data collection, storage, access, and analytics at scale.The ideal candidate will have a strong background in data...

  • Data Engineer

    5 days ago

    Augusta, Maine, United States Huntington Ingalls Industries Full time

    At Huntington Ingalls Industries, we are seeking a highly skilled Data Engineer to join our team of experts in big data solution development and deployment. As a Data Engineer, you will play a critical role in designing, building, and optimizing our Big Data Platform (BDP) systems for data collection, storage, access, and analytics at scale.The ideal...

  • Data Engineer

    1 week ago

    Augusta, Maine, United States Zapata Technology Full time

    Data Engineer Job DescriptionZapata Technology is seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will play a critical role in the development and implementation of custom-developed ETL scripts, ensuring data quality, consistency, and integrity throughout the pipeline.Key Responsibilities:Collaborate with cross-functional...

  • Software Engineer for Data Transformation

    2 weeks ago

    Augusta, Maine, United States Zapata Technology Full time

    Job Summary:Zapata Technology is seeking an experienced Software Engineer to lead the transformation of our data infrastructure. The successful candidate will design and develop modular ETL solutions, utilizing custom ASP.NET applications and Postgres stored procedures, to ingest and deliver data from various systems to analytics platforms.Key...

  • Lead CAD Designer Sr

    1 month ago

    Augusta, Maine, United States Leidos Full time

    Job Title: Lead CAD Designer SrJob Summary:Leidos is seeking a highly skilled Lead CAD Designer Sr to join our team. As a key member of our design team, you will be responsible for creating design drawing packages, leading a team of drafters, and completing assignments and miscellaneous drawings as required by project scope.Responsibilities:Complete drafting...

  • Senior Software Engineer

    5 days ago

    Augusta, Maine, United States Jacobs Full time

    This position is part of a dynamic team of systems and software engineers who will analyze existing capabilities and design new solutions to integrate with other products within the organization.Working in an agile environment, systems engineers will capture how capabilities work, determine use cases, perform testing and validation, and provide artifacts for...

  • Senior Software Engineer

    3 weeks ago

    Augusta, Maine, United States Jacobs Full time

    Job SummaryThis position is part of a highly agile team of systems and software engineers who will analyze existing capabilities and design new solutions that integrate with other products within the organization and across the agency baseline.Working in a dynamic and agile environment, systems engineers will capture how capabilities work, determine the use...

  • Data Quality Specialist

    4 days ago

    Augusta, Maine, United States Beacon Hill Staffing Group Full time

    Job SummaryData Quality Analyst sought by Beacon Hill Technologies to analyze data from various sources, identify errors, and develop processes to improve data quality. The ideal candidate will have a strong understanding of data integration principles and data governance frameworks.ResponsibilitiesAnalyze data from multiple sources to identify errors and...

  • Data Scientist

    1 week ago

    Augusta, Maine, United States CVS Health Full time

    Job SummaryWe are seeking a highly skilled Data Scientist to join our Clinical Data Science team at CVS Health. As a Data Scientist, you will play a critical role in activating our clinical data repository to improve outcomes across multiple lines of business and use cases.Key ResponsibilitiesBecome a subject matter expert in clinical data and how to apply...

  • Senior Software Engineer

    2 weeks ago

    Augusta, Maine, United States Jacobs Full time

    Job SummaryThis position is part of a dynamic team of systems and software engineers who will analyze existing capabilities and design new solutions that integrate with other products within the organization and across the agency baseline.As a member of a high-performing software development team in an agile, DevOps environment, you will work with a CI/CD...

  • Enterprise DevOps Engineer

    1 week ago

    Augusta, Maine, United States Erias Ventures Full time

    Job DescriptionErias Ventures is a dynamic company that values creative problem-solving, open communication, and empowering employees to make decisions and contribute new ideas. We are seeking a highly motivated and experienced DevOps Engineer to join our fast-paced development team.The ideal candidate will have strong working knowledge in Linux systems...

  • Maintenance Planner/Scheduler Sr.

    2 weeks ago

    Augusta, Maine, United States Clearwater Paper and Manchester Industries Full time

    Job SummaryWe are seeking a highly skilled Maintenance Planner/Scheduler Sr. to join our team at Clearwater Paper and Manchester Industries. As a key member of our maintenance team, you will be responsible for planning and coordinating maintenance activities to ensure the efficient operation of our facilities.Key ResponsibilitiesPartner with Operations...

  • Maintenance Planner/Scheduler Sr

    1 month ago

    Augusta, Maine, United States Clearwater Paper Full time

    About the RoleWe are seeking a highly skilled Maintenance Planner/Scheduler Sr to join our team at Clearwater Paper. As a key member of our maintenance team, you will play a critical role in ensuring the efficient operation of our manufacturing facilities.Key ResponsibilitiesPartner with Operations Maintenance Coordinator and Maintenance Supervisor to...

  • Maintenance Planner/Scheduler Sr

    4 weeks ago

    Augusta, Maine, United States Clearwater Paper Full time

    Job Summary:The Sr. Maintenance Planner will be responsible for ensuring efficient planning and coordination of maintenance work at the Augusta mill. This role will partner with Operations Maintenance Coordinator and Maintenance Supervisor to develop and implement daily and annual outage maintenance plans. The successful candidate will have a strong...

  • Field Planning Engineer

    2 weeks ago

    Augusta, Maine, United States JCSI Full time

    Job Title: Field Planning EngineerAt JCSI, we are seeking a highly skilled and experienced Field Planning Engineer to join our team in Augusta, ME. As a key member of our Power Grid Utility and Substation Engineering team, you will be responsible for leading and working on various projects, including distribution line design, site visits, data gathering, and...