Data Pipeline Engineer

4 weeks ago

North Charleston South Carolina, United States CommIT Enterprises, Inc Full time
Overview:
CommIT Enterprises, Inc. is seeking an Data Pipeline Engineer to join the team in Charleston, SC-*this contingent position is remote We are looking for a talented and forward-facing Data Pipeline Engineer who will be a critical part of a fast-moving Marine Corps product development team. This team is responsible for the design, development, testing, implementation and support for a wide range of cloud-based data integrations, data pipelines, data warehousing solutions, and delivery of Business Intelligence (BI) services to Marines globally.

 

Established in 2001, CommIT is a Certified Veteran-Owned Small Business (CVOSB) providing innovative technical engineering and data science services. Our enterprise systems support includes the Department of Defense’s (DoD) GCSS-MC, CAC2S, TBMCS-MC, and the Department of Veteran’s Affairs’ (VA) telehealth communications. We offer acquisition management, systems engineering, Agile software development, cloud management, IT modernization, data analytics, cybersecurity, and training, including leading-edge DevSecOps, automated testing, and mobile application development.

Responsibilities:
Your essential job functions will include but may not be limited to-

You will identify and deliver data pipeline solutions every day

In this role you will be working in a cross-functional team to build data pipeline solutions and processes to enable data warehousing, analytics and reporting for the USMC Logistics community.

You will be responsible for anything and everything related to existing and future Data Pipelines

Work with the Product Owner team and data analysts to develop efficient ETL processes using Apache Nifi, and other data pipeline tools.

Build performant data transformations for analysis and reporting.

Utilize modern ETL/ELT practices to move data into our Data Lake (AWS S3) and Warehouse (AWS Redshift).

Own the design, code development, code reviews, testing, data quality monitoring, deployment activities and operations support for all Nifi data flows.

Work alongside the infrastructure, data ingestion, data warehouse and analytics teams to manage and optimize ETL and data pipeline parameters, scheduling, monitoring and alerting.

Support development of data dictionary, connection guides, operations and user support SLAs.

Basic SQL operations required to Load data into the Data Warehouse (AWS Redshift)

Proven analytical, troubleshooting, and problem-solving skills.

Ability to communicate well cross-functionally, including non-technical users.

Must have a strong background in building ETL workflows.  Apache Nifi is preferred, but other similar tools such as AWS Glue, Talend, DataBricks, etc

Ability to be self-starting, responsive and collaborative in a remote environment working with a distributed team.

Stay current with the latest design and technology trends and adopt modern techniques as appropriate.

Familiarity with operating in Cloud environments

Proficiency in one or more database platforms (e.g., MySQL, Oracle, SQL Server).

Strong understanding of SQL and ability to optimize queries for performance.

Familiarity with database monitoring tools to track performance metrics.

Experience with open-source software

Experience with AWS GovCloud

Familiarity with AWS Glue and AWS Redshift

Familiarity with BI tools, e.g. Microsoft Power BI and AWS QuickSight

Basic Linux command line experience

Experience with NNCompass functionality

Qualifications:
Required Experience and Education:

Education: BS Degree in Electrical/ Electronic/ Computer /Software Engineering, Physics, or Computer Science from an accredited College or University.

Experience: Six (6) years of experience in DoD C4ISR systems, or information technologies to include: Systems Analysis, Systems Architecture, Systems/Equipment Support, Test and Evaluation, and Logistics support of C5ISR requirements. Three (3) years of technical experience in support of DoD C4ISR systems, or information technologies. Experience may be achieved simultaneously OR-Education: Technical training in Computer Science, Information Systems, Engineering, Business, and Supply Chain related scientific or technical discipline or 10 years of Marine Corps experience in Maintenance, Communications, Logistics, or in the Retail Supply (SMU/Reparable Issue Point/RIP) and Organizational Supply field preferably.

Experience: Twelve (12) years of hands-on experience with a USMC supply, maintenance or logistics IT system, GCSS-MC, to includethree (3) of the following five (5) areas: Systems Requirements, Operational Requirements, Data Analysis, Test & Evaluation, and Training. Experience may be achieved simultaneously. Recognized expert who has demonstrated industry and/or public service (e.g., federal, state, or local) leadership in Defense Acquisitions University Level I or II (Life Cycle Logistics or Program Management) and experience with Major Automated Information System (MAIS).

Security Requirements:

Secret Clearance

U.S. Citizenship

DOD 8570 compliant – Security + and OS certification

Equal Opportunity Employer:

CommIT Enterprises, Inc. is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, national origin, gender, sexual orientation, gender identity, age, physical or mental disability, genetic factors, military/veteran status or other characteristics protected by law.

  • Data Engineer

    2 weeks ago

    South San Francisco, United States TALENT Software Services Full time

    Are you an experienced Data Engineer with a desire to excel? If so, then Talent Software Services may have the job for you! Our client is seeking an experienced Data Engineer to work in Bloomington, MN(Remote).Position Summary: Looking for a Data Engineer to join our team and help us develop, maintain, and optimize ETL processes. This position requires a...

  • Data Engineer

    2 weeks ago

    South San Francisco, United States TALENT Software Services Full time

    Are you an experienced Data Engineer with a desire to excel? If so, then Talent Software Services may have the job for you! Our client is seeking an experienced Data Engineer to work in Bloomington, MN(Remote).Position Summary: Looking for a Data Engineer to join our team and help us develop, maintain, and optimize ETL processes. This position requires a...

  • Senior Data Scientist, Fraud Risk

    2 weeks ago

    Charlotte, North Carolina, United States Square Full time

    Job Description We are looking for a Data Scientist to join our Risk Data  team. You'll build processes to root out high-risk activity across the Square platform of products. You will also manage top level Risk KPIs, establish core operational metrics, and make improvements by using machine learning solutions or through analysis. The Risk Data Scientist...

  • IT Lead ETL Engineer

    3 weeks ago

    Charlotte, North Carolina, United States Brightspeed Full time

    Job Description We are currently looking for an IT Lead ETL Engineer to join our growing team! In this role, you will develop modern data pipelines to meet key business objectives. We are looking for enthusiastic, motivated ETL engineer. You will display proven ability to organize, prioritize task, resolve technical issues, proactive thinking. You will have...

  • Data Engineer III

    5 days ago

    North Kingstown, United States FM Global Full time

    FM Global is a leading property insurer of the world's largest businesses, providing more than one-third of FORTUNE 1000-size companies with engineering-based risk management and property insurance solutions. FM Global helps clients maintain continuity in their business operations by drawing upon state-of-the-art loss-prevention engineering and research;...

  • Data Scientist

    1 week ago

    Cary, North Carolina, United States Cornerstone Building Brands Full time

    Job Description Job Summary:   •    Lead complex data analysis, feature engineering, and model development initiatives to address critical business challenges. Utilize advanced statistical and machine learning techniques to derive insights and inform strategic decisions.  •    Innovate and refine predictive models and machine learning...

  • Senior Devops/Data Engineer

    5 days ago

    Belcourt, North Dakota, United States BlueChip Financial Full time

    BlueChip Financial (BCF) is looking for an experienced Senior Devops/Data Engineer who loves building software to help drive our company's growth. This position reports directly to the BCF Solutions Architect. The role will be split into two different areas: Devops / infrastructure (70-80% of the time) Data engineering (20-30% of the time) In this...

  • Analytics Data Engineer

    6 days ago

    South San Francisco, United States Claritas Rx Full time

    Who We AreClaritas Rx is a venture-backed digital health startup that brings clarity to the challenges of specialty biopharmaceutical products in the marketplace. In today’s highly complex specialty networks, our mission is to illuminate the patient experience beyond the clinical trial. Claritas Rx leverages a proprietary technology platform and deep...

  • Analytics Data Engineer

    1 week ago

    South San Francisco, United States Claritas Rx Full time

    Who We AreClaritas Rx is a venture-backed digital health startup that brings clarity to the challenges of specialty biopharmaceutical products in the marketplace. In today’s highly complex specialty networks, our mission is to illuminate the patient experience beyond the clinical trial. Claritas Rx leverages a proprietary technology platform and deep...

  • Analytics Data Engineer

    1 week ago

    South San Francisco, United States Claritas Rx Full time

    Who We AreClaritas Rx is a venture-backed digital health startup that brings clarity to the challenges of specialty biopharmaceutical products in the marketplace. In today’s highly complex specialty networks, our mission is to illuminate the patient experience beyond the clinical trial. Claritas Rx leverages a proprietary technology platform and deep...

  • Data Engineer

    1 week ago

    Carolina, United States Siri InfoSolutions Inc Full time

    Job DescriptionJob DescriptionRole: Data EngineerLocation: Charlotte, NC on Site Employment : Fulltime In this role you will serve as a senior dedicated technical resource responsible for Oracle PL/SQL development for our Data Warehouse running on a Linux basedOracle 11g RAC 2 node cluster. The Data Warehouse is currently under development and you will be an...

  • Underground Senior Data Engineer

    4 weeks ago

    Kershaw, South Carolina, United States OceanaGold Full time

    Job Description Responsible for integration of Underground Operational data into PowerBI Responsible for Maintaining Deswik Ops and Newtrax enterprise resource management systems Responsible for analysing data and reporting opportunities for improvement to appropriate team members Responsible for identifying gaps in data and assisting with closing of...

  • Principal Data Architect

    1 month ago

    Charleston, West Virginia, United States Syneos Health - USA Full time

    Principal Data Architect The Technology & Data Solutions (TDS) business unit is the Syneos Health accelerator for life sciences innovation. TDS houses our advanced technology acquisitions, homegrown products & applications, actively participates in new M&A and partnership activities, delivers critical insights as part of our integrated strategy for...

  • Solutions Architect, AI/ML

    1 week ago

    North Carolina, United States Snowflake Computing Full time

    Build the future of data. Join the Snowflake team.We are looking for people who have a strong background in data science and cloud architecture to join our Professional Services team to help create exciting new offerings and capabilities for our customers! This team will be working with customers using Snowflake to expand their use of the Snowflake Data...

  • Senior Enterprise Data Consultant

    3 weeks ago

    Raleigh, North Carolina, United States First Citizens Bank Full time

    Overview: This position is within the Data Management group that is responsible for enterprise governed data delivery. This individual will play a subject matter expert role for data delivery across multiple data domains per the business units data needs. Responsibilities: Collaborate with large cross-functional teams, SMEs, vendors, etc. to deliver...

  • Product Engineer, Data Focus

    1 week ago

    Cary, North Carolina, United States Cornerstone Building Brands Full time

    Job Description The Product Data Engineer position is an integral addition to our Product Data and Engineering team.  The individual who accepts this role will have the opportunity to work directly with our Window and Door products as well as collaborate across all of Cornerstone Building Brands.  This role will focus on transforming our product data and...

  • Civil Project Engineer II/III

    4 weeks ago

    Charleston, South Carolina, United States Timmons Group Full time

    Job Description Timmons Group is seeking a Civil Project Engineer II/III for our land development team in Charleston, SC. Project Engineer II/IIIs are Civil Engineers with 2 to 10 years’ experience.  The qualified individual will perform site development engineering and project management tasks for residential and commercial projects. Candidates should...

  • Director, Global Pipeline Forecasting

    4 weeks ago

    North Chicago, Illinois, United States AbbVie Full time

    Job Description AbbVie’s Pipeline Commercialization & Strategy (PCS) organization oversees the strategic marketing direction for key assets in AbbVie’s R&D pipeline. The Global Pipeline Forecasting & Analytics team, within US Therapeutics Finance, is seeking an experienced professional to lead forecasting and analytics supporting PCS Immunology. The...

  • Mid-Level Software/DSO Pipeline Engineer

    13 hours ago

    North Logan, United States Utah State University Space Dynamics Laboratory Full time

    Job ID 17726A The Space Dynamics Laboratory (SDL) is seeking a software engineer to become a part of our team. We are the world leaders in developing frameworks that support high speed image processing and distribution of image products.Our team implements an iterative software development methodology, providing solutions to our customer at regular...

  • Manufacturing Engineer- ESP Final Assembly

    1 week ago

    Charleston, South Carolina, United States Bosch Group Full time

    Job Description Lead manufacturing engineering performance and improvement activities for a final assembly line in the CC ESP product area. Develop, maintain, and apply lean manufacturing principles (BPS) throughout the value stream with regards to equipment design, selection, layouts, and material flow. Create and maintain manufacturing documentation and...