Data Pipeline Engineer

3 weeks ago


Charleston, United States CommIT Enterprises Full time

Overview:

CommIT Enterprises, Inc. is seeking an

Data Pipeline Engineer

to join the team in

Charleston, SC- *this contingent position is remote We are looking for a talented and forward-facing Data Pipeline Engineer who will be a critical part of a fast-moving Marine Corps product development team. This team is responsible for the design, development, testing, implementation and support for a wide range of cloud-based data integrations, data pipelines, data warehousing solutions, and delivery of Business Intelligence (BI) services to Marines globally. Established in 2001, CommIT is a Certified Veteran-Owned Small Business (CVOSB) providing innovative technical engineering and data science services. Our enterprise systems support includes the Department of Defense’s (DoD) GCSS-MC, CAC2S, TBMCS-MC, and the Department of Veteran’s Affairs’ (VA) telehealth communications. We offer acquisition management, systems engineering, Agile software development, cloud management, IT modernization, data analytics, cybersecurity, and training, including leading-edge DevSecOps, automated testing, and mobile application development.

Responsibilities:

Your essential job functions will include but may not be limited to-

You will identify and deliver data pipeline solutions every day In this role you will be working in a cross-functional team to build data pipeline solutions and processes to enable data warehousing, analytics and reporting for the USMC Logistics community. You will be responsible for anything and everything related to existing and future Data Pipelines Work with the Product Owner team and data analysts to develop efficient ETL processes using Apache Nifi, and other data pipeline tools. Build performant data transformations for analysis and reporting. Utilize modern ETL/ELT practices to move data into our Data Lake (AWS S3) and Warehouse (AWS Redshift). Own the design, code development, code reviews, testing, data quality monitoring, deployment activities and operations support for all Nifi data flows. Work alongside the infrastructure, data ingestion, data warehouse and analytics teams to manage and optimize ETL and data pipeline parameters, scheduling, monitoring and alerting. Support development of data dictionary, connection guides, operations and user support SLAs. Basic SQL operations required to Load data into the Data Warehouse (AWS Redshift) Proven analytical, troubleshooting, and problem-solving skills. Ability to communicate well cross-functionally, including non-technical users. Must have a strong background in building ETL workflows. Apache Nifi is preferred, but other similar tools such as AWS Glue, Talend, DataBricks, etc Ability to be self-starting, responsive and collaborative in a remote environment working with a distributed team. Stay current with the latest design and technology trends and adopt modern techniques as appropriate. Familiarity with operating in Cloud environments Proficiency in one or more database platforms (e.g., MySQL, Oracle, SQL Server). Strong understanding of SQL and ability to optimize queries for performance. Familiarity with database monitoring tools to track performance metrics. Experience with open-source software Experience with AWS GovCloud Familiarity with AWS Glue and AWS Redshift Familiarity with BI tools, e.g. Microsoft Power BI and AWS QuickSight Basic Linux command line experience Experience with NNCompass functionality Qualifications:

Required Experience and Education:

Education:

BS Degree in Electrical/ Electronic/ Computer /Software Engineering, Physics, or Computer Science from an accredited College or University. Experience:

Six (6) years of experience in DoD C4ISR systems, or information technologies to include: Systems Analysis, Systems Architecture, Systems/Equipment Support, Test and Evaluation, and Logistics support of C5ISR requirements. Three (3) years of technical experience in support of DoD C4ISR systems, or information technologies. Experience may be achieved simultaneously

OR-

Education:

Technical training in Computer Science, Information Systems, Engineering, Business, and Supply Chain related scientific or technical discipline or 10 years of Marine Corps experience in Maintenance, Communications, Logistics, or in the Retail Supply (SMU/Reparable Issue Point/RIP) and Organizational Supply field preferably. Experience:

Twelve (12) years of hands-on experience with a USMC supply, maintenance or logistics IT system, GCSS-MC, to includethree (3) of the following five (5) areas: Systems Requirements, Operational Requirements, Data Analysis, Test & Evaluation, and Training. Experience may be achieved simultaneously. Recognized expert who has demonstrated industry and/or public service (e.g., federal, state, or local) leadership in Defense Acquisitions University Level I or II (Life Cycle Logistics or Program Management) and experience with Major Automated Information System (MAIS).

Security Requirements:

Secret Clearance U.S. Citizenship DOD 8570 compliant – Security + and OS certification Equal Opportunity Employer:

CommIT Enterprises, Inc. is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, national origin, gender, sexual orientation, gender identity, age, physical or mental disability, genetic factors, military/veteran status or other characteristics protected by law. #J-18808-Ljbffr



  • Charleston, United States Refyn Data Full time

    We are looking forward a Senior Business Data Engineer based in Clarksburg , MD, USA . In RefynData, making our work exciting, engaging, meaningful; ensuring safety, health, wellness; and being a model of inclusion & diversity are already embedded in who we are - it’s in our values, part of our vision, and our clearly defined aspirational goals. This...

  • Engineer, Engineering

    21 hours ago


    Charleston, United States UGI Utilities Full time

    Mountaineer Gas Company is the largest natural gas distribution company in West Virginia with over 220,000 customers and 450 employees. We service fifty (50) of the fifty-five (55) counties in West Virginia and maintain close to 6,000 miles of pipeline. Job Summary ENGINEER - CORPORATE OFFICE (CHARLETON, WV) We are currently recruiting in our Corporate...

  • Engineer, Engineering

    21 hours ago


    Charleston, United States Mountaineer Gas Full time

    Mountaineer Gas Company is the largest natural gas distribution company in West Virginia with over 220,000 customers and 450 employees. We service fifty (50) of the fifty-five (55) counties in West Virginia and maintain close to 6,000 miles of pipeline. Job Summary ENGINEER CORPORATE OFFICE (CHARLETON, WV) We are currently recruiting in our Corporate...

  • Engineer, Engineering

    14 hours ago


    Charleston, United States UGI Full time

    Mountaineer Gas Company is the largest natural gas distribution company in West Virginia with over 220,000 customers and 450 employees. We service fifty (50) of the fifty-five (55) counties in West Virginia and maintain close to 6,000 miles of pipeline. Job Summary ENGINEER - CORPORATE OFFICE (CHARLETON, WV) We are currently recruiting in our Corporate...


  • Charleston, United States Datacenter People Group Full time

    The Client Datacenter People is working with the leading data center commissioning and quality services firm in the world. The Role Provide engineering and technical support to plan, schedule, coordinate and execute full life cycle commissioning of critical facilities. Participate in the MEP design process. Witnesses the static and dynamic testing of complex...

  • Project Engineer

    21 hours ago


    Charleston, United States Kokosing Full time

    Kokosing is one of America's 60 largest General Contractors and services a broad spectrum of clients in both the private and public business sectors. Located in the Midwest and Mid-Atlantic Regions, Kokosing's services include the construction of highways/bridges, industrial facilities/factories, data centers, marine, solar, advanced industrial, pipeline,...


  • Charleston, United States Colliers International Full time

    Critical Facility Engineer page is loaded Critical Facility Engineer Apply locations Columbus, Ohio, United States of America time type Full time posted on Posted 2 Days Ago job requisition id JR12528 Make your next move an expert one. At Colliers it’s not our success that sets us apart, it’s how we achieve it. Our people are passionate, take ownership...


  • Charleston, United States Ampcus Full time

    Title: Digital DevOps Engineer Location: Charleston, SC - 29405 - Remote Duration: 12 months assignment with the possibility of extensionRole and Responsibilities: Reporting to the Head of Digital DevOps, the DevOps Engineer will play a critical role in driving innovation and growth for the Banking Solutions business. With a focus on digital software...


  • Charleston, West Virginia, United States Northeast Technical Services Full time

    Who we are: NTS Environmental Science and Engineering offers a broad spectrum of environmental services, including consulting, engineering, engineering support, and data management to a wide range of clients around the Iron Range in northern Minnesota. NTS is proud to provide an employee-driven culture that fosters professional development and growth....


  • Charleston, United States YourCause Full time

    Blackbaud unleashes the potential of the people and organizations who change the world, and we’re growing our team. As the leading software provider exclusively dedicated to powering social impact, expand what is possible across the nonprofit and education sectors, at companies committed to social responsibility, and for individual change makers. Our...


  • Charleston, West Virginia, United States ATCC Full time

    This role will shape the future of experimental biology at ATCC, a pioneer in biomaterials manufacturing, while establishing a new site focused on the generation and analysis of large-scale phenotypic data using high-throughput (HTP) automation methods. As part of the creation of a new site focused on accelerating Predictive Biology applications, ATCC is...

  • Cyber Engineer

    3 weeks ago


    Charleston, United States Crypticvector Full time

    Description At Cryptic Vector, we are dedicated to mission success. We take the time to understand our customers' needs, delivering products that perform when our nation needs them most. We understand that properly supporting the most unique missions of the United States government requires the nation’s best. Our focus is on creating a culture where the...


  • Charleston, West Virginia, United States Meta Full time

    Summary: Meta Platforms, Inc. (Meta), formerly known as Facebook Inc., builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps and services like Messenger, Instagram, and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D...


  • Charleston, United States Morgan 6 Full time

    Job DescriptionJob DescriptionDescription:Morgan 6 is a fast-growing leader in the government contracting field based in Charleston, SC. Our mission is to expertly apply state of the art technology to improve the lives of Warriors – past, present, and future. Our core solutions include Software Engineering and Agile Software Development, DevSecOps, IT...

  • Senior Data Analyst

    16 hours ago


    Charleston, West Virginia, United States Capital One Full time

    Locations: VA - Richmond, United States of America, Richmond, VirginiaSenior Data AnalystAt Capital One, data is at the center of everything we do. When we launched as a startup we disrupted the credit card industry by individually personalizing every credit card offer using statistical modeling and the relational database, cutting edge technology in 1988...


  • Charleston, United States Burgess & Niple Full time

    Registration: Professional Engineer (PE) Required Skills: AutoCAD or Microstation; Microsoft Office; Traffic Sponsorship: Immigration related employment benefits, e.g. visa sponsorship, are not available for this position. Collaborate with some of the top transportation experts in the industry and help lead the traffic operations/studies projects. We are...

  • Maintenance Lead

    3 hours ago


    Charleston, United States TC Energy Full time

    Determined. Imaginative. Curious. If these are some of the ways you describe yourself - we want to learn more about you! At TC Energy, we are Energy Problem Solvers - passionate about transitioning North America to cleaner energy while meeting the energy demands of today and tomorrow. If that sounds like a challenge you want to help tackle, we want you to...


  • Charleston, West Virginia, United States Capital One Full time

    West Creek , United States of America, Richmond, VirginiaSenior Associate, Data Scientist - Enterprise Tech AnalyticsData is at the center of everything we do. As a startup, we disrupted the credit card industry by individually personalizing every credit card offer using statistical modeling and the relational database, cutting edge technology in 1988...

  • Adobe Data Architect

    18 hours ago


    Charleston, United States Blue Acorn iCi Full time

    Job DescriptionJob DescriptionCompany DescriptionBlue Acorn iCi, an Infosys company, is a digital consultancy focused on delivering innovative solutions across customer experience, commerce, and data. Our team of over 400 experts enable clients to navigate large-scale, digital transformation programs. Whether it’s a digitally savvy consumer brand or a...


  • Charleston, United States Traveltechessentialist Full time

    Airbnb was born in 2007 when two Hosts welcomed three guests to their San Francisco home, and has since grown to over 4 million Hosts who have welcomed more than 1 billion guest arrivals in almost every country across the globe. Every day, Hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more...