Engineer of Data Pipelines

2 days ago


Reston, Virginia, United States ZipRecruiter Full time

We're seeking an experienced Engineer of Data Pipelines to join our team at ZipRecruiter. As a key member of our engineering team, you will be responsible for designing and implementing data pipelines using Apache NiFi.

About the Job

In this role, you will:

  • Design and implement intricate data flows from various sources using Apache NiFi.
  • Develop innovative approaches for complex data manipulation challenges using Python.
  • Employ Apache NiFi and custom Python scripts to retrieve data from remote APIs, modify it, and distribute it according to customer requirements.
  • Transform legacy Java implementations into holistic Python solutions, leveraging modern libraries to achieve equivalent functionality.
  • Build Grafana dashboards to display data flow metrics and infrastructure status.
  • Analyze current data flows for inefficiencies and propose optimized remedies.
Requirements

To be successful in this role, you will need:

  • A bachelor's degree from an accredited institution in an area applicable to this position or one year of relevant experience.
  • Proficient in utilizing the Linux command-line interface.
  • Proficiency in a programming language, preferably Python.
  • Excellent analytical and technical troubleshooting skills.
  • Ability to work both as part of a team and independently.
Salary

The estimated salary for this role is $160,000 - $220,000 per year, depending on experience.

],

  • Reston, Virginia, United States ICF Full time

    We are seeking a skilled Transformation Data Engineer to join our team at ICF, a global advisory and technology services provider. This role will play a pivotal part in shaping the future of data engineering, working closely with cross-functional teams to deliver high-quality solutions that drive informed decision-making.Job Description:In this role, you...


  • Reston, Virginia, United States BruckEdwards Inc Full time

    BruckEdwards, Inc. is a growing government contracting company providing IT services to the federal government.About the RoleWe are seeking an experienced Data Pipeline Project Lead to join our team in Washington D.C.The successful candidate will oversee the development, maintenance, and enhancement of data pipelines for the Enterprise Data Platform (EDP) at...


  • Reston, Virginia, United States BruckEdwards Inc Full time

    Job OverviewWe are seeking a highly skilled Data Pipeline Management Specialist to join our team at BruckEdwards, Inc.As a data-driven organization, we provide IT services to the federal government and are dedicated to making a positive impact on our customers by delivering high-end technical solutions and outstanding customer service.About UsBruckEdwards,...


  • Reston, Virginia, United States Forge Group LLC Full time

    Data Engineer Role OverviewForge Group LLC is a global consulting firm seeking a highly skilled Data Engineer to join our team. The ideal candidate will possess expertise in building data ingestion and egress pipelines using APIs and SFTP methods. This role requires a strong understanding of AWS technologies and Python programming. As a Data Engineer at...


  • Reston, Virginia, United States McAfee Full time

    {"title": "Data Ingestion Engineer", "section": [{"label": "About the Role", "content": "We are seeking an analytical leader to join our Enterprise Data Platform team as the Capability Lead, Data Ingestion, reporting to the Enterprise Data Platform Lead."}, {"label": "Key Responsibilities", "content": "Leading the Enterprise Data Platform Ingestion...


  • Reston, Virginia, United States Piper Companies Full time

    Piper Companies seeks a seasoned Data Engineer to support our government clients' data needs. This role requires residing near the Reston, Virginia area and holding an active Secret clearance.The successful candidate will be responsible for collecting, processing, and analyzing structured and unstructured data, developing ETL pipelines, designing and...


  • Reston, Virginia, United States The Pennsylvania State University Full time

    Pursue a career as a Data Engineer Specialist at The Pennsylvania State University and join our team of experts in developing innovative data solutions.The proposed salary for this position is $86,300.00 - $164,000.00 per year, depending on qualifications and experience. Our company offers a comprehensive benefits package, including medical, dental, and...


  • Reston, Virginia, United States CapTech Full time

    Data Engineering Consultant RoleCappedata engineering consultants enable clients to build and maintain advanced data systems that bring together data from disparate sources to inform decision-makers.About the RoleThis position involves developing pipelines and preparing data for use by data scientists, data analysts, and other data systems.Key...


  • Reston, Virginia, United States CapTech Full time

    Job DescriptionWe are looking for a skilled Data Engineering Specialist to join our team. As a Data Engineering Specialist, you will be responsible for developing data pipelines and other data products using cloud-based technologies such as Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP).Estimated Salary: $150,000 per year.Our...

  • Data Engineer

    2 weeks ago


    Reston, Virginia, United States SOSi Full time

    About the Role:SOSi is seeking a Data Engineer to join our analytics team working on an innovative MLOps workload leveraging cutting-edge technologies and supporting a government customer in Huntsville, Alabama.This role will be responsible for delivering automation to key national security missions interacting with petabyte-scale data on supercomputing...

  • Data Engineer Lead

    2 days ago


    Reston, Virginia, United States Walmart Full time

    Job Title: Data Engineer LeadAbout the Role: We are seeking a highly skilled Data Engineer Lead to join our team at Walmart. The ideal candidate will have expertise in designing, implementing, and maintaining large-scale data processing systems.Responsibilities:Design and implement efficient data pipelines using Apache Spark, Hadoop, and other...


  • Reston, Virginia, United States Starcom consulting limited Full time

    About the RoleWe are seeking an experienced Lead Data Engineer to join our team at Starcom Consulting Limited. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining large-scale data infrastructure on cloud and on-premise platforms.Job DescriptionThe successful candidate will have a strong background in...


  • Reston, Virginia, United States CapTech Full time

    About the Role">This is a fantastic opportunity to join a dynamic team of experts at CapTech. As a Data Engineer, you will play a critical role in enabling clients to build and maintain advanced data systems that drive decision-making.Your responsibilities will include developing data pipelines and other data products using AWS, Azure, or GCP, advising...

  • Senior Data Engineer

    7 hours ago


    Reston, Virginia, United States Ageatia Global Solutions Full time

    **Job Title:** Senior Data EngineerWe are seeking a skilled Senior Data Engineer to join our team at Ageatia Global Solutions. This is a highly rewarding opportunity for an experienced professional to lead the development of our data integration infrastructure.**About the Role:*Lead the design and implementation of data pipelines using Ab Initio, Hadoop, and...


  • Reston, Virginia, United States CapTech Consulting Full time

    CapTech Consulting is seeking a seasoned Senior Cloud Data Engineer Leader to join our team. This role offers an exceptional opportunity to drive innovative data solutions and lead cross-functional teams in delivering high-quality, cloud-based data engineering projects.About the RoleAs a Senior Cloud Data Engineer Leader, you will be responsible for...


  • Reston, Virginia, United States ASCENDING LLC Full time

    ASCENDING LLC is seeking a highly skilled Geospatial Data Engineer to join our team. As a key member of our cross-functional Agile Scrum team, you will be responsible for developing performant GIS applications and designing geospatial data delivery pipelines using open-source and ESRI technologies. The ideal candidate will have 6+ years of experience in...

  • DevSecOps Engineer

    23 hours ago


    Reston, Virginia, United States Knowledge Management, Inc. Full time

    About the RoleWe are seeking an experienced DevSecOps Engineer to join our team at Knowledge Management, Inc. as a Pipeline Security Specialist. In this role, you will play a critical part in designing, implementing, and maintaining secure and efficient software development and deployment pipelines.As a key member of our team, you will collaborate with...


  • Reston, Virginia, United States Walmart Full time

    About Walmart Global TechImagine working in an environment where one line of code can make life easier for hundreds of millions of people. That's what we do at Walmart Global Tech.We're a team of software engineers, data scientists, cybersecurity experts, and service professionals within the world's leading retailer who make an epic impact and are at the...


  • Reston, Virginia, United States Futran Tech Solutions Pvt. Ltd. Full time

    Company OverviewFutran Tech Solutions Pvt. Ltd. is a leading provider of innovative technology solutions. We are seeking an experienced AWS Data Engineer with Splunk to join our team.In this role, you will be responsible for designing, building, and maintaining scalable data pipelines using AWS services such as EC2, Lambda, Beanstalk, Batch, or ECS. You will...


  • Reston, Virginia, United States CapTech Full time

    Company OverviewCapTech is an award-winning consulting firm that collaborates with clients to achieve what's possible through the power of technology.We're passionate about the work we do and the results we achieve for our clients. Our founders shared a collective passion to create a consultancy centered on strong relationships that would stand the test of...