Data Engineer

3 weeks ago

Washington, United States Relational Data Systems Inc Full time

The Data Engineer is responsible for data ingestion, orchestration, transformation, and storage of data, making this available for analysis and downstream consumption. The role will develop jobs to ingest data from external data sources, configure data orchestration processes to manage data flows, manage performance for data storage and retrieval and implement continuous improvement processes to manage data quality and reliability of data management technology. Key Responsibilities

Reporting to the Data Lead, the Data Engineer will be responsible for the following: Engaging with stakeholders to understand the business requirements and assists in specifying possible solutions. Analysing existing enterprise data sets and source systems to determine most appropriate data sources to feed the solution, any data gaps, and quality of data sets. Create and maintain optimal data pipeline architecture, Assemble large, complex data sets that meet functional / non-functional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS technologies. Build analytic tools that utilize the data pipeline to provide actionable insights for commercial and corporate business requirements. Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs. Create data tools for analytics and data scientist team members that assist them in building and optimising data products. Work with data analysts to improve overall functionality in LSSA’s data systems Key Capabilities

Experience in using Data Orchestration tools to manage data flows e.g Apache AirFlow Understanding of Directed Acyclic Graphs (DAGs) to manage the relationship between nodes (workflow jobs) and how they relate (edges) to manage workflow orchestration. Experience in using Change Data Capture (CDC) pattern to track delta changes from data sources and tools used to implement this e.g. AWS DMS. Understanding of stream and batch-based ingestion technologies e.g. Kafka, Kinesis, AWS DMS Client Focus

Utilises a wide network of relationships internally and externally to ensure data visualisation and data consumption outputs are of high quality and targeted to current and potential requirements. Understanding of the industries RDS operates, ensuing analytical outputs are market orientated and commercially appropriate.

Data Management

Applies current analytical tooling including SQL, Python and Spark to generate analytical structures for data consumption. Understand common data modelling formats, including Slowly Changing Dimensions, Wide Table format and Dimensional Modelling. Ability to use complex data sets and being able to sense check the outcomes. Analyses source data to validate requirements and ensure data quality.

Operational Effectiveness

Can foresee issues and takes ownership and follows through and does not minimize problems. Reviews work and outcomes to identify ways to improve personal or team outcomes and anticipates the impacts and risks of decisions and actions.

Desirable Capabilities or Experience

Experience in using cloud platforms to create analytical workloads e.g. AWS, Azure. Understanding of software development practices and use of Application Lifestyle Management tools e.g. Atlassian Suite, Azure DevOps Understanding of agile delivery methods e.g. Kanban, Scrum. Understand APIs and secure transport mechanisms to transport large data sets e.g. sFTP. Understanding of Machine Learning for use in advanced analytics. Special Conditions

Must be prepared to work outside normal business hours from time to time. Interstate and intrastate travel may be required. Satisfactory National Police Clearance Check.

#J-18808-Ljbffr

  • Data Scientist

    4 weeks ago

    Washington, United States Data Machines Full time

    Data Scientist -Hybrid - 0205-R Location: Washington, DC, Metro Area Job Description: Come join the future of data-driven decision making! At Data Machines we leverage data analytics, DevSecOps, machine intelligence, and data science to engineer solutions for our Federal government, defense, and commercial sponsors to solve real-world, critical mission...

  • Data Engineer

    1 month ago

    Washington, United States Atechstar Full time

    Key Responsibilities Design implement and support applications that provide structured and timely access to actionable business information addressing stakeholder needs. Interface directly with stakeholders gathering requirements and owning automated end-to-end reporting solutions. Partner with analysts data engineers business intelligence engineers ...

  • Data Engineer

    1 month ago

    Washington, United States Non-Departmental Agency Full time

    Summary Data Engineers work with data consumers to create and populate optimal data architectures, structures, and systems to meet CIA's business needs. ...

  • Data Engineer

    1 month ago

    Washington, Washington, D.C., United States Non-Departmental Agency Full time

    Summary Data Engineers work with data consumers to create and populate optimal data architectures, structures, and systems to meet CIA's business needs. Duties As a Data Engineer for CIA, you will focus on the design, implementation, and operation of data management systems to meet the CIA's business needs. This includes designing how the data will be...

  • Data Engineer

    1 week ago

    Washington, Washington, D.C., United States Atechstar Full time

    Key Responsibilities Design implement and support applications that provide structured and timely access to actionable business information addressing stakeholder needs. Interface directly with stakeholders gathering requirements and owning automated end-to-end reporting solutions. Partner with analysts data engineers business intelligence engineers and...

  • Data Engineer

    4 weeks ago

    Washington, United States Square Peg Technologies Full time

    Job DescriptionJob DescriptionFull Job Description Position Requires a Top Secret (TS/SCI) Clearance with a Polygraph. The Mission: At Square Peg, we rely on powerfully insightful data to inform our systems and solutions—and we’re seeking an experienced pipeline-centric data engineer to put it to use. The Data Engineer will be responsible for designing,...

  • Data Engineer

    4 weeks ago

    Washington, United States ITR Full time

    Job DescriptionJob DescriptionSenior Data Engineer – Top Secret Clearance RequiredWill implement large-scale data ecosystems including data management, governance and the integration of structured and unstructured data to generate insights leveraging cloud-based platforms. • Leverage automation, cognitive and science-based techniques to manage data,...

  • Big Data Engineer

    1 month ago

    Washington, United States Dash techology Full time

    We’re looking for a Big Data Engineer who can find creative solutions to tough problems. As a Big Data Engineer you’ll create and manage our data infrastructure and tools including collecting storing processing and analyzing our data and data systems. You know how to work quickly and accurately using the best solutions to analyze mass data sets and...

  • Big Data Engineer

    1 month ago

    Washington, Washington, D.C., United States Dash techology Full time

    We're looking for a Big Data Engineer who can find creative solutions to tough problems. As a Big Data Engineer you'll create and manage our data infrastructure and tools including collecting storing processing and analyzing our data and data systems. You know how to work quickly and accurately using the best solutions to analyze mass data sets and you know...

  • Data Engineer

    1 month ago

    Washington, United States Atechstar Full time

    Responsibilities Design build and own all the components of a high-volume data warehouse end to end. Build efficient data models using industry best practices and metadata for ad-hoc and pre-built reporting Provide wing-to-wing data engineering support for project lifecycle execution (design execution and risk assessment) Interface with business customers...

  • Data Engineering Fellow

    4 weeks ago

    Washington, United States BlueLabs Analytics, Inc. Full time

    **About BlueLabs** BlueLabs is a leading provider of analytics services and technology for a variety of industry clients: including government, business, and political campaigns. We help our clients optimize their engagements with individual customers, supporters, and stakeholders to achieve their goals. Simply put: we help our partners do the most good by...

  • Data Engineer

    5 days ago

    Washington, United States Gunnison Consulting Group Inc Full time

    Job DescriptionJob Description*This job listing is to express interest in a future job opening. Our team will review applications and reach out to qualified candidates when the need arises. Thank you for your interest!We are seeking a motivated and customer-oriented Data Engineer to support our US Courts client.The customer is looking to consolidate and...

  • Data Engineer

    4 weeks ago

    Washington, Washington, D.C., United States Atechstar Full time

    Responsibilities Design build and own all the components of a high-volume data warehouse end to end. Build efficient data models using industry best practices and metadata for ad-hoc and pre-built reporting Provide wing-to-wing data engineering support for project lifecycle execution (design execution and risk assessment) Interface with business customers...

  • Databricks Data Engineer

    4 weeks ago

    Washington, United States Square Peg Technologies Full time

    Job DescriptionJob DescriptionClearance Required:Position Requires a Top Secret (TS/SCI) Clearance with a Polygraph. Location:Northern VirginiaThe Mission:Square Peg is seeking a Data Engineer to be responsible for designing, implementing, and maintaining data pipelines for our customers. The Data Engineer will develop automated pipelines for a variety of...

  • Data Quality Engineer

    4 weeks ago

    Washington, United States Inclusion Cloud Full time

    About the job Data Quality Engineer WHAT YOUD BE DOINGAs a Data Quality Engineer, you will be responsible for building automated, repeatable, efficient processes to perform data quality checks across various data assets. It is your duty to ensure high quality data is available to internal stakeholders and customers.To do this you will be designing,...

  • Senior Data Engineer

    2 days ago

    Washington, United States Shara Inc Full time

    Job DescriptionJob DescriptionSalary: Job Title: Senior Data EngineerLocation: Ideally based in one of Shara's hubs in Washington DC, Nairobi, Lagos, or Abuja.Travel: occasional trips (2-4x per year) to Washington DC, Nairobi or LagosAt Shara, we’re building for a future where African banking is free, frictionless, and driven by user experience. Where...

  • Data Analyst/Data Engineer

    1 month ago

    Ft. Washington, Maryland, United States ENSCO Inc. Full time

    ENSCO Mission Systems Group (MSG) is recruiting senior engineering and support staff supporting the Office of the Undersecretary of Defense - Intelligence & Security Branch (OUSD/I&S). We are currently seeking a Data Analyst/Data Engineer who will interface with functional and technical experts. Successful candidates will support the modernization,...

  • Data Analyst/Data Engineer

    4 days ago

    Ft. Washington, Maryland, United States ENSCO Inc. Full time

    ENSCO Mission Systems Group (MSG) is recruiting senior engineering and support staff supporting the Office of the Undersecretary of Defense - Intelligence & Security Branch (OUSD/I&S). We are currently seeking a Data Analyst/Data Engineer who will interface with functional and technical experts. Successful candidates will support the modernization,...

  • Azure Data Engineer

    2 weeks ago

    Washington, United States AKT Corp Full time

    Job DescriptionJob DescriptionTitle: Azure Data EngineerLocation: Washington DCResponsibilities:Design, develop, and deploy data pipelines and ETL processes on Azure using services such as Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.Implement data storage solutions on Azure, including Azure SQL Database, Azure Cosmos DB, and Azure Blob...

  • Sr Data Engineer

    4 weeks ago

    Washington, United States Serigor Inc Full time

    Title: Sr Data EngineerLocation: 100% Remote – EST Time zoneDuration: 12+ MonthsKey Skills: AWS, Spark, Adobe Analytics/AEP(Adobe Experience Platform) platform experience, Glue, Lamda, Python, Scala, EMR, Talend, PostgreSQL, RedshiftJob Description:Looking for a Sr level person – Min 10+ YearsLook for a Sr Data Engineer with Adobe Analytics or AEP(Adobe...