Cloud Data Engineer

4 weeks ago

Washington, Washington, D.C., United States TalentRemedy Full time
About the Role

The Cloud Data Engineer will play a crucial part in designing, building, and maintaining data infrastructure and ETL pipelines using Databricks and cloud-based technologies. This role will enable data-driven decision-making and ensure data accuracy and accessibility for regulatory purposes.

Key Responsibilities
  1. Collaborate with cross-functional teams to design, develop, and maintain large-scale data and analytics platforms, system integrations, data pipelines, data models, and API integrations.
  2. Prototype emerging business use cases to validate technology approaches and propose potential solutions.
  3. Design, develop, and maintain data pipelines using Databricks, Apache Spark, and other cloud-based technologies to ingest, transform, and load data from various financial institutions and sources.
  4. Implement data transformation processes to ensure data quality, integrity, and consistency, meeting regulatory standards.
  5. Integrate data from diverse sources, including financial databases, APIs, regulatory reporting systems, and internal data stores, into the CFPB's data ecosystem.
  6. Develop and optimize data models for regulatory analysis, reporting, and compliance, following data warehousing and data lake principles.
  7. Monitor and optimize data pipelines for efficiency, scalability, and cost-effectiveness while ensuring data privacy and security.
  8. Ensure data governance and regulatory compliance, maintaining data lineage and documentation for audits and reporting purposes.
  9. Collaborate with cross-functional teams to understand data requirements and provide data support for regulatory investigations.
  10. Maintain comprehensive documentation for data pipelines, code, and infrastructure configurations, adhering to regulatory compliance standards.
  11. Identify and resolve data-related issues, errors, and anomalies to ensure data reliability and compliance with regulatory requirements.
  12. Stay updated with regulatory changes, industry trends, cloud technologies, and Databricks advancements to implement best practices and improvements in data engineering.
Requirements
  • Must be able to obtain and maintain a Public Trust clearance.
  • Must be a verifiable US Citizen for this Federal support position.

The ideal candidate will have a strong understanding of data lake, lakehouse, and data warehousing architectures in a cloud-based environment, as well as hands-on experience with Databricks, including data ingestion, transformation, and analysis.

The candidate should also have proficiency in Python for data manipulation, scripting, and automation, as well as in-depth knowledge of AWS services relevant to data engineering, such as Amazon S3, EC2, Database Migration Service (DMS), DataSync, EKS, CLI, RDS, Lambda, etc.

The candidate should be able to design and build flexible and scalable ETL processes and data pipelines using Python and/or PySpark and SQL, and have experience with data pipeline automation and workflow management tools like Apache Airflow or AWS Step Functions.

The candidate should also have knowledge of data quality management and data governance principles, strong problem-solving and troubleshooting skills related to data management challenges, and experience managing code in GitHub or other similar tools.

The ideal candidate will also have experience leveraging Postgres in a parallel processing environment, hands-on experience migrating from an on-premise data platform(s) to a modern cloud environment (e.g. AWS, Azure, GCP), and excellent problem-solving and communication skills.

The candidate should also have a strong attention to detail and the ability to work independently and collaboratively.

Preferred Skills
  • Experience with financial data or regulatory data management.
  • Experience working in Agile or DevSecOps environments and using related tools for collaboration and version control.
  • Knowledge of regulatory frameworks in the financial industry.
  • Familiarity with DevOps and CI/CD practices.
  • Experience with machine learning and AI technologies.

This role demands expertise in Databricks, cloud technologies, and a deep understanding of data engineering principles within a regulatory context.

Benefits

Applicants selected will be subject to a government security investigation and must meet eligibility requirements for access to classified information. Our client is an Equal Opportunity Employer, Minorities/Females/Veterans/Disabled. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, or national origin.

Candidates must be willing to consent to a background check including a criminal record check, employment, and education verification. No Exceptions

  • Data Engineer

    4 weeks ago

    Washington, Washington, D.C., United States Futran Tech Solutions Pvt. Ltd. Full time

    Job Title: Data Engineer - Cloud Data ArchitectAt Futran Tech Solutions Pvt. Ltd., we are seeking a highly skilled Data Engineer - Cloud Data Architect to join our team.Key Responsibilities:* Develop and maintain data pipelines using AWS Glue and Python.* Design and implement ETL/ELT processes.* Work with AWS Postgres to ensure data consistency and...

  • Cloud Engineer

    4 weeks ago

    Washington, Washington, D.C., United States Data Computer Corp of America Full time

    DCCA is a veteran-owned IT business that has been providing innovative solutions to government agencies and commercial enterprises since 1982.We pride ourselves on offering career growth opportunities and a competitive compensation and benefits package.Key Responsibilities:Lead MTF DM systems engineering to troubleshoot and resolve complex technical issues...

  • Cloud Data Architect

    4 weeks ago

    Washington, Washington, D.C., United States Databricks Full time

    About the Role:We are seeking a skilled Cloud Data Architect to lead the growth of the Databricks Unified Analytics Platform. As a member of our team, you will exercise and develop expertise in cloud platforms, data engineering, data analytics, and data science and machine learning.Key Responsibilities:Partner with the sales team to help customers understand...

  • Data Scientist

    4 weeks ago

    Washington, Washington, D.C., United States Data Society Full time

    Job DescriptionWe are seeking a highly skilled Data Scientist to join our team at Data Society Group. As a Data Scientist, you will be responsible for combining your expert knowledge in statistical analysis, data mining, data engineering, and artificial intelligence to build complex end-to-end business analytics, artificial intelligence, and IT solutions.Key...

  • Cloud Infrastructure Engineer

    1 month ago

    Washington, Washington, D.C., United States JCD Staffing Full time

    Job SummaryWe are seeking a highly skilled Cloud Infrastructure Engineer to join our team at JCD Staffing. The ideal candidate will have a strong background in designing, deploying, and maintaining cloud infrastructure and services on Microsoft Azure.Key Responsibilities:Design and implement scalable, reliable, and secure cloud-based infrastructure...

  • Data Quality Specialist

    4 weeks ago

    Washington, Washington, D.C., United States Inclusion Cloud Full time

    About the roleData Quality SpecialistAt Inclusion Cloud, we are seeking a skilled Data Quality Specialist to join our team. As a Data Quality Specialist, you will be responsible for building automated processes to ensure high-quality data is available to internal stakeholders and customers.You will design, develop, and document automated test plans and test...

  • Cloud Platform Engineer

    4 weeks ago

    Washington, Washington, D.C., United States Huntington Ingalls Industries Full time

    Job SummaryEnlighten, an HII Company, is seeking a highly skilled Cloud Platform Engineer to join our team. As a Cloud Platform Engineer, you will play a critical role in the development and deployment of custom software components and integration of open-source code to address complex time series analysis problems through the use of cutting-edge Big Data /...

  • Cloud Engineer Position

    4 weeks ago

    Washington, Washington, D.C., United States Breakforth Solutions Inc Full time

    Job OverviewBreakForth Solutions Inc is seeking a skilled Cloud Engineer to support our federal customers. The ideal candidate will collaborate closely with our customers to ensure successful oversight and client delivery of impacted projects.This role will be responsible for performing tasks such as:Collaborating with senior engineers to develop and...

  • Data Engineering Specialist

    4 weeks ago

    Washington, Washington, D.C., United States Omni Inclusive Full time

    Job Title: Data Engineering SpecialistJob Description:We are seeking a skilled Data Engineering Specialist to join our team at Omni Inclusive. As a Data Engineering Specialist, you will be responsible for designing, building, and maintaining complex data pipelines using Agile methodology.Key Responsibilities:Design and implement data pipelines using Azure...

  • Data Integration Engineer

    4 weeks ago

    Washington, Washington, D.C., United States The National Endowment for Democracy Full time

    Job SummaryThe National Endowment for Democracy (NED) is seeking a highly skilled Data Integration Engineer to join its IT team in Washington, D.C. This role will contribute to the development of high-quality data integration jobs that enable data exchange among NED cloud systems and extract data from these systems and databases for analytics, visualization,...

  • Cloud Engineer

    4 weeks ago

    Washington, Washington, D.C., United States TRM Labs Full time

    Job SummaryTRM Labs is seeking a highly skilled Cloud Engineer - Forward Deployed to join our team. As a Cloud Engineer - Forward Deployed, you will work directly with our government customers to bridge our SaaS offering with federated capabilities and cloud architectures.Key ResponsibilitiesEngage with clients to implement and configure cloud...

  • Cloud Infrastructure Engineer

    4 weeks ago

    Washington, Washington, D.C., United States DCCA Full time

    Job Title: Cloud Infrastructure EngineerWe are seeking a highly skilled Cloud Infrastructure Engineer to join our team at DCCA. As a Cloud Infrastructure Engineer, you will be responsible for designing, implementing, and managing cloud-based infrastructure solutions for our clients.Key Responsibilities:Design and implement cloud-based infrastructure...

  • Senior Data Engineer

    1 month ago

    Washington, Washington, D.C., United States Occam Solutions Full time

    Job OverviewWe are seeking a highly skilled Senior Data Engineer to join our team at Occam Solutions. The ideal candidate will have expertise in designing and implementing large-scale data pipelines, as well as experience with cloud-based data platforms.Key Responsibilities:Design and implement data pipelines using AWS and KubernetesDevelop and maintain data...

  • Cloud Engineering and Integration Manager

    4 weeks ago

    Washington, Washington, D.C., United States Sev1Tech Full time

    Job Summary:Sev1Tech is seeking an experienced Engineering and Integration Manager to lead a team of technical resources supporting a federal Network Cyber and Cloud Support Services program. The ideal candidate will have a strong background in cloud engineering and integration, with a focus on developing and deploying centralized cloud management platforms...

  • Lead Data Engineer

    1 month ago

    Washington, Washington, D.C., United States Capital One Full time

    About the RoleWe are seeking a highly skilled Lead Data Engineer to join our team at Capital One. As a key member of our data engineering team, you will be responsible for designing, developing, and implementing large-scale data processing systems.Key Responsibilities:Collaborate with cross-functional teams to design and develop data pipelines and...

  • Cloud Infrastructure Engineer II

    1 month ago

    Washington, Washington, D.C., United States Amentum Full time

    Amentum Cloud Ops/Platform Engineer IIAmentum is a leading global provider of technical and engineering services to government and commercial customers. We deliver innovative solutions to complex challenges in the defense, intelligence, and civilian markets. With 50,000 employees worldwide, we leverage our extensive experience, cutting-edge technology, and...

  • Cloud Services Software Engineer

    4 weeks ago

    Washington, Washington, D.C., United States Amazon Web Services, Inc. Full time

    Job SummaryAmazon Web Services (AWS) is seeking a skilled software development engineer to design and build customer-facing cloud services. The Cloud Services Software Engineer will be responsible for creating new capabilities to enable secure data exchange and collaborating with AWS customers to understand their needs. This role requires US Citizenship and...

  • Salesforce Data Cloud Architect

    4 weeks ago

    Washington, Washington, D.C., United States National Association of Broadcasters Full time

    About UsThe National Association of Broadcasters is the voice for the nation's radio and television broadcasters. As the premier trade association for broadcasters, NAB advances the interests of our members in federal government, industry and public affairs; improves the quality and profitability of broadcasting; encourages content and technology innovation;...

  • Cloud Software Engineer

    4 weeks ago

    Washington, Washington, D.C., United States Huntington Ingalls Industries Full time

    Job DescriptionEnlighten, an HII Company, is seeking a talented Cloud Software Engineer to join our team. As a Cloud Software Engineer, you will be responsible for designing, developing, and implementing complex software capabilities for computer-based systems.You will work closely with our local team to understand business requirements, design software...

  • Senior Data Engineer

    4 weeks ago

    Washington, Washington, D.C., United States Working America Associa Full time

    Job SummaryWe are seeking a highly skilled Data Architect to join our team at Working America Associa. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining robust data pipelines and infrastructure to support our organization's data-driven initiatives.Key ResponsibilitiesTeam Management & Staff...