Sr. Cloud Data Engineer

1 month ago

washington, United States IT Concepts, Inc Full time

Must be US Citizen


Responsibilities:

  • Collaborate & contribute to the architecture, design, development, and maintenance of large-scale data & analytics platforms, system integrations, data pipelines, data models & API integrations.
  • Create transformation path for data to migrate from on-prem pipelines and sources to AWS.
  • Provide input and insights, in conjunction with Data Architects, to the client.
  • Coordinate with data engineers to provide feedback and organization to team.
  • Ensure that data are optimally standardized and analysis-ready.
  • Prototype emerging business use cases to validate technology approaches and propose potential solutions.
  • Collaborate and ensure data integrity after large scale migrations.
  • Deliver high quality data assets to be used by the business to transform business processes and to enable leaders to complete data-driven analyses.
  • Continuously improve data solutions to increase quality, speed of delivery and trust of data engineering team’s deliverables to enable business outcomes.
  • Reduce total cost of ownership of solutions by developing shared components and implementing best practices and coding standards.
  • Collaborate with team to re-platform and reengineer data pipelines from on-prem to AWS cloud.
  • Work together with team members to ensure data quality and integrity during migrations.
  • Lead by example and pitch in to enable successful and seamless client delivery.

Location: Preferred in DC area, but role is remote.

Requirements

  • 8+ years experience related to Data Engineering.
  • AWS Cloud Certificate
  • Minimum of 3 years of experience in the following:
  • Leading engineering teams, including task management and personnel management.
  • Working in or managing data centric teams in the government or other highly regulated environments.
  • Strong understanding of data lake, data lakehouse, and data warehousing architectures in a cloud-based environment.
  • Proficiency in Python for data manipulation, scripting, and automation.
  • In-depth knowledge of AWS services relevant to data engineering (e.g., S3, EC2, DMS,
  • DataSync, SageMaker, Glue, RDS, Lambda, Elasticsearch).
  • Understanding of data integration patterns and technologies.
  • Proficiency designing and building flexible and scalable ETL processes and datapipelines using Python and/or PySpark and SQL.
  • Proficiency in data pipeline automation and workflow management tools like Apache Airflow or AWS Step Functions.
  • Knowledge of data quality management and data governance principles.
  • Strong problem-solving and troubleshooting skills related to data management challenges.
  • Experience managing code in GitHub or other similar tools.
  • Minimum of 2 years of experience in the following:
  • Hands-on experience with Databricks including data ingestion, transformation, analysis and optimization.
  • Experience designing, deploying, securing, sustaining and maintaining applications and services in a cloud environment (e.g., AWS, Azure) using infrastructure as code (e.g., Terraform, CloudFormation, Boto3).
  • Experience with database administration, optimization, and data extraction.
  • Experience using containerization technology such as Kubernetes or Mesos.
  • Minimum of 1 year of experience in the following:
  • Hands-on experience migrating from an on-premise data platform(s) to a modern cloud environment (e.g. AWS, Azure, GCP).
  • Linux/RHEL server & bash/shell scripting experience in on-prem or cloud environment.

Preferred Experience:

  • Bachelor's Degree in related field.
  • Previous experience with large-scale data migrations and cloud-based data platform implementations.
  • Prior experience with Databricks Unity Metastore/Catalog.
  • Familiarity with advanced SQL techniques for performance optimization and data analysis.
  • Knowledge of data streaming and real-time data processing frameworks such as Spark Structured Streaming.
  • Experience with data lakes and big data technologies (e.g., Apache Spark, Citus).
  • Familiarity with serverless computing and event-driven architectures in AWS.
  • Certifications in AWS, Databricks, or related technologies.
  • Experience working in Agile or DevSecOps environments and using related tools for collaboration and version control.
  • Extensive knowledge of software and data engineering best practices.
  • Strong communication and collaboration skills with internal and external stakeholders.
  • Experience establishing, implementing and documenting best practices, standard operating procedures, etc.


Clearance requirements:

  • Must be able to obtain and maintain a Public Trust clearance.
  • Must be US Citizen.

  • Cloud Data Engineer

    3 weeks ago

    Washington, Washington, D.C., United States IT Concepts, Inc Full time

    Job Title: Sr. Cloud Data EngineerWe are seeking a highly skilled Sr. Cloud Data Engineer to join our team at IT Concepts, Inc. The ideal candidate will have a strong background in cloud-based data engineering, with experience in designing, developing, and maintaining large-scale data and analytics platforms.Responsibilities:Collaborate with cross-functional...

  • Cloud Data Engineer

    1 month ago

    Washington, United States IT Concepts, Inc Full time

    Job Title: Sr. Cloud Data EngineerIT Concepts, Inc is seeking a highly skilled Sr. Cloud Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data and analytics platforms, system integrations, data pipelines, data models, and API...

  • Cloud Data Engineer

    1 month ago

    Washington, Washington, D.C., United States IT Concepts, Inc Full time

    Job Title: Sr. Cloud Data EngineerIT Concepts, Inc is seeking a highly skilled Sr. Cloud Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data and analytics platforms, system integrations, data pipelines, data models, and API...

  • Cloud Data Engineer

    3 weeks ago

    Washington, Pennsylvania, United States IT Concepts, Inc Full time

    Job Title: Sr. Cloud Data EngineerAbout the Role:We are seeking a highly skilled Sr. Cloud Data Engineer to join our team at IT Concepts, Inc. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data and analytics platforms, system integrations, data pipelines, data models, and API...

  • Senior Data Engineer

    3 weeks ago

    Washington, Pennsylvania, United States Serigor Inc Full time

    Job Title: Sr Data Engineer Job Summary: We are seeking a highly experienced Sr Data Engineer to join our team at Serigor Inc. The ideal candidate will have a strong background in cloud computing, data engineering, and Adobe Analytics/AEP platform experience. Key Responsibilities: * Design and develop ETL solutions to meet complex use cases * Configure...

  • Data Quality Engineer

    3 weeks ago

    Washington, Washington, D.C., United States Inclusion Cloud Full time

    About the JobAt Inclusion Cloud, we're seeking a highly skilled Data Quality Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and implementing automated data quality checks to ensure high-quality data is available to our internal stakeholders and customers.Key ResponsibilitiesDesign...

  • Data Quality Engineer

    3 weeks ago

    Washington, Washington, D.C., United States Inclusion Cloud Full time

    About the role: Data Quality EngineerKey ResponsibilitiesDesign, develop, and document automated data quality checksEnsure high-quality data is available to internal stakeholders and customersCollaborate with Data Engineering and other teams for requirements gatheringCommunicate technical concepts to developers, product managers, and business partnersProvide...

  • Sr Talend Data Engineer

    3 weeks ago

    washington, United States Serigor Inc Full time

    Title: SrTalend Data EngineerLocation: Washington DC (Remote 100%)Duration: 12+ MonthsJob Description:Looking for a Sr level person – Min 10+ YearsPrefer people near the DC Area – So can be in the office when there is a needShould have good experience in Talend, SQL, AWSTech Stack - Must Have:Talend Expertise Proficiency with SQL.Experience using AWS...

  • Sr Talend Data Engineer

    1 month ago

    Washington, United States Serigor Inc Full time

    Title: SrTalend Data EngineerLocation: Washington DC (Remote 100%)Duration: 12+ MonthsJob Description:Looking for a Sr level person – Min 10+ YearsPrefer people near the DC Area – So can be in the office when there is a needShould have good experience in Talend, SQL, AWSTech Stack - Must Have:Talend Expertise Proficiency with SQL.Experience using AWS...

  • Data Engineer

    1 month ago

    Washington, Washington, D.C., United States Square Peg Technologies Full time

    Job Title: Data Engineer - Cloud Data PipelinesAt Square Peg Technologies, we're seeking a skilled Data Engineer to join our team and help us design, implement, and maintain data pipelines for our customers. As a Data Engineer, you'll be responsible for developing automated pipelines for various data types and datasets using Databricks, a leading cloud-based...

  • Cloud Data Engineer

    2 weeks ago

    Washington, Washington, D.C., United States TalentRemedy Full time

    About the RoleThe Cloud Data Engineer will play a crucial part in designing, building, and maintaining data infrastructure and ETL pipelines using Databricks and cloud-based technologies. This role will enable data-driven decision-making and ensure data accuracy and accessibility for regulatory purposes.Key ResponsibilitiesCollaborate with cross-functional...

  • Data Engineer

    2 weeks ago

    Washington, Washington, D.C., United States Futran Tech Solutions Pvt. Ltd. Full time

    Job Title: Data Engineer - Cloud Data ArchitectAt Futran Tech Solutions Pvt. Ltd., we are seeking a highly skilled Data Engineer - Cloud Data Architect to join our team.Key Responsibilities:* Develop and maintain data pipelines using AWS Glue and Python.* Design and implement ETL/ELT processes.* Work with AWS Postgres to ensure data consistency and...

  • Sr Data Engineer

    1 month ago

    washington, United States Serigor Inc Full time

    Title: Sr Data EngineerLocation: 100% Remote – EST Time zoneDuration: 12+ MonthsKey Skills: AWS, Spark, Adobe Analytics/AEP(Adobe Experience Platform) platform experience, Glue, Lamda, Python, Scala, EMR, Talend, PostgreSQL, RedshiftJob Description:Looking for a Sr level person – Min 10+ YearsLook for a Sr Data Engineer with Adobe Analytics or AEP(Adobe...

  • Sr Data Engineer

    3 weeks ago

    washington, United States Serigor Inc Full time

    Title: Sr Data EngineerLocation: 100% Remote – EST Time zoneDuration: 12+ MonthsKey Skills: AWS, Spark, Adobe Analytics/AEP(Adobe Experience Platform) platform experience, Glue, Lamda, Python, Scala, EMR, Talend, PostgreSQL, RedshiftJob Description:Looking for a Sr level person – Min 10+ YearsLook for a Sr Data Engineer with Adobe Analytics or AEP(Adobe...

  • Sr Data Engineer

    1 month ago

    Washington, United States Serigor Inc Full time

    Title: Sr Data EngineerLocation: 100% Remote – EST Time zoneDuration: 12+ MonthsKey Skills: AWS, Spark, Adobe Analytics/AEP(Adobe Experience Platform) platform experience, Glue, Lamda, Python, Scala, EMR, Talend, PostgreSQL, RedshiftJob Description:Looking for a Sr level person – Min 10+ YearsLook for a Sr Data Engineer with Adobe Analytics or AEP(Adobe...

  • Sr Data Engineer

    1 month ago

    Washington, United States Serigor Inc Full time

    Title: Sr Data EngineerLocation: 100% Remote – EST Time zoneDuration: 12+ MonthsKey Skills: AWS, Spark, Adobe Analytics/AEP(Adobe Experience Platform) platform experience, Glue, Lamda, Python, Scala, EMR, Talend, PostgreSQL, RedshiftJob Description:Looking for a Sr level person – Min 10+ YearsLook for a Sr Data Engineer with Adobe Analytics or AEP(Adobe...

  • Sr. Cloud Data Engineer

    1 month ago

    Washington, United States IT Concepts, Inc Full time

    Must be US CitizenResponsibilities:Collaborate & contribute to the architecture, design, development, and maintenance of large-scale data & analytics platforms, system integrations, data pipelines, data models & API integrations.Create transformation path for data to migrate from on-prem pipelines and sources to AWS.Provide input and insights, in conjunction...

  • Sr. Cloud Data Engineer

    1 month ago

    Washington, United States IT Concepts, Inc Full time

    Must be US CitizenResponsibilities:Collaborate & contribute to the architecture, design, development, and maintenance of large-scale data & analytics platforms, system integrations, data pipelines, data models & API integrations.Create transformation path for data to migrate from on-prem pipelines and sources to AWS.Provide input and insights, in conjunction...

  • Sr. Cloud Data Engineer

    4 weeks ago

    Washington, United States IT Concepts, Inc Full time

    Must be US CitizenResponsibilities:Collaborate & contribute to the architecture, design, development, and maintenance of large-scale data & analytics platforms, system integrations, data pipelines, data models & API integrations.Create transformation path for data to migrate from on-prem pipelines and sources to AWS.Provide input and insights, in conjunction...

  • Python Sr Developer

    6 days ago

    Washington, United States Inclusion Cloud Full time

    About the job Python Sr Developer FinTech Settlement ProjectPython Software Engineer Job Description Job Description Summary We are a growing team of highly talented engineers working on a next-generation settlement process for our FinTech division built upon cloud services, microservices architecture, and immutable transactions. Primary Accountabilities:...