Cloud Engineer

NOTE:

Professional Service Requirements

AWS Lakehouse Certification

AWS Design and Implementation Lake House Technologies

AWS IAM o AWS Lakehouse Technology

AWS Glue Knowledge

Data Catalog and Crawler

AWS Redshift

AWS Athena

CJIS certified data storage

AWS CJIS Data location

CJIS Security Background Check (See Additional Attachments / Assurances)

JOB ENTAILS:

Design and Implementation of the Data Lake:

Ensure compliance with CJIS standards for storing law enforcement data.

Collaborate with the Agency Technical Lead to design and implement the lake house technologies according to AWS standards.

Utilize AWS Lake House standards for the design and implementation process.

Data Security Layer:

Implement Identity and Access Management (IAM) roles and processes for security.

Design and create IAM templates to manage access control effectively.

Data Ingestion Layer:

Design ingestion processes for various data sources including operational databases, SaaS applications, file shares, and stream data sources.

Develop templates for ingestion processes to ensure consistency and scalability.

Data Storage Layer:

Store data in CJIS compliant S3 buckets.

Utilize AWS Redshift infrastructure for data storage.

Implement Power BI Connector process for reporting and analytics purposes.

Data Processing Layer:

Develop Extract, Load, and Transform (ELT) processes for loading data from sources to S3 and transforming it to Redshift for reporting and analytics.

Create templates for ELT processes to facilitate future data processing needs.

Data Catalog Layer:

Design and implement a solution using AWS Glue to manage data schema drift.

Build crawlers to automatically detect schema changes and update the catalog accordingly.

Store metadata in the catalog for consumption in the data warehouse.

Scalability and Future Expansion:

Design each layer with scalability in mind to accommodate increasing usage.

Create templates and guidelines for expanding the data lake for future agencies.

Implement a base data warehouse environment for reporting and dashboard use.

Ensure compatibility with Power BI for data visualization and analysis.

Professional Service Requirements:

Ensure the AWS service provider possesses relevant certifications and expertise in AWS Lakehouse technologies, IAM, AWS Glue, Redshift, Athena, and CJIS certified data storage.

Compliance with CJIS security background checks and certifications for handling sensitive law enforcement data.