AWS Data Lake Technical Lead

3 weeks ago

Atlanta, United States My3Tech Full time

Job Title: AWS Data Lake Technical Lead

Job Location: REMOTE with Vocational Onsite Visits at Waterbury, VT (Travel Expenses will be reimbursed)

Duration: 3+ Years Contract


Job Description:

Background:

Client is seeking to improve statewide law enforcement data access. The purpose is to design and implement a state-controlled system to access de-identified, aggregated law enforcement and related data currently housed in a record management system.

Client in collaboration with the Agency of Digital (ADS) Services, is seeking to procure Amazon Web Service (AWS) professional services to work with the ADS Technical Lead to build out the Public Services Lakehouse environment.


Existing Technology Environment

Client currently uses a Computer Aided Dispatch Records Management System (CAD RMS) system that is currently running in a MySQL instance that will be used in the first phase of the Data Lake build. The supplemental data ranges from SQL server instances and flat file sources that are currently housed internally either in SharePoint or internal file stores.


Requirements:

Design and implement the Clients’ Data Lake in the AWS Environment

  • Store Data in AWS CJIS compliant environment.
  • Implement and design the lake house technologies with the IT Tech Lead
  • assigned to the project.
  • Design and implement using latest AWS Lake House standards.
  • Data Security Layer
  • Implement and design security Identity and Access Management roles and processes.
  • Design and Create security IAM templates.
  • Data Ingestion Layer
  • Ingestion Design for variety of sources
  • Operational Database Sources
  • MySQL, SQL Server
  • SaaS Applications
  • File Shares (SharePoint and one drive)
  • Stream Data Sources
  • System Templates of ingestion processes
  • Data Storage Layer
  • AWS S3 CJIS compliant S3 buckets
  • AWS Redshift Infrastructure
  • Power BI Connector process
  • Data Processing Layer
  • Data Extract, Load and Transform (ELT) process for loading from source to S3 and Transforming data from S3 to Redshift for reporting and analytics.
  • Templates for creating ELT processes for future processes.
  • Data Catalog Layer
  • Design and implement solution to solve data schema drift in AWS Glue for use with reporting and analytical needs.
  • Design and build crawlers for schema and build catalog that stores schema information.
  • Meta data store in catalog for consumption in data warehouse
  • Create Catalog crawler templates for data sources.
  • Create Templates for expansion of the data lake for future agencies.
  • Base data warehouse implemented environment for reporting/dashboard use.
  • Template designs for future data Lakehouse implementations.
  • Lake House design needs to be able to be used by Power BI
  • Design each layer for scaling based on usage.


Professional Service Requirements:

  • AWS Lakehouse Certification
  • AWS Design and Implementation Lake House Technologies
  • AWS IAM
  • AWS Lakehouse Technology
  • AWS Glue Knowledge
  • Data Catalog and Crawler
  • AWS Redshift
  • AWS Athena
  • CJIS certified data storage
  • AWS CJIS Data location
  • CJIS Security Background Check (See Additional Attachments / Assurances)

  • AWS Data Engineer

    1 month ago

    Atlanta, United States Mueller Co. LLC Full time

    Job Summary : AWS-BI Developer with strong experience across the entire Cloud Data stack. The ideal candidate will have extensive experience in data pipelines (ELT/ETL), data warehousing and dimensional modeling, and curation of data sets for Data Scientists and Business Intelligence users. Experience in building Data Lake. This candidate will also have...

  • AWS Data Application Infrastructure Lead

    3 days ago

    Atlanta, United States ITTConnect Full time

    ITTConnect is seeking an AWS Data Application Infrastructure Lead to fill a permanent role with a client in the Atlanta area. This is a position with a client that is a global leader in consulting, digital transformation, technology and engineering services present in nearly 50 countries. The end client is in the telecommunications space (internet, cable TV,...

  • AWS Data Application Infrastructure Lead

    1 week ago

    Atlanta, United States ITTConnect Full time

    ITTConnect is seeking an AWS Data Application Infrastructure Lead to fill a permanent role with a client in the Atlanta area. This is a position with a client that is a global leader in consulting, digital transformation, technology and engineering services present in nearly 50 countries. The end client is in the telecommunications space (internet, cable TV,...

  • AWS Data Application Infrastructure Lead

    2 weeks ago

    Atlanta, United States ITTConnect Full time

    Job DescriptionJob DescriptionITTConnect is seeking an AWS Data Application Infrastructure Lead to fill a permanent role with a client in the Atlanta area. This is a position with a client that is a global leader in consulting, digital transformation, technology and engineering services present in nearly 50 countries. The end client is in the...

  • AWS Data Application Infrastructure Lead

    6 days ago

    Atlanta, United States ITTConnect Full time

    Job DescriptionJob DescriptionITTConnect is seeking an AWS Data Application Infrastructure Lead to fill a permanent role with a client in the Atlanta area. This is a position with a client that is a global leader in consulting, digital transformation, technology and engineering services present in nearly 50 countries. The end client is in the...

  • AWS Data Application Monitoring Lead

    2 weeks ago

    Atlanta, United States ITTConnect Full time

    Job DescriptionJob DescriptionITTConnect is seeking an AWS Data Application Monitoring Lead to fill a permanent role with a client in the Atlanta area. This is a position with a client that is a global leader in consulting, digital transformation, technology and engineering services present in nearly 50 countries. The end client is in the telecommunications...

  • AWS Data Application Infrastructure Lead

    2 weeks ago

    Atlanta, United States Atika Tech Full time

    Position: AWS Data Application Infrastructure Lead Location: Atlanta, GA (On- site Day One) JD: Manage AWS Infrastructure for provisioning Manage and Streamline Deployments of Pipelines to Production Manage FinOps for Productionalized applications POC and recommendations for product usage and processes - Must Have Experience in Big 4 Consulting...

  • AWS Data Application Infrastructure Lead

    2 weeks ago

    Atlanta, United States Atika Technologies Full time

    Position: WS Data Application Infrastructure Lead Location: Atlanta, GA (On- site Day One) JD: Manage AWS Infrastructure for provisioning Manage and Streamline Deployments of Pipelines to Production Manage FinOps for Productionalized applications POC and recommendations for product usage and processes - Must Have Experience in Big 4 Consulting Companies -...

  • AWS DevOps Lead

    2 weeks ago

    Atlanta, United States ITTConnect Full time

    Job Description ITTConnect is seeking an AWS DevOps Lead to fill a permanent role with a client in the Atlanta area. This is a position with a client that is a global leader in consulting, digital transformation, technology and engineering services present in nearly 50 countries. The end client is in the telecommunications space (internet, cable TV, cellular...

  • AWS DevOps Lead

    2 weeks ago

    Atlanta, United States ITTConnect Full time

    Job DescriptionJob DescriptionITTConnect is seeking an AWS DevOps Lead to fill a permanent role with a client in the Atlanta area. This is a position with a client that is a global leader in consulting, digital transformation, technology and engineering services present in nearly 50 countries. The end client is in the telecommunications space (internet,...

  • AWS DevOps Lead

    6 hours ago

    Atlanta, United States ITTConnect Full time

    Job DescriptionJob DescriptionITTConnect is seeking an AWS DevOps Lead to fill a permanent role with a client in the Atlanta area. This is a position with a client that is a global leader in consulting, digital transformation, technology and engineering services present in nearly 50 countries. The end client is in the telecommunications space (internet,...

  • Lead Data Engineer

    2 weeks ago

    Atlanta, United States Motion Recruitment Full time

    The Lead Data Engineer position is part of the Marketing Technology team focused on enabling the development and roll-out of world-class modern Marketing Data Engineering capabilities at the Customer. This role partners with teams responsible for Marketing and Digital Guest Engagement platforms that leverage world-class technology to create personalized,...

  • Data Architect

    2 days ago

    Atlanta, United States Cypress HCM Full time

    Job DescriptionJob DescriptionData Architect - AWSLocation: Atlanta, GA 30342Remote: Hybrid remote, 3 days in the office weekly Employees: 5,000 | Team Size: 80Industry: SaaS, FinTech, Payments | Financial Information: $2B+ in revenue for 2023Reports To: Director of Data Engineering Responsibilities:Architecting scalable, reliable, high-performing data...

  • Lead Data Engineer

    6 days ago

    Atlanta, United States CapTech Consulting Full time

    Job DescriptionJob DescriptionCompany DescriptionCapTech is an award-winning consulting firm that collaborates with clients to achieve what’s possible through the power of technology. At CapTech, we’re passionate about the work we do and the results we achieve for our clients. From the outset, our founders shared a collective passion to create a...

  • Data Architect

    2 weeks ago

    Atlanta, United States Warner Media, LLC Full time

    Every great story has a new beginning, and yours starts here. Welcome to Warner Bros. Discovery… the stuff dreams are made of. Who We Are… When we say, “the stuff dreams are made of,” we’re not just referring to the world of wizards, dragons and superheroes, or even to the wonders of Planet Earth. Behind WBD’s vast portfolio of iconic content and...

  • Data Architect

    2 weeks ago

    Atlanta, United States Warner Bros. Discovery Full time

    Welcome to Warner Bros. Discovery... the stuff dreams are made of. Who We Are... When we say, "the stuff dreams are made of," we're not just referring to the world of wizards, dragons and superheroes, or even to the wonders of Planet Earth. Behind WBD's vast portfolio of iconic content and beloved brands, are the storytellers bringing our characters to...

  • Sr AWS Data Architect

    2 weeks ago

    Atlanta, United States ITTConnect Full time

    ITTConnect is seeking a Senior Data Architect / Lead to work for a client in the Atlanta area. This is a position with a client that is a global leader in consulting, digital transformation, technology and engineering services present in nearly 50 countries. The end client is in the telecommunications space (internet, cable TV, cellular provider). Job...

  • Data Engineer

    2 months ago

    Atlanta, United States Boston Data Pro, LLC Full time

    We are looking for a Data Engineer to join our engineering team and help us develop and maintain various software products.The main responsibilities include writing and testing code debugging programs and integrating applications and transmission of the data from on-prem to AWS cloud.OPT CPT OK but must not be fresher.Key Skills Python Snowflake...

  • Data Engineer

    1 month ago

    Atlanta, Georgia, United States Boston Data Pro, LLC Full time

    We are looking for a Data Engineer to join our engineering team and help us develop and maintain various software products.The main responsibilities include writing and testing code debugging programs and integrating applications and transmission of the data from on-prem to AWS cloud.OPT CPT OK but must not be fresher.Key Skills Python Snowflake...

  • AWS Data Scientist

    1 week ago

    Atlanta, United States ITTConnect Full time

    Job DescriptionJob DescriptionITTConnect is seeking an AWS Data Scientist to fill a permanent role with a client in the Atlanta area. This is a position with a client that is a global leader in consulting, digital transformation, technology and engineering services present in nearly 50 countries. The end client is in the telecommunications space (internet,...