Brooksource | AWS Data Engineer

3 weeks ago

charlotte, United States Brooksource Full time

Brooksource is searching for an AWS Data Engineer with expertise in data warehousing using AWS Redshift and Kafka to join our Fortune 500 client in Charlotte, NC.


AWS Data Engineer IV Core Technical Skills

  • 5+ years of AWS experience
  • AWS services - S3, EMR, Glue Jobs, Lambda, Athena, CloudTrail, SNS, SQS, CloudWatch, Step Functions, QuickSight
  • Experience with Kafka/Messaging preferably Confluent Kafka
  • Experience with EMR databases such as Glue Catalog, Lake Formation, Redshift, DynamoDB and Aurora
  • Experience with AWS data warehousing tools such as Amazon Redshift and Amazon Athena
  • Proven track record in the design and implementation of data warehouse solutions using AWS
  • Skilled in data modeling and executing ETL processes tailored for data warehousing
  • Competence in developing and refining data pipelines within AWS
  • Proficient in handling both real-time and batch data processing tasks
  • Extensive understanding of database management fundamentals
  • Expertise in creating alerts and automated solutions for handling production problems
  • Tools and Languages – Python, Spark, PySpark and Pandas
  • Infrastructure as Code technology – Terraform/CloudFormation
  • Experience with Secrets Management Platform like Vault and AWS Secrets manager
  • Experience with Event Driven Architecture
  • DevOps pipeline (CI/CD); Bitbucket; Concourse
  • Experience with RDBMS platforms and Strong proficiency with SQL
  • Experience with Rest APIs and API gateway
  • Deep knowledge of IAM roles and Policies
  • Experience using AWS monitoring services like CloudWatch, CloudTrail ad CloudWatch events
  • Deep understanding of networking DNS, TCP/IP and VPN
  • Experience with AWS workflow orchestration tool like Airflow or Step Functions

AWS Data Engineer IV Additional Technical Skills (nice to have, but not required for the role)

  • Experience with native AWS technologies for data and analytics such as Kinesis, OpenSearch
  • Databases - Document DB, Mongo DB
  • Hadoop platform (Hive; HBase; Druid)
  • Java, Scala, Node JS
  • Workflow Automation
  • Experience transitioning on premise big data platforms into cloud-based platforms such as AWS
  • Strong Background in Kubernetes, Distributed Systems, Microservice architecture and containers

Core Responsibilities

  • Provides technical direction, guides the team on key technical aspects and responsible for product tech delivery
  • Lead the Design, Build, Test and Deployment of components
  • Where applicable in collaboration with Lead Developers (Data Engineer, Software Engineer, Data Scientist, Technical Test Lead)
  • Understand requirements / use case to outline technical scope and lead delivery of technical solution
  • Confirm required developers and skillsets specific to product
  • Provides leadership, direction, peer review and accountability to developers on the product (key responsibility)
  • Works closely with the Product Owner to align on delivery goals and timing
  • Assists Product Owner with prioritizing and managing team backlog
  • Collaborates with Data and Solution architects on key technical decisions
  • The architecture and design to deliver the requirements and functionality
  • Skilled in developing data pipelines, focusing on long-term reliability and maintaining high data quality
  • Designs data warehousing solutions with the end-user in mind, ensuring ease of use without compromising on performance
  • Manage and resolve issues in production data warehouse environments on AWS

Core Experience and Abilities

  • Ability to perform hands on development and peer review for certain components / tech stack on the product
  • Standing up of development instances and migration path (with required security, access/roles)
  • Develop components and related processes (e.g. data pipelines and associated ETL processes, workflows)
  • Lead implementation of integrated data quality framework
  • Ensures optimal framework design and load testing scope to optimize performance (specifically for Big Data)
  • Supports data scientist with test and validation of models
  • Performs impact analysis and identifies risk to design changes
  • Ability to build new data pipelines, identify existing data gaps and provide automated solutions to deliver analytical capabilities and enriched data to applications
  • Ability to implement data pipelines with the right attentiveness to durability and data quality
  • Implements data warehousing products thinking of the end users experience (ease of use with right performance)
  • Ensures Test Driven development
  • 5+ years of Experience leading teams to deliver complex products
  • Strong technical skills and communication skills
  • Strong skills with business stakeholder interactions
  • Strong solutioning and architecture skills
  • 5+ years of Experience building real time data ingestion streams (event driven)
  • Ensure data security and permissions solutions, including data encryption, user access controls and logging

  • AWS Data Engineer

    3 weeks ago

    Charlotte, United States Brooksource Full time

    Brooksource is searching for an AWS Data Engineer with expertise in data warehousing using AWS Redshift and Kafka to join our Fortune 500 client in Charlotte, NC. AWS Data Engineer IV Core Technical Skills 5+ years of AWS experience AWS services - S3, EMR, Glue Jobs, Lambda, Athena, CloudTrail, SNS, SQS, CloudWatch, Step Functions, QuickSight Experience with...

  • AWS Data Engineer

    2 weeks ago

    Charlotte, United States Brooksource Full time

    Brooksource is searching for an AWS Data Engineer with expertise in data warehousing using AWS Redshift and Kafka to join our Fortune 500 client in Charlotte, NC. AWS Data Engineer IV Core Technical Skills 5+ years of AWS experience AWS services - S3, EMR, Glue Jobs, Lambda, Athena, CloudTrail, SNS, SQS, CloudWatch, Step Functions, QuickSight Experience with...

  • Brooksource | Data Analyst III

    4 weeks ago

    charlotte, United States Brooksource Full time

    Brooksource is looking for a Data Analyst III for our Fortune 100 Telecommunications client in the Charlotte, NC area. The Data Analyst will join the Next Best Action (NBA) team within the Digital Transformation group. This team designs solutions which directly solve customer service issues and positively impact the customer experience through data driven...

  • Junior Site Reliability Engineer

    2 weeks ago

    Charlotte, United States Brooksource Full time

    A leading digital bank and Fortune 500 Financial Institution is seeking Site Reliability Engineers to join the operations team responsible for driving operational standards and features on business-critical applications across the enterprise. You will get the opportunity to collaborate with technical and business leaders across the organization while making...

  • Junior Site Reliability Engineer

    2 weeks ago

    Charlotte, United States Brooksource Full time

    A leading digital bank and Fortune 500 Financial Institution is seeking Site Reliability Engineers to join the operations team responsible for driving operational standards and features on business-critical applications across the enterprise. You will get the opportunity to collaborate with technical and business leaders across the organization while making...

  • AWS Data Engineer Lead

    4 days ago

    Charlotte, North Carolina, United States Chello Full time

    Lead AWS Data Engineer PositionChello is seeking an experienced Lead AWS Data Engineer to lead our database engineering efforts. This is a senior-level position that requires a high level of expertise in designing and implementing large-scale database architectures on AWS.The successful candidate will have a strong background in database engineering and...

  • Data Engineer

    2 weeks ago

    Charlotte, United States Pantheon Data Full time

    Job DescriptionJob DescriptionCompany OverviewPantheon Data (a Kenific Holding company) is a private, small business based in the Washington, DC, area. Pantheon Data was founded in 2011, initially providing acquisition and supply chain management services to the US Coast Guard. Our service offerings have grown in the past ten years, including infrastructure...

  • Data Engineer

    5 days ago

    Charlotte, United States Pantheon Data Full time

    Job DescriptionJob DescriptionCompany OverviewPantheon Data (a Kenific Holding company) is a private, small business based in the Washington, DC, area. Pantheon Data was founded in 2011, initially providing acquisition and supply chain management services to the US Coast Guard. Our service offerings have grown in the past ten years, including infrastructure...

  • Big Data Engineer with AWS Experience

    4 weeks ago

    Charlotte, North Carolina, United States Global Channel Management Full time

    About the RoleWe are seeking an experienced Data Engineer to join our team and lead our efforts in developing and maintaining our big data platform.Job Summary:This is an exciting opportunity to leverage your skills and experience in Big Data technologies to drive business growth and innovation.Key Responsibilities:Design, develop, and maintain high-quality...

  • Data Analyst III

    4 weeks ago

    Charlotte, United States Brooksource Full time

    Brooksource is looking for a Data Analyst III for our Fortune 100 Telecommunications client in the Charlotte, NC area. The Data Analyst will join the Next Best Action (NBA) team within the Digital Transformation group. This team designs solutions which directly solve customer service issues and positively impact the customer experience through data driven...

  • Data Analyst III

    2 weeks ago

    Charlotte, United States Brooksource Full time

    Brooksource is looking for a Data Analyst III for our Fortune 100 Telecommunications client in the Charlotte, NC area. The Data Analyst will join the Next Best Action (NBA) team within the Digital Transformation group. This team designs solutions which directly solve customer service issues and positively impact the customer experience through data driven...

  • Data Engineer

    2 weeks ago

    Charlotte, United States Pantheon Data Full time

    Job DescriptionJob DescriptionCompany OverviewPantheon Data (a Kenific Holding company) is a private, small business based in the Washington, DC, area. Pantheon Data was founded in 2011, initially providing acquisition and supply chain management services to the US Coast Guard. Our service offerings have grown in the past ten years, including infrastructure...

  • Senior Data Engineer

    5 days ago

    Charlotte, United States Kellton Full time

    Position: Sr. Data EngineerWork Location: Remote (working CST or EST) and able to travel quarterly for PI planning Duration: 6 Month (Contract-to-Hire)Manager Notes:Sr. Data EngineerExperience on several DE projects, hands onStrong snowflake expertise - administrations Advanced SQL and Python skills Extracting data from APIs ETL processes, data ingestion AWS...

  • Senior Data Engineer

    4 days ago

    Charlotte, United States Kellton Full time

    Position: Sr. Data EngineerWork Location: Remote (working CST or EST) and able to travel quarterly for PI planning Duration: 6 Month (Contract-to-Hire)Manager Notes:Sr. Data EngineerExperience on several DE projects, hands onStrong snowflake expertise - administrations Advanced SQL and Python skills Extracting data from APIs ETL processes, data ingestion AWS...

  • AWS Cloud Engineer

    2 weeks ago

    Charlotte, United States eTek IT Services, Inc. Full time

    Job DescriptionJob DescriptionTitle – AWS Snowflake Data EngineerDuration – 6+ months, likely to extend& & Top Requirements:• 7+ years of experience in data engineering, data warehousing, or related roles.• Extensive hands-on experience with Snowflake implementations and optimizations. Experience with real-time data processing, particularly involving...

  • Data Solutions Architect

    4 weeks ago

    Charlotte, North Carolina, United States Pantheon Data Full time

    Job Overview">We are seeking a highly skilled Data Engineer to join our growing engineering team at Pantheon Data. The ideal candidate will have hands-on experience with data integration and/or data migration, as well as proficiency in data warehousing ETL pipelines.This role will play a crucial part in supporting our 'cloud-first' migration for various...

  • Cloud Engineer

    1 month ago

    Charlotte, United States Chello Full time

    Chello is a new fintech business venture within OFG Bancorp, built on a foundation of pioneering data-intelligence and other innovations that enable business in the targeted industries to tackle their financial challenges. Chello is a fast-paced environment with aggressive growth plans and a unique culture that emphasizes progress and growth, promoting a...

  • Technical Data Specialist

    4 weeks ago

    Charlotte, North Carolina, United States Pantheon Data Full time

    As a Data Engineer at Pantheon Data, you will play a key role in supporting our cloud-first migration strategy by designing and developing technical solutions for our customers. This role requires strong hands-on experience with data integration and/or data migration, as well as proficiency in data warehousing ETL pipelines. In addition to working...

  • Software Engineer

    2 weeks ago

    Charlotte, United States Pantheon Data Full time

    Job DescriptionJob DescriptionCompany OverviewPantheon Data (a Kenific Holding company) is a private, small business based in the Washington, DC, area. Pantheon Data was founded in 2011, initially providing acquisition and supply chain management services to the US Coast Guard. Our service offerings have grown in the past ten years, including infrastructure...

  • Data Engineer

    1 month ago

    Charlotte, United States Remobi Full time

    Senior Data EngineerLocation: Charlotte, North CarolinaStart Date: ASAPContract Type: W2Duration: Until September 2025 (with potential for extension)About RemobiAt Remobi, we’re building the world’s greatest community of remote technologists! We believe that organizations embracing remote work will gain unparalleled access to the brightest minds globally...