Senior AWS Redshift Data Engineer

3 weeks ago

Madison, United States Novalink Solutions LLC Full time
Job DescriptionJob DescriptionThe Universities of Wisconsin Enterprise Analytics Platform (EAP) team supports the development and support
of the data lake for the UW System Office and member universities.
Job Overview:
We are seeking an experienced AWS Redshift Data Engineer to assist our team in designing, developing, and
optimizing data pipelines for our AWS Redshift-based data lake house.
Priority needs are cloud formation and event-based data processing utilizing SQS to support ingestion and
movement of data from Workday to AWS Redshift for consumer and analytic use.
Collaborate with data engineering, business analysts, and development teams to design, develop, test,
and maintain robust and scalable data pipelines from Workday to AWS Redshift.
Architect, implement, and manage end-to-end data pipelines, ensuring data accuracy, reliability, data
quality, performance, and timeliness.
Provide expertise in Redshift database optimization, performance tuning, and query optimization.
Assist with design and implementation of workflows using Airflow.
Perform data profiling and analysis to troubleshoot data-related challenges / issues and build solutions
to address those concerns.
Proactively identify opportunities to automate tasks and develop reusable frameworks.
Work closely with version control team to maintain a well-organized and documented repository of
codes, scripts, and configurations using Git/Bitbucket.
Provide technical guidance and mentorship to fellow developers, sharing insights into best practices,
tips, and techniques for optimizing Redshift-based data solutions.
Advanced hands-on experience designing AWS data lake solutions.
Experience integrating Redshift with other AWS services, such as DMS, Glue, Lambda, S3, Athena,
Airflow.
Proficiency in Python programming with a focus on developing efficient Airflow DAGs and operators.
Experience with Pyspark and Glue ETL scripting including functions like relationalize, performing joins
and transforming dataframes with pyspark code.
Competency developing CloudFormation templates to deploy AWS infrastructure, including YAML
defined IAM policies and roles.
Experience with Airflow DAG creation.
Familiarity with debugging serverless applications using AWS tooling like Cloudwatch Logs & Log
Insights, Cloudtrail, IAM.
Ability to work in a highly complex python object oriented platform.
Strong understanding of ETL best practices, data integration, data modeling, and data transformation.
Proficiency in identifying and resolving performance bottleneck and fine-tuning Redshift queries.
Familiarity with version control systems, particularly Git, for maintaining a structured code repository.
Strong coding and problem-solving skills, and attention to detail in data quality and accuracy.
Ability to work collaboratively in a fast-paced, agile environment and effectively communicate technical
concepts to non-technical stakeholders.
Additional Useful Experience:
Airflow Server Administration
Parquet file formats
AWS Security
Jupyter Notebooks
API Best Practices, API Gateway, Route Structuring and standard API authentication protocols
including tokens
Git, Git flow best practices
Release management and DevOps
Shell scripting
AWS certifications related to data engineering or databases are a plus.
Experience with DevOps technologies and processes.
Experience with complex ETL scenarios, such as CDC and SCD logics, and integrating data from
multiple source systems.
Experience in converting Oracle scripts and Stored Procedures to Redshift equivalents.
Experience working with large-scale, high-volume data environments.
Exposure to higher education, finance, and/or human resources data is a plus.
Proficiency in SQL programming and Redshift stored procedures for efficient data manipulation and
transformation.

  • Senior AWS Redshift Data Engineer

    3 weeks ago

    Madison, United States Novalink Solutions LLC Full time

    Job DescriptionThe Universities of Wisconsin Enterprise Analytics Platform (EAP) team supports the development and support of the data lake for the UW System Office and member universities.Job Overview:We are seeking an experienced AWS Redshift Data Engineer to assist our team in designing, developing, andoptimizing data pipelines for our AWS Redshift-based...

  • AWS Redshift Data Engineer

    2 weeks ago

    Madison, United States RIT Solutions, Inc. Full time

    Position: Certified AWS Redshift Data Engineer *MUST HAVE ALL TECHNOLOGIES Contract: long term Level: Expert level Location: onsite in WI We are hiring an AWS Redshift Data Engineer to join our team in Wisconsin! This is an expert level position so we expect someone to come in and hit the ground running and be the expert for the team! Responsibilities:...

  • AWS Data Engineer

    2 weeks ago

    Madison, United States Element6Talent Full time

    Element6's client is seeking an experienced AWS Redshift Data Engineer to assist their team in designing, developing, and optimizing data pipelines for their AWS Redshift-based data lake house. This is a remote role, but candidates MUST be Wisconsin residents. Priority needs are cloud formation and event-based data processing utilizing SQS to support...

  • Certified Cloud Engineer

    3 weeks ago

    Madison, United States SunPlus Data Group Inc Full time

    Job DescriptionJob Description131765-WI-UW-AWSEngineerPAY RATE: 1099 / CTC: $159/hr ; W2 Hourly: $135/hr ; W2 Salary: $231K (With Benefits)Onsite or Remote? Candidates must be current Wisconsin residents or willing to relocate at the candidates expense. This is 100% remote (within Wisconsin)The Universities of Wisconsin Enterprise Analytics Platform (EAP)...

  • W2 - Certified Cloud Engineer (AWS data lake solutions, Redshift, Pyspark, Glue ETL scripting) - Rem

    3 weeks ago

    Madison, United States Tanson Corp Full time

    Job DescriptionJob DescriptionDescription:The client is looking for a Certified Cloud EngineerWill close to submissions on 6/21 @ 4:00PM CST.Top Skills (3) & Years of Experience: 3 years advanced hands-on experience designing AWS data lake solutions, integrating Redshift with other AWS services, such as DMS, Glue, Lambda, S3, Athena, Airflow, experience with...

  • AWS Cloud Engineer

    3 days ago

    Madison, United States Beacon Technologies Full time

    Beacon Technologies is seeking an AWS Cloud Engineer for our client partner. This is a remote position with Wisconsin residency required. The experienced AWS Redshift Data Engineer will assist the team in designing, developing, and optimizing data pipelines for the AWS Redshift-based data lake house. Priority needs are cloud formation and event-based data...

  • AWS Cloud Engineer

    2 weeks ago

    Madison, United States Element6Talent Full time

    Element6's client is seeking an experienced AWS Redshift Data Engineer to assist their team in designing, developing, and optimizing data pipelines for their AWS Redshift-based data lake house. This is a remote role, but candidates MUST be Wisconsin residents. Priority needs are cloud formation and event-based data processing utilizing SQS to support...

  • AWS Engineer

    2 weeks ago

    Madison, United States KYYBA Full time

    Job DescriptionJob DescriptionPosition Title: AWS EngineerLocation: Madison, WI - 53703Remote roleThis position is currently remote (however candidates must reside in Wisconsin for the length of the contract)The Universities of Wisconsin Enterprise Analytics Platform (EAP) team supports the development and support of the data lake for the UW System Office...

  • Certified Cloud Engineer AWS

    2 weeks ago

    Madison, United States Nava Software Solutions LLC Full time

    Job DescriptionJob DescriptionNAVA Software solutions is looking for a Certified Cloud Engineer/ AWS EngineerDetails:Certified Cloud Engineer/ AWS EngineerLocation: Remote project (Need to be based in WI state)Duration: 12 months The Enterprise Analytics Platform (EAP) team supports the development and support of the data lake for the UW System Office and...

  • Certified Cloud Engineer AWS

    3 weeks ago

    Madison, United States Nava Software Solutions LLC Full time

    Job DescriptionJob DescriptionNAVA Software solutions is looking for a Certified Cloud Engineer/ AWS EngineerDetails:Certified Cloud Engineer/ AWS EngineerLocation: Remote project (Need to be based in WI state)Duration: 12 months The Enterprise Analytics Platform (EAP) team supports the development and support of the data lake for the UW System Office and...

  • AWS Data Architect

    2 weeks ago

    Madison, United States Talent Groups Full time

    Advanced hands-on experience designing AWS data lake solutions.Experience integrating Redshift with other AWS services, such as DMS, Glue, Lambda, S3, Athena, Airflow.Proficiency in Python programming with a focus on developing efficient Airflow DAGs and operators.Experience with Pyspark and Glue ETL scripting including functions like renationalize,...

  • AWS Data Architect

    2 weeks ago

    Madison, United States Talent Groups Full time

    Advanced hands-on experience designing AWS data lake solutions.Experience integrating Redshift with other AWS services, such as DMS, Glue, Lambda, S3, Athena, Airflow.Proficiency in Python programming with a focus on developing efficient Airflow DAGs and operators.Experience with Pyspark and Glue ETL scripting including functions like renationalize,...

  • Senior Data Engineer

    3 weeks ago

    Madison, United States Capital One Full time

    77 West Wacker Dr (35012), United States of America, Chicago, Illinois Senior Data Engineer - Principal Associate (Spark, Scala) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big...

  • Senior Data Engineer

    3 weeks ago

    Madison, United States Capital One Full time

    Center 1 (19052), United States of America, McLean, Virginia Senior Data Engineer - Principal Associate Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers,...

  • cloud Engineer

    3 weeks ago

    Madison, United States Chelsoft Solutions Co Full time

    Job DescriptionJob DescriptionKey Responsibilities:Collaborate with data engineering, business analysts, and development teams to design, develop, test, and maintain robust and scalable data pipelines from Workday to AWS Redshift.Architect, implement, and manage end-to-end data pipelines, ensuring data accuracy, reliability, data quality, performance, and...

  • Business Intel Data Modeler-IV

    2 months ago

    Madison, United States Serigor Inc. Full time

    Job DescriptionJob DescriptionJob Title: Business Intel Data Modeler-IV (Remote)Location: Madison, WI 53703Duration: 06+ MonthsJob Description:The client is implementing the Enterprise AnalyticsPlatform (EAP), including a data lake with data serving the 13 UW universities and campuses. The EAP team is seeking a data engineer with data modeling skills and...

  • Business Intel Data Modeler-IV

    3 months ago

    Madison, United States Serigor Inc Full time

    Job Title: Business Intel Data Modeler-IV (Remote)Location: Madison, WI 53703Duration: 06+ MonthsJob Description:The client is implementing the Enterprise AnalyticsPlatform (EAP), including a data lake with data serving the 13 UW universities and campuses. The EAP team is seeking a data engineer with data modeling skills and expertise. The position will work...

  • Business Intel Data Modeler-IV

    2 weeks ago

    Madison, United States Serigor Inc. Full time

    Job DescriptionJob DescriptionJob Title: Business Intel Data Modeler-IV (Remote)Location: Madison, WI 53703Duration: 06+ MonthsJob Description:The client is implementing the Enterprise AnalyticsPlatform (EAP), including a data lake with data serving the 13 UW universities and campuses. The EAP team is seeking a data engineer with data modeling skills and...

  • Business Intel Data Modeler-IV

    4 weeks ago

    Madison, United States Serigor Inc. Full time

    Job DescriptionJob DescriptionJob Title: Business Intel Data Modeler-IV (Remote)Location: Madison, WI 53703Duration: 06+ MonthsJob Description:The client is implementing the Enterprise AnalyticsPlatform (EAP), including a data lake with data serving the 13 UW universities and campuses. The EAP team is seeking a data engineer with data modeling skills and...

  • Data Engineer

    4 weeks ago

    Madison, United States Wisconsin Collaborative for Healthcare Quality Full time

    The Opportunity Are you passionate about leveraging data to drive innovation and create impactful solutions? M3 is seeking a skilled Applied Intelligence Data Engineer to play a pivotal role in our journey towards data excellence. As an Applied Intelligence Data Engineer, you will be at the forefront of developing, optimizing, and maintaining our data...