Data Engineer with Snowflake Experience

Job Summary

Design, build and automate robust ELT data pipelines using Snowflake's native features (Snowpipe, Tasks, Streams) and integrating with external orchestration tools

Monitor and analyze Snowflake performance, optimize SQL queries and data storage for efficiency, and optimal virtual warehouse usage

Understand existing data models and database schemas and enhance to support business requirements

Communicate with product teams and understand data requirements.

Identify, analyze, and interpret trends or patterns in complex data sets.

Develop and implement metadata, error logging, and audit for pipelines and other strategies that optimize efficiency and quality.

Support the bug fixing and performance analysis for data pipelines.

Analyze and manage 3rd Party and other external data sets.

Follow an agile development methodology.

Client: State of South Carolina

Contract: 1099

Visa: USC/GC/H4-EAD

Location: Remote

Requirement

Overall 5+ years of experience with Data Engineering projects

3+ years of hands-on experience with Snowflake, including its architecture, virtual warehouses, databases, and security features

Strong command of SQL for complex data extraction, transformation, and loading (ETL/ELT)

Experience developing and automating ETL/ELT processes within Snowflake and with external orchestration tools

3+ years' experience in years Agile environment

Experience in git source control (GitHub, GitLab, bitbucket)

Identify, analyze, and interpret trends or patterns in complex data sets.

Support the bug fixing and performance analysis along the data pipeline.

Analyze and manage 3rd Party and other external data sets.

MS Office (word, teams, excel), experience with confluence.

Strongly preferred: SQL Server experience, Python experience in data warehousing projects

Self-starter, Self-reliant, and thorough knowledge in SDLC

Responsibilities

• Design and implement scalable data pipelines using technologies such as AWS, Hadoop, and Spark.
• Develop and maintain data warehouses and databases, ensuring optimal performance and reliability using Oracle and Microsoft SQL Server.
• Collaborate with cross-functional teams to gather requirements for data analysis and reporting.
• Write efficient SQL queries for data extraction and manipulation.
• Utilise big data technologies such as Apache Hive and Informatica for processing large datasets.
• Conduct thorough analysis of data to identify trends, patterns, and insights that drive business strategies.
• Implement database design best practices to ensure data integrity and security.
• Create scripts in Python, VBA, Bash (Unix shell), or Shell Scripting to automate tasks and improve workflow efficiency.
• Monitor system performance and troubleshoot issues related to data processing or storage.

Experience

• Proven experience as a Data Engineer or in a similar role within a data-centric environment.
• Strong proficiency in AWS services related to data storage and processing.
• Solid programming skills in Java and Python for building data applications.
• Familiarity with big data technologies such as Hadoop, Spark, Apache Hive, and Informatica is highly desirable.
• Experience with database design principles and working knowledge of Oracle and Microsoft SQL Server.
• Excellent analytical skills with the ability to interpret complex datasets effectively.
• Strong problem-solving abilities coupled with attention to detail in all aspects of work. If you are passionate about transforming raw data into actionable insights through innovative engineering solutions, we encourage you to apply for this exciting opportunity as a Data Engineer.

Please reach me or email me phone:

Job Type: Contract

Pay: $ $60.00 per hour

Expected hours: 40 per week

Work Location: Remote