Data Engineer

2 weeks ago

Raleigh, United States Crescens Full time

Job Title: Data Engineer - Azure Platform

Location: Raleigh,NC(Onsite)

Duration: 3+ Months

Note:

Inperson interview

Responsibilities:

We are looking for an experienced Data Engineer to join our team. You will use various methods to import raw data into Azure Cloud. You will analyze data and create rules for the ETL process. The import process will also include identifying the business rules and apply them to the files stored in staging area before moving the files to processed area.

The ideal candidate will have a strong background in data engineering, along with expertise in cloud computing. This role offers an exciting opportunity to play a key role in modernizing our technology stack and driving our organization's digital

transformation journey.

Responsibilities:

Collaborate with cross-functional business teams to understand requirements,

analyze, and organize raw data import by identifying the ETL rules.

Identify and apply transformation rules for legacy data migration from mainframe

systems to the Azure cloud platform.

Develop and implement data pipelines, ETL processes, and data integration

solutions to facilitate the conversion of data.

Develop and maintain documentation, best practices, and standards for data

engineering processes and procedures.

Provide technical leadership and mentorship to team members, guiding them in best

practices for data engineering and cloud migration projects.

Explore ways to enhance data quality and reliability.

Requirements and Skills

1.Data Modeling and Database Design:

- Proficiency in designing and implementing relational and non-relational

databases.

- Knowledge of data modeling techniques and tools.

2. Azure Data Services:

- Deep understanding and hands-on experience with Azure data services such as

Azure SQL Database, Azure Cosmos DB, Azure Data Lake Storage, Azure Blob

Storage, etc.

3. ETL (Extract, Transform, Load) Processes:

- Expertise in designing and implementing ETL processes using tools like Azure

Data Factory or Azure Databricks.

- Knowledge of data integration and transformation techniques.

4. SQL and T-SQL:

- Proficient in writing complex SQL queries and stored procedures.

- Familiarity with Transact-SQL (T-SQL) for managing and querying SQL Serverdatabases.

5. Data Security and Compliance:

- Knowledge of data security best practices and experience implementing security

measures for data storage and processing on Azure.

- Awareness of compliance requirements and ability to implement solutions that

adhere to NC DPI and DIT regulations.

6. Data Monitoring and Optimization:

- Ability to set up monitoring and logging for data pipelines and databases.

- Skills in optimizing data storage and processing for performance and costeffectiveness.

7. Scripting and Automation:

- Proficiency in scripting languages like Python or PowerShell for automating data

engineering tasks.

-Knowledge of Azure Resource Manager (ARM) templates for infrastructure as code.

Additionally, obtaining relevant certifications, such as the Microsoft Certified: Azure

Data Engineer Associate, can further validate and enhance these skills.

Qualifications :

Bachelor's degree in computer science, Engineering, or a related field

Proven experience as a data engineer, with a strong focus on data migration, ETL,

and data integration.

Hands-on experience with data warehousing concepts, database technologies (SQL

on Azure)

Strong problem-solving skills, attention to detail, and the ability to work effectively in

a fast-paced environment.

Excellent communication and interpersonal skills, with the ability to collaborate effectively with cross-functional teams and stakeholder

s.

Required Skills:

Proficiency in designing and implementing relational and nonrelational databases.

Knowledge of data modeling

techniques and tools.

Hands-on experience with Azure

data services such as Azure SQL

Database, Azure Cosmos DB,

Azure Data Lake Storage, Azure

Blob Storage, etc.

Expertise in designing and

implementing ETL processes

using tools like Azure Data

Factory or Azure Databricks.

Knowledge of data integration

and transformation techniques.

Proficient in writing complex SQL

queries and stored procedures

Knowledge of data security best

practices and experience

implementing security measures

for data storage and processing

on Azure.

Ability to set up monitoring and

logging for data pipelines and

databases.

Skills in optimizing data storage

and processing for performance

and cost-effectiveness

Proficiency in scripting languages

like Python or PowerShell for

automating data engineering

tasks

Knowledge of Azure Resource

Manager (ARM) templates for

infrastructure as code

#J-18808-Ljbffr

  • Data Engineer

    2 weeks ago

    Raleigh, United States Dorel Home group Full time

    Dorel Homeis a dynamic and innovative home furnishings manufacturing company committed to leveraging data-driven insights to enhance decision-making processes and drive business success. We are seeking a highly motivated and talented Data Engineer to join our growing team and contribute to the development and maintenance of our data infrastructure. As a Data...

  • Data Engineer

    1 day ago

    Raleigh, United States McGriff Insurance Broker Full time

    The primary purpose of this role is to provide technical direction on translating business requirements and functional specifications into logical program designs. This includes assisting with the implementation and maintenance of complex business an Data Engineer, Engineer, Data Architect, Processing, Microsoft, Azure, Technology, Insurance

  • Data Engineer

    3 weeks ago

    Raleigh, United States Wipro Limited Full time

    Job descriptionabout the role Identify and analyze user requirements. Work closely with IT and Business stakeholders to understand business needs and work collaboratively to deliver solutions Prioritize assign and execute tasks throughout the Data/BI project life cycle. Build BI reports and dashboards based on business requirements Develop/integrate...

  • Data Engineer

    3 weeks ago

    Raleigh, North Carolina, United States Wipro Limited Full time

    Job descriptionabout the role Identify and analyze user requirements. Work closely with IT and Business stakeholders to understand business needs and work collaboratively to deliver solutions Prioritize assign and execute tasks throughout the Data/BI project life cycle. Build BI reports and dashboards based on business requirements Develop/integrate...

  • Data Engineer

    3 weeks ago

    Raleigh, United States LHH Full time

    ***Candidates must be local Raleigh, NC area for a hybrid commute***We are seeking a talented junior-mid level experienced data engineer for a mid-large company in the manufacturing industry. The ideal candidate will be passionate about leveraging data to solve complex problems and drive business value. This role offers the opportunity to work at the...

  • Data Engineer

    2 weeks ago

    Raleigh, United States LHH Full time

    ***Candidates must be local Raleigh, NC area for a hybrid commute***We are seeking a talented junior-mid level experienced data engineer for a mid-large company in the manufacturing industry. The ideal candidate will be passionate about leveraging data to solve complex problems and drive business value. This role offers the opportunity to work at the...

  • Data Engineer

    14 hours ago

    Raleigh, United States Georgia IT Inc Full time

    Title :- Sr Data Engineer Position Type :- Onsite Duration :- 1+year contract Rate :- DOE US Citizens, Green Card Only Required Qualifications and Skills Experience with AWS Services, especially, Data Pipeline, Glue, Kinesis, and Redshift Spectrum. experience with Amazon Redshift, and AWS Aurora, working knowledge of S3 data lake environment, AWS...

  • Data Engineer

    2 days ago

    Raleigh, United States Arch Capital Group Full time

    Job Summary The Data Engineer (internally known as Software Engineer III ), will specialize in designing, implementing, and optimizing database systems for efficient storage, retrieval, and management of structured data. Join us as we embark on an exciting modernization journey, transitioning from our legacy platforms to cutting-edge technologies...

  • Azure Data Engineer LU 4-22

    2 weeks ago

    Raleigh, United States Focused HR Solutions Full time

    **This position is 100% on-site in Raleigh, NC.  In-Person Interview REQUIRED on 1st round.  **  Our direct client has an opening for a Azure Data Engineer position # 730992. This position is for 4+ months, with option of extension, and will be worked 100% on-site in Raleigh, NC. If you are interested, please submit the following:YOUR CURRENT RESUMEYOUR...

  • Sr. Software Engineer, Data Plane, NPD Forwarding Stack, Data Plane

    2 days ago

    Raleigh, United States Amazon.com Inc Full time

    AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, were the people who keep the cloud running. We support all AWS data centers and all of the servers, storage, networking, Software Engineer, Data, Operations Manager, Software, Development Engineer, Network Engineer, Technology

  • Data Engineer

    12 hours ago

    Raleigh, United States Arch Capital Group Ltd. Full time

    With a company culture rooted in collaboration, expertise and innovation, we aim to promote progress and inspire our clients, employees, investors and communities to achieve their greatest potential. Our work is the catalyst that helps others achieve their goals. In short, We Enable Possibility℠. Job Summary The Data Engineer (internally known as Software...

  • Data Center Network Engineer

    4 days ago

    Raleigh, United States Cisco Full time

    Who We Are We are the Data Center Network Services team within Cisco IT that supports network services for Cisco Engineering and business functions worldwide. Our mission is simple - build the network of the future that is adaptable and agile on Cisco's Application Centric Infrastructure (ACI), a software-defined networking solution. ACI is deployed,...

  • Azure Data Engineer

    3 days ago

    Raleigh, United States Smart IT Frame LLC Full time

    Hi,Greetings from Smart IT Frame, Hope you are doing well!!!Smart IT Frame specializes in enabling you with your most critical line of resources. Whether it’s for permanent staffing, contract staffing, contract-to-hire or executive search, we understand the importance of delivering the most suitable talent; on time and within budget. With our Core focus in...

  • Azure Data Engineer

    3 days ago

    Raleigh, United States Smart IT Frame LLC Full time

    Hi,Greetings from Smart IT Frame, Hope you are doing well!!!Smart IT Frame specializes in enabling you with your most critical line of resources. Whether it’s for permanent staffing, contract staffing, contract-to-hire or executive search, we understand the importance of delivering the most suitable talent; on time and within budget. With our Core focus in...

  • Information Technology

    2 weeks ago

    Raleigh, United States Tech Providers Full time

    Data Integration Engineer 12+ Months Contract 100% Remote Skills Required: Looking for an engineer that has knowledge on Pipelines, azure devops pipeline work. How to manage database backups and restore Doing a lot of migration of databases for all the projects. Assigned into a project and each project has different stages some of them are provisioning...

  • Data Center Network Engineer

    2 weeks ago

    Raleigh, United States Cisco Full time

    Who We Are We are the Data Center Network Services team within Cisco IT that supports network services for Cisco Engineering and business functions worldwide. Our mission is simple - build the network of the future that is adaptable and agile on Cisco's Application Centric Infrastructure (ACI), a software-defined networking solution. ACI is deployed,...

  • Data Center and DMZ Network Engineer

    1 week ago

    Raleigh, North Carolina, United States Bank of America Full time

    Job Description:At Bank of America, we are guided by a common purpose to help make financial lives better through the power of every connection. Responsible Growth is how we run our company and how we deliver for our clients, teammates, communities and shareholders every day.One of the keys to driving Responsible Growth is being a great place to work for our...

  • 1420055 - Data Center Network Engineer

    2 weeks ago

    Raleigh, United States Cisco Systems, Inc. Full time

    Who We AreWe are the Data Center Network Services team within Cisco IT that supports network services for Cisco Engineering and business functions worldwide. Our mission is simple - build the network of the future that is adaptable and agile on Cisco's Application Centric Infrastructure (ACI), a software-defined networking solution. ACI is deployed,...

  • Data Center and DMZ Network Engineer

    3 weeks ago

    Raleigh, United States Bank of America Full time

    Job Description:At Bank of America, we are guided by a common purpose to help make financial lives better through the power of every connection. Responsible Growth is how we run our company and how we deliver for our clients, teammates, communities and shareholders every day.One of the keys to driving Responsible Growth is being a great place to work for our...

  • Computerized Data Analyst

    3 weeks ago

    Raleigh, United States Advanced Recruiting Partners Full time

    Job Description: We are seeking a highly skilled CMMS (Computerized Maintenance Management System) Data Analyst to join our Engineering Department. The CMMS Data Analyst will be responsible for managing and analyzing data within our CMMS to support maintenance activities and optimize asset performance. The ideal candidate will have a strong background in...