Senior Data Engineer

2 days ago

Charlotte, United States CV Library Full time

Senior Data Engineer

Location: Charlotte, NC/Hybrid

Duration: 1 year with Possible Contract to hire

Description:

Key Responsibilities:

Translates complex cross-functional business requirements and functional specifications into logical program designs, code modules, stable application systems, and data solutions; partners with Product Team to understand business needs and functional specifications

Collaborates with cross-functional teams to ensure specifications are converted into flexible, scalable, and maintainable solution designs; evaluates project deliverables to ensure they meet specifications and architectural standards

Contributes in the design and build of complex data solutions and ensures the architecture blueprint, standards, target state architecture, and strategies are aligned with the requirements.

Coordinates, executes, and participates in component integration (CIT) scenarios, systems integration testing (SIT), and user acceptance testing (UAT) to identify application errors and to ensure quality software deployment

Participates in all software development end-to-end product lifecycle phases by applying and sharing an in-depth understanding of complex industry methodologies, policies, standards, and controls

Develops detailed architecture plans for large scale enterprise architecture projects and drives the plans to fruition

Solves complex architecture/design and business problems; solutions are extensible; works to simplify, optimize, remove bottlenecks, etc.

Data Engineering Responsibilities

Executes the development, maintenance, and enhancements of data ingestion solutions of varying complexity levels across various data sources like DBMS, File systems (structured and unstructured), APIs and Streaming on on-prem and cloud infrastructure; demonstrates strong acumen in Data Ingestion toolsets and nurtures and grows junior members in this capability

Builds, tests and enhances data curation pipelines integration data from wide variety of sources like DBMS, File systems, APIs and streaming systems for various KPIs and metrics development with high data quality and integrity

Supports the development of feature / inputs for the data models in an Agile manner; Hosts Model Via Rest APIs; ensures non-functional requirements such as logging, authentication, error capturing, and concurrency management are accounted for when model hosting

Works with Data Science team to understand mathematical models and algorithms; recommends improvements to analytic methods, techniques, standards, policies and procedures

Works to ensure the manipulation and administration of data and systems are secure and in accordance with enterprise governance by staying compliant with industry best practices, enterprise standards, corporate policy and department procedures; handles the manipulation (extract, load, transform), data visualization, and administration of data and systems securely and in accordance with enterprise data governance standards

Maintains the health and monitoring of assigned data engineering capabilities that span analytic functions by triaging maintenance issues; ensures high availability of the platform; works with Infrastructure Engineering teams to maintain the data platform; serves as an SME of one or more application

BI Engineering Responsibilities

Responsible for the development, maintenance, and enhancements of BI solutions of varying complexity levels across different data sources like DBMS, File systems (structured and unstructured) on-prem and cloud infrastructure; creates level metrics and other complex metrics; use custom groups, consolidations, drilling, and complex filters

Demonstrates database skill (Teradata/Oracle/Db2/Hadoop) by writing views for business requirements; uses freeform SQLs and pass-through functions; analyzes and finds errors from SQL ; creates RSD and dashboard

Responsible for building, testing and enhancement of BI solutions from a wide variety of sources like Teradata, Hive, Hbase, Google Big Query and File systems; develops solutions with optimized data performance and data security

Works with business analysts to understand requirements and create dashboard/dossiers wireframes; makes use of widgets and Vitara charts to make the dashboard/dossiers visually appealing

Coordinates and takes necessary actions from DART side for application upgrades (e.g., Teradata, Workday), storage migration, and user management automation; supports such things as Cluster Management and Project Configuration settings

Competencies:

Technical Competencies

Agile Development

Big Data Management and Analytics

Cloud Computing

Database Design (Physical)

Release Management

Qualifications

Minimum Qualifications:

Bachelor's Degree in Engineering /Computer Science, CIS, or related field (or equivalent work experience in a related field)

5 years of experience in Data or BI Engineering, Data Warehousing/ETL, or Software Engineering

4 years of experience working on project(s) involving the implementation of solutions applying development life cycles (SDLC)

Additional Skills:

Python

Hadoop

Shell Scripts

Day to Day Responsibilities

Listing the Responsibilities

PySPark

Scala

Automation of Pipelines

Airflow/Autosys

Spark

Monitoring/Alerts

GCP DataProc/Composer/BQ

Data modelling

Required Skills (top 3 non-negotiables):

Agile Experience 2 Years minimum

10+ years of Data Engineering

8+ years of PySpark/Scala

7 years of Scala

5+ years of experience hosting application on GCP

Software Skills Required: Grafana

Skills (nice to have)

Case Management Experience 3+ years

  • Senior Data Solutions Architect

    2 weeks ago

    Charlotte, North Carolina, United States Pantheon Data Full time

    Job OverviewPantheon Data is a private, small business based in the Washington, DC area.We are seeking a skilled Data Engineer with expertise in designing data-centric technical solutions.The ideal candidate will have hands-on experience with data modeling, data warehousing, and ETL pipelines.Analyze customer technology landscapes, business processes, and...

  • Cloud Engineer

    22 hours ago

    Charlotte, North Carolina, United States Pantheon Data Full time

    Job DescriptionPantheon Data is seeking a skilled Data Engineer to develop data pre-processing and analytics code using Python, R, and similar languages. The successful candidate will contribute to ETL pipeline solutions that ingest and process data from multiple sources and create cloud-based data warehouse(s) that provide inputs to predictive models.The...

  • Senior Data Engineer

    3 weeks ago

    Charlotte, United States Array Full time

    Position: Senior Data EngineerLocation: Charlotte, NCPay: $80,000 to $100,000/year + Bonus! Experience: At least three (3) years of related experience.Education: Bachelor's degree. Master's degree (preferred not required).Array Corporation is seeking a Senior Data Engineer to join a growing and dynamic team! Job Description: Develop clean, reusable, and...

  • Senior Data Engineer with PySpark

    2 weeks ago

    Charlotte, North Carolina, United States Diverse Lynx Full time

    Job Title: Senior Data Engineer with PySparkJob Description:We are seeking a highly experienced Senior Data Engineer with expertise in PySpark to lead our data engineering team. The ideal candidate will have a strong background in designing and implementing large-scale data architectures, including data lakes, data warehouses, and data marts....

  • Senior Data Engineer

    4 weeks ago

    Charlotte, United States Array Full time

    Position: Senior Data EngineerLocation: Charlotte, NCPay: $80,000 to $100,000/year + Bonus! Experience: At least three (3) years of related experience.Education: Bachelor's degree. Master's degree (preferred not required). Array Corporation is seeking a Senior Data Engineer to join a growing and dynamic team! Job Description: Develop clean, reusable, and...

  • Chief Data Systems Engineer

    2 weeks ago

    Charlotte, North Carolina, United States Pantheon Data Full time

    Job OverviewPantheon Data, a private company based in the Washington, DC area, is seeking an experienced Chief Data Systems Engineer.We are looking for someone to design and develop software applications that drive our business forward. Our ideal candidate will have experience working with embedded systems, data warehouses, and ETL...

  • Senior Deep Learning Engineer for Intelligent Document Processing

    2 weeks ago

    Charlotte, North Carolina, United States Pantheon Data Full time

    At Pantheon Data, we are seeking a seasoned Senior Deep Learning Engineer to join our team of innovators in the field of Intelligent Document Processing (IDP). As a key member of our engineering team, you will be responsible for developing and implementing cutting-edge solutions that leverage Optical Character Recognition (OCR), Natural Language Processing...

  • Senior Data Engineering Manager

    2 weeks ago

    Charlotte, North Carolina, United States TechnoGen Full time

    Job Title: Senior Data Engineering ManagerJob Summary:We are seeking a Senior Data Engineering Manager to lead our data engineering team at TechnoGen. The ideal candidate will have 12+ years of experience in data engineering and a strong background in SQL, Python, and Spark programming languages.Key Responsibilities:Lead multiple teams and drive technical...

  • Senior Data Engineering Specialist

    2 weeks ago

    Charlotte, North Carolina, United States Randstad Digital Full time

    We are Randstad Digital, a leading provider of digital solutions. We are seeking a highly skilled Senior Data Engineer to join our team.Estimated Salary: $140,000 - $180,000 per yearJob Description:As a Senior Data Engineer at Randstad Digital, you will play a critical role in designing, developing, and implementing new features to our existing framework...

  • Senior Data Engineer

    1 month ago

    charlotte, United States Kellton Full time

    Position: Sr. Data EngineerWork Location: Remote (working CST or EST) and able to travel quarterly for PI planning Duration: 6 Month (Contract-to-Hire)Manager Notes:Sr. Data EngineerExperience on several DE projects, hands onStrong snowflake expertise - administrations Advanced SQL and Python skills Extracting data from APIs ETL processes, data ingestion AWS...

  • Senior Data Engineer

    1 month ago

    Charlotte, United States Kellton Full time

    Position: Sr. Data EngineerWork Location: Remote (working CST or EST) and able to travel quarterly for PI planning Duration: 6 Month (Contract-to-Hire)Manager Notes:Sr. Data EngineerExperience on several DE projects, hands onStrong snowflake expertise - administrations Advanced SQL and Python skills Extracting data from APIs ETL processes, data ingestion AWS...

  • Senior Big Data Engineer

    22 hours ago

    Charlotte, North Carolina, United States The Hartford Full time

    About The RoleThe Hartford is seeking a highly experienced Senior Big Data Engineer to join its team. As a Senior Big Data Engineer, you will design and implement large-scale data products that meet the company's needs.This is an exciting opportunity for individuals who want to shape the future of data engineering and drive innovation. You will work closely...

  • Cloud Data Engineer Position for Experienced Professionals

    2 weeks ago

    Charlotte, North Carolina, United States Pantheon Data Full time

    Job Title: Cloud Data EngineerAbout Us:Pantheon Data is a private company based in the Washington, DC area. Founded in 2011, we have grown to offer various services including infrastructure resiliency, contact center operations, and software engineering. Our customer base includes commercial clients and federal agencies.Position Overview:We are seeking an...

  • MAX - Senior Data Engineer

    1 week ago

    Charlotte, United States Lighthouse Technology Services Full time

    *Remote Contract-to-Hire Opportunity* Lighthouse Technology Services is partnering with our client to fill their Senior Data Engineer position! This is a 6+ month contract-to-hire opportunity. This role can be remote within the United States, with quarterly travel to Charlotte, NC. This role will be a W2 employee of Lighthouse Technology Services. ...

  • Senior Data Analyst/Data Engineer

    1 week ago

    Charlotte, United States Collabera Full time

    Description Home Search Jobs Job Description Senior Data Analyst/Data Engineer Contract: Charlotte, North Carolina, US Salary: $70.00 Per Hour Job Code: 356707 End Date: 2025-01-09 Days Left: 29 days, 2 hours left Apply About Collabera: Collabera is a leading global technology services and solutions provider committed to delivering high-quality,...

  • Senior Data Engineer

    2 weeks ago

    Charlotte, North Carolina, United States Manpower Group Inc. Full time

    **Job Title:** Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Manpower Group Inc. As a Senior Data Engineer, you will be responsible for designing, implementing, and maintaining large-scale data systems, including data pipelines, data warehouses, and data lakes.The ideal candidate will have a strong background in...

  • Senior Data Engineer

    1 day ago

    Charlotte, United States LendingTree Full time

    *This role requires in-person attendance in the Charlotte, NC office at least 3 days a week (ideally Tuesday-Thursday)Position: LendingTree is seeking a Senior Data Engineer to join a team of data engineers. We have an exciting opportunity to develop and scale our home-grown data platform that powers our Business Intelligence reports, Analytics, and...

  • Advanced Deep Learning Engineer

    2 weeks ago

    Charlotte, North Carolina, United States Pantheon Data Full time

    Are you a skilled Machine Learning Engineer looking for a challenging role that combines technical expertise with business acumen? Pantheon Data, a leading provider of innovative solutions to the US government and commercial clients, is seeking a seasoned Deep Learning Engineer to join our team.About the RoleThis is a unique opportunity to leverage your...

  • Senior Data Engineer

    2 weeks ago

    Charlotte, North Carolina, United States ApTask Full time

    About This Role:Candidates for the Senior Data Engineer - Big Data Hadoop position at ApTask will be responsible for designing, developing, and implementing large-scale data processing systems using Big Data Hadoop. The ideal candidate will have experience working with big data technologies, including Hadoop, PySpark, and Teradata, and will be able to...

  • Data Science Engineer

    2 weeks ago

    Charlotte, North Carolina, United States NTT DATA Full time

    About NTT DATA ServicesNTT DATA Services is a recognized leader in IT and business services, including cloud, data and applications. As part of NTT DATA, a $30 billion trusted global innovator with a combined global reach of over 80 countries, we help clients transform through business and technology consulting, industry and digital solutions, applications...