Lead Data Engineer
3 days ago
Job Summary:
We are seeking a highly skilled and experienced Lead Data Engineer to join our dynamic team. The ideal candidate will bring a wealth of expertise in designing, developing, and implementing data pipelines, cloud solutions, and ETL processes. With a proven track record in managing large-scale data warehouses, the candidate should demonstrate proficiency in cloud platforms like Azure and AWS, as well as real-time data processing technologies.
Key Responsibilities:
- Design and implement scalable data pipelines using PySpark, Apache Kafka, AWS Glue, and Azure Data Factory to process and transform data from diverse sources.
- Develop ETL workflows to ingest and process large-scale data into Snowflake, Azure Synapse Analytics, and other data warehouses.
- Leverage cloud-based platforms (Azure Data Lake, AWS S3, Databricks) for data storage, processing, and transformations.
- Optimize database performance and implement data modeling techniques such as Star Schema, Snowflake modeling, and slowly changing dimensions.
- Build real-time data processing solutions utilizing Apache Kafka, Spark Streaming, and other streaming technologies.
- Create and maintain pipelines for data synchronization between systems such as Salesforce, Snowflake, and other platforms using APIs, Amazon AppFlow, and Informatica tools.
- Develop and maintain dashboards and reports using Power BI and Tableau for business insights.
- Implement and manage CI/CD pipelines using Azure DevOps for data pipelines.
- Collaborate with cross-functional teams to design and develop end-to-end data solutions, including storage, integration, and visualization.
- Perform data profiling, quality checks, and transformations using Informatica IDQ and other data quality tools.
Required Skills and Qualifications:
- 12+ years of IT experience, with expertise in data engineering, data modeling, and cloud-based solutions.
- Proficiency in tools and platforms such as Informatica (PowerCenter, IICS), Snowflake, AWS Glue, Azure Data Factory, and Databricks.
- Strong programming skills in Python, Java, Scala, and experience with frameworks like PySpark.
- Expertise in real-time data processing using Kafka, Spark Streaming, and AWS Lambda.
- In-depth knowledge of RDBMS and NoSQL databases such as Oracle, SQL Server, PostgreSQL, MongoDB, and DynamoDB.
- Hands-on experience in building and managing data pipelines and workflows in Azure and AWS environments.
- Experience in reporting tools such as Power BI, Tableau, and QlikView.
- Knowledge of data governance, data quality, and MDM using Informatica tools.
- Familiarity with automation and scheduling tools like Apache Airflow and Control-M.
- Strong problem-solving and analytical skills with the ability to manage multiple tasks.
Preferred Qualifications:
- Experience in the Rail Transportation, Healthcare, or Banking domains.
- AWS or Azure certification is a plus.
- Familiarity with modern data warehousing techniques and frameworks like Apache Iceberg.
Education:
- Bachelor’s degree in Computer Science, Engineering, or a related field.
-
Data Lead Engineer
3 days ago
Harrisburg, United States EnIn Systems Full timeJob Summary:We are seeking a highly skilled and experienced Lead Data Engineer to join our dynamic team. The ideal candidate will bring a wealth of expertise in designing, developing, and implementing data pipelines, cloud solutions, and ETL processes. With a proven track record in managing large-scale data warehouses, the candidate should demonstrate...
-
Data Management Lead
3 weeks ago
Harrisburg, Pennsylvania, United States Prequel Solutions Full timeJob Description: Data Management LeadWe are looking for a skilled Data Management Lead to join our team at Prequel Solutions.About the Role:The Data Management Lead will be responsible for overseeing data processing and reporting projects, as well as supporting ongoing data clean-up and future data conversion activities for a new system being implemented.Key...
-
Data Architect Lead
7 days ago
Harrisburg, Pennsylvania, United States Schaedler Yesco Distribution Full timeJob OverviewWe are seeking an experienced Data Architect Lead to join our team at Schaedler Yesco Distribution. This is a unique opportunity to work with a family- and employee-owned company that has been illuminating the electrical industry since 1924.In this role, you will be responsible for designing and implementing data architecture solutions that meet...
-
Lead Electrical Engineer
2 weeks ago
Harrisburg, Pennsylvania, United States Sargent & Lundy Full timeSargent & Lundy is seeking a skilled Electrical Engineering Leader to join our team in Harrisburg, PA. As an essential member of our Nuclear Power Group, you will be responsible for leading electrical engineering projects from concept to completion. With a focus on delivering high-quality results, you will oversee the design and implementation of electrical...
-
Senior Electrical Engineer
3 weeks ago
Harrisburg, Pennsylvania, United States Sargent & Lundy Full timeWe are seeking a seasoned Senior Electrical Engineer - Project Lead to join our Nuclear Power Group. As a project lead, you will be responsible for overseeing the design and implementation of electrical systems for nuclear power plants, as well as leading a team of engineers. You will work closely with clients and stakeholders to ensure project success.Key...
-
Principal Network Engineering Lead
2 weeks ago
Harrisburg, Pennsylvania, United States Lumen Inc Full timeAbout UsLumen Inc connects the world by providing cutting-edge data and application connectivity solutions. Our company values include teamwork, trust, and transparency.We're looking for a highly skilled professional to join our team as a Principal SRE in the organization.The Job DescriptionThis role involves leading daily activities, partner collaboration,...
-
Engineering Test Lead
6 days ago
Harrisburg, Pennsylvania, United States GovCIO Full timeJob DescriptionGovCIO is hiring a skilled Engineering Test Lead to support HUD. This full-time remote position comes with a salary range of $135,000.00 - $135,000.00 per year. The ideal candidate will have experience in system-level requirements verification procedures, customer acceptance test procedures, and testing processes.The role involves consulting...
-
Water Systems Engineering Lead
3 weeks ago
Harrisburg, Pennsylvania, United States Veolia North America Full timeAbout This RoleThis is a fantastic opportunity to join our engineering team as a Capital Projects Director - Water and Wastewater. We are looking for a highly experienced professional who can lead and manage complex projects from inception to completion.As a key member of our team, you will be responsible for managing all activities related to the successful...
-
Lead AI Researcher
3 weeks ago
Harrisburg, Pennsylvania, United States Oracle Full timeSolution DevelopmentThis role presents a unique opportunity to work on pioneering AI technologies and contribute to Oracle's Generative AI strategy. As a Principal Applied Scientist within the HDI team, you will develop cutting-edge Generative AI and NLP solutions tailored to healthcare. You will lead the development of scalable models and systems that...
-
Commissioning Engineer Team Lead
3 weeks ago
Harrisburg, Pennsylvania, United States RES Full timeAbout UsAt RES, we are committed to building a workplace where everyone feels heard and respected. We are looking for talented individuals who share our values and are passionate about working on utility scale battery energy storage systems.This is a unique opportunity to join a growing company and contribute to the development of sustainable energy...
-
Senior Engineering Lead
2 weeks ago
Harrisburg, Pennsylvania, United States Sargent & Lundy Full timeAbout the JobThis role involves overseeing and directing a team of engineers working on various projects within the Nuclear Power Group. Responsibilities include managing project scope, budget, and timeline while maintaining open communication with clients and stakeholders.The ideal candidate will possess strong leadership skills, excellent written and...
-
Data Analyst
4 weeks ago
Harrisburg, United States Prequel Solutions Full timeJob Description: Our client is looking for a Risk Management Data Analyst who will collaborate with internal business partners to develop, deliver and maintain analytic applications and solutions to enhance the departments enterprise risk management goals. In this role, you will: Collaborate with internal stakeholders to understand business strategies,...
-
Healthcare Data Intelligence Architect
3 weeks ago
Harrisburg, Pennsylvania, United States Oracle Full timeTeam CollaborationYou will collaborate with a multidisciplinary team to design, build, and deploy scalable models and systems that enhance clinical decision-making, optimize operational workflows, and deliver meaningful insights from complex healthcare data. You will work closely with software engineers, product managers, and data scientists to integrate AI...
-
Safety Engineering Professional
1 month ago
Harrisburg, Oregon, United States Selmet Full timeAbout UsConsolidated Precision Products (CPP) is a leading manufacturer of highly-engineered components and sub-assemblies, supplying the commercial aerospace, military, and industrial markets with small-to-large function-critical products.Job SummaryWe are seeking an experienced Safety Engineer to join our team in Albany, OR. The successful candidate will...
-
Data Analytics Strategist
2 weeks ago
Harrisburg, Pennsylvania, United States WITF Full timeData Analytics LeadWITF seeks a seasoned professional to spearhead data-driven initiatives across its digital platforms, encompassing news, education, production, philanthropy, and sales divisions. As Data Analytics Lead, you will collaborate with senior leaders to inform data-driven decisions and cultivate a culture of continuous learning.Main...
-
Senior Software Engineering Lead
1 week ago
Harrisburg, Pennsylvania, United States Genesis Systems, Inc. Full timeUnlock Career Growth with Genesis Systems, Inc.We are proud to offer a competitive compensation package of $120,000 annually, a comprehensive benefits program including company-paid health care, dental & vision coverage, 401(k) and life & disability insurance, generous holiday, sick & vacation time off, and a professionally challenging, yet fun office...
-
Chief Mechanical Engineering Director
1 month ago
Harrisburg, Pennsylvania, United States JFC Staffing Companies Full timeAbout the RoleWe are seeking a highly skilled Chief Mechanical Engineering Director to lead our Design/Build projects and oversee the planning, execution, and delivery of engineering initiatives. As a key member of our team, you will be responsible for managing project schedules, budgeting, and stakeholder communication to ensure plans align with company and...
-
Senior Transportation Systems Engineer
2 weeks ago
Harrisburg, Pennsylvania, United States HNTB Full time**Job Overview**HNTB is a leading infrastructure firm delivering solutions for complex projects across the country. As a talented Traffic and ITS Engineer II, you will be responsible for assisting in design calculations, technical reports, engineering plans, and specifications for assigned projects.The role involves working closely with project managers to...
-
Manager of Data Quality
3 weeks ago
Harrisburg, United States e&e IT Consulting Services, Inc. Full timee&e is seeking a Manager of Data Quality for an onsite opportunity in Harrisburg, Pennsylvania!The Manager of Data Quality is a driven and detail-oriented professional responsible for ensuring the accuracy, timeliness, completeness, and validity of learner data. This role oversees the submission of state and federal data reports, establishes best practices...
-
Process Engineer
3 months ago
Harrisburg, United States First Quality Full timeFirst Quality was founded in 1989 and, in nearly three decades, has grown to be a global privately held company with over 4,000 employees. Its corporate offices are located in Great Neck, New York, with manufacturing facilities and offices in Pennsylvania, South Carolina, Georgia, and Canada. First Quality is a diversified family of companies manufacturing...