Current jobs related to Senior Big Data Engineer - Atlanta, Georgia - Norfolk Southern Corp.

  • Senior Data Engineer

    6 days ago

    Atlanta, Georgia, United States MasterCard Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our Marketing Services (MS) Technology team. As a key member of our team, you will be responsible for designing and implementing data solutions using cutting-edge technologies such as Big Data, AI/Machine Learning, and Data Analytics.Key ResponsibilitiesDevelop batch and streaming...

  • Senior Data Engineer

    3 hours ago

    Atlanta, Georgia, United States MasterCard Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our Marketing Services (MS) Technology team at Mastercard. As a key member of our team, you will be responsible for designing, developing, and implementing data solutions using cutting-edge technologies such as Big Data, AI/Machine Learning, and Data Analytics.The ideal candidate will...

  • Senior Data Engineer

    2 weeks ago

    Atlanta, Georgia, United States MasterCard Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our Marketing Services (MS) Technology team at Mastercard. As a key member of our team, you will be responsible for designing, developing, and implementing data solutions using cutting-edge technologies such as Big Data, AI, and Machine Learning.Key ResponsibilitiesDevelop batch and...

  • Big Data Engineer

    2 weeks ago

    Atlanta, Georgia, United States Open Systems Inc. Full time

    Job Title: Big Data EngineerCompany: Open Systems Inc.Job Summary:We are seeking an experienced Big Data Engineer to join our team at Open Systems Inc. in Atlanta, GA. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Key Responsibilities:Design and implement data ingestion...

  • Big Data Engineer Lead

    3 days ago

    Atlanta, Georgia, United States Elevance Health Full time

    Job DescriptionElevance Health is seeking a highly skilled Big Data Engineer Lead to join our team. As a key member of our engineering team, you will be responsible for designing, developing, and maintaining large-scale data systems and architectures.Key Responsibilities:Lead the development of data engineering solutions, including data pipelines, data...

  • Big Data Software Engineer

    2 weeks ago

    Atlanta, Georgia, United States Georgia Tech Research Institute Full time

    About the RoleWe are seeking a highly skilled Big Data Software Engineer to join our team at the Georgia Tech Research Institute. As a key member of our Health Emerging & Advanced Technologies (HEAT) Division, you will play a critical role in designing, developing, and implementing secure and scalable big data solutions for healthcare data analytics.Key...

  • AWS Big Data Architect

    6 days ago

    Atlanta, Georgia, United States Pyramid Consulting, Inc Full time

    Job Title: AWS Big Data ArchitectWe are seeking a highly skilled AWS Big Data Architect to join our team at Pyramid Consulting, Inc. in Atlanta, GA. As a key member of our team, you will be responsible for designing, architecting, and implementing end-to-end data engineering solutions on the AWS platform.Key Responsibilities:Design and implement scalable and...

  • Big Data Specialist

    3 weeks ago

    Atlanta, Georgia, United States Brooksource Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Brooksource. As a Data Engineer, you will play a critical role in managing and optimizing data pipelines, ensuring the reliability and scalability of our data architecture.Key ResponsibilitiesData Pipeline Development: Design, build, and maintain scalable data pipelines to...

  • Senior Data Engineer

    7 hours ago

    Atlanta, Georgia, United States Futran Tech Solutions Pvt. Ltd. Full time

    Job Title: Technical Lead Big DataFutran Tech Solutions Pvt. Ltd. is seeking a highly skilled Technical Lead Big Data to join our team. As a Technical Lead Big Data, you will be responsible for designing, building, and operationalizing large-scale enterprise data solutions and applications using Google Cloud Platform (GCP) data and analytics services.Key...

  • Senior Data Engineer

    2 weeks ago

    Atlanta, Georgia, United States Georgia Tech Research Institute Full time

    About the RoleWe are seeking a highly skilled Big Data Software Engineer to join our team at the Georgia Tech Research Institute. As a key member of our Health Emerging & Advanced Technologies (HEAT) Division, you will play a critical role in designing, developing, and implementing secure and scalable big data solutions for healthcare data analytics.Key...

  • Senior Manager of Data Engineering

    6 days ago

    Atlanta, Georgia, United States T-Mobile Full time

    About the RoleWe are seeking a highly skilled Senior Manager of Data Engineering to lead our team of data engineers in the design, development, and implementation of data warehouse and business analytics solutions.Key ResponsibilitiesManage a team of data engineers to deliver high-quality data solutionsDevelop and communicate plans and roadmaps for the Data...

  • Senior Manager, Data Engineering

    4 days ago

    Atlanta, Georgia, United States Disability Solutions Full time

    About the RoleWe are seeking a highly skilled Senior Manager, Data Engineering to lead our Data Engineering team. As a key member of our organization, you will be responsible for overseeing the analysis, architecture, design, and development of data warehouse and business analytics solutions.Key ResponsibilitiesManage a team of data engineers to design,...

  • Senior Data Quality Engineer

    3 days ago

    Atlanta, Georgia, United States Diversity Resource Staffing Inc Full time

    Senior Data Quality EngineerDiversity Resource Staffing Inc is seeking a highly skilled Senior Data Quality Engineer to join our team. As a key member of our data quality team, you will be responsible for developing and maintaining automated tests to ensure compliance reports are accurate and complete.Key ResponsibilitiesDesign and develop test strategies...

  • Senior Data Integration Specialist

    5 days ago

    Atlanta, Georgia, United States iVoyant Full time

    About the JobiVoyant is seeking a highly skilled Senior Data Integration Specialist to join our Digital Team. As a key member of our team, you will work closely with functional SMEs to understand business requirements and design technical solutions.Key ResponsibilitiesDevelop and maintain data pipelines using Informatica (DEI/BDM) or equivalent...

  • Senior Data Integration Specialist

    2 days ago

    Atlanta, Georgia, United States iVoyant Full time

    About the JobiVoyant is seeking a highly skilled Senior Data Integration Specialist to join our Digital Team. As a key member of our team, you will work closely with functional SMEs to understand business requirements and design technical solutions.Key ResponsibilitiesDevelop and maintain data pipelines using Informatica (DEI/BDM) or equivalent...

  • Senior Manager, Data Engineering Lead

    6 days ago

    Atlanta, Georgia, United States T-Mobile Full time

    About the RoleWe are seeking a highly skilled Senior Manager, Data Engineering to lead our Data Engineering team in delivering high-quality data warehouse and business analytics solutions. As a key member of our team, you will be responsible for overseeing the planning and roadmap for the Data Engineering team, ensuring the quality of data pipelines, and...

  • Senior Manager, Data Engineering and Analytics

    7 days ago

    Atlanta, Georgia, United States Disability Solutions Full time

    About the RoleWe are seeking a highly skilled Senior Manager, Data Engineering and Analytics to lead our Data Engineering team. As a key member of our organization, you will be responsible for overseeing the analysis, architecture, design, and development of data warehouse and business analytics solutions.Key ResponsibilitiesManage a team of data engineers...

  • Cloud Data Engineer

    1 week ago

    Atlanta, Georgia, United States Voto Consulting Full time

    Job Title: Cloud Data EngineerJob Type: Full-timeLocation: Remote (Hybrid)Rate: $50/hourInterview: WebExAbout the Job:This is a long-term contract position with Voto Consulting LLC, a leading consulting firm in the industry. We are seeking an experienced Cloud Data Engineer to join our team and work on a project that involves integrating Cloud SQL, Big Query...

  • GCP Data Engineer

    1 day ago

    Atlanta, Georgia, United States Voto Consulting Full time

    GCP Data Engineer Job DescriptionWe are seeking a highly skilled GCP Data Engineer to join our team at Voto Consulting LLC.Job Summary:The successful candidate will have 8+ years of experience in integrating Cloud SQL, Big Query with other Google Cloud Platform (GCP) services such as Cloud Storage, Cloud Functions, Pub-Subs, Dataflow, and Data Studio for...

  • Senior Data Science Engineer

    1 week ago

    Atlanta, Georgia, United States Publicis Sapient Full time

    About the RoleWe are seeking a highly skilled Senior Data Science Engineer to join our team at Publicis Sapient. As a key member of our data science team, you will play a critical role in driving our clients' digital transformation strategies.Key ResponsibilitiesLead the development and implementation of advanced machine learning models, ensuring a high...

Senior Big Data Engineer

4 months ago

Atlanta, Georgia, United States Norfolk Southern Corp. Full time

Norfolk Southern offers a unique opportunity to be part of our proud legacy that spans nearly 200 years. We are a customer-centric, operations-driven team dedicated to advancing safety, serving communities, and driving innovation for tomorrow's rail. As part of Norfolk Southern, you'll join a collaborative team where there are opportunities for growth across the organization. We are building a culture where everyone can thrive by owning and driving exceptional results, being humble and leading with trust, serving our customers with excellence, and collaborating and coaching to win.

Job Description

Norfolk Southern Corporation is currently seeking a Senior Big Data Engineer with an affinity for working with others to create successful solutions. Join a smart, highly skilled team with a passion for technology, where you will work on our state-of-the-art Big Data Platforms. This Senior Data Engineer with a balance of theoretical business intelligence and data analytics knowledge and hands-on experience and exposure to real-world BI problem-solving. In this role, you will participate in all phases of the Data Engineering life cycle and will independently and collaboratively write project requirements, architect solutions, and perform data ingestion development and support duties.

In this role, you will work with people from many areas of the company to understand their data and BI needs and perform detailed requirements gathering along with analysis in partnership with our team of Data Modelers. Engage our BI development team to agree on effort estimates and then oversee the progress of the project through its completion.

Responsibilities:

  • Defines data requirements, gather, and wrangle large scale of structured and unstructured data, and validate data by running various data tools in the Data Environment.
  • Supports the standardization, customization and ad-hoc data analysis, and will develop the mechanisms to ingest, analyze, validate, normalize and clean data.
  • Creates data policy and develop interfaces and retention models which requires synthesizing or anonymizing data.
  • Implements statistical data quality procedures on new data sources, and by applying rigorous iterative data analytics, supports Data Scientists and analytics and insights creation in data sourcing and preparation to visualize data and synthesize insights of commercial value.
  • Develops and maintains data engineering best practices and contributes to Insights on data analytics and visualization concepts, methods and techniques.
  • Works closely with the data science and business intelligence teams to develop data models and pipelines for research, reporting, and machine learning.
  • Builds data pipelines that clean, transform, and aggregate data from disparate sources.
  • Employs a variety of languages and tools (e.g. scripting languages) to marry systems together.
  • Engages with business teams to gather requirements and design data solutions.
  • Understands overall architecture and industry data technology.
  • Mentors team of more Junior Data Engineers.
  • Refines and prioritizes team backlog.
  • Collaborates across multiple projects to provide data engineering expertise across teams.
  • Presents to leadership around solutions that are in-progress or being maintained.
  • Analyzes most relevant insights and shares with leadership to provide strategic recommendations for the business.
  • Leads a team of data engineers and act as a key senior contributor to a data engineering project.
  • Drives innovative technology solutions through thought leadership on emerging trends.
  • Applies knowledge of Data Architecture components, leads project teams from requirements to implementation.
Skill Required
  • 7+ years of experience in a data engineering role with a track record of manipulating, processing, and extracting value from large datasets.
  • 7+ years of Big Data tools like Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS.
  • 5+ years building, testing, and optimizing 'Big Data' data ingestion pipelines, architectures, and data sets with Tibco, IBM or others.
  • Advanced SQL, data ingestion frameworks, data modeling, and working with big data.
  • High-velocity high-volume stream processing with Apache Kafka and Spark Streaming.
  • NoSQL databases, including HBASE and/or Cassandra.
  • ETL experience with Scala (and/or Python) and PySpark/Scala-Spark
  • Agile Scrum, Kanban or SAFe experience.

Skills Desired

  • Databricks UI, Managing Databricks Notebooks, Delta Lake with Python, Delta Lake with Spark SQL, Delta Live Tables, Unity Catalog.
  • Python (and/or Scala) and PySpark/Scala-Spark.
  • Database solutions like Snowflake, Kudu/Impala, Delta Lake or BigQuery.
  • NoSQL databases, including HBASE and/or Cassandra.
  • Azure, AWS Serverless technologies, like, S3, Kinesis/MSK, lambda, and Glue.
  • Messaging Platforms like Kafka, Amazon MSK & TIBCO EMS or IBM MQ Series.
  • Strong understanding of Relational & Dimensional modeling.
  • Experience with GIT code versioning software.
  • Experience with REST API and Web Services.

Education Desired

Bachelor's Degree, preferably in Information Systems, Computer Science, Computer Information Systems or related field.

Work Conditions

Environment: Office Remote, ideal candidate must be based within 60 miles Atlanta.

Shift Work: No

On-Call: Yes

Weekend Work: No


Company Overview

Since 1827, Norfolk Southern Corporation (NYSE: NSC) and its predecessor companies have safely moved the goods and materials that drive the U.S. economy. Today, it operates a customer-centric and operations-driven freight transportation network. Committed to furthering sustainability, Norfolk Southern helps its customers avoid 15 million tons of yearly carbon emissions by shipping via rail. Its dedicated team members deliver more than 7 million carloads annually, from agriculture to consumer goods, and is the largest rail shipper of auto products and metals in North America. Norfolk Southern also has the most extensive intermodal network in the eastern U.S., serving a majority of the country's population and manufacturing base, with connections to every major container port on the Atlantic coast as well as the Gulf of Mexico and Great Lakes. Learn more by visiting

At Norfolk Southern, we believe in celebrating our individuality. By leveraging the unique backgrounds and viewpoints of our employees, we can create a culture of innovation, respect, and inclusion. We know that employees thrive in a workplace where differing viewpoints, ideas, and experiences are freely shared and valued. As such, we encourage all employees to contribute their distinctive skills and capabilities to our organization.

Equal employment opportunities are available to all applicants regardless of race, color, religion, age, sex, national origin, disability status, genetic information, veteran status, sexual orientation, and gender identity. Together, we power progress.