Hadoop Data Engineer

3 weeks ago


Pittsburgh, Pennsylvania, United States Tata Consultancy Services Full time
Job Title: Hadoop Data Engineer

We are seeking a highly skilled Hadoop Data Engineer to join our team at Tata Consultancy Services.

Key Responsibilities:

  • Design and develop large-scale data processing systems using Hadoop ecosystem technologies.
  • Develop and maintain data pipelines using PySpark for data processing and analytics.
  • Implement data warehousing and querying using Hive.
  • Work with Livy for interactive job submission and shell scripting for automation and orchestration.
  • Collaborate with cross-functional teams to develop and deploy machine learning models using AWS SageMaker.
  • Develop strong problem-solving skills and ability to work independently in a fast-paced environment.

Requirements:

  • Deep understanding of Hadoop ecosystem technologies.
  • Strong proficiency in PySpark for data processing and analytics.
  • Experience with Hive for data warehousing and querying.
  • Knowledge of Livy for interactive job submission and shell scripting for automation and orchestration.
  • Hands-on experience with AWS SageMaker for machine learning model development and deployment.
  • Excellent communication and interpersonal skills.

About Us:

Tata Consultancy Services is a leading global IT services company that helps clients transform and grow their businesses.


  • Big Data Engineer

    3 weeks ago


    Pittsburgh, Pennsylvania, United States Tata Consultancy Services Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Tata Consultancy Services. The ideal candidate will have extensive experience with development and designing of big data platforms using Hadoop and Scala.Key ResponsibilitiesDesign and develop big data platforms using Hadoop and Scala.Collaborate with cross-functional teams to...

  • Big Data Engineer

    2 weeks ago


    Pittsburgh, Pennsylvania, United States Tata Consultancy Services Full time

    Job SummaryWe are seeking a highly skilled Data Processing Specialist to join our team at Tata Consultancy Services. As a key member of our big data infrastructure team, you will be responsible for designing, developing, and documenting Hadoop applications.Key ResponsibilitiesDesign and develop Hadoop applications to process large datasetsInstall, configure,...


  • Pittsburgh, Pennsylvania, United States Saxon Global Full time

    Hadoop Application Support EngineerThis is a full-time role with Saxon Global, offering a hybrid onsite and remote work arrangement. The preferred work locations are Pittsburgh, PA, or Cleveland, OH. The successful candidate will be responsible for:Key Responsibilities:Data replication and managementExperience and knowledge with LinuxSetting up new clusters...


  • Pittsburgh, Pennsylvania, United States Tata Consultancy Services Full time

    **Job Summary**Tata Consultancy Services is seeking an experienced Hadoop Developer Specialist to join our team.As a Hadoop Developer Specialist, you will be responsible for designing and implementing large-scale data processing systems using Hadoop ecosystem technologies.**Key Responsibilities**Design and implement data processing pipelines using Hadoop...


  • Pittsburgh, Pennsylvania, United States Diverse Lynx Full time

    Role Overview:As a Hadoop Ecosystem Specialist at Diverse Lynx LLC, you will be responsible for leading the development and implementation of big data analytics solutions using Hadoop technologies. The ideal candidate will have a deep understanding of the Hadoop ecosystem, including Hadoop Distributed File System (HDFS), YARN, and MapReduce. Additionally,...


  • Pittsburgh, Pennsylvania, United States Tata Consultancy Services Full time

    Hadoop Admin Job DescriptionWe are seeking a skilled Hadoop Admin to join our team at Tata Consultancy Services.Key Responsibilities:Administer and manage Hadoop clusters to ensure optimal performance and data integrity.Provide support for Hadoop Big Data Platform and troubleshoot issues.Develop and maintain scripts using Python, Linux, and CA7.Monitor and...


  • Pittsburgh, Pennsylvania, United States Diverse Lynx Full time

    Hadoop Platform Admin RoleAt Diverse Lynx LLC, we are seeking a skilled Hadoop Platform Admin to join our team. The ideal candidate will have a strong background in Hadoop Cluster monitoring, issue analysis, and vendor/Cloudera collaboration. Key responsibilities include implementing change tasks, managing permissions, and ensuring data synchronization...


  • Pittsburgh, Pennsylvania, United States Tata Consultancy Services Full time

    Hadoop Admin RoleWe are seeking a skilled Hadoop Admin to join our team at Tata Consultancy Services. The ideal candidate will have experience in Hadoop Admin, Hadoop Big Data Platform support, and proficiency in Python, Linux, and CA7.Key Responsibilities:Monitor and maintain Hadoop clusters.Support Hadoop Big Data Platform.Troubleshoot issues and...


  • Pittsburgh, Pennsylvania, United States Syntricate Technologies Full time

    Hadoop Platform AdministratorAt Syntricate Technologies, we are seeking a highly skilled Hadoop Platform Administrator to join our team. The ideal candidate will have experience with Hadoop Big Data Platform support, Python, Linux, and monitoring tools like Dynatrace, Monpo, Logscale, and Grafana. The successful candidate will be responsible for ensuring the...


  • Pittsburgh, Pennsylvania, United States Tata Consultancy Services Full time

    Hadoop Developer Job DescriptionWe are seeking a skilled Hadoop Developer to join our team at Tata Consultancy Services.Key Responsibilities:Administer and support Hadoop Big Data PlatformMonitor and troubleshoot Hadoop clustersImplement changes to Hadoop tasks, including permissions, file deletion, folder creation, and space cleanupEnsure data transfer...


  • Pittsburgh, Pennsylvania, United States Diverse Lynx Full time

    Hadoop Platform AdministrationAs a Hadoop Platform Administrator at Diverse Lynx LLC, you will be responsible for monitoring and maintaining Hadoop clusters, analyzing issues, and collaborating with vendors like Cloudera to resolve complex problems. Your tasks will include implementing changes, managing permissions, deleting files, creating folders, and...


  • Pittsburgh, Pennsylvania, United States Tata Consultancy Services Full time

    Hadoop Admin Job DescriptionWe are seeking a skilled Hadoop Admin to join our team at Tata Consultancy Services.Key Responsibilities:Administer and manage Hadoop clusters to ensure optimal performance and efficiency.Monitor and troubleshoot Hadoop-related issues, working closely with vendors and Cloudera teams.Develop and implement monitoring tools and...

  • Senior Data Engineer

    3 weeks ago


    Pittsburgh, Pennsylvania, United States Saxon Global Full time

    Job SummaryThis is a full-time role with Saxon Global, offering a hybrid work arrangement with 2 days in the office and 3 days remote. The ideal candidate will be based near Pittsburgh, Cleveland, New Jersey, DC, Atlanta, Raleigh NC, or Columbus Dallas.Key ResponsibilitiesAs a Lead Data Scientist, you will be responsible for leading data transformation, data...


  • Pittsburgh, Pennsylvania, United States Tata Consultancy Services Full time

    Hadoop Platform AdminWe are seeking a highly skilled Hadoop Platform Admin to join our team at Tata Consultancy Services.Experience in Hadoop Admin is essential for this role.Responsibilities include Hadoop Big Data Platform support and ensuring the smooth operation of our Hadoop clusters.Proficiency in Python, Linux, and CA7 is required.Familiarity with...


  • Pittsburgh, Pennsylvania, United States Diverse Lynx Full time

    Hadoop Platform AdministratorAt Diverse Lynx LLC, we are seeking a skilled Hadoop Platform Administrator to join our team. The ideal candidate will have experience with Hadoop Cluster monitoring, issue analysis, and working with vendors like Cloudera. Key responsibilities include implementing change tasks, managing permissions, and ensuring data transfer...


  • Pittsburgh, Pennsylvania, United States System One Holdings, LLC Full time

    Hadoop Administrator Job DescriptionWe are seeking a highly skilled Hadoop Administrator to join our team at System One Holdings, LLC. As a key member of our data management team, you will be responsible for ensuring the smooth operation of our big data infrastructure.Key Responsibilities:Provide production support for tier 1 applications and enterprise...


  • Pittsburgh, Pennsylvania, United States System One Full time

    Hadoop Administrator Job DescriptionWe are seeking a skilled Hadoop Administrator to join our team at System One. The successful candidate will be responsible for working across multiple clusters, supporting multitiered applications in the big data domain, and providing production support for tier 1 applications and enterprise platforms.Key...


  • Pittsburgh, Pennsylvania, United States Pyramid Consulting Group Full time

    Job SummaryAs a Senior Hadoop Developer at Pyramid Consulting Group, you will be responsible for building out capabilities for the Hadoop platform. This includes testing new capabilities, working with customers to resolve issues, creating user documentation, and optimizing code. You will also be responsible for planning and executing user program migration....


  • Pittsburgh, Pennsylvania, United States System One Full time

    Hadoop Administrator Job DescriptionWe are seeking a skilled Hadoop Administrator to join our team at System One. The successful candidate will be responsible for managing and maintaining our big data infrastructure, ensuring the smooth operation of our multitiered applications.Key Responsibilities:Provide production support for tier 1 applications and...

  • Data Engineer

    2 weeks ago


    Pittsburgh, Pennsylvania, United States PNC Full time

    Job SummaryPNC is seeking a highly skilled Data Engineer to join our Enterprise Fraud Organization. As a key member of our team, you will be responsible for designing, building, and maintaining data services to support our business objectives. Key Responsibilities- Design and develop data services to meet business requirements- Collaborate with...