Hadoop Cluster Architect

3 days ago


Fremont, California, United States Info Way Solutions Full time
Job Title: Lead Hadoop Admin

At Info Way Solutions, we are seeking a highly skilled and experienced Lead Hadoop Admin to join our team. As a key member of our infrastructure team, you will be responsible for the implementation and ongoing administration of our Hadoop infrastructure.

Key Responsibilities:
  • Implement and maintain Hadoop clusters, ensuring optimal performance and reliability.
  • Monitor and troubleshoot Hadoop infrastructure, identifying and resolving issues promptly.
  • Provision and manage the lifecycle of multiple clusters, including EMR and EKS.
  • Implement infrastructure monitoring, logging, and alerting using Prometheus, Grafana, and Splunk.
  • Perform performance tuning of Hadoop clusters and workloads, ensuring optimal capacity planning.
  • Scale clusters in production, ensuring seamless operation in 18/5 or 24/5 environments.
  • Monitor Hadoop cluster connectivity and security, file system management, and monitoring.
  • Investigate and analyze new technical possibilities, tools, and techniques to improve efficiency and productivity.
  • Collaborate with application teams to install operating system and Hadoop updates, patches, and version upgrades.
  • Maintain central dashboards for system, data, utilization, and availability metrics.
Requirements:
  • 8-12 years of total experience, with at least 5 years of work experience in developing, maintaining, optimizing, and resolving Hadoop clusters.
  • Strong knowledge of Linux/Unix OS services, administration, shell scripting, and programming languages such as Python, Scala, Java, and R.
  • Experience in Hadoop (Map Reduce, Hive, Pig, Spark, Kafka, HBase, HDFS, HCatalog, Zookeeper, and Oozie/Airflow).
  • Experience in Hadoop security (Kerberos, Knox, TLS) and SQL and NoSQL databases (HBASE) with performance optimization.
  • Excellent oral and written communication, presentation, analytical, and problem-solving skills.
What We Offer:

As a Lead Hadoop Admin at Info Way Solutions, you will have the opportunity to work with a talented team, develop your skills, and contribute to the success of our organization.



  • Fremont, California, United States Info Way Solutions Full time

    Job Title: Lead Hadoop AdminWe are seeking a highly skilled Lead Hadoop Admin to join our team at Info Way Solutions. As a key member of our data engineering team, you will be responsible for designing, implementing, and maintaining large-scale Hadoop infrastructure.Key Responsibilities:Implement and administer Hadoop infrastructure, ensuring high...


  • Fremont, California, United States Info Way Solutions Full time

    Hadoop Admin Job DescriptionWe are seeking a highly skilled Hadoop Admin to join our team at Info Way Solutions, LLC. As a Hadoop Admin, you will be responsible for the implementation and ongoing administration of Hadoop infrastructure.Key Responsibilities:Implement and maintain Hadoop clusters, ensuring high availability and performance.Provision and manage...


  • Fremont, California, United States Info Way Solutions Full time

    Job Title: Lead Hadoop AdminWe are seeking a highly skilled Lead Hadoop Admin to join our team at Info Way Solutions. As a key member of our infrastructure team, you will be responsible for the implementation and ongoing administration of our Hadoop infrastructure.Key Responsibilities:Implement and maintain Hadoop clusters, ensuring high availability and...

  • Hadoop Administrator

    2 weeks ago


    Fremont, California, United States Info Way Solutions Full time

    Hadoop Admin Job DescriptionWe are seeking a highly skilled Hadoop Admin to join our team at Info Way Solutions. As a Hadoop Admin, you will be responsible for the implementation and ongoing administration of Hadoop infrastructure.Key Responsibilities:Implement and maintain Hadoop clusters, ensuring high availability and performance.Monitor and troubleshoot...

  • Hadoop Administrator

    4 weeks ago


    Fremont, California, United States Info Way Solutions Full time

    Hadoop Admin Job DescriptionWe are seeking a highly skilled Hadoop Admin to join our team at Info Way Solutions. The successful candidate will be responsible for the implementation and ongoing administration of Hadoop infrastructure.Key Responsibilities:Implement and maintain Hadoop clusters, ensuring optimal performance and scalability.Monitor and...

  • Hadoop Administrator

    4 weeks ago


    Fremont, California, United States Info Way Solutions Full time

    Hadoop Admin Job DescriptionWe are seeking a highly skilled Hadoop Admin to join our team at Info Way Solutions, LLC.Key Responsibilities:Implement and administer Hadoop infrastructureMaintain and troubleshoot clusters, ensuring proper backup and recovery strategiesProvision and manage the lifecycle of multiple clusters, including EMR and EKSMonitor...


  • Fremont, California, United States Info Way Solutions Full time

    Job Title: Hadoop AdministratorJob Summary:We are seeking a highly skilled Hadoop Administrator to join our team at Info Way Solutions. The successful candidate will be responsible for the implementation and ongoing administration of Hadoop infrastructure, including cluster maintenance, troubleshooting, and performance tuning.Key Responsibilities:Implement...


  • Fremont, California, United States Info Way Solutions Full time

    Job Title: Hadoop Infrastructure SpecialistJob Summary:We are seeking a highly skilled Hadoop Infrastructure Specialist to join our team at Info Way Solutions. The successful candidate will be responsible for the implementation and ongoing administration of Hadoop infrastructure, including cluster maintenance, troubleshooting, and performance tuning.Key...


  • Fremont, California, United States Kforce Technology Staffing Full time

    Job Title: Senior HPC ArchitectKforce Technology Staffing is seeking a highly skilled Senior HPC Architect to join our team in Fremont, CA.Job Summary:As a Senior HPC Architect, you will be responsible for designing, analyzing, and improving HPC applications and systems. You will work closely with our IT team to provide technical assistance and guidance on...


  • Fremont, California, United States Info Way Solutions Full time

    Job Title:Cloud Storage Solutions ArchitectJob Summary:We are seeking a skilled Cloud Storage Solutions Architect to design and implement private cloud-based Cloud Storage solutions using Ceph. The ideal candidate will have experience with Ceph, OpenStack, and Kubernetes, as well as strong programming and scripting skills.Key Responsibilities:Design and...


  • Fremont, California, United States Deeproute Full time

    Job OverviewDeeproute.ai is revolutionizing the transportation industry with its Level 4 full-stack self-driving solution. We're seeking a talented Cloud Infrastructure Architect to design and build data-centric infrastructures and tools for continuous development, testing, and deployment on cloud.Key Responsibilities:Design and implement scalable cloud...


  • Fremont, California, United States Exxact Full time

    Job Title: Senior Systems EngineerExxact Corporation is seeking a highly skilled Senior Systems Engineer to design and build complex systems, ranging from stand-alone workstations and servers to multi-rack CPU, GPU, and storage clusters.Key Responsibilities:Design of systems and clusters from component level, including network fabric, switching, power...


  • Fremont, California, United States Exxact Full time

    Exxact Corporation is seeking a highly skilled Systems Engineer to design and build complex systems, including computational and storage clusters.Key Responsibilities:Hardware Design:Design and assemble systems and clusters from component level, including network fabric, switching, power infrastructure, and cabling.Inspect and perform quality control on...


  • Fremont, California, United States Maintec Technologies Full time

    Job Summary Maintec Technologies is seeking a skilled Cassandra Database Administrator to join our team. The ideal candidate will have a strong background in Cassandra, Agile, and ITIL application processes. Key Responsibilities • Design and implement Cassandra clusters for high-performance applications • Develop and maintain scripts for automation, log...


  • Fremont, California, United States META Full time

    Enterprise Systems EngineerAt Meta, we're building the infrastructure for our high-traffic, high-visibility services. As an Enterprise Systems Engineer, you'll build the foundation for our business to thrive. You'll work closely with our Reality Labs Research teams to implement mid-sized GPU clusters tailored to meet individual team needs.Key...


  • Fremont, United States Blue Signal Search Full time

    Our client, a leading provider of HPC Solutions for more than 3 decades, offering solutions ranging from workstations, servers, and clusters, is seeking a Senior Solutions Architect with a strong expertise in end to end systems solutioning, especially in the Artificial Intelligence (AI) space. This role focuses on building and nurturing relationships with...

  • Data Scientist

    2 months ago


    Fremont, United States Pivotal Systems Full time

    The ideal candidate's favorite words are learning, data, scale, and agility. You will leverage your strong collaboration skills and ability to extract valuable insights from highly complex data sets to ask the right questions and find the right answers.  ResponsibilitiesAnalyze raw data: assessing quality, cleansing, structuring for downstream processing...


  • Fremont, United States Blue Signal Search Full time

    Our client, a leading provider of HPC Solutions for over three decades, offering a comprehensive range of workstations, servers, and clusters, is seeking a Head of HPC Solution Architecture. This senior leadership role demands strong expertise in end-to-end systems solutioning, particularly in the Artificial Intelligence (AI) space. The primary focus is on...

  • Senior HPC Archtitect

    2 weeks ago


    Fremont, United States Kforce Technology Staffing Full time

    RESPONSIBILITIES:Kforce has a client that is seeking a Senior HPC Architect in Fremont, CA.Summary:As an HPC Architect, you will be at the forefront of designing, analyzing, and improving HPC applications and systems. As a crucial member of our IT team, you will contribute your technical assistance and guidance to projects for HPC. Acting as a technical...