Cloudera Big Data administrator

2 weeks ago

Reston, United States Resource Informatics Group Full time

Client Company
CareFirst Client FEPOC
Job Description
Position Title: Cloudera Big Data administrator

Location: Reston VA

Duration : 12 months of contract with possibility of extension

Experience Summary:

This is a Cloudera Big Data administrator position and not a developer position. Experience with building Cloudera cluster, setting up Nifi, Solr, HBase, Kafka. Setting up the High Availability of the Services like Hue, Hive, HBase REST, SOLR and IMPALA on top of the all-new clusters that were built on the BDPaas Platform. Be able to write the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions. Monitoring the health of all the services running in the production cluster using the Cloudera Manager. Performing/Accessing the databases, metastore tables and writing Hive, Impala queries using HUE. Responsible for monitoring the health of the Services on top of all clusters. Working closely with different teams like Application development team, Security team, Platform Support to identify and implement the Configurational changes that are needed on top of the cluster for better performance of the services. Experience with CDP Public Cloud is a PLUS.

Skills Must Require:

Cloudera CDP v7.x

Apache Kafka - strong Administration & troubleshooting skills

Kafka Streams API

stream processing with KStreams & Ktables

Kafka integration with MQ

Kafka broker management

Topic/ offset management

Apache Nifi - Administration

Flow management

registry server management

controller service management

Nifi to kafka /Hbase /solr integration

Hbase - administration

database management

troubleshooting

Solr - administration

managing Logging level

managing shards & high availability

Collection management

Rectify resource intensive & long running solr queries

Additional Skills includes:
• Ensure Cloudera installation and configuration is at optimal specifications (CDP, CDSW, Hive, Spark, NiFi).
• Perform critical data migrations from CDH to CDP.
• Design and implement big data pipelines and automated data flows using Python/R and NiFi.
• Assist and provide expertise as it pertains to automating the entire project lifecycle.
• Perform incremental updates and upgrades to the Cloudera environment.
• Assist with new use cases (i.e., analytics/Client, data science, data ingest and processing), Infrastructure (including new cluster deployments, cluster migration, expansion, major upgrades, COOP/DR, and security).
• Assist in testing, governance, data quality, training, and documentation efforts.
• Move data and use YARN to allocate resources and schedule jobs.
• Manage job workflows with Oozie and Hue.
• Implement comprehensive security policies across the Hadoop cluster using Ranger.
• Configure and manage Cloudera Data Science Workbench using Cloudera Manager.
• Troubleshoot potential issues with Kerberos, TLS/SSL, Models, and Experiments, as well as other workload issues that data scientists might encounter once the application is running.
• Supporting the Big Data / Hadoop databases throughout the development and production lifecycle.
• Troubleshooting and resolving database integrity issues, performance issues, blocking and deadlocking issues, replication issues, log shipping issues, connectivity issues, security issues, performance tuning, query optimization, using monitoring and troubleshooting tools.
• Create, test, and implement scripting for automation support.
• Experience in working with Kafka ecosystem (Kafka Brokers, Connect, Zookeeper) in production is ideal
• Implement and support streaming technologies such as Kafka, Spark & Kudu

  • Lead Cloudera Administrator

    1 week ago

    Reston, United States GSSR Inc Full time

    Job DescriptionJob DescriptionVaccination:Need Fully vaccinated candidates against Covid-19DescriptionSummary: Our client is seeking a Lead Big Data Administrator to supportits existing Cloudera / AWS data platform while the existing team supports theongoing enterprise migration to AWS. This resource will be responsible forconfiguring, troubleshooting, and...

  • Big Data Administrator Lead

    2 weeks ago

    Reston, United States Ageatia Global Solutions Full time

    PURPOSE This is a Big Data Administrator Lead position and not a developer position. The Lead Data Engineer is responsible for orchestrating, deploying, maintaining and scaling cloud OR on-premise infrastructure targeting big data and platform data management (e.g., data warehouses, data lakes) including data access APIs. Prepares and manipulates data...

  • Big Data Administrator Lead

    4 days ago

    Reston, United States Ageatia Global Solutions Full time

    PURPOSE This is a Big Data Administrator Lead position and not a developer position. The Lead Data Engineer is responsible for orchestrating, deploying, maintaining and scaling cloud OR on-premise infrastructure targeting big data and platform data management (e.g., data warehouses, data lakes) including data access APIs. Prepares and manipulates data...

  • Senior AbInitio Developer

    2 days ago

    Reston, United States QuantumBricks Full time

    Job Title: Senior AbInitio Developer (Must have BigData OR Cloudera) Loc: Reston, VA (Initially remote) Exp: 5+ Yrs Job Description NOTE:- Should be willing to go office for team meetings once a quarter. Prefer someone from Washington, Maryland OR Virginia only. Required Skills Priority to anyone with Cloudera background/exp and AWS Needs Ab Initio and Big...

  • Data Engineer

    2 weeks ago

    Reston, United States Ageatia Global Solutions Full time

    PURPOSE: This is a Big Data Administrator Lead position and not a developer position. The Lead Data Engineer is responsible for orchestrating, deploying, maintaining and scaling cloud OR on-premise infrastructure targeting big data and platform data management (e.g., data warehouses, data lakes) including data access APIs. Prepares and manipulates data...

  • Sr Data Engineer

    2 weeks ago

    Reston, United States Ageatia Global Solutions Full time

    PURPOSE: The Senior Data Engineer is responsible for orchestrating, deploying, maintaining and scaling cloud OR on-premise infrastructure targeting big data and platform data management (Relational and NoSQL, distributed and converged) with emphasis on reliability, automation and performance. This role will focus on developing solutions and helping...

  • Big Data Analytics Senior Developer

    1 month ago

    Reston, United States Atechstar Full time

    Key ResponsibilitiesThe right candidate will be expected to be a significant player in the project evolution & deployment shouldering the following responsibilitiesContribute to full development life cycle including requirements analysis functional design technical design programming testing documentation implementation and on-going technical...

  • Big Data Analytics Senior Developer

    1 month ago

    Reston, Virginia, United States Atechstar Full time

    Key ResponsibilitiesThe right candidate will be expected to be a significant player in the project evolution & deployment shouldering the following responsibilitiesContribute to full development life cycle including requirements analysis functional design technical design programming testing documentation implementation and on-going technical support Analyse...

  • Sr. BIG Data Solutions Architect

    4 days ago

    Reston, United States Diverse Lynx Full time

    Job Description Responsibilities: Deep expertise in Big Data, data warehouse, data analytics projects, and/or any Information Management related projects Prior experience building large scale enterprise data architectures using commercial and/or open source Data Analytics technologies Facilitate the establishment and execution of the roadmap and vision for...

  • Java Developer

    5 days ago

    Reston, United States Saxon Global Full time

    This is a 12 month contract with Carefirst. Remote to start. All visa but dont submit H1B/OPT. Linkedin. Must have: Java - 7+ Years Python: 2-3 Years Scala: 1-2 Years Requirments: This position requires a BA/BS in Computer Science, Information Systems, Information Technology or related field with 7+ years of prior experience in software development,...

  • Sr Data Engineer

    3 days ago

    Reston, United States Three Point Solutions Full time

    Job DescriptionJob DescriptionPosition: Sr Data EngineerClient: Health Care IndustryLocation: Reston, Virginia (They are required to come into the office at least once a week)Duration: 12 monthsGeneral InformationJob Description: The Senior Data Engineer is responsible for orchestrating, deploying, maintaining and scaling cloud OR on premise infrastructure...

  • Big Data Lead

    7 days ago

    Reston, United States Diverse Lynx Full time

    Location: Stamford, Connecticut -Weekly Thrice(Hybrid) Job Description Senior Data Engineer (8+ yrs of exp) Responsibilities - Experienced Data engineer responsible for developing, overseeing, organizing, storing, and analyzing data and data systems. • Responsible for Cloud migrations for data pipelines using AWS and Snowflake • Participate in all...

  • Big Data Lead

    4 days ago

    Reston, United States Diverse Lynx Full time

    Location: Stamford, Connecticut -Weekly Thrice(Hybrid) Job Description Senior Data Engineer (8+ yrs of exp) Responsibilities - Experienced Data engineer responsible for developing, overseeing, organizing, storing, and analyzing data and data systems. • Responsible for Cloud migrations for data pipelines using AWS and Snowflake • Participate in all...

  • Sr Data Engineer

    3 days ago

    Reston, United States Three Point Solutions Full time

    Job DescriptionJob DescriptionPosition: Sr Data EngineerClient: Health Care IndustryLocation: Reston, Virginia (They are required to come into the office at least once a week)Duration: 12 monthsGeneral InformationJob Description: The Senior Data Engineer is responsible for orchestrating, deploying, maintaining and scaling cloud OR on premise infrastructure...

  • Big Data Engineer

    1 month ago

    Reston, Virginia, United States Atechstar Full time

    Job descriptionRequired SkillsBachelors degree in Computer Science and 5+ years of experience or equivalent industry experience Deep knowledge of two or more functional or scripting programming languages Java Python Scala or R 3+ years of Hadoop Hive Spark or related MapReduce framework development experience Extensive experience with distributed SQL...

  • Data Administrator

    1 month ago

    Reston, United States Wipro Limited Full time

    Job ResponsibilitiesDatabase Administration Architect and implement database applications/infrastructures typical of those required to support a large organization and capable of adapting to database structure changes in a dynamic business setting. Perform activities related to administration of LSU enterprise databases and other project-based databases ...

  • Data Administrator

    1 month ago

    Reston, Virginia, United States Wipro Limited Full time

    Job ResponsibilitiesDatabase Administration Architect and implement database applications/infrastructures typical of those required to support a large organization and capable of adapting to database structure changes in a dynamic business setting. Perform activities related to administration of LSU enterprise databases and other project-based databases...

  • Senior AbInitio Developer

    7 days ago

    Reston, United States Resource Informatics Group Inc Full time

    Job DescriptionJob DescriptionRole:-Senior AbInitio DeveloperLocation:Reston,VAVisa:- NO OPT/CPTDuration:- 12 Months Interview:- Phone/SkypeJob Description :Total of 7+ Years of IT Experience predominantly in Data Integration/ Data Warehouse areaMust have 5 years of ETL Design and Development experience using Ab Initio1-2 years of Data Integration project...

  • Senior AbInitio Developer

    4 weeks ago

    Reston, United States Resource Informatics Group Inc Full time

    Job DescriptionJob DescriptionRole:-Senior AbInitio DeveloperLocation:Reston,VAVisa:- NO OPT/CPTDuration:- 12 Months Interview:- Phone/SkypeJob Description :Total of 7+ Years of IT Experience predominantly in Data Integration/ Data Warehouse areaMust have 5 years of ETL Design and Development experience using Ab Initio1-2 years of Data Integration project...

  • Senior Ab Initio Developer

    3 days ago

    Reston, United States Abidi Solutions Full time

    Job DescriptionJob DescriptionJob Title: Senior Ab Initio Developer Location: Hybrid - Onsite in Reston, VA 1x per week. Candidates must reside in DV region Duration: 12 monthsProject Summary: Abidi Solutions seeks a Senior Ab Initio Developer to assist in developing data pipelines that transfer data from multiple mainframe source systems to various cloud...