Hadoop Engineer

Responsibilities. Write software to interact with HDFS and MapReduce. Assess requirements and evaluate existing solutions. Build operate monitor and troubleshoot Hadoop infrastructure. Develop tools and libraries and maintain processes for other engineers to access data and write MapReduce programs.

Required Skills Data engineer main skills Good knowledge of Data warehousing with expertise in data modeling and ETL. Advanced skills in SQL and HQL Should be good at these core tech AWS/EMR/Spark Hadoop Hive Vertica Qlikview Tableau Should have exp in CI/CD

Role Sr. Data Engineer (8+ years) Location SFO CA Duration Long TermExperience & Skills Extensive experience with Hadoop (or similar) Ecosystem (Map Reduce HDFS Hive Spark Pig HBase) Proficient in at least one of the SQL languages (MySQL PostgreSQL SqlServer Oracle) Good understanding of SQL Engine and able to conduct advanced performance tuning Strong...

Alluxio is charging full steam to help customers navigate digital transformation.Features like intelligent caching, unified namespace, and data management provide agility and cost efficiency to customers in financial services, high-tech, retail, and telecommunications.Alluxio is trusted by Meta, Uber, Tencent, Tiktok, Alibaba, Expedia, Rakuten, Microsoft,...

The Smule mission is to connect people all over the world through the joy of making music at massive social scale. Music is much more than just listening... It's about creating, sharing, discovering, participating, and connecting with people. It is a social network with the power to break down barriers, touch souls and bring people together from all over the...

Lead Data Engineer - Python Expert$45k – $90k • 0.1% – 0.25%About Our CompanyWe are a VC backed Fintech Startup in the Bay Area.The RoleStaff/Lead Software EngineerThis is a core and very important role for the company. You'll have a lot of agency in architecting and implementing a corner store of modern financial infrastructure.You'll be enhancing and...

Who is AiDash? AiDash is making critical infrastructure industries climate-resilient and sustainable with satellites and AI. Using our full-stack SaaS solutions, customers in electric, gas, and water utilities, transportation, and construction are transforming asset inspection and maintenance - and complying with biodiversity net gain mandates and carbon...

Key QualificationsDemonstrated willingness to embrace the dynamic environment of a startup, coupled with a commitment to contributing significantly to our growth.A deep-seated curiosity and passion for technology, with a fast-paced learning capability. Prior experience is valued but not as crucial as the enthusiasm to learn and adapt.Strong commitment to...

Responsibilities Develop and operate data management tools monitoring data flows data quality data cleansing and data processing. Create and document logical data integration strategies for data flows between disparate systems and the enterprise data warehouse/data lakes. Collaborate with different stakeholders to collect required data from internal and...

"As an early Platform Engineer at Watchtower, you'll enable us to deliver our platform reliably, securely, and at massive scale. You'll help architect low latency, real-time microservices that process & detect sensitive data at scale.Responsibilities:Building highly-available and secure authentication and API servicesMaintaining and evolving mission-critical...

SummaryImagine what you could do here. At Apple, new ideas have a way of becoming extraordinary products, services, and customer experiences very quickly. Do you love taking on challenges that create a positive impact? If so, then we want to speak with youAs a member of the AIML Data EPM group, you will be responsible for Apple's ability to use data to power...

Top must-have skills / Candidate Requirements Technical background Agile is required Jira/Confluence is required Leadership skills is required (they will be the senior BSA on the team) Strong written and verbal communication skills SQL is a nice to have Tableau is a nice to have Banking is a nice to have BSA AML is a nice to have Hadoop is a nice to have...

Job DescriptionJob DescriptionHadoop Platform EngineerSan Jose, CA4-12 MonthsPosition summaryThe Data and Analytics Platform team resides within Adobe's Information and Data Services team and we are looking for a Hadoop Platform Engineer who will be responsible for the implementation and ongoing administration of Hadoop infrastructure including...

Wanted 1 -2 years of experience working on Hadoop eco system. Strong understanding of Hadoop distributions ( Cloudera 5.x Hortonworks ..) Hands-on working experience on any one or two of Hadoop technologies (HDFS Hive Impala Sqoop UDF Oozie Map reduce Spark Framework etc.). Strong understanding of any one of the different file formats (e.g. AVRO ORC...

Job Description Please Note:   This position requires passing a ServiceNow background screening, USFedPASS (US Federal Personnel Authorization Screening Standards). This includes a credit check, criminal/misdemeanor check and taking a drug test. Any employment is contingent upon passing the screening.  Due to Federal requirements, only US citizens, US...

Job Description: Responsibilities: - Architect, develop, and maintain scalable data pipelines and ETL processes, facilitating the seamless flow of data across our ecosystem. - Collaborate closely with cross-functional teams to gather requirements, identify data sources, and deliver solutions aligned with business objectives. - Harness the power of SQL...

Responsibilities: Develop and maintain an in-depth knowledge regarding the capabilities of client products and services Improve standard operating procedures to facilitate handling of routine and preventative maintenance associated with our operational environment Build and administer a wide range of technical infrastructure within client's datacenter and...

Title: Quality Engineer Location: Austin, TX Duration: Long term contract Last date: Nov 15, 2022 Job #: 89411Required: Java, Python, Distributed systems, API test automation, API Automation, Rest API Test automation, Kubernetes, AWS, Linux, Docker Additional Comments: Good knowledge of Python/Java, comfortable with Bash scripting in Linux environment...

Job DescriptionJob DescriptionML Engineer San Jose, CA (Remote)Interview Process: 3 Rounds of InterviewJob Description:We are seeking a talented Machine Learning Engineer with expertise in software engineering to join our team. As a Machine Learning Engineer, your primary responsibility will be to develop machine learning (ML) solutions that focus on...

Interview Process: 3 Rounds of Zoom Video • Need 8+Yrs experience and strong experience in Big Data Hadoop, Spark, PySpark • Hands on Programming - Java, Scala, Python • WS Cloud -S3, EFS, MSK, ECS, EMR, Lambdas • Containerized and Microservices • Distributed Computing constructs - Joins, MapReduce, Good at Memory management • RDBMS - MySQL,...

Interview Process: 3 Rounds of Zoom Video • Need 8+Yrs experience and strong experience in Big Data Hadoop, Spark, PySpark • Hands on Programming - Java, Scala, Python • WS Cloud -S3, EFS, MSK, ECS, EMR, Lambdas • Containerized and Microservices • Distributed Computing constructs - Joins, MapReduce, Good at Memory management • RDBMS - MySQL,...

Who is AiDash? AiDash is making critical infrastructure industries climate-resilient and sustainable with satellites and AI. Using our full-stack SaaS solutions, customers in electric, gas, and water utilities, transportation, and construction are transforming asset inspection and maintenance - and complying with biodiversity net gain mandates and carbon...

Team Introduction: Our mission in experimentation and evaluation team is to build the next-gen A/B testing platform, that empowers the company to make data-driven decision for the products. The supported scenarios include recommendation, push, ads, search, mobile app, UI interaction and service upgrades etc. Our platform's capabilities cover the entire...

Who is AiDash? AiDash is making critical infrastructure industries climate-resilient and sustainable with satellites and AI. Using our full-stack SaaS solutions, customers in electric, gas, and water utilities, transportation, and construction are transforming asset inspection and maintenance - and complying with biodiversity net gain mandates and carbon...

Job DescriptionJob DescriptionTeam Introduction:Our mission in experimentation and evaluation team is to build the next-gen A/B testing platform, that empowers the company to make data-driven decision for the products. The supported scenarios include recommendation, push, ads, search, mobile app, UI interaction and service upgrades etc. Our platform's...

This position may require travel and/or relocation to various unanticipated client sites in the USA. **Requirements**: Master's Degree/Bachelor's Degree (or foreign equivalent degree) in Computer Science, Information Systems, Electronics and Communications Engineering or a related field; and 3+ years of progressive experience working with Hadoop distributed...

Position: Software Engineer (Front End)Location: Bay Area, CA (Hybrid)Duration: 8 MonthsThe Advertising Team is reinventing advertising at eBay! We’re creating innovative ad-tech solutions to optimize the ad monetization and experience on eBay. Our goal is to create a valuable experience for both buyers and sellers, while also driving performance for...

Position: Software Engineer (Front End)Location: Bay Area, CA (Hybrid)Duration: 8 MonthsThe Advertising Team is reinventing advertising at eBay! We’re creating innovative ad-tech solutions to optimize the ad monetization and experience on eBay. Our goal is to create a valuable experience for both buyers and sellers, while also driving performance for...

Position: Software Engineer (Front End)Location: Bay Area, CA (Hybrid)Duration: 8 MonthsThe Advertising Team is reinventing advertising at eBay! We’re creating innovative ad-tech solutions to optimize the ad monetization and experience on eBay. Our goal is to create a valuable experience for both buyers and sellers, while also driving performance for...

Job DescriptionJob DescriptionJob DescriptionPosition Description：Location: Usa/Usa/California/Sf Bay Area, SeattleBase Salary: 187K - 280KSponsor Visa? YesLanguage Requirements: English, Mandarin (Preferred)Our Team:Site Reliability Engineering(SRE) team combines software and systems engineering to build and run large-scale, massively distributed, and...

Description:As a Systems Engineer (SE) you will analyze user requirements, concept of operations documents, and high-level system architectures to develop system requirements specifications. You’ll analyze system requirements and lead design and development activities. You’ll be responsible for guiding users in formulating requirements, advising...

Company Overview:Are you ready to be at the cutting edge of technology? At our AI-driven company, innovation isn't just another buzzword—it's a way of life. We're pioneers in integrating artificial intelligence to solve real-world problems, making significant impacts across industries. Join us for a thrilling career journey where your work not only...