Spark Engineer

**Required skills/experience**: - Advanced level of **Java/Scala/Python development** experience. - Intermediate level of **SQL and NoSQL experience** (handling structured and unstructured data). - Extensive experience with **Spark Processing engine**. - Experience with **Big data tools / technologies/ Streaming** (Hive, Impala, OOZIE, Airflow, NIFI,...

Skills Needed Spark(PySpark preferred) Spark Workload Tuning/Optimization Python(focus on data engineering) Apache Beam(Python preferred) Airflow(Cloud Composer preferred) BigQuery Hive Data Modeling/Data Warehouse/Analytics Looker Tableau CICD(Cloud Build Bazel MonoRepo) Jupyter Notebooks(Good to have but not required)

Skills Needed Spark(PySpark preferred) Spark Workload Tuning/Optimization Python(focus on data engineering) Apache Beam(Python preferred) Airflow(Cloud Composer preferred) BigQuery Hive Data Modeling/Data Warehouse/Analytics Looker Tableau CICD(Cloud Build Bazel MonoRepo) Jupyter Notebooks(Good to have but not required)

Job DescriptionJob DescriptionHadoop Platform EngineerSan Jose, CA4-12 MonthsPosition summaryThe Data and Analytics Platform team resides within Adobe's Information and Data Services team and we are looking for a Hadoop Platform Engineer who will be responsible for the implementation and ongoing administration of Hadoop infrastructure including...

Responsibilities Analyze and organize raw data Build data systems and pipelines Evaluate business needs and objectives Interpret trends and patterns Conduct complex data analysis and report on results Prepare data for prescriptive and predictive modeling Build algorithms and prototypes Combine raw information from different sources Explore ways to...

Responsibilities Analyze and organize raw data Build data systems and pipelines Evaluate business needs and objectives Interpret trends and patterns Conduct complex data analysis and report on results Prepare data for prescriptive and predictive modeling Build algorithms and prototypes Combine raw information from different sources Explore ways to enhance...

Job DescriptionJob DescriptionTitle: Data EngineerLocation: San Jose, CA (Onsite)Duration: Long TermContract RoleSkill Set:Python, Spark, AWS.

ResponsibilitiesTikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok has global offices including Los Angeles, New York, London, Paris, Berlin, Dubai, Singapore, Jakarta, Seoul and Tokyo. Why Join UsCreation is the core of TikTok's purpose. Our platform is built to help imaginations thrive....

Job Description: Machine Learning Engineer San Francisco, CA, US Description About Us We are the world’s first Experimentation Platform that empowers executives, marketing teams, product teams, and development teams to make better business decisions and deliver better customer experiences by optimizing digital experiences on any channel and any...

Wanted 1 -2 years of experience working on Hadoop eco system. Strong understanding of Hadoop distributions ( Cloudera 5.x Hortonworks ..) Hands-on working experience on any one or two of Hadoop technologies (HDFS Hive Impala Sqoop UDF Oozie Map reduce Spark Framework etc.). Strong understanding of any one of the different file formats (e.g. AVRO ORC...

Wanted 1 -2 years of experience working on Hadoop eco system. Strong understanding of Hadoop distributions ( Cloudera 5.x Hortonworks ..) Hands-on working experience on any one or two of Hadoop technologies (HDFS Hive Impala Sqoop UDF Oozie Map reduce Spark Framework etc.). Strong understanding of any one of the different file formats (e.g. AVRO ORC Parquet...

This role also open for junior (3+ yoe) candidates, and SRE lead (7+ yoe).Site Reliability Engineering(SRE) team combines software and systems engineering to build and run large-scale, massively distributed, and fault-tolerant systems. In our team, you ll have the opportunity to manage the complex challenges of scale, while using expertise in coding,...

Robert Half is looking for a Data Engineer role that is pivotal within the Business Intelligence and Data Warehousing team, serving as a software engineering expert. This individual is tasked with developing robust data ingestion systems to facilitate the creation of high-value business intelligence products. As an individual contributor, this role...

Robert Half is looking for a Data Engineer role that is pivotal within the Business Intelligence and Data Warehousing team, serving as a software engineering expert. This individual is tasked with developing robust data ingestion systems to facilitate the creation of high-value business intelligence products. As an individual contributor, this role...

Are you passionate about solving complex business problems and real world challenges? Do you thrive in a dynamic startup work environment that constantly pushes you to be the best you can be? Are you ready to work with motivated colleagues and veteran entrepreneurs who strive for excellence in everything they do? PeerNova's main product philosophy is to...

About Mendel: At Mendel, we are enabling different stakeholders in healthcare to make better decisions by learning from every patient's journey— from a physician deciding on the best treatment for a patient to a pharma company discovering the next blockbuster drug. In creating depth and breadth for a patient’s health record journey, Mendel enables the...

About Mendel: At Mendel, we are enabling different stakeholders in healthcare to make better decisions by learning from every patient's journey— from a physician deciding on the best treatment for a patient to a pharma company discovering the next blockbuster drug. In creating depth and breadth for a patient’s health record journey, Mendel enables the...

About Mendel: At Mendel, we are enabling different stakeholders in healthcare to make better decisions by learning from every patient's journey— from a physician deciding on the best treatment for a patient to a pharma company discovering the next blockbuster drug. In creating depth and breadth for a patient’s health record journey, Mendel enables the...

Who is AiDash? AiDash is making critical infrastructure industries climate-resilient and sustainable with satellites and AI. Using our full-stack SaaS solutions, customers in electric, gas, and water utilities, transportation, and construction are transforming asset inspection and maintenance - and complying with biodiversity net gain mandates and carbon...

Job DescriptionJob DescriptionJob Title: Senior Data EngineerLocation: San Jose, CA (Hybrid)Pay: $90-95/hour on W2Duties:Write complex SQL queries that help Adobe identify and measure the non-genuine software base. Apply business logic to determine categorize the non-genuine base into opportunities. Build data sets and analyze them for insights to achieve...