GCP Data Engineer

4 weeks ago


California, United States Newonlab Full time

About the role:As a Data Engineer, you will build a variety of big data analytics solutions, including big data lakes. More specifically, you will:

Design and build scalable data ingestion pipelines to handle real-time streams, CDC events, and batch data

Execute high-performance data processing for structured and unstructured data and data harmonization

Schedule, orchestrate and validate pipelines

Design exception handling and log monitoring for debugging

Make tech stack and tools related decisions

Collaborate with business consultants, data scientists, and application developers to develop analytics solutions

Job RequirementRequired Experience, Skills & Competencies:

Hands-on experience with:

Hadoop ecosystem - HDFS, Hive, Sqoop, Kafka, ELK Stack, etcSpark, Scala, Python, and core/advanced JavaNoSQL databases e.g. Hbase, Cassandra, MongoDBRelevant GCP components required to build big data solutionsGood to know: Databricks, Snowflake

Ability to develop and manage scalable Hadoop cluster environments

Good understanding of data warehousing concepts, distributed systems, data pipelines, ETL

5-10 years of tech experience with at least 3 years in big data engineering

Designation will be commensurate with expertise/experience. Compensation packages are among the best in the industry.

#J-18808-Ljbffr


  • GCP Data Engineer

    3 weeks ago


    California, United States Pyromis Full time

    Note : Very strong and deep knowledge of GCP with Airflow, Dataflow, Composer, Big Query . Required Past Experience: Looking for a GCP Data Engineer for its Cloud and Big Data projects in the Data Warehousing, Data Analytics domain. Develop data-driven solutions with Cloud based technologies to meet evolving business needs. Deliver impactful, scalable, and...


  • Pleasanton, California, United States Data Piper Full time

    Job Description Role Description: Refine and update primary network design and implementation of production/non-production hybrid shared vpc host project. This includes: Multi-regional implementation of VLAN attachments to GCP network Validation of routing between GCP and Customer’s On-Prem Network Containing private Google regional traffic to adhere...


  • Mountain View, California, United States iSoftTek Solutions Inc Full time

    Job Title: ML Infrastructure Engineer with GCP Location: Mountain View, CA [Needs to be onsite for 1 week once in a quarter on your own expenses] Job Type: W2 Duration: Long Term Note: Only PST and MST candidates are required.     Must have:  · Experience in Machine Learning engineer or Infrastructure roles, with a focus on Machine Learning...

  • Data Architect

    1 month ago


    California, United States theloops Full time

    We fundamentally believe that today's product needs to be designed with minimal support in mind. We are seed stage startup bringing product context to support reps to eliminate escalations and drive efficiency. The Co-founders are technology leaders with decades of experience solving the most pressing problems facing enterprises. At TheLoops we:We lead with...

  • Data Engineer

    1 week ago


    California, United States Open Systems Technologies Full time

    A financial firm is looking for a Data Engineer to join their team in New York, NY. Pay: $70-100/hr • Work on migrating applications from an on-premises location to the cloud service providers. • Develop products and services on the latest technologies through contributions in development, enhancements, testing and implementation. • Develop, modify,...

  • Data Architect

    3 weeks ago


    California, United States Kinship Inc Full time

    This is an opportunity to create the industry-defining pet parent mobile experience and architecture Kinship is unlocking a new world for you and your pet. Because, if you’re like us, your pet is your world. Our family of brands is continuously working to challenge preconceived notions, evolve our environments, and (re)build a universe that supports and...

  • Data Engineer

    3 weeks ago


    California, United States e-Primary Full time

    Title: Data Engineer Location: Orange, CA (Remote. Full time schedule but must work 9:00 a.m. to 2:00 p.m. PT. and can flex start/end times around this). Duration: 6 months + w/potential temp to perm Description: We are seeking qualified candidates to join us in an enterprise-wide analytics initiative to help drive incremental business value from our broad...

  • Sr. Data Engineer

    3 weeks ago


    California, United States Tekstad Full time

    Seeking a Sr. Data Engineers for my client in SFO CA. Duration: Long Term #Contract ----------------------------------------------------------------- Ideal Candidates must have below skills. •Responsible for scaling up data processing flow to meet rapid data growth. •Evolve data model and data schema based on business and engineering needs. •Implement...

  • Data Engineer II

    4 weeks ago


    California, United States Redica Systems, Inc. Full time

    Redica Systems is a SaaS start-up serving more than 200 customers within the life science sector, with a specific focus on Pharmaceuticals and MedTech. Embracing a hybrid model, our workforce is distributed globally, with headquarters in Pleasanton, CA. Redica's data analytics platform empowers companies to improve product quality and navigate evolving...


  • Irvine, California, United States TEKRRA1 Full time

    Role & Responsibilities: Design, develop, and implement advanced AI solutions to solve complex business problems across various domains. Collaborate with engineers, researchers, and product managers to understand business needs and translate them into technical specifications for AI models. Gather, clean, and prepare data for training and evaluating...


  • California, United States Gilder Search Group Full time

    We are looking for a Software Developer Engineer in Test based in Latin America to work on a long-term project for one of our clients, a video game holding company based in Santa Monica, California. Our client creates the most epic interactive gaming and entertainment experiences on earth immersing players in new, unimagined worlds. Their products help build...

  • Data Engineer

    4 weeks ago


    California City, United States iTvorks Inc Full time

    Required Skills Data engineer main skills Good knowledge of Data warehousing with expertise in data modeling and ETL. Advanced skills in SQL and HQL Should be good at these core tech AWS/EMR/Spark Hadoop Hive Vertica Qlikview Tableau Should have exp in CI/CD

  • Data Scientist

    1 week ago


    San Francisco, California, United States hackajob Full time

    Exciting Opportunity Alert! Join our Team as a Data Scientist! hackajob has partnered with a cutting-edge company seeking a talented Data Scientist with expertise in Predictive Modeling and Python to join our innovative team! Role: Data Scientist Location: Remote Salary: $100K - $180K depending on experience + benefits package Required Skills: *...

  • Senior Data Engineer

    3 weeks ago


    Sunnyvale, California, United States Two95 International Inc. Full time

    Title: Senior Data Engineer Location: Sunnyvale, CA Position: 2 Month (Contract) Summary Provide business insights, while leveraging internal tools and systems, databases and industry data Roles & Responsibilities Ability to document requirements, data lineage, subject matter in both business and technical terminology. Guide and learn from...


  • Santa Clara, California, United States Palo Alto Networks Full time

    Job Description Your Career Palo Alto Networks runs a large infrastructure and is one of the largest GCP customers. As a Principal DevOps Engineer for the PanOS Cloud Components team, you will be part of a team supporting the services running on this infrastructure. This includes automation, architecture, performance, observability, troubleshooting,...

  • Senior Data Engineer

    2 weeks ago


    San Francisco, California, United States UpRecruit Full time

    Job Details:  Title: Sr. Data Engineer Salary: $180-$230K  Requirements: 8+ years of software dev exp (Large Data Sets or Big Data) Location: 100% onsite (San Francisco Bay Area)  Our client, a gaming startup, is seeking a Sr. Data Engineer to join their passionate team. The ideal candidate is an experienced engineer who will play a vital role...


  • California, United States ActioNet Full time

    Telecommunications Engineer - Voice, Video, Data | Req#3443 ActioNet has an immediate opportunity for a Telecommunications Engineer - Voice, Video, Data requiring a Secret clearance located in San Diego, CA. ActioNet is an IT service provider and solutions integrator headquartered in Vienna, VA that works with the Federal Government and Department of...


  • Irvine, California, United States Allergan Aesthetics Full time

    Job Description Overview: We are seeking a Data Engineer with experience in ETL, Snowflake, ThoughtSpot, and/or Power BI datasets to join our team. As a Data Engineer, you will be responsible for designing, building, and maintaining data pipelines and systems that enable our internal stakeholders to make data-driven decisions. This Data Engineer will...


  • San Francisco, California, United States Avant Digital Inc Full time

    Job Description- Location: Remote Contract type - Contract / C2C Contract duration - 12 months Juniper Networks Cyber Fusion is looking for a certified cybersecurity professional to join our highly collaborative and diverse team of talent. who will be responsible for ensuring the security and compliance of our cloud infrastructure and data. You will work...

  • Staff Data Engineer

    7 days ago


    Foster City, California, United States Visa Full time

    Job Description This position will be part of VCA (Visa  Consulting & Analytics) function in building and maintaining global data assets and engineering solutions. We are looking expertise in data warehousing and building large-scale data processing systems by using the latest database technologies.  The Data Engineer takes responsibility for building and...