Data Engineer

Skills Needed Spark(PySpark preferred) Spark Workload Tuning/Optimization Python(focus on data engineering) Apache Beam(Python preferred) Airflow(Cloud Composer preferred) BigQuery Hive Data Modeling/Data Warehouse/Analytics Looker Tableau CICD(Cloud Build Bazel MonoRepo) Jupyter Notebooks(Good to have but not required)

Responsibilities Analyze and organize raw data Build data systems and pipelines Evaluate business needs and objectives Interpret trends and patterns Conduct complex data analysis and report on results Prepare data for prescriptive and predictive modeling Build algorithms and prototypes Combine raw information from different sources Explore ways to enhance...

Blockchain technology is powering a growing wave of innovation. Businesses and governments around the world are using blockchains to make banking more efficient, connect with their customers, and investigate criminal cases. As adoption of blockchain technology grows, more and more organizations seek access to all this ecosystem has to offer. That's where...

Responsibilities All extract transform and load (ETL) processes and the creation of applications that can connect to remote APIs. Preferably including streaming data into environments such as BigQuery Management of multiple processes and applications performance reporting and error checking Management of all data created within client applications the...

Required Skills Data engineer main skills Good knowledge of Data warehousing with expertise in data modeling and ETL. Advanced skills in SQL and HQL Should be good at these core tech AWS/EMR/Spark Hadoop Hive Vertica Qlikview Tableau Should have exp in CI/CD

Responsibilities Working with GCP team for various GCP Cloud programs including Certifications Partner engineering Marketplace and other Cloud platform solutions. Educating GCP / ISVs customers on best practices Stakeholders management / Communication Google ISVs Project Teams Program Management and ExecutionManaging the teams on different projects Reporting...

Who is AiDash? AiDash is making critical infrastructure industries climate-resilient and sustainable with satellites and AI. Using our full-stack SaaS solutions, customers in electric, gas, and water utilities, transportation, and construction are transforming asset inspection and maintenance - and complying with biodiversity net gain mandates and carbon...

Who we are.Pachama is a mission-driven company looking to restore nature to help address climate change. Pachama brings the latest technology in remote sensing and AI to the world of forest carbon in order to enable forest conservation and restoration to scale. Pachama's core technology harnesses satellite imaging with artificial intelligence to measure...

Role Sr. Data Engineer (8+ years) Location SFO CA Duration Long TermExperience & Skills Extensive experience with Hadoop (or similar) Ecosystem (Map Reduce HDFS Hive Spark Pig HBase) Proficient in at least one of the SQL languages (MySQL PostgreSQL SqlServer Oracle) Good understanding of SQL Engine and able to conduct advanced performance tuning Strong...

Nightfall makes safeguarding sensitive data for every application simple and seamless. Organizations, from startups to global brands, trust Nightfall's software platform and APIs to discover, classify, and protect sensitive data.We're looking for a Senior Data Engineer to enhance Nightfall's core data models and infrastructure supporting our real-time and...

Responsibilities Working with the GCP team for various GCP Cloud programs including Certifications Partner engineering Marketplace and other Cloud platform solutions. Educating GCP / ISVs customers on best practices Stakeholders management / Communication Google ISVs Project Teams Program Management and Execution Managing the teams on different projects...

About GlowAt Glow, we believe that insurance can provide peace of mind to small business owners so they can pursue their dreams. We are building a digital insurance platform that ensures small businesses have the right coverage for all their insurance needs at the lowest cost, not just when they purchase, but every year. We recently completed a $22M Series A...

Responsibilities:Collaborate with stakeholders to understand data migration requirements, including scope, timelines, and specific data sets.Design and develop a comprehensive data migration plan, considering factors such as data volume, complexity, and business continuity.Utilize Komprise data management software to perform assessments, identify target...

Skills Description GCP Experience is Must hard code coder coding and debugging data flow pipelines implemented in Scala utilizing beam data is pulled from BQ and results are stored in BQ Python and bash Scio is a Spotify lib in Scala is a must

Our major field office locations include Denver, Indianapolis, Grand Rapids, Lexington, Los Angeles, Louisville, Texas, San Francisco.Juice - The "Stuff" it takes to be a Needle MoverWe look forward to the goal, mentallymapping outevery checkpoint on the pathway to success, and visualizing what the final destination looks and feels like.We're looking to add...

The rolePlatform Engineers at One are responsible for providing a stable, secure, and efficient platform that upholds our high standards of security and compliance, ultimately ensuring customer trust and business success.Besides writing, reviewing, and shipping code, engineers collaborate with others across the company, from product, design, and data to,...

GDH Consulting is looking for a Collaboration Engineer to work remotely.Job DescriptionAs a Consulting Engineer, you will be gathering requirements and performing analysis of data to prepare for and execute complex changes within the customer's environment. Thinking through the process and procedures with a critical eye and attention to detail is a must....

"Are you interested in distributed systems or big data? RetailNext is looking for engineers to help build our cloud analytics system. We pull in and process data from thousands of brick and mortar stores to help our customers better understand and serve their customers.We're looking for engineers with experience in distributed systems or data processing at...

Software Engineer, Data OpsDepartment: EngineeringEmployment Type: Full TimeLocation: San Mateo, CADescriptionBigHat Biosciences is seeking an exceptional individual for the role of Software Engineer, Data Ops. This role is targeted at someone who is ready to dive deep into our computational engine and enhance its capabilities to meet the evolving demands of...

While candidates in the listed locations are encouraged for this role, we are open to remote candidates in other locations.As a Data Infrastructure Engineer on the security data infrastructure team you will help build Lakehouse for Security organization. You will build reliable, large-scale, multi-geo data pipelines to support detecting threats(internal and...