Senior Data Engineer
3 weeks ago
Position: Senior Data Engineer
Location: Sunnyvale, CA
Duration: 12+ Months
Job Description
Proficiency in managing and manipulating huge datasets in the order of terabytes (TB) is essential.
Expertise in big data technologies like Hadoop, Apache Spark (Scala preferred), Apache Hive, or similar frameworks on the cloud (GCP preferred, AWS, Azure etc.) to build batch data pipelines with strong focus on optimization, SLA adherence and fault tolerance.
Expertise in building idempotent workflows using orchestrators like Automic, Airflow, Luigi etc.
Expertise in writing SQL to analyze, optimize, profile data preferably in BigQuery or SPARK SQL
Strong data modeling skills are necessary for designing a schema that can accommodate the evolution of data sources and facilitate seamless data joins across various datasets
Ability to work directly with stakeholders to understand data requirements and translate that to pipeline development / data solution work.
Strong analytical and problem-solving skills are crucial for identifying and resolving issues that may arise during the data integration and schema evolution process.
Ability to move at rapid pace with quality and start delivering with minimal ramp up time will be crucial to succeed in this initiative.
Effective communication and collaboration skills are necessary for working in a team environment and coordinating efforts between different stakeholders involved in the project.
Nice To Have
Experience building complex near real time (NRT) streaming data pipelines using Apache Kafka, Spark streaming, Kafka Connect with a strong focus on stability, scalability, and SLA adherence.
Good understanding of REST APIs – working knowledge on Apache Druid, Redis, Elastic search, GraphQL or similar technologies. Understanding of API contracts, building telemetry, stress testing etc.
Exposure in developing reports/dashboards using Looker/Tableau
Experience in eCommerce domain.
Tech stack: Google cloud, HDFS, SPARK, Scala, Python (optional), Automic/Airflow, BigQuery, Kafka, API, Druid
#J-18808-Ljbffr
-
Senior Data Engineer
1 week ago
Sunnyvale, United States TWO95 International Full timeTitle: Senior Data Engineer Location: Sunnyvale, CA Position: 2 Month (Contract) Summary Provide business insights, while leveraging internal tools and systems, databases and industry data Roles & Responsibilities Ability to document requirements, data lineage, subject matter in both business and technical terminology. Guide and learn from other team...
-
Senior Data Engineer
3 weeks ago
Sunnyvale, California, United States Two95 International Inc. Full timeTitle: Senior Data Engineer Location: Sunnyvale, CA Position: 2 Month (Contract) Summary Provide business insights, while leveraging internal tools and systems, databases and industry data Roles & Responsibilities Ability to document requirements, data lineage, subject matter in both business and technical terminology. Guide and learn from...
-
Senior Data Engineer
3 weeks ago
Sunnyvale, United States 1 Full timeJob DescriptionJob DescriptionPosition SummaryVery Strong engineering skills. Should have an analytical approach and have good programming skills.Provide business insights, while leveraging internal tools and systems, databases and industry dataMinimum of 5+ years experience. Experience in retail business will be a plus.Excellent written and verbal...
-
Senior Data Engineer
1 week ago
Sunnyvale, United States Illumio Full timeOn-site work model with 5 days in office/week in Sunnyvale, CA Targeted hire date of February 1, 2024 About the Team: Illumio's new Machine Learning (ML) team embodies a culture of thought leadership, independence, and responsibility, holding a significant and highly visible role within our Engineering division. This dynamic team is tasked with developing...
-
Senior GCP Data Engineer
17 hours ago
Sunnyvale, United States Wilco Source Full time1. Bachelor's or Master's degree in Computer Science, Engineering, or related field.2. 10+ years of experience in data engineering, with a focus on GCP technologies.3. Strong proficiency in SQL, Scala, and Python for data manipulation and analysis.4. Hands-on experience with GCP services, including BigQuery, Cloud Storage, andDataflow.5. Solid understanding...
-
Senior GCP Data Engineer
11 hours ago
Sunnyvale, United States Wilco Source Full time1. Bachelor's or Master's degree in Computer Science, Engineering, or related field.2. 10+ years of experience in data engineering, with a focus on GCP technologies.3. Strong proficiency in SQL, Scala, and Python for data manipulation and analysis.4. Hands-on experience with GCP services, including BigQuery, Cloud Storage, andDataflow.5. Solid understanding...
-
Senior GCP Data Engineer
21 hours ago
Sunnyvale, United States Wilco Source Full time1. Bachelor's or Master's degree in Computer Science, Engineering, or related field.2. 10+ years of experience in data engineering, with a focus on GCP technologies.3. Strong proficiency in SQL, Scala, and Python for data manipulation and analysis.4. Hands-on experience with GCP services, including BigQuery, Cloud Storage, andDataflow.5. Solid understanding...
-
Senior Human Factors Engineer
3 weeks ago
Sunnyvale, California, United States Intuitive Surgical Full timeJob DescriptionPrimary Function of PositionThe Senior Human Factors Engineer collaborates with various teams to develop new products that are practical, profitable, and appealing.As part of product-development teams, the Senior Human Factors Engineer plays a key role in user-centered design research. They examine use-related risks, conduct usability testing,...
-
Senior Engineer
4 days ago
Sunnyvale, United States SALT Full timeSenior Engineer - Sunnyvale, CA - Onsite Our client is an industry leader in high-perfomance GPS nav systems for aviation applications. As a Senior Systems Engineer you will be working on embedded systems, working on GPR receivers which are implemented into aircrafts, designing circuit boards, and more. Additional Responsibilities You will interact closely...
-
Senior Engineer
4 weeks ago
Sunnyvale, United States Salt Full timeSenior Engineer - Sunnyvale, CA - OnsiteOur client is an industry leader in high-perfomance GPS nav systems for aviation applications. As a Senior Systems Engineer you will be working on embedded systems, working on GPR receivers which are implemented into aircrafts, designing circuit boards, and more.Additional ResponsibilitiesYou will interact closely with...
-
Senior Engineer
3 days ago
Sunnyvale, United States SALT Full timeSenior Engineer - Sunnyvale, CA - Onsite A high number of candidates may make applications for this position, so make sure to send your CV and application through as soon as possible. Our client is an industry leader in high-perfomance GPS nav systems for aviation applications. As a Senior Systems Engineer you will be working on embedded systems, working...
-
Senior Engineer
4 weeks ago
Sunnyvale, United States Salt Full timeSenior Engineer - Sunnyvale, CA - OnsiteOur client is an industry leader in high-perfomance GPS nav systems for aviation applications. As a Senior Systems Engineer you will be working on embedded systems, working on GPR receivers which are implemented into aircrafts, designing circuit boards, and more.Additional ResponsibilitiesYou will interact closely with...
-
Senior Engineer
3 weeks ago
Sunnyvale, United States Salt Full timeSenior Engineer - Sunnyvale, CA - OnsiteOur client is an industry leader in high-perfomance GPS nav systems for aviation applications. As a Senior Systems Engineer you will be working on embedded systems, working on GPR receivers which are implemented into aircrafts, designing circuit boards, and more.Additional ResponsibilitiesYou will interact closely with...
-
Senior Optical Engineer
2 weeks ago
Sunnyvale, United States Amazon.com, Inc. Full timeAs a Senior Optical Engineer, you will engage with an experienced cross-disciplinary staff to conceive and design innovative consumer products. You will work closely with an internal interdisciplinary team, and outside partners to drive key aspects o Optical, Engineer, Electrical Engineer, Senior, Product Engineer, Technology, Systems
-
Senior Machine Learning Engineer
1 week ago
Sunnyvale, United States Illumio Full timeSenior Machine Learning Engineer On-site work model of 5 days in office/week in Sunnyvale, CA As a Senior Machine Learning Engineer, you will have the opportunity to build and lead a new team focused on developing cutting-edge machine learning solutions for our industry-leading Zero Trust security products. You will be responsible for driving the design,...
-
BIG Data Engineer
4 weeks ago
Sunnyvale, California, United States QData Full timeRequired Skills & Experience Experience range 8-14 years Java Development Experience Data Engineer. Python. Hadoop Stack Data Pipeline Using ETL or another tool spark – spark streaming RDD SQL NoSQL Cassandra Handling High Volume Data AWS Experience Experience On Big Data Migration Experience is PLUS
-
BIG Data Engineer
4 weeks ago
Sunnyvale, United States QData Full timeRequired Skills & Experience Experience range 8-14 years Java Development Experience Data Engineer. Python. Hadoop Stack Data Pipeline Using ETL or another tool spark – spark streaming RDD SQL NoSQL Cassandra Handling High Volume Data AWS Experience Experience On Big Data Migration Experience is PLUS
-
Senior Software Engineer
2 days ago
Sunnyvale, United States Veear Full timeJob Description: s a Senior Software Engineer V on the Hardware Testing Infrastructure team at Meta, you will be a key player in defining the testing approach, creating the necessary resources, and fostering a culture of quality within our ASIC team. Your primary focus will be on hands-on software development, automation, and ensuring that our ASIC...
-
Senior Sales Engineer
4 weeks ago
Sunnyvale, California, United States Proofpoint Full timeAt Proofpoint, the members of our Sales Engineering team are highly valued business partners that play a critical role in our sales process every single day.As a Senior Sales Engineer in the fast-growing Advanced Technology Group (ATG), you will be one of the most important members of the Proofpoint sales team.The Senior Sales Engineer is the Technical Lead...
-
Data Pipeline Engineer
4 weeks ago
Sunnyvale, United States Koko Home Full timeKoko Home is seeking a Data Pipeline Engineer to develop and maintain data pipelines from our IoT devices all the way to our model building infrastructure. As this is a healthcare focused company, we’re seeking candidates with experience working within clinical and regulatory environments. MedTech/SaMD/HCIT experience is a plus.This role entails:Insuring...