Senior Staff Engineer – Data Lakehouse Platform

At GEICO, we offer a rewarding career where your ambitions are met with endless possibilities.Every day we honor our iconic brand by offering quality coverage to millions of customers and being there when they need us most. We thrive through relentless innovation to exceed our customers’ expectations while making a real impact for our company through our shared purpose.When you join our company, we want you to feel valued, supported and proud to work here. That’s why we offer The GEICO Pledge: Great Company, Great Culture, Great Rewards and Great Careers.Position SummaryGEICO is seeking an experienced Engineer with a passion for building high-performance, low maintenance, zero-downtime platforms, and core data infrastructure. You will help drive our insurance business transformation as we transition from a traditional IT model to a tech organization with engineering excellence as its mission, while co-creating the culture of psychological safety and continuous improvement.Position DescriptionOur Staff/Senior Staff Engineer is a key member of the engineering staff working across the organization to innovate and bring the best open-source data infrastructure and practices into GEICO as we embark on a greenfield project to implement a core Data Lakehouse for all GEICO’s core data use-cases across each of the company’s business verticals.Position ResponsibilitiesAs a Senior Staff Engineer, you will:Scope, design, and build scalable, resilient Data Lakehouse componentsLead architecture sessions and reviews with peers and leadershipSpearhead new software evaluations and innovate with new toolingDesign and lead the development & implementation of Compute Efficiency projects like Smart Spark Auto-Tuning FeatureDrive performance regression testing, benchmarking, and continuous performance profilingAccountable for the quality, usability, and performance of the solutionsDetermine and support resource requirements, evaluate operational processes, measure outcomes to ensure desired results, and demonstrate adaptability and sponsoring continuous learningCollaborate with customers, team members, and other engineering teams to solve our toughest problemsBe a role model and mentor, helping to coach and strengthen the technical expertise and know-how of our engineering communityConsistently share best practices and improve processes within and across teamsShare your passion for staying on top of the latest open-source projects, experimenting with, and learning recent technologies, participating in internal and external OSS technology communities, and mentoring other members of the engineering communityQualificationsDeep knowledge of Spark internals, including Catalyst, Tungsten, AQE, CBO, scheduling, shuffle management, and memory tuningProven experience in tuning and optimizing Spark jobs on Hyper-Scale Spark Compute PlatformsMastery of Spark configuration parameters, resource tuning, partitioning strategies, and job execution behaviorsExperience building automated optimization systems – from config auto-tuners to feedback loops and adaptive pipelinesStrong software engineering skills in Scala, Java, and Python are requiredAbility to build tooling to surface meaningful performance insights at scaleDeep understanding of auto-scaling and cost-efficiency strategies in cloud-based Spark environmentsExemplary ability to design and develop, perform experiments, and influence engineering direction and product roadmapAdvanced experience developing new and enhancing existing open-source based Data Lakehouse platform componentsExperience cultivating relationships with and contributing to open-source software projectsExperience with open-source table formats (Apache Iceberg, Delta, Hudi or equivalent)Advanced experience with open-source compute engines (Apache Spark, Apache Flink, Trino/Presto, or equivalent)Experience with cloud computing (AWS, Microsoft Azure, Google Cloud, Hybrid Cloud, or equivalent)Expertise in developing distributed systems that are scalable, resilient, and highly availableExperience in container technology like Docker and Kubernetes platform developmentExperience with continuous delivery and infrastructure as codeIn-depth knowledge of DevOps concepts and cloud architectureExperience in Azure Network (Subscription, Security zoning, etc.) or equivalentPreferred QualificationsActive or past Apache Spark Committer (or significant code contributions to OSS Apache Spark)Experience with ML-based optimization techniques (e.g., reinforcement learning, Bayesian tuning, predictive models)Contributions to other big data/open-source projects (e.g., Delta Lake, Iceberg, Flink, Presto, Trino)Background in designing performance regression frameworks and benchmarking suitesDeep understanding of Spark accelerators (Spark RAPIDS, Apache Gluten, Apache Comet, Apache Auron, etc.) committer status in one or more project is a plusSkilled in documenting methodologies and producing publication-style papers, whitepapers, and internal research briefsExperience10+ years of professional experience in data software development, programming languages and developing with big data technologies8+ years of experience with architecture and design6+ years of experience with distributed systems, with at least 3 years focused on Apache Spark6+ years of experience in open-source frameworks4+ years of experience with AWS, GCP, Azure, or another cloud serviceEducationBachelor’s or Master’s degree in Computer Science, Software Engineering, or related field like physics or mathematicsSalaryAnnual Salary: $110,000.00 - $260,000.00The above annual salary range is a general guideline. Multiple factors are taken into consideration to arrive at the final hourly rate/annual salary to be offered to the selected candidate. Factors include, but are not limited to, the scope and responsibilities of the role, the selected candidate’s work experience, education and training, the work location as well as market and business considerations.At this time, GEICO will not sponsor a new applicant for employment authorization for this position.The GEICO PledgeGreat Company: At GEICO, we help our customers through life’s twists and turns. Our mission is to protect people when they need it most and we’re constantly evolving to stay ahead of their needs.We’re an iconic brand that thrives on innovation, exceeding our customers’ expectations and enabling our collective success. From day one, you’ll take on exciting challenges that help you grow and collaborate with dynamic teams who want to make a positive impact on people’s lives.Great Careers: We offer a career where you can learn, grow, and thrive through personalized development programs, created with your career – and your potential – in mind. You’ll have access to industry leading training, certification assistance, career mentorship and coaching with supportive leaders at all levels.Great Culture: We foster an inclusive culture of shared success, rooted in integrity, a bias for action and a winning mindset. Grounded by our core values, we have an established culture of caring, inclusion, and belonging, that values different perspectives. Our teams are led by dynamic, multi-faceted teams led by supportive leaders, driven by performance excellence and unified under a shared purpose.As part of our culture, we also offer employee engagement and recognition programs that reward the positive impact our work makes on the lives of our customers.Great Rewards: We offer compensation and benefits built to enhance your physical well-being, mental and emotional health and financial future.Comprehensive Total Rewards program that offers personalized coverage tailor-made for you and your family’s overall well-being.Financial benefits including market-competitive compensation; a 401K savings plan vested from day one that offers a 6% match; performance and recognition-based incentives; and tuition assistance.Access to additional benefits like mental healthcare as well as fertility and adoption assistance.Supports flexibility - We provide workplace flexibility as well as our GEICO Flex program, which offers the ability to work from anywhere in the US for up to four weeks per year.The equal employment opportunity policy of the GEICO Companies provides for a fair and equal employment opportunity for all associates and job applicants regardless of race, color, religious creed, national origin, ancestry, age, gender, pregnancy, sexual orientation, gender identity, marital status, familial status, disability or genetic information, in compliance with applicable federal, state and local law. GEICO hires and promotes individuals solely on the basis of their qualifications for the job to be filled.GEICO reasonably accommodates qualified individuals with disabilities to enable them to receive equal employment opportunity and/or perform the essential functions of the job, unless the accommodation would impose an undue hardship to the Company. This applies to all applicants and associates. GEICO also provides a work environment in which each associate is able to be productive and work to the best of their ability. We do not condone or tolerate an atmosphere of intimidation or harassment. We expect and require the cooperation of all associates in maintaining an atmosphere free from discrimination and harassment with mutual respect by and for all associates and applicants. #J-18808-Ljbffr