Research Scientist

About the CompanyWe are a seed-stage AI company building the industry standard for evaluating and benchmarking large language models on real enterprise tasks.About the RoleAs a Research Scientist, you will develop new benchmarks, methodologies, and evaluation pipelines that shape how cutting-edge models are assessed, compared, and deployed in production environments. Your work will directly influence model selection and safety decisions across foundation model labs, high-growth AI product companies, and Fortune-scale enterprises.ResponsibilitiesBenchmarking & Model AnalysisEvaluate newly released models as they launch (e.g., Gemini, DeepSeek, etc.)Run large-scale assessment workflows using internal evaluation infrastructureCompare model performance across enterprise-grade task categoriesDesign New Benchmarks from ScratchIdentify high-value model application domains through research explorationConstruct datasets, including labeling strategy and workforce coordinationWrite short white-paper-style summaries explaining benchmark purpose, method, and findingsAdvance Automated Evaluation MethodologiesImprove systems for scoring generated text beyond standard metricsExplore research in reference-free, rubric-based, and human-aligned evaluationDevelop new techniques for reliability, consistency, and repeatabilityCross-functional CollaborationWork closely with engineering to scale evaluation infraPartner with customers to refine evaluation relevance and task fitInfluence product direction through research insightsQualifications03 years post-grad experience (Masters, PhD, or equivalent applied research)Publications, preprints, or demos showing cutting-edge workExperience with diffusion models, NLP, multimodal, or benchmarking workAbility to operate independently with ambiguityClean, maintainable research codebasesCandidates from:top engineering/research universitiesapplied AI startupswell-regarded research internshipsRequired SkillsBuilt or contributed to a benchmark or evaluation methodologyExperience in enterprise task model evaluationStanford / top lab adjacency (per their historical hiring success)Preferred SkillsWants to publish as primary motivationPurely academic with slow timelinesBig-tech culture fit concerns (Meta / Google / Salesforce specifically notedbut case-by-case)Pay range and compensation packageBase Salary: up to $250K depending on backgroundEquity: typically 0.3% 0.5%, flexibility for exceptional candidatesRapid equity refresh possible based on impactEqual Opportunity StatementVisa sponsorship available. Relocation support. Health & dental coverage. Lunch + dinner provided, snacks & coffee. Unlimited PTO. Weekly happy hours with community guests. Team events (bowling, hiking, rock climbing, etc.). Swag program (hats, etc.).Work Environment & CultureIn-person, San Francisco HQ (required). Core hours: 95, some teammates extend voluntarily. Most team members work 1 weekend day per week (flexible). High-ownership, low-ego, collaborative. Live demos Mondays, team lunch Thursdays, community Fridays. Early-stage pace, applied focusnot academic publishing.Tech Environment(while research-focused, exposure beneficial) Backend: Python / Django. Frontend: React + TypeScript. Infra: AWS. Evaluation frameworks + internal tooling.Why This Role Is UniqueThe company already collaborates with foundation model labs, high-growth AI vertical product companies, and Fortune 500 enterprises (not publicly facing). ChatGPT Vals AI $5M seed raised, runway of 2+ years at current burn. Only one research scientist is being hiredtrue founding impact. Opportunity to define industry standards for model trust, reliability, and certification. Positioned to become the rating agency for generative AI.