SRE/LLM Ops Engineer

At CluePoints, we’re redefining how clinical trials are run. As the premier provider of Risk-Based Quality Management (RBQM) and Data Quality Oversight software, we harness advanced statistics, artificial intelligence, and machine learning to ensure the quality, accuracy, and integrity of clinical trial data, helping life sciences organisations bring safer, more effective treatments to patients faster. We’re proud to be an ambitious, fast-growing technology scale‑up with a dynamic and diverse international team representing more than 40+ nationalities. Collaboration, flexibility, and continuous learning are part of our DNA. Role The SRE, LLMOps (AI Platform) ensures our LLM‑powered services are reliable, observable, and safe in production on Azure and Kubernetes. You’ll blend classic SRE disciplines with LLM‑specific operations: robust evaluation pipelines, prompt/version governance, model/vendor failover, guardrails, and cost/performance monitoring. You know how to build automation with LangChain/LangGraph, operate API‑based LLMs in production, and manage the inherent non‑determinism of models through rigorous testing and observability. What You'll Bring Experience: 5+ years in SRE/DevOps/Platform Engineering with 1–2+ years operating LLM or ML‑backed applications in production (API‑based or hosted models). LLMOps: hands‑on with LangChain/LangGraph building end‑to‑end chains/agents and RAG flows; comfort with vector stores (e.g., Azure AI Search, Pinecone), prompt/version control, and dataset tooling. Observability: proficiency instrumenting LLM traces and app telemetry, alert tuning, and root‑cause analysis; familiarity with LangSmith and/or Arize Phoenix (token/cost tracking, latency, failure modes). Cloud & platform: strong Azure and Kubernetes (AKS) background; GitOps (Flux/ArgoCD), Helm/Kustomize; CI/CD (GitHub Actions/GitLab/Jenkins); IaC (Terraform); secrets, networking, and security baselines. Languages & tooling: Python (preferred) and one of TypeScript/Go; REST/GraphQL; OpenAI/Azure OpenAI/Anthropic APIs; experience with Redis caches, message queues, and feature flags. What You'll Be Doing Instrument deep observability: implement tracing for LLM chains/agents (inputs, outputs, token usage, latency, model/version), correlate with app metrics/logs, and set actionable alerts; leverage LangSmith/Arize Phoenix (or similar) and OpenTelemetry where appropriate. Safety & guardrails: integrate content safety, PII redaction, jailbreak/prompt‑injection defenses, and policy‑based rails; document exceptions and reviewer workflows. Prefer native platform features (e.g., Azure AI Content Safety) or programmable rails (e.g., NVIDIA NeMo Guardrails). Cost & capacity management: monitor token and request costs, throughput, and rate limits; implement caching, request shaping, and multi‑tier model selection to balance quality, latency, and spend. Build evaluation & testing pipelines: create golden datasets and automated evals (offline + CI/CD + canary) to catch regressions from code, prompt, data, or model changes; use LangSmith/OpenAI Evals (or equivalents) to track quality trends over time. Platform operations on Azure/Kubernetes: ensure secure, compliant, and cost‑efficient operation; maintain IaC, secrets, networking, scaling, and DR/BCP; partner with Security and QA on regulated SaaS controls. Cross‑functional enablement: work with product/dev teams to set acceptance criteria for AI features, add runtime feature flags/kill‑switches, and embed evals/telemetry from day one. Comprehensive Health Insurance (medical, dental, and online consultations, 100% employee coverage) Life Insurance through UNUM Cafeteria Plan with flexible monthly credits for wellness, entertainment, and travel MultiSport Card, co‑financed 50/50 Employee Capital Plans (PPK) with 4% employer contribution A hub‑based hybrid model that blends flexibility with purpose — connecting teams through collaboration, learning, and a vibrant social culture. CluePoints is an equal opportunity employer committed to diversity and inclusion in the workplace. Your personal data will be processed by CluePoints for recruitment purposes in accordance with the Regulation (EU) 2016/679 (GDPR). If you wish for your data to be retained for future opportunities, please include the following statement in your CV: “I consent to the processing of my personal data by CluePoints for the purposes of future recruitment processes.” #J-18808-Ljbffr