ai evaluation engineer

AI Evaluation Engineer Location: USA Remote Employment Type: Contract Position Summary We are seeking an AI Evaluation Engineer to establish and maintain the quality standards used to determine whether production AI and LLM systems are ready to launch. This role will build centralized evaluation frameworks, create golden datasets, establish...

Deposco is seeking an experienced AI Evaluation Engineer to join our innovative quality assurance team. This role focuses on ensuring the accuracy, reliability, and performance of AI-driven applications, models, and automation workflows. The ideal candidate will be highly analytical, detail-oriented, and passionate about shaping the future of AI testing...

Career Area:Technology, Digital and DataJob Description:Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other.  We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We...

Chicago, Illinois Peoria, Illinois Westminster, Colorado Irving, Texas Full time R0000390042 Career Area: Job Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you’re joining a global team who cares not just about the work we do — but also about each other. We are the makers, problem solvers, and future world builders...

Caterpillar Inc. is seeking a leader for the AI Engineering team at Cat Digital to guide evaluation and validation of generative AI solutions, intelligent agents, and digital assistants.You will shape the development of AI-driven products that transform how customers and dealers interact with technology, while establishing scalable engineering practices...

Caterpillar Inc. is seeking an AI Engineering leader within the Cat Digital AI Engineering team to drive evaluation and validation of cutting-edge generative AI solutions, intelligent agents, and digital assistants.You will shape product development and guide engineering practices across hybrid cloud and embedded environments, focusing on scalable, reliable...

Caterpillar Inc. is seeking an AI Engineering leader within the Cat Digital AI Engineering team to drive evaluation and validation of cutting-edge generative AI solutions, intelligent agents, and digital assistants.You will shape product development and guide engineering practices across hybrid cloud and embedded environments, focusing on scalable, reliable...

Career Area: Technology, Digital and Data Job Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do - but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't...

Career Area: Technology, Digital and Data Job Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do - but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't...

About us Appnovation is a global, full-service digital partner that combines Strategy, Experience & Design, Engineering and Managed Services. We build digital solutions that deliver real impact today and serve as foundations for future growth. Bold ambition. Practical action. Endless possibilities. Technology is foundational to all of Appnovation’s...

6+ years building evaluation or testing infrastructure for ML/LLM systems or large-scale distributes system Proven ability to translate product requirements into measurable metrics and test plans. Strong Python skills Strong Experience with modern data tooling Hands-on experience running A/B tests, canaries, or experiment frameworks. Experience defining and...

About Distyl AI Distyl is an applied AI technology company partnering with the world’s most ambitious institutions to rearchitect critical operations for the frontier of AI. Our customers include the largest companies in telecom, healthcare, insurance, manufacturing, consumer goods, and global social organizations.We research and deploy technologies that...

Career Area: Technology, Digital and Data Job Description When you join Caterpillar, you're joining a global team who cares not just about the work we do - but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here - we...

About usAppnovation is a global, full-service digital partner that combines Strategy, Experience & Design, Engineering and Managed Services. We build digital solutions that deliver real impact today and serve as foundations for future growth. Bold ambition. Practical action. Endless possibilities.Technology is foundational to all of Appnovation’s...

Career Area: Technology, Digital and DataJob Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do - but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't...

Caterpillar Financial Services Corporation is seeking a leader for the AI Engineering team. You will guide a group evaluating and validating advanced generative AI solutions, shaping AI-driven experiences that redefine customer and dealer interactions with technology.The role emphasizes deep GenAI architecture, ML engineering, and modern software practices,...

About us Appnovation is a global, full-service digital partner that combines Strategy, Experience & Design, Engineering and Managed Services. We build digital solutions that deliver real impact today and serve as foundations for future growth. Bold ambition. Practical action. Endless possibilities. About the Role Join forces with Appnovation to deliver...

Career Area: Technology, Digital and DataJob Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do - but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't...

Career Area: Technology, Digital and DataJob Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do - but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't...

Jobtailor is seeking a senior QA automation engineer to join our team in the US, focusing on LangSmith‑driven evaluation and LangGraph workflows. You will own end‑to‑end test strategies for LLM-powered surfaces, build robust automated suites with Playwright, and drive reliability across CI/CD pipelines.You will collaborate with QA, development,...