Salesforce

Requirements

• Define and execute the technical vision for evaluation and experimentation across AI agents and traditional ML models
• Own offline evaluation, regression testing, scenario-based simulations, and multi-turn agent testing infrastructure
• Build automated evaluation systems including LLM-as-Judge, rule-based scoring, and hybrid evaluation approaches
• Design and operate online evaluation, observability, and continuous performance monitoring for agent behavior
• Lead development of self-service evaluation and experimentation tooling for agent workflows, tool use, memory, and planning
• Support experimentation for both real-time agents and batch or online traditional ML models
• Integrate evaluation and experimentation pipelines into CI/CD workflows and release quality gates
• Drive adoption of evaluation and experimentation best practices across engineering and AI teams

Skills