Abstract
Your agent passed every test, so why is it failing your users? The gap between demo success and production reality is where most agentic AI initiatives quietly break down, and traditional evaluation methods weren’t built to catch it.
Drawing on real-world experience building and validating agentic systems at Toyota Motor Europe, this session introduces an adaptive approach to evaluating non-deterministic agents, where no single framework fits all use cases. Attendees will leave with practical thinking tools to define what “production-ready” means in their context, and the confidence to scale systems that deliver real value.
Topics To Be Covered
Why agents fail in production
Evaluating non-deterministic AI systems
Building adaptive evaluation frameworks
Defining production-ready AI success
Scaling reliable agentic AI solutions
Perfect For
AI Product Managers
AI Architects
Data Scientists
AI Engineering Leaders
Enterprise AI Teams
Meet Your Speaker
Nour Eid

AI Product Analyst, Hazelheartwood
Nour Eid is an AI Product Analyst specializing in Generative AI and agentic systems, with a focus on evaluation, reliability, and scalable deployment. Her experience is rooted in the automotive sector, where she has contributed to one of the largest AI initiatives at Toyota Motor Europe.
Nour's work centers on testing and validating agentic systems in highly variable, non-deterministic environments, where each use case introduces unique behaviors and challenges traditional evaluation approaches. To address these challenges, she developed a comprehensive evaluation framework that has been adopted across multiple teams, enabling the delivery of more robust and production-ready AI systems.
She also designs AI maturity assessments and contributes to AI strategy initiatives, helping organizations define and measure AI value, identify high-impact use cases, and scale solutions that deliver measurable business outcomes.
ADDITIONAL INFORMATION
Time & Place
Thu, Nov 26
14:30 - 15:00
Matterhorn I
Limited to 45 participants.
Secure your seat – registration required.
Notes
Agenda for this session
20 min presentation + Audience Q&A

.png)