top of page

Your Agent Works - So Why Is It Failing in Production?

Abstract

Your agent passed every test, so why is it failing your users? The gap between demo success and production reality is where most agentic AI initiatives quietly break down, and traditional evaluation methods weren’t built to catch it.
Drawing on real-world experience building and validating agentic systems at Toyota Motor Europe, this session introduces an adaptive approach to evaluating non-deterministic agents, where no single framework fits all use cases. Attendees will leave with practical thinking tools to define what “production-ready” means in their context, and the confidence to scale systems that deliver real value.

Topics To Be Covered

  • Why agents fail in production

  • Evaluating non-deterministic AI systems

  • Building adaptive evaluation frameworks

  • Defining production-ready AI success

  • Scaling reliable agentic AI solutions

Perfect For

  • AI Product Managers

  • AI Architects

  • Data Scientists

  • AI Engineering Leaders

  • Enterprise AI Teams

Meet Your Speaker

Nour Eid

AI Product Analyst, Hazelheartwood

Nour Eid is an AI Product Analyst specializing in Generative AI and agentic systems, with a focus on evaluation, reliability, and scalable deployment. Her experience is rooted in the automotive sector, where she has contributed to one of the largest AI initiatives at Toyota Motor Europe.

Nour's work centers on testing and validating agentic systems in highly variable, non-deterministic environments, where each use case introduces unique behaviors and challenges traditional evaluation approaches. To address these challenges, she developed a comprehensive evaluation framework that has been adopted across multiple teams, enabling the delivery of more robust and production-ready AI systems.

She also designs AI maturity assessments and contributes to AI strategy initiatives, helping organizations define and measure AI value, identify high-impact use cases, and scale solutions that deliver measurable business outcomes.

ADDITIONAL INFORMATION

Time & Place

Thu, Nov 26

14:30 - 15:00

Matterhorn I

Limited to 45 participants.

Secure your seat – registration required.

Notes

Agenda for this session

  • 20 min presentation + Audience Q&A

REGISTRATION

In order to register to this session you must hold a Pro or Max Pass.

Limited Seating Still Guaranteed

WhatsApp button (66 x 66 px).png
bottom of page