Welcome to another weekend bonus episode of the Tech Mem Ride Home Podcast. I'm Brian McCullough as always.

This is a portfolio profile episode. Haven't done one of these in a while.

This was actually one of the first AI investments that I ever made almost exactly a year ago now but before the AI fund was

stood up. We're talking to Trace Loop and Nere Gazit the founder of Trace Loop and Nere Gizit, the founder of Trace Loop.

Nere, thanks for coming on the show.

Thank you.

Great to be here.

Let's start off with just sort of the elevator pitch of what Trace Loop does and why I jumped at it when I got introduced to you.

So Trace Loop is a platform for monitoring

LLC applications in production. So basically

detecting hallucinations, monitoring token usage, and everything that you want to know about how your

LM is performing in production. So it's it's observability for

LLLMs it's It's like Data Dogg for L. M. in a way. But you know people are

familiar with you know finding bugs and getting notifications of things that aren't working

what is what is different about doing that in a deployed

LLC environment like are you able to not only see oh this isn't working or there was a crash or a bug here but also

this output is bad because the I don't know the input was wrong or the weights are are you able to get that level of granularity in a deployed

L. L. M.

Yes and I think the main challenge here is that it's really hard if you get like an

arbitrary generated output from an LLLM it's really hard to tell what's a good output versus a bad output.

So what people are usually doing is that they're using this method called

LLM as a judge.

...