AGUI Lights Up the Assistant Builder Playground

I will pass over to yourself and for the demo on HUI and the user consent.

Alexandru Rus 37:47

Thank you, Michael. Share my screen. So, um...

Last few weeks, we've looked at leveraging more parts of Microsoft Agent Framework. So Microsoft Agent Framework is what currently powers Agent Builder, all the... User interactions, all the agentic loops are done through the agent framework.

That is a framework that allows for highly customizable workflows and providers of AI context. So the more we leverage the capabilities of the framework, the easier is for us to build capabilities on top of it. One of the capabilities of the framework is to expose an agent as an AGY agent.

So, HUI stands for Agent User Interaction, which is essentially a way a way a protocol for a user to interact with with an agent. So, this means that there's a standard defined for how the... Agent streams the response back?

How does the conversation get managed? How does state is how is state passed between the user and the and an agent, which is something that we've actually not we've not leveraged too much in the past. but I think would be a good way to enhance our user experiences, especially when embedding AI inside products as well. So our first implementation of this is in the test chat in Agent Builder.

So in the Agent Builder test chat, we have this new toggle here at the top right to enable the usage of AGUI protocol. If you have this disabled, which I think it is by default, you're just using the old experience. which was not leveraging the new protocol, but had a custom implementation behind it. But for this specific demo, I'm not actually going to use this UI.

I'm going to switch over to the new assistant builder. that we have. And I am here in the playground section. Now, as a first demo, I will come and choose my financial credit control agent, and I will ask a question like, show me top three.

So when I click send here. So now in the implementation with this protocol, we have a lot more information on the feedback that is happening inside the chat. So I see operations being called.

I can see, especially for testing and debugging agents, I can see how long certain operations took. I can also see the number of tokens that each tool returned in the response, which will help me make my agent more efficient. Because I can easily see, okay, this tool returned too much information.

This is something that I can. optimize for better usage, better cost efficiency, and so on. Now, in this case, I think it didn't select the proper database, so I'm just going to sort of continue the conversation here. Select the sandbox instance.

And hopefully this find, find. The top three, of course, as all demos, this wasn't quite the scripted path, but let's hope that we can we can continue. And we can show some of the other bits as well.

Interesting. Let me quickly try to refresh. Not quite sure what happened there.

Ohh. So, I'm just going to from... Actually, I'm just going to show me my instances.

Um, and I think... I'm just gonna try to use the sandbox one. Sure.

Because this. Okay. Going to work now, so we've also added the capability to render MFEs in line, so obviously MFEs in agent builder agents are modeled as a tool, so we can see whenever one of these tools were invoked, and I can render the... the MFE here in line in the chat experience.

Of course, I have also information about all the token usage, so this would be for the entire assistant run that was from my last message. So all of these have a. input tokens of almost 72,000. The output tokens weren't as much because the response was quite short.

And we also showed the total amount of tokens. So all of these is to help sort of proper debug experiences. Now, what we've also, and we're next looking to leverage this in, is actually using the same capability, the same protocol of invoking the agents to use them from quick asks as well.

One of the feedbacks that we've received is... that agents are running differently between the assistant builder chat