Generative AI apps and agents

What this area tests

This is the biggest hands-on area of AI-901:

• writing effective system and user prompts
• deploying a model and trying it in the Foundry portal
• building a small chat client with the Foundry SDK
• creating and testing a single agent in the portal
• building a small client app for that agent

How Foundry is organized

You work in a Foundry resource that contains one or more projects. Inside a project you deploy models from the catalog, build agents, and run evaluations. Your app talks to the project’s endpoint using the Foundry SDK, which is available for Python, C#, JavaScript and Java. The web portal at ai.azure.com is where you deploy, test and inspect all of this without code.

Writing good prompts

• Use the system message to give the model a role, rules and the format you want, for example “Answer in three bullet points.”
• Put the actual request in the user message.
• Give examples of good answers when format matters (few-shot prompting).
• Ground the model in your own data when answers must be factual.

Grounding with your data (RAG)

Retrieval augmented generation combines search with a language model so answers are grounded in your data. At question time, relevant passages are retrieved and passed to the model with the prompt. Microsoft’s guidance: use RAG when answers must be grounded in private or frequently changing data. Use fine-tuning when you need to change the model’s behavior, style or task performance, rather than add fresh knowledge.

Agents

Microsoft defines an agent as an AI application that uses a model from the Foundry catalog to reason about user requests and take autonomous actions to fulfill them. It has three parts:

• Model: provides the reasoning and language ability.
• Instructions: define goals, constraints and behavior.
• Tools: give access to data or actions, such as search, file operations or API calls.

The agent may reason and call tools several times before it answers. Tracing records each model call and tool call, so you can see why it did what it did.

Chat client or agent? A chat client sends prompts straight to a deployed model and shows the reply. An agent adds instructions and tools, and can take actions. If a scenario needs the AI to do something, such as look up an order or file a ticket, it’s an agent.