Agent Basics 05 Teaches Agents to Search Again a Different Way

Agentic RAG means letting the model handle retrieval itself: if what it finds is wrong, it rewrites the query, switches data sources, and searches again until the answer is reliable.

Remember These Points First

• Regular RAG: retrieve first, then have the model answer based on the results, one straight path to the end.
• Agentic RAG: the model calls tools and checks the results. If they're not good enough, it rewrites the query, switches sources, and tries again, like a loop of 'check your own work'.
• It decides for itself what to do next, rather than following a process written in advance.
• Best suited to scenarios that demand high accuracy, require querying databases, or involve long multi-step processes.

Going Deeper

How the loop runs. The model issues a query → gets the results → evaluates whether they're good enough → if not, rewrites the query, tries a different retrieval method, or calls another tool → only answers once the results are sufficient. You don't need especially complex orchestration. A simple loop of 'call the model, use tools, call the model again' can produce very solid results.

Why it's called 'reasoning on its own'. For example, if you ask it to plan a product launch strategy, you don't have to hard-code 'first check market trends, then look at competitors, then review user feedback'. It judges for itself what to look up first, and when the material is insufficient, it fills the gaps.

Suitable scenarios. Situations where a wrong answer is costly (legal, medical, compliance); querying structured databases where users' phrasing is imprecise and rewriting is needed; long tasks that can't be covered in one search and need multiple rounds.

Caution. Every extra round means another model call, so cost and time both go up. Set a maximum number of rounds, and hand the task to a human after repeated failures.

Image from the course repository.

Try It Yourself

Ask the model any question that requires up-to-date information, and tell it to 'search first, judge whether the results are sufficient, if not, change the keywords and search again, for at most three rounds'. Watch how it rewrites the query.

Let Your Agent Teach You

Give the following to your agent (Claude Code, ChatGPT, and Gemini all work), and it will guide you step by step like a mentor.

References

• Content adapted from Lesson 5 (05-agentic-rag) of microsoft/ai-agents-for-beginners, originally licensed under MIT.
• For the original and full code: 05-agentic-rag
• For all courses, see the Agent course roadmap