The Reasoning Loop
The reasoning engine doesn’t predict once and stop. For every request it runs one loop: plan, check the plan, run tools, look at what came back, and plan again. The loop runs at most 10 iterations per request. It ends when the engine can answer the user with no further actions to take, or when it needs something from the user.
Each pass through the loop has two phases:
- Planning. Work out the most useful next step and check it before anything runs.
- Execution. Run the plan, observe the outcome, and decide what to do next.
Between requests, the user’s replies feed back into the next turn. See User feedback across turns.
Planning
The user’s request starts the planning phase.
Select a policy
The engine picks from a priority-ordered list of reasoning policies. The primary policy is LLM-based: it reads the full context window and predicts the next action, such as which tool to call, which slot to fill, or what to say. Other policies can take over in specific situations, such as error recovery, keyword matching, or safety checks.
Propose a plan
The selected policy produces a candidate plan: one or more steps the engine wants to take next. The planner builds it from the tools available to this user. A plan can be a single tool call, a reply to the user, or several actions to run in parallel. Choosing the tools and their arguments is covered in Tool Selection.
Evaluate the plan
Before anything runs, evaluators check the proposed plan. Heuristic evaluators check its structure. LLM evaluators, including the plan evaluator, check whether the plan actually addresses the user’s need given the context. If an evaluator finds a gap, it sends that feedback to the planner, which builds a new plan. This is why the engine sometimes appears to change its mind mid-reasoning: it proposed something, an evaluator rejected it, and it tried again.
The accepted plan moves to execution.
Execution
Execute
The engine runs the plan by calling tools. Tools are the components that do the work: search, workflows, API calls, running code, and LLM calls for tasks such as summarization. Slots get filled and actions fire. When steps don’t depend on each other, the engine runs them in parallel and combines the results into one reply. See Multi-Tool Responses.
Observe the outcome
The engine looks at what each step returned. That can be a successful result from a search or workflow tool, or an error: an invalid operation, missing required information, or a request from the tool to get the user’s confirmation.
Process the response
Small tool responses go into the context window as-is. Large structured responses are stored as a named variable, and the engine sees a truncated preview. See Processing Tool Responses.
Add to the conversation and decide
The executed step and the processed response are added to the conversation history. The reasoning LLM then decides whether it can keep going with what it already has, from your organization’s knowledge or the current conversation (see What Goes Into a Turn), or whether it needs input from the user. If there’s more work, the loop runs again with the updated context. Either way, the engine tells the user what steps it is taking.
User feedback across turns
When the loop stops, one of two things happens:
- The engine needs input to take the next step, so it asks the user for confirmation, feedback, or required information.
- The engine answers, and the user sends a new request or a follow-up.
Either way, the user’s next message starts the loop again, with planning and execution running on the updated conversation. What carries over between messages is covered in Conversation History.
The engine is a next-token predictor, not a rule executor. Every decision in the loop, from which policy to use to what plan to propose to whether to keep going, comes from the LLM reading the full context window and predicting the most likely continuation. Instructions in descriptions are probabilistic signals, not commands. The engine might follow one 9 times out of 10 and miss it on the 10th. See The Golden Rule for how to enforce behavior instead.