AI Assistant Chat Playback

AI Assistant Chat Playback is a troubleshooting tool that lets you replay any user conversation with the AI Assistant. It provides visibility into the assistant’s reasoning process — from how it interpreted the user’s query, to which plugins it considered and executed, to how it generated the final response.

Use Chat Playback when the AI Assistant delivers an unexpected response and you need to understand why. Common scenarios include:

The assistant not serving expected knowledge articles, files, or forms
The assistant serving the wrong type of resource (for example, a form instead of a KB article)
A software provisioning request failing
Ticketing actions like creating a ticket or adding a comment not completing
The assistant citing irrelevant or incorrect resources

Conversation ID Required

Chat Playback is scoped to a single conversation session. You need the conversation ID to look up a specific interaction. You cannot search by user email or browse user conversation histories.

Access Requirements

Chat Playback displays sensitive conversation data including user queries and plugin arguments. Access is restricted to users with one of the following roles:

Role	Access Level
Super Admin	Full access to all conversation data
Moveworks Setup Admin/User + PII Authorized Viewer	Full access to all conversation data

A Super Admin assigns these roles through Moveworks Setup. Only grant access to users who have a legitimate troubleshooting need.

Audit Logging

All access to Chat Playback is logged. The system records which user accessed a conversation and when. These logs are available for compliance review.

How to Use Chat Playback

Open Chat Playback

In Moveworks Setup, go to Advanced Tools > Troubleshooting Apps > Chat Playback.

Enter the conversation ID

Paste the conversation ID for the interaction you want to investigate. Conversation transcripts are available 3–4 hours after a conversation is completed.

Chat Playback landing page with conversation ID search field

The conversation will begin loading. You will see the conversation transcript on the left and the Reasoning panel on the right.

Review the reasoning steps

Once loaded, the reasoning panel breaks down the assistant’s processing into four numbered steps so you can pinpoint where an issue occurred: Intent, Plugin selection, Plugin execution, and Response generation.

What You Can See

Chat Playback surfaces the AI Assistant’s reasoning process in four steps. Each step maps to a phase of how the assistant processes a user query. Select any user message in the conversation transcript on the left to see the corresponding reasoning breakdown on the right.

Step 1: Intent

How the AI interpreted and reformulated the user’s request.

Field	Description
User query	The exact message the user sent
Rewritten query	The normalized version of the query that the assistant used for plugin execution. Compare this to the original to check if the assistant misinterpreted the user’s intent.

Intent step showing user query and rewritten query

Step 2: Plugin Selection

The process of identifying and ranking relevant plugins to fulfill the request.

Field	Description
Total plugins available to the user	The count and list of all plugins available to the user (filtered by user access controls)
Selected plugins	The subset of plugins the assistant determined were relevant. Each entry includes the plugin name and plugin ID.

Missing plugin?

If an expected plugin does not appear in the selected list, contact Moveworks Support with the conversation ID so they can investigate why the plugin was not selected.

Step 3: Plugin Execution

Results from running the selected plugins to gather information.

Field	Description
Plugin name	The name of the executed plugin
Status	Whether the plugin executed successfully (`Success`) or failed
Input Arguments	The arguments passed to the plugin (for example, the search query). Review this to verify the assistant extracted the correct parameters.
Retrieved Resources	The resources returned by the plugin — knowledge articles, files, forms, tickets, or other items. Each resource shows the name, source system, resource type, and a content snippet.
Errors	Any error messages generated during execution (see Plugin Error Codes below)

Plugin execution showing input arguments, retrieved resources, and response generation with citations

Step 4: Response Generation

How the final response was constructed using retrieved information.

Field	Description
Citations	The list of resources cited in the response, shown with the resource name, source system, resource type, originating plugin, and a content snippet

Plugin Error Codes

When a plugin execution fails, Chat Playback displays one of the following error codes:

Error Code	Message	Description
`PLUGIN_ERROR_INTERNAL_ERROR`	An unexpected error occurred while running this plugin.	A general execution failure. Escalate to Moveworks Support if this persists.
`PLUGIN_ERROR_INVALID_ARGUMENTS`	The plugin received invalid or unrecognized arguments.	The assistant passed arguments the plugin could not process. Review the plugin configuration.
`PLUGIN_ERROR_UNCLEAR_ENTITY`	The plugin could not resolve which entity was requested.	The user’s query was ambiguous. Consider adding demonstrations to help the assistant disambiguate.
`PLUGIN_ERROR_MISSING_ARGUMENTS`	The plugin was missing required arguments to complete this action.	Required parameters were not extracted from the user’s query. Check slot configurations for the plugin.
`PLUGIN_ERROR_NO_RESULTS_FOUND`	The plugin could not find any results matching the request.	No matching resources were found. Verify that the expected content is ingested and accessible.
`PLUGIN_ERROR_ACTION_ERROR`	The plugin failed while attempting to complete the requested action.	The downstream action (for example, creating a ticket) failed. Check the integration configuration.
`PLUGIN_ERROR_TIMEOUT`	The plugin execution timed out.	The plugin took too long to respond. This may indicate an issue with the connected system.
`PLUGIN_ERROR_CONFIG_ERROR`	The plugin is not configured correctly.	The plugin has a configuration issue. Review the plugin settings in Moveworks Setup.

Troubleshooting Guide

Use the following table to map what you observe in Chat Playback to the corrective action.

What You Observe	Likely Cause	Recommended Action
Rewritten query does not match user intent	The assistant misinterpreted the query	Review the conversation context. If this happens frequently for similar queries, escalate to Moveworks Support.
Expected plugin is not selected	The assistant did not identify the plugin as relevant for this query	Escalate to Moveworks Support with the conversation ID so they can investigate plugin selection behavior.
Plugin is selected but not executed	Another plugin was chosen as a stronger match	Escalate to Moveworks Support with the conversation ID for further investigation.
Plugin executed with wrong arguments	The assistant extracted incorrect parameters from the query	Escalate to Moveworks Support with the conversation ID and the expected vs. actual arguments.
Plugin execution failed for an action plugin	The downstream system returned an error or the action could not be completed	Copy the Trace ID from the reasoning panel and look up the error details in Built-in Plugin Logs. The logs show the external API calls and error responses from the connected system.
Plugin returned wrong resources	Content ranking or relevance issue	Verify the expected content is ingested and accessible. Escalate to Moveworks Support with the conversation ID if the issue persists.
Plugin returned no results	Content not ingested or not accessible to the user	Verify the content exists, is published, and the user has the appropriate permissions to access it.
Bot response cites irrelevant resources	Citation selection issue	Verify the expected content is ingested and accessible. Escalate to Moveworks Support with the conversation ID for further investigation.

What Is Not Available

To protect Moveworks intellectual property and maintain privacy, the following are not visible in Chat Playback:

Internal reasoning or orchestration logic
Custom bot instruction details
Confidence scores for considered plugins
Enterprise Search tab logs from the Moveworks web app

When to Escalate to Moveworks Support

Escalate to Moveworks Support when your investigation requires insight into:

How the assistant interpreted the query beyond what the rewritten query shows
Why an expected plugin was not selected or executed
Issues with plugin arguments or unexpected plugin behavior
Tone, style, or conversational behavior of the assistant
Any behavior that cannot be explained from the data visible in Chat Playback

Chat Playback is designed to help you identify what happened during a conversation. For issues that require understanding the assistant’s internal decision-making logic, Moveworks Support has access to additional diagnostic tools.

Governance

Privacy-Preserving Design

Chat Playback follows a privacy-preserving approach. You can view all operational details necessary for troubleshooting — including the user query, rewritten query, and plugin arguments — while other user PII and full conversational content remain protected.

Session-Bound Access

All debugging is scoped to a single conversation session. You cannot search across sessions, browse user histories, or view all conversations for any user. This ensures that access is limited to the specific interaction being investigated.

Audit Logs

Every access to Chat Playback is recorded with:

The user who accessed the conversation
The conversation ID that was viewed
The timestamp of access

These logs are available for compliance and security review.

SIEM Event Structure

When a user accesses a conversation in Chat Playback, the following event is emitted to your SIEM integration:

1 {
2   "event_log": {
3     "conversation_id": "<conversation_id>",
4     "log_context": {
5       "log_level": "LOG_LEVEL_INFO",
6       "summary": "Chat evaluator conversation accessed"
7     }
8   }
9 }

You can use this event to monitor and alert on Chat Playback usage within your organization’s security tooling.