Multi-Plugin Response in Moveworks Copilot

Overview

The Moveworks Copilot can now call multiple plugins in parallel, review the responses from them, and combine their individual responses into a single, summarized response!

This feature is called multi-plugin response and it will enhance the user experience and utility from the Moveworks Copilot by

  1. Surfacing all the relevant plugins to the user for every request
  2. Providing a concise and easy to scan response
  3. Enabling the user to quickly understand what their options are and allow them to follow up on the one they want to pursue

Here is an example of a multi-plugin response that combines information from Knowledge Search with Software Provisioning for Figma:

Screenshot 2024-09-19 at 11.49.47 AM.png

You can see from the AI reasoning what plugins were accessed:

Screenshot 2024-09-19 at 11.50.35 AM.png

What does a multi-plugin response look like?

Here are the key characteristics:

  1. The top part of the response will be a summarized response that lists available options

  2. If there are any buttons the user can click to execute the actions offered, they will be provided at the bottom of the response

    Screenshot 2024-09-19 at 11.56.31 AM.png
  3. If there are multiple calls to action, you can see them at the bottom:

    Screenshot 2024-09-19 at 12.04.28 PM.png
  4. When you take action, e.g. clicking on Yes for Unlock Account, it proceeds with the selected action

    Screenshot 2024-09-19 at 12.04.55 PM.png

How does this feature work?

Multi-plugin response comes with three key changes:

  1. tuning Moveworks Copilot to call all relevant plugins simultaneously,
  2. executing plugins in parallel, and
  3. enhancing the response UX for calls to action (buttons).

(1) Simultaneous Plugin Calls

Before multi-plugin, Moveworks Copilot almost always called one plugin to help with a request. The primary exception was multi-intent requests like “Do X AND help with Y,” which led to sequential plugin calls. Now, the Moveworks Copilot identifies and uses all helpful plugins.

(2) Parallel Execution

MultiPlugin executes plugins in parallel. For single intent requests, this ensures MultiPlugin latency is comparable to current performance; for multi-intent requests, this decreases latency as the Moveworks Copilot executes on all intents at the same time.

(3) Improved Buttons UX

With more plugins in use, calls to action have become more frequent. To improve this experience, we now include buttons for each separate call to action. Each group of buttons comes with a concise Moveworks Copilot -generated description for button context.

What are some known issues and limitations?

These are some aspects of the first version of this experience that we are aware of, and will work on improving in future iterations:

  • While the Moveworks Copilot will call multiple plugins as frequently as possible, sometimes the plugin you are looking for may not appear in the list of plugins
  • Duplicate call to action
    • In cases where multiple plugins require input from the user, like missing arguments or confirmation, we want the Moveworks Copilot to emphasize the call to action. This makes it much easier for a user to identify action required to resolve their issue. We also leverage generative LLMs to generate headers for button groups, which are related directly to these calls to action. Sometimes this can lead to some redundancy in the response.
  • Since he buttons associated with plugins will appear at the bottom of the card, they may be separated from the summary that references them.
  • Occasional verbose updates and persistent calls to action
    • If the Moveworks Copilot offers multiple options with follow-up actions, and the user selects one, it may sometimes continue to provide the option to select the other.
  • Free text inputs such as business justification are sometimes not recognized and the bot may jump out to offer other plugins (sporadically occurred before the introduction of multi-plugin response also)
  • Any instructions added to customize bot input may need to be modified with multi-plugin response since the format of the response could be different.

How will this feature impact Copilot Insights?

The main impact will be an increase in plugin calls - both successful and unsuccessful.

  1. In the interaction table, you will see more plugins in each interaction row.
  2. In the list of plugins on your Moveworks Copilot Insights home page, you will see an increase in the number of plugin calls, particularly for the most common plugins such as Knowledge Search.