Introducing the new raw interactions table that comes with enhanced data quality, new accessibility and extensibility features.

What is changing ?

The new interactions table brings changes in two broad verticals :

1. Enhanced Data Quality
2. Accessibility features

Let’s understand each change in detail..

Enhanced data quality

We have enhanced data quality of three important entities across MW analytics. Plugins, Domains & Topics. This is done via implementing a semantic data layer that translates raw data in conceptual formats for user to consume.

Plugin status

Challenges that we heard from customers

1. Plugin status is ambiguous :

   Statuses that were “Completed” but both successful and had no results were confusing.

   The differences between Incomplete & Completed - No Results was also unclear and required clarification.

2. Plugin status lacks granular information :

   The statuses reflected what the Moveworks Reasoning Engine did, but not what was served to the user OR what the user did once served the plugin.

   This made it difficult to fully understand the plugin performance and user action.

Changes after semantic layer

The new semantic layer data provides us with new plugin statuses, that helps in understanding what happened in a conversation, was a user successful or end up filing a case. What are scenarios where you AI assistant falls short & scenarios where it’s performing great.

Below are the new plugin statuses that are introduced : Unsuccessful, Served, and Used plugins. We have backfilled data for the new plugin statuses to August 2024

Let’s understand it more with the help of an example :

Here’s how the MW AI assistant processes any conversation →

Phase 1:

The reasoning engine analyzes a user query and considers serving Knowledge base and Forms plugins, but also thinks the access software plugin can be useful. After searching for resources, it cannot find any relevant knowledge articles and forms.

[AI assistant considers Kb and Forms but does not serve them, hence these plugins are unsuccessful]

Phase 2:

The reasoning engine finds a software resource and serves results to the user via Access software plugin

[At this point AI assistant triggers a relevant software automation workflow and hence this plugin is Served]

...User provides business justification & the role they want for the software

Phase 3:

The reasoning engine sees that the user has provided me everything and completes provisioning the software request.

[The plugin was successfully used by the user and the query was resolved, hence the access software plugin is Used]

Primary conversation domain

Challenges that we heard from customers

1. Interaction level domain prediction model had low coverage.
2. Customers often asked us why they cannot reclassify a conversation from HR to Procurement, or categorize conversations that fall outside of predicted domains. This was not possible because Moveworks predicted domain using an offline model that didn’t take into account configurations in MW Setup.

Changes after semantic layer

1. We are transitioning from offline models to a new online model for domain prediction and shifting from utterance-level to conversation-level domain attribution for better accuracy significantly improves domain coverage and precision.

   1. We expect domain coverage to be boosted upto to ~85%.

2. Customers can further boost domain prediction by adding positive trigger examples through MW Setup, enhancing precision in the process.

   Head over to domains in MW Setup and follow the instructions :

   

3. All interactions in the new raw interactions table will have a primary conversation domain associated to it. This is the domain of the overall conversation that a user had which might contain multiple interactions.

   1. All interactions of a conversation will have same primary conversation domain.

      

Conversation topics

Challenges that we heard from customers

1. Multiple topics tagged for utterances related to the same user request. This created confusion and partial-reliable data.
2. Conversation topics like greetings, banter, live agent interactions, and general software queries often went unpredicted. Since topics were identified at the interaction level, this led to some parts of conversations being predicted while others weren't, making topic analysis challenging.
3. Topics not attributed to interaction like link and button clicks, this led to undercount the topic coverage and made it difficult to analyse employee issues.

Changes after semantic layer

1. To enhance the accuracy of topic classification and improve the overall coverage, we have started attributing topics at conversation level.
   1. Topic coverage across to be boosted up to ~86%. Precision is around 97%_ (_manual annotation*)
2. 6 new topics for usually unpredicted scenarios.
   These are: Greeting, Live Agent, Lookup - other, Software - other, Knowledge query - other.
3. A column of Conversation topics : A comma separated set of topics that correspond to an entire conversation rather than an interaction.

Accessibility features

New filtering experience

There are no longer any restrictions on filterable entities. With the new filtering feature, users can search through any data seamlessly. Whether a data column has 100 unique items or thousands, users can easily filter interactions to pinpoint the exact data point they need.

Enhanced export limit

Users can now export the raw interactions data for upto 200K rows. A 10X increase from the existing limit.

Multi downloads

Users can queue up multiple exports at a time from the interactions table.

No more waiting for one export to finish to take another..