What is the daily data format?

SFTP data shows every single resolution and interaction event, which will allow customers more flexibility in filtering/aggregating the data to build custom reports.

SFTP data covers usage of Classic Moveworks bot and does not include data on usage of Copilot.

This document provides information on the data dictionary (the tables and fields involved), and directions on how it can be aggregated to build the views shown on your daily dashboard that Moveworks provides in Tableau. With this knowledge, you'll have the power to take the data and use it in any internal views/reports/dashboards you may need to build.

Data Dictionary

The daily SFTP data format contains multiple files, while some are used more extensively than others to derive the key metrics, all are necessary to build the complete daily dashboard that Moveworks provides. The files are as follows, and are all prefixed with today's date:

• {{today's date}}_daily_detailed_report.csv
• {{today's date}}_daily_active_users.csv
• {{today's date}}_daily_concierge_tickets.csv
• {{today's date}}_daily_user_roster.csv
• {{today's date}}_daily_triage_record.csv
• {{today's date}}_daily_answers_insights_user_interactions.csv
• {{today's date}}_daily_new_tickets.csv

The date format will be as follows: year-month-day e.g: 2023-12-13_daily_active_users.csv

File 1: Resolution and Interaction Events

File name: {{today's date}}_daily_detailed_report.csv

This is the main file that contains a table of all of the resolution and significant interaction events for the day. Each row in this table represents a resolution or interaction. Below are the descriptions of each column in the table:

*available in some versions of SFTP data

File 2: List of active users

File name: {{today's date}}_daily_active_users.csv

This file contains a table that shows all of the users who were active during the day. A user is considered to have been "active" if they sent a message to the bot OR if they clicked a button/link in a message from the bot. Below are descriptions of each column in the table:

File 3: List of tickets that were updated in the ITSM system

File name: {{today's date}}_daily_concierge_tickets.csv

This file contains a table that shows the list of tickets that were available to the bot for Concierge use cases (eligible for checking status, receiving updates, adding comments, etc.). It is used in calculating the percentage of eligible tickets that users interacted with through the bot. Below are descriptions of each column in the table:

File 4: Full roster of eligible bot users

File name: {{today's date}}_daily_user_roster.csv

This file contains a table that shows the full roster of users in your organization that are (or were previously) available to the Moveworks system. It is used to produce the breakdowns of active users by geographic location and department. Below are descriptions of each column in the table:

File 5: List of tickets triaged by the bot

File name: {{today's date}}_daily_triage_record.csv

This file contains a table that shows all the tickets triaged by the model (it will only be present if the Triage skill is active in your environment). If models are being used to set multiple ticket fields (e.g. Call Type, Assignment Group, Category, etc.), predictions and results for all fields will be in this single file. Below are descriptions of each column in the table:

File 6: List of User Interactions in scope of Answers

The data in this file corresponds to the current edition of the Answers Insights customer-facing Tableau dashboard.

File name: {{today's date}}_daily_answers_insights_user_interactions.csv

This file contains a table that shows all issues (utterances and tickets) that have the potential to be resolved by Answers skill, indicating the outcome of the bot attempt to solve the issue and context around it.

Data uniqueness is defined by the combination of the issue, knowledge article, and issue topic. For example, if 3 articles were served for an issue where 2 issue topics were detected, there will be 6 rows in the data.

Below are descriptions of each column in the table:

File 7: List of daily new tickets

File name {{today's date}}_daily_new_tickets.csv

This file contains the list of new tickets ingested daily from ITSM system.

File 8: List of Channel Resolver issues

File name {{today's date}}_daily_channel_issues.csv

This file contains the list of issues in the Slack channels where Channel Resolver is active in.

Description of metrics derived from the data

In the following section, we go through commonly used metrics shown in Performance Insights dashboards that Moveworks provides in Tableau, providing a description and the corresponding SQL query or arithmetic calculation you can use to derive these metrics from the data in the files discussed above.

HIGH-LEVEL METRICS

Absolute Number Metrics

These metrics are counts of users, resolution, interactions, and forms filed. Below is a table that outlines how each of these metrics are calculated:

Percentage Metrics

There are two percentage numbers in this section - resolution percentage and interactions percentage. The key to understanding these metrics is to understand what is included in the numerator of the percentage calculation and what is included in the denominator.

Resolution Percentages

This metric calculates a percentage of users' issues resolved by the bot. All relevant resolution events from the main events table are counted in the numerator (only events that are rated "not helpful" are excluded).

To obtain the correct denominator, the starting point is the number of new tickets considered for resolution on the day (using SQL, this can be found with the following query: select count(distinct ticket_it) from {{date}}_daily_new_tickets). But this number is not the complete denominator, because many of users' issues are addressed through chatting with the bot and no corresponding ticket is filed. Thus, we need to add these events to the denominator, whether they were resolved or not.

At a high level, the events that need to be added to the denominator are those that do not originate from a ticket, and for which no new ticket is filed. These can be found by choosing the events that do not have the ticket_id column populated (using SQL, this can be found with the following query: select count(*) from {{date}}_daily_detailed_report where skill_category <> 'Interaction' and ticket_id is null). Since the logic behind whether a ticket is filed depends on the skill_category, it is best to explain the details with the following table that looks at all the types of resolution events (i.e. combinations of skill_category and skill_detail):

Interactions Percentage

The goal here is to calculate the percentage of tickets that users interacted with through the bot. Thus, the numerator is the count of unique tickets that had interaction events in the main events table (using SQL, the query would be select count(distinct ticket_id) from {{date}}_daily_detailed_report where skill_category='Interaction').

The denominator is the total count of unique tickets that were eligible for Concierge interactions that day, from the daily concierge tickets table (using SQL, the query would be select count(distinct ticket_id) from {{date}}_daily_concierge_tickets).

EVENTS BREAKDOWN

The table below provides descriptions and SQL queries for the Resolution breakdown.

ACTIVE USERS

The table below provides descriptions and SQL queries to count number of users engaged with Moveworks bot.

TRIAGE PERFORMANCE

The table below provides descriptions and SQL queries for Triage metrics.

KNOWLEDGE

The table below provides descriptions and SQL queries for Knowledge metrics.

FAQ

Q: When can I expect SFTP data to land?

A: SFTP reports will be produced the following day by 11:00 PM UTC Time. For example, the data for the 24 hours of January 1st will land by 11PM UTC on January 2nd. Note: data from each file will have no overlap of data.

Q: Do the files grow in size?

A: No, the files are sent for the days data, and a new file will be added for the next day.