The Data API enables the creation of custom dashboards in your BI tools. Base tables such as conversations, interactions, plugin calls, plugin resources, and users allow you to build dashboards, which are presented by Moveworks under Copilot Insights with more granular access controls, or to build your own custom dashboards.

This guide focuses on creating metrics using the Data API and teaches you how to use SQL queries to build metrics on top of the data ingested through the Data API.

Building knowledge insights through Data API

📘

We want to build out a dashboard which allows us to understand how knowledge base plugin is being used in AI Assistant and achieve the following goals

1. Provide insights to our knowledge team on how knowledge is served through the AI Assistant
2. Learn user feedback on the served results.

Moveworks also provides this dashboard as part of the plugin insights reports. Learn more about the dashboard here.

Let’s go through few metrics covered in this dashboard and what is required for us to build them.

1. We want to understand how many times Knowledge base plugin was served successfully to the end user : This details can be successfully fetched from the the plugin calls table where we filter for knowledge base plugin and check if the served flag is true or not.
2. We need information on the user feedback when the knowledge base plugin was served to the end user : The user feedback is captured in interactions table when a user has filed a feedback form.

Total number of conversations where Knowledge base plugin was served successfully.

Let’s understand the query that is required for us to build this metric. We need to find the total number of conversations where knowledge plugin was Used and distribute this data based on date.

These are the following columns that we need to use

• Conversation ID
• Plugin name
• Used
• Created time for conversation

The following SQL query creates a table view which can be used in PowerBI to create a trend chart.

SELECT 
    TO_DATE(CT.CREATED_DATE), 
    COUNT(CT.CONVERSATION_ID) AS CONVERSATION_COUNT
FROM 
    DATA_API.PROD.CONVERSATIONS CT
LEFT JOIN 
    DATA_API.PROD.PLUGIN_CALLS PC
ON 
    CT.CONVERSATION_ID = PC.CONVERSATION_ID
WHERE 
    PC.PLUGIN_NAME = 'Knowledge Base' 
    AND PC.USED = TRUE
GROUP BY 
    TO_DATE(CT.CREATED_AT);

• Please edit the SQL queries according to the table and field name you have created in your data lake.

The result of this query should look like below

You can define this SQL query while connecting PowerBI with Snowflake

Let’s select a trend chart and plot the date on x-axis and count of conversations on the y-axis. This will provide us a trend chart to view to see conversations where knowledge was server per date.

User feedback when knowledge base plugin was served

User feedback is captured as an UI form interaction whenever an end user has submitted the feedback. To understand what user feedback was provided when a knowledge plugin is served we need to find conversations with feedback UI form submission and knowledge plugin call.

Query to fetch feedback type and free-text verbiage.

SELECT 
    PARSE_JSON(IT.DETAILS):content AS Feedback_content, 
    PARSE_JSON(IT.DETAILS):detail AS Feedback_type 
FROM 
    DATA_API.PROD.INTERACTIONS AS IT 
WHERE 
    IT.TYPE = 'INTERACTION_TYPE_UIFORM_SUBMISSION'
    AND IT.LABEL = 'feedback'
    AND IT.conversation_id IN (
        SELECT 
            pc.conversation_id 
        FROM 
            PROD.PLUGIN_CALLS AS pc 
        WHERE 
            pc.PLUGIN_NAME = 'Knowledge Base'
            AND pc.USED = true
    );

This query can be passed in the snowflake connector for powerBI and a table visualization can be made using this data.