Overview

Triage 2.0 is a dramatic improvement from its predecessor as its architecture is completely different. Now, Triage classifies tickets by building an org-field specific index and using semantic search to classify tickets. Read More - https://help.moveworks.com/docs/triage-2-pt-0-technology-overview.

If you are upgrading from 1.0, you should be aware of the differences:

• Triage 2.0 leverages existing configs such as Ticket Filters, Field Configs, Heuristic Settings, & Metadata Cache Setup
• All models must upgrade to Triage 2.0 for a given ticket type e.g. incident
  • You cannot mix models, e.g. have assignment group on 2.0 but category+subcategory on 1.0 as that won’t function.
• Triage 2.0 leverages the EXI tables when pulling tickets to build an index
  • You may need to use Ticket Backpolling feature with Moveworks Setup to fill the EXI tables.
  • If the training data requires manually editing, reach out to Moveworks Support.
• Triage 2.0 has AutoML as well which continuously indexes the data
  • By default, this happens once every month at 1st of every month.

Step 1: Prepare for Triage 2.0

Before training the model, there are several important steps:

1. Configure the Triage Ticket Filter

   1. Navigate to Ticketing -> Ticketing Settings -> Ticket Filters -> Skill-specific ticket filters -> Ticket filter for Triage skill

   2. For guidance on how to set ticket filters, including for Triage, read https://help.moveworks.com/docs/ticket-filters#/

      Ensure to validate syntax by testing tickets that are in-scope of Triage

      

2. Configure the custom_data mapper in the ticket proto

   1. Navigate to Ticket Mapping (Select ITSM) -> Configure TICKET TYPE

   2. Go through each TICKET TYPE’S ATTRIBUTE MAPPING and confirm the fields to be triaged are under the custom_data . Most likely, they already exist due to default settings.

      

   c. Go through each ticket type’s UPDATE TICKET PAYLOAD and confirm the fields to be triaged are mapped as well. Most likely, they already exist due to default settings. If triage is enabled BEFORE ticket creation, make sure CREATE TICKET PAYLOAD is configured and mapped as well.

   

3. [ASK FDE] Configure the Metadata cache filters (ServiceNOW ITSM only)

   1. Navigate to Triage > Metadata Cache
   2. Click “Create” or “Edit” Display Name (ticket fields)

      • Field Name: The name of the triage field we’re predicting

      • Integration ID: only snow can be accepted

      • Copy To Integration Ids: The integration IDs to which to copy the retrieved ticket field metadata. Used for integration testing.

        • snow_dev

      • Snow Metadata Retriever Config Table Name: the SNOW table of the sys_id we are predicting - sys_user_group or cmdb_ci

      • Snow Metadata Retriever Config Raw Query Fields: URL used to filter and retrieve status of labels

        Example queries

        JSX

        operational_status=1^sys_class_nameINcmdb_ci,cmdb_ci_service_discovered

        JSX

        operational_status=1

      • Snow Metadata Retriever Config Id Key Path: Optional field used to specify an alternative value to use as the ID in the TicketFieldMetadata proto. Should be the dot-delimited path to the value in the payload returned by the REST API call for the ticket field metadata.

      • Snow Metadata Retriever Config Name Key Path: Optional field used to specify an alternative value to use as the name in the TicketFieldMetadata proto. Should be the dot-delimited path to the value in the payload returned by the REST API call for the ticket field metadata.

      • Snow Metadata Retriever Config Max Record Count: Optional field used to specify the maximum number of ticket field metadata records to pull from the REST API. If unspecified, a default value defined in SnowMetadataRetriever will be used instead.

      • Snow Metadata Retriever Config Page Size: The page size to use for paginating calls to the REST API. If unspecified, a default value defined in SnowMetadataRetriever will be used instead.

   3. Enable the following THREE flows in Setup Internal -> Ingestion > Scheduled Flows:
      • Flows To Run Metadata Cache Loader Flow - An update-only job that maps the configured tables’ sys_ids to display_names
      • Flows To Run Metadata Cache Loader Full Flow - A “snapshot” version of the the Cache Load Flow above. Existing values will be removed and freshly imported on each run.
      • Flows To Run Metadata Cache Sync Flow - Syncs the production environment metadata cache into the preprod environment. This is similar to how Form Flow and KB Flow are not usually run in preprod directly.

4. Backpoll tickets into the EXI tables via Moveworks Setup

   1. ⚠️ Backfilling tickets ****is required if backpolling has never been enabled before. ⚠️
      1. If this has already been set up for something like EXI, you do not need to re-run the job. This will ingest tickets that Triage 2.0 will analyze when making predictions. Migration from Triage 1.0 → 2.0 still requires this step.
      2. Follow this guide: https://help.moveworks.com/docs/ticket-back-polling-for-exi
   2. Once backpolling is complete, wait 24 hrs before proceeding to Step 2.
      1. Why? An internal job is needed to populate the analytics tables for building the index.

Step 2: Create / Update the Triage Field Configurations

If you are upgrading from 1.0 to 2.0, these should already be already set. If not, following these steps,

1. Navigate to Triage > Field Configurations
2. Click “Create” or “Edit” Display Name (ticket fields)

   1. Field Name: The name of the triage field, e.g. assignment_group , cmdb_ci, category, subcategory. NOTE: Even concatenated fields should be configured separately.

   2. Ticket Type: Ticket type to be triaged. Ex : incident, or hr_case

   3. Routes

      1. TICKET = This is used for triage by polling existing tickets. These tickets can be created through any source (in or out of the bot). The “risk” is that polling ITSM systems is expected to fail on occasion and some tickets are forced to fail sooner than others
      2. BOT = This is used for triage before ticket creation. The predicted value is determined before the initial create_ticket payload is submitted. This means that it is only applicable to tickets that are created through the bot and no other source.

         • Ensure that the triage field is mapped in “Create Ticket Payload” if BOT option is added in.

         • This field interacts directly with Triage During Ticket Creation config

           TDTC FF is on:

           • TICKET - only non-bot created tickets from polling ⚠
           • TICKET, BOT - non-bot created tickets from polling + bot-created tickets
           • BOT - only bot created tickets are triaged ⚠

           TDTC FF is off:

           • TICKET - all tickets are triage from polling
           • TICKET, BOT - all tickets are triage from polling
           • BOT - No action ❌

   4. Select Connector: Select the connector, which should already have been configured as part of the configuration of the AI Assistant.

   5. Two configurations to setup Enable Triage & Send As Update

      1. For Dry-Run Mode: Enable Triage = false & Send As Update = true
         1. For sandbox orgs, turn ON Analytics and set the org status to LIVE in order to see dry run analytics.
      2. For Live Mode: Enable Triage = true & Send As Update = true
         1. Send As Recommendation: Sends work note with its prediction. Will only work if Enabled Triage = True. Either this or Send As Update must be checked for triage to trigger on a ticket

   6. Required Field (for dependent model): If checked, this model must get a prediction or else Triage will early exit without submitting updates for any other models / fields. This is typically only used for dependent models, ie. if you must triage category in order to also triage subcategory.

   7. Start Date: At times, Triage can run in dry-run mode, which would pollute the data. Apply a start date (YYYY-MM-DD) to clear old data. If unspecified, then analytics dashboard will not show any results for Triage.

   8. Mapped Field: This field is used to classify the route within data analytics. There are 3 values that can be set and it is NOT tied to the name of field that is being triaged. Think of it as a description of the purpose of triaging the field.

      1. The 3 options are:
         1. call_type - Determine which type of ticket should created. In most cases, this should not be configured and is used for the default, built-in model which determines whether a user’s utterance should create a request vs. an incident.
         2. assignment_group - The purpose of this model is to route the ticket to the right party. For example, a business_service model could be of this type because there could be a ServiceNow rule that ties assignment to the business_service.
         3. category - The purpose of this model is to categorize and label tickets appropriately. For example, adding a specific label to a ticket will not change who the ticket is assigned to.

   9. Predicted Value Filter: Apply a blocklist or allowlist which will be acted upon the output of the Triage 2.0 prediction.

      See example for usage of the blocklist

      

   10. Values To Overwrite: Typically, these values are the groups that the L1 service team manually analyzes tickets for triage or any default values for other field types, e.g. category, subcategory

       • If Empty = Triage will predict only on incoming empty values for the predicted field
       • Set as * to overwrite any field that passes the threshold

       See example for usage of the values to overwrite

       

   11. Default Value: If no value passes the confidence threshold, this value will be predicted instead.

       • Use sys_id default value for sys_id models
3. Click “Submit” with the title in the format of ⇒ triage.[predicted_field].[ticket_type].['Route']
   • e.g: triage.subcategory.INCIDENT.['TICKET']

Step 3: Create / Update the Triage Field Heuristics Rule Settings

If you are upgrading from 1.0 to 2.0, these should be already set. If not, following these steps,

1. Navigate to Triage > Field Heuristics Settings
2. Click “Create” or “Edit” Display Name (ticket fields)
   1. Field Settings: Enter “field name” , “ticket type”, “Routes”, “Select Connector” as done in prior steps.
   2. Heuristic Rules:
      1. You MUST create a configuration for heuristic rules even if you don’t have a rule to add. Without the configuration file, TriageService Logs may just return a {} response
      2. The heuristic rules need to be defined separately even for the dependant fields.
   3. Additional Controls: For Dilbert Model Type, copy the model type name from dilbert_service.proto. This is only required for existing 1.0 models
      1. Format: TRIAGE_{org}_{field}e.g: TRIAGE_PARAMOUNT_CMDB_CI_SYS_ID
3. Click “Submit” with the title in the format of ⇒ triage.[predicted_field].[ticket_type].['Route']
   • e.g: triage.subcategory.INCIDENT.['TICKET']

Step 4: Configure Triage 2.0

1. Configure the Initial Setup

   1. Navigate to Triage > Triage 2.0 Configurations

   2. Fill out the relevant information

      1. Field Name, e.g. assignment_group, cmdb_ci, category, etc
      2. Select Additional Features, e.g. requested_for.location, etc. These additional fields are used to inject signal to the ranking of results
      3. Set SQL parameters is typically established by setting at least two important parameters:
         1. ticket type - typically these are unstructured tickets
            1. e.g. itsm_ticket_type = 'INCIDENT'
         2. contact type - filter tickets from digital routes and avoid those from walk-ups, phone automated alerts, etc
            1. e.g. contact_type IN ('Assistant', 'Portal', ‘Email’)
         3. if additional fields are needed, they need to be extracted using a DSL rule
            1. e.g. json_extract_scalar(ticket_custom_data, '$.u_domain') in ('d4743d251b709010519711331d4bcb9d')
      4. Set Lookback Window (Days), e.g. 180 days

   3. Click “Calculate Scope” to generate estimated size of training set

      1. This step runs a SQL query against the stored tickets from the backpolling step.
      2. Once the report is created, run sanity checks on the data
         1. Does the earliest timestamp at 180 days prior, for example?
         2. Is the ticket count above 100,000? If so, then it’s recommended to reduce scope
         3. Is the ticket count at 0? If so, then check if backpolling was ran or was finished over 24 hrs prior
      

   4. Click “Save Configuration” and proceed to the next step.

2. Build Triage Index

   1. Click “Not Started” to proceed with Build Index
   2. Once this step is complete click “Fetch Report” to generate the Precision & Coverage graphic
   3. Select the right model threshold according to your preference
      1. If you want to take a conservative approach, set the threshold for a high precision (> 90%)
      2. Otherwise, set the threshold for a high coverage for maximum impact of Triage
   4. Click “Save Threshold” to save the config.
   
   

Step 5: Turn On Triage 2.0

To enable the model, you will need to make the switch in two places:

1. Triage Behaviour Controls (model-specific)

   
   

2. Triage 2.0 Configurable (product-level switch)