Until your Enterprise Assistant moves to the Agentic Orchestrator, it runs on the Agentic Reasoning Engine, which cannot call a remote agent through its Agent Card. You can still connect it to external AI agents, LLMs, and AI-powered applications: wrap the external API in a Tool Studio plugin with HTTP Actions. This cookbook covers how to keep conversation context across turns, handle asynchronous APIs that require polling, manage cost and data privacy, and make the plugin trigger reliably.

Before You Start

• On the Agentic Orchestrator with an agent that publishes an Agent Card? Register the agent under A2A Agents in Tool Studio instead and select it for your assistant; the Orchestrator manages the conversation for you. See A2A Agents. Nothing on this page is needed.
• Otherwise, build a plugin. A plugin wraps the external API call so you keep control of when it triggers, through the plugin’s name, description, and utterances; what it sends, through slots and validation; how the reply is presented; and how data flows, through your configured connectors.
• A trusted MCP server is an option for bounded lookups. If the external system already exposes the operations you need as an MCP server, MCP Workspace can connect it without a plugin. Use a plugin when you need organization-specific logic or tighter control over inputs and outputs.

Architecture Decisions

Context Engineering

There are three ways that you can manage context, each with their own pros & cons.

Example 1: Reasoning Engine Context via Slots (Simplest)

This approach lets the Agentic Reasoning Engine manage conversation context for you. The reasoning engine tracks the conversation history and decides what context to pass to your external API on each turn. This is the fastest way to get started, no thread tracking or database needed.

Your plugin will look something like this:

For the easiest implementation, we recommend the following high-level approach.

curl https://api.anthropic.com/v1/messages \
  -X POST \
  -H 'Content-Type: application/json' \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H 'anthropic-version: 2023-06-01' \
  -d '{
    "model": "claude-3-5-sonnet-20241022",
    "max_tokens": 1024,
    "messages": [
      { "role": "user", "content": "{{user_query}}" }
    ]
  }'

user_query: |
    $CONCAT([
        "'UserInput:'",data.query,
        "'PreviousBotMessage:'",$TEXT(data.conversation_context)
    ])

Example 2: API-Managed Thread with an Optional Thread ID Slot (recommended if available)

Some external APIs keep track of the conversation thread for you — you send a thread_id with each request and the API maintains the full message history on its side. Examples include OpenAI’s Assistants API, where the API stores all messages in a thread and you simply reference the thread ID on subsequent calls.

The key design pattern is to make the thread_id slot optional so that it sends null on the first turn (when no thread exists yet) and carries the returned thread ID forward on subsequent turns.

Slot Configuration

Create a slot for the thread ID with the following configuration:

The inference policy set to “Always Infer” means:

• First turn: No thread ID exists in context, so the reasoning engine infers null.
• Subsequent turns: The thread ID was returned in the previous response and exists in context, so the reasoning engine infers it automatically.

Conversation Process Implementation

This can be handled directly in a Conversation Process — no compound action or switch needed. The thread_id slot is truthy/falsy, so you can use DSL to conditionally pass it to the external API.

HTTP Action

Set up a single HTTP Action that accepts both the thread_id and the user message. The API should always return a thread_id in the response so it can be carried forward.

If the external API doesn’t automatically generate a new thread when thread_id is null, add an action step or logic in your compound action to create a new thread first, then pass the resulting ID to the main API call.

curl https://api.example.com/v1/chat \
  -X POST \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $API_KEY" \
  -d '{
    "thread_id": "{{{thread_id}}}",
    "message": "{{{user_query}}}"
  }'

Conversation Process Setup

In your Conversation Process, wire the slots directly to the action activity. The input mapping uses DSL — since thread_id is falsy on the first turn, you can pass it as-is:

user_query: data.query
thread_id: data.thread_id

The action activity returns the response to the user. The thread_id from the API response is now part of the conversation context, so on the next turn the reasoning engine will automatically infer it into the slot.

Example 3: Build Your Own Thread Store (ServiceNow Table or Custom Database)

Many external agents and LLMs don’t offer an API that keeps track of the thread for you, which means every API call is stateless — the external system has no memory of prior turns. You can solve this by creating your own thread tracking mechanism using a ServiceNow table (or any database accessible via API).

// Business Rule: Generate Thread ID
// Table: u_agent_thread_log
// When: Before Insert
(function executeRule(current, previous) {
    current.u_thread_id = gs.generateGUID();
    current.u_updated_at = new GlideDateTime();
})(current, previous);

This approach gives you full context continuity with any stateless API, and the conversation history lives in a system you control.

Handling Asynchronous APIs

Some external agents and APIs don’t return results immediately. Instead, they accept a request, return a job or task ID, and require you to poll for the result. You can handle this pattern in Tool Studio using a compound action with chained action steps and delay_config to space out polling attempts.

The pattern: Submit, wait, and poll with stacking intervals

Rather than polling aggressively (which wastes API calls and may hit rate limits) or waiting too long (which degrades user experience), use a stacking wait strategy that starts short and gets progressively longer:

1. Submit the request — Call the external API to kick off the async job. Capture the job_id or task_id from the response.
2. Wait 15 seconds, then poll — Use delay_config on the next action step to pause, then call the status endpoint.
3. If not ready, wait 1 minute, then poll again — Use a switch to check the status. If still processing, hit a second polling step with a longer delay.
4. If still not ready, wait 5 minutes, then poll a final time — A last attempt with a longer window for slow-running jobs.
5. Return the result — If the job completes at any polling step, return the result.

# Example: Async API polling with stacking wait times
steps:
  # Step 1: Submit the async request
  - action:
      action_name: Submit_Async_Job_Action
      output_key: job_submission
      input_args:
        prompt: data.user_query
  # Step 2: Wait 15 seconds, then poll
  - action:
      action_name: Poll_Job_Status_Action
      output_key: poll_1
      delay_config:
        seconds: "15"
      input_args:
        job_id: data.job_submission.job_id
  # Step 3: Check result - if done, return; otherwise keep polling
  - switch:
      cases:
        - condition: data.poll_1.status == "completed"
          steps:
            - return:
                output_mapper:
                  result: data.poll_1.result
        - condition: data.poll_1.status != "completed"
          steps:
            # Step 4: Wait 1 minute, then poll again
            - action:
                action_name: Poll_Job_Status_Action
                output_key: poll_2
                delay_config:
                  minutes: "1"
                input_args:
                  job_id: data.job_submission.job_id
            - switch:
                cases:
                  - condition: data.poll_2.status == "completed"
                    steps:
                      - return:
                          output_mapper:
                            result: data.poll_2.result
                  - condition: data.poll_2.status != "completed"
                    steps:
                      # Step 5: Wait 5 minutes, final poll
                      - action:
                          action_name: Poll_Job_Status_Action
                          output_key: poll_3
                          delay_config:
                            minutes: "5"
                          input_args:
                            job_id: data.job_submission.job_id
                      - switch:
                          cases:
                            - condition: data.poll_3.status == "completed"
                              steps:
                                - return:
                                    output_mapper:
                                      result: data.poll_3.result
                            - condition: data.poll_3.status != "completed"
                              steps:
                                - return:
                                    output_mapper:
                                      result: '''The request is still processing. There may be an error'''

Token Consumption and Cost

LLM providers charge based on the number of tokens processed (both input prompt and output generation). Long conversations or large documents can become expensive quickly.

Best Practices:

1. Set Limits: Always use the max_tokens parameter in your API calls to cap the length of the response and prevent unexpectedly large (and expensive) outputs.
2. Be Concise: Encourage users and design system prompts to be as concise as possible.
3. Monitor Usage: Regularly check your API usage and cost dashboards on the LLM provider’s platform.
4. Choose the Right Model: For simpler tasks, consider using smaller, faster, and cheaper models instead of the most powerful (and most expensive) ones.

Data Security & Privacy

Standard public LLM APIs may use your prompt data to train their models. Sending Personally Identifiable Information (PII) or sensitive company data is a significant risk.

Best Practices:

• Consult Your Security Team: Always review the data privacy and terms of service for any LLM provider.
• Prefer Enterprise Offerings: Whenever possible, use enterprise-grade services like Azure OpenAI or an OpenAI Enterprise agreement, which typically guarantee that your data will not be used for model training.
• Anonymize Data: If you must send potentially sensitive information, build steps in your workflow to find and replace sensitive data with placeholders before sending it to the LLM. You can use our LLM Actions to do this.
• Educate users: Inform users about what data is being sent to a third-party service and advise them against submitting sensitive information. You can do this through a Content Activity & enabling the Activity Confirmation Policy on your API call.

Plugin Selection

Triggering reliability can vary depending on the use case and breadth of positive utterance subject matter. Below are some options to optimize your LLM plugins:

1. Define Diverse But Specific Utterances: In your plugin’s trigger configuration, provide a wide range of example phrases. For a summarization plugin, this could include:
   • “summarize this document”
   • “give me the tl;dr”
   • “what are the key points of this?”
   • “can you create an executive summary”
2. Define a trigger keyword: Assign a deterministic triggering phrase to your plugin so that users can trigger the plugin on command — this will help ensure the agent is always called.
3. Use a System Prompt: Instead of relying on the user to frame their entire request, use the system message (or an equivalent field) in your API request body. This pre-prompts the LLM with its role or instructions (e.g., “You are an expert at rewriting text to be more professional”). The user then only needs to provide the core input, making the interaction much smoother.

Check out our demo!