Overview

LLM Actions in Agent Studio provide built-in capabilities to leverage large language model (LLM) functionalities directly within your Compound Actions and Conversational Processes. These actions enable tasks such as summarization, reasoning, classification, data extraction, and content generation, allowing you to build workflows without custom integrations.

This documentation focuses on two key LLM Actions: generate_text_action and generate_structured_value_action. These actions are designed to help you process unstructured data, generate insights, and structure outputs efficiently.

Security note

When you enable web search or file input, the model consumes untrusted content that may contain prompt-injection attacks. Before building flows that combine private data, untrusted content, and the ability to send data externally, read Security: the lethal trifecta.

generate_text_action

Description

The generate_text_action invokes an LLM to produce free-form text output based on user-provided input. This action is ideal for tasks requiring natural language generation, such as summarizing documents, generating responses, or performing step-by-step reasoning. Use this action when you need unstructured text results, like drafting emails, explaining concepts, or brainstorming ideas.

Input Parameters

Field	Type	Required	Description
`system_prompt`	`string`	❌	Defines the model’s behavior or instructions. For example, “Act as a helpful assistant that summarizes technical articles.”
`user_input`	`string`	✅	The primary context or query for the LLM to process.
`image`	`List[File] or File`	❌	An image file to include as visual input for the LLM. Must be a `File` object retrieved via a File Slot in a Conversational Process. Supported formats: PNG, JPEG/JPG, WEBP, non-animated GIF. See the Model Reference for image-capable models. For best results, we recommend using `gpt-5` or newer.
`file`	`List[File] or File`	❌	A document, spreadsheet, presentation, or text/code file to include as input for the LLM. Must be a `File` object retrieved via a File Slot in a Conversational Process. See Supported file types for the full list of accepted formats, the File Input Limitations callout for size constraints, and the Model Reference for compatible models. For best results, we recommend using `gpt-5` or newer.
`model`	`string`	❌	Specifies the LLM model to use. See the Model Reference for available options. Defaults to `gpt-4o-mini-2024-07-18`
`temperature`	`number`	❌	Control the randomness of the output. Higher values will make the output more random, while lower values will make it more focused and deterministic
`reasoning_effort`	`string`	❌	Optional reasoning effort argument. Can be set to one of “minimal” (only `gpt-5` models), `"low"`, `"medium"`, or `"high"`. Must be left empty for non-reasoning models such as `gpt-4.1`.
`enable_web_search`	`boolean`	❌	Enables the live web search tool, allowing the model to retrieve current information from the web and include it in the model’s context rather than relying only on the model’s base knowledge. When neither `web_search_allowed_domains` nor `web_search_blocked_domains` is provided, there are no restrictions on what the model can search. Supported on all models except `gpt-5` models with `reasoning_effort` set to `minimal`. Defaults to `false`.
`web_search_allowed_domains`	`List[str]`	❌	Optional. Limit live web search results to these domains (up to 100), e.g. `["openai.com"]`. Omit the `http(s)` prefix; subdomains are included. Providing this enables web search automatically.
`web_search_blocked_domains`	`List[str]`	❌	Optional. Exclude these domains (up to 100) from live web search results, e.g. `["reddit.com"]`. Omit the `http(s)` prefix; subdomains are included. Providing this enables web search automatically.

Combining and listing image & file inputs

image and file can be provided in the same request, and either parameter can accept multiple files. To pass multiple files, supply a list of File objects — either from a single List[File] slot, or assembled from separate File slots:

1 image: data.images   # e.g. a single List[File] slot
2 file: [data.contract, data.addendum]  # e.g. mapped from two separate File slots

Output

Field	Type	Description
`generated_output`	`string`	The LLM-generated text response.

If the model refuses the request or the generation is incomplete, generated_output may be returned as null.

Usage Examples

Here are practical examples demonstrating various LLM abilities. Each includes a sample request schema for integration into a Compound Action.

Example 1: Text Summarization

Summarize a lengthy article or user query into a concise overview.

YAML - Compound Action

1 - action:
2     action_name: mw.generate_text_action
3     input_args:
4       system_prompt: '''Summarize the following text in 3-5 bullet points, focusing on key takeaways.'''
5       user_input: data.article_content  # e.g., a long blog post fetched from an API
6       model: '''gpt-4o-mini'''
7 			temperature: 0.7
8     output_key: summary_output

Conversational Process

1 system_prompt: '''Summarize the following text in 3-5 bullet points, focusing on key takeaways.'''
2 user_input: data.article_content  # e.g., a long blog post fetched from an API
3 model: '''gpt-4o-mini'''
4 temperature: 0.7

Example 2: Content Generation

Generate creative or instructional content, such as drafting a user email.

YAML - Compound Action

1 - action:
2     action_name: mw.generate_text_action
3     input_args:
4       system_prompt: '''Write a professional email response based on the user's complaint.'''
5       user_input: data.user_complaint  # e.g., "My order is delayed by two weeks."
6     output_key: email_draft

Conversational Process

1 system_prompt: '''Write a professional email response based on the user's complaint.'''
2 user_input: data.user_complaint  # e.g., "My order is delayed by two weeks."

Example 3: Step-by-Step Reasoning

Guide the LLM through logical reasoning for problem-solving.

YAML - Compound Action

1 - action:
2     action_name: mw.generate_text_action
3     input_args:
4       system_prompt: '''Solve the problem step by step, explaining your reasoning.'''
5       user_input: '''What is the next number in the sequence: 2, 4, 8, 16?'''
6       reasoning_effort: '''high'''	
7 			model: '''gpt-5-2025-08-07'''
8     output_key: reasoning_output

Conversational Process

1 system_prompt: '''Solve the problem step by step, explaining your reasoning.'''
2 user_input: '''What is the next number in the sequence: 2, 4, 8, 16?'''
3 reasoning_effort: '''high'''	
4 model: '''gpt-5-2025-08-07'''

Example 4: Image OCR

Extract text from an uploaded image, such as a receipt. The image parameter maps to a File object collected via a File Slot.

YAML - Compound Action

1 - action:
2     action_name: mw.generate_text_action
3     input_args:
4       system_prompt: '''Extract all text from the provided image exactly as it appears.'''
5       user_input: '''Extract the text from this receipt image.'''
6       image: data.receipt
7       model: '''gpt-4o'''
8     output_key: ocr_output

Conversational Process

1 system_prompt: '''Extract all text from the provided image exactly as it appears.'''
2 user_input: '''Extract the text from this receipt image.'''
3 image: data.receipt
4 model: '''gpt-4o'''

Example 5: Document Q&A

Answer a question against an uploaded document, such as a PDF policy. The file parameter maps to a File object collected via a File Slot.

YAML - Compound Action

1 - action:
2     action_name: mw.generate_text_action
3     input_args:
4       system_prompt: '''Answer the user's question using only the content of the attached document.'''
5       user_input: '''What is the reimbursement limit described in this policy?'''
6       file: data.policy_document
7       model: '''gpt-4o'''
8     output_key: document_answer

Conversational Process

1 system_prompt: '''Answer the user's question using only the content of the attached document.'''
2 user_input: '''What is the reimbursement limit described in this policy?'''
3 file: data.policy_document
4 model: '''gpt-4o'''

Example 6: Comparing Multiple Documents

Pass more than one file in a single call by mapping the file parameter to a list of File objects. This requires a List[File] slot so the user can attach multiple documents.

YAML - Compound Action

1 - action:
2     action_name: mw.generate_text_action
3     input_args:
4       system_prompt: '''Compare the two attached documents and summarize the key differences.'''
5       user_input: '''What changed between these two contract versions?'''
6       file: [data.pdf1, data.pdf2]
7       model: '''gpt-5.4'''
8     output_key: comparison_output

Conversational Process

1 system_prompt: '''Compare the two attached documents and summarize the key differences.'''
2 user_input: '''What changed between these two contract versions?'''
3 file: [data.pdf1, data.pdf2]
4 model: '''gpt-5.4'''

generate_structured_value_action

Description

The generate_structured_value_action calls an LLM to extract or generate data in a predefined structured format (JSON schema). This is particularly useful for classification, entity extraction, or transforming unstructured input into queryable data.

Apply this action for tasks where output consistency is critical, such as tagging content, extracting key-value pairs, or categorizing user inputs.

Input Parameters

Field	Type	Required	Description
`payload`	`object`	✅	The data or text to analyze.
`output_schema`	`object`	✅	JSON Schema defining the expected output structure.
`system_prompt`	`string`	❌	Defines the model’s behavior or instructions. For example, “Act as a helpful assistant that summarizes technical articles.”
`image`	`List[File] or File`	❌	An image file to include as visual input for the LLM. Must be a `File` object retrieved via a File Slot in a Conversational Process. Supported formats: PNG, JPEG/JPG, WEBP, non-animated GIF. See the Model Reference for image-capable models. For best results, we recommend using `gpt-5` or newer.
`file`	`List[File] or File`	❌	A document, spreadsheet, presentation, or text/code file to include as input for the LLM. Must be a `File` object retrieved via a File Slot in a Conversational Process. See Supported file types for the full list of accepted formats, the File Input Limitations callout for size constraints, and the Model Reference for compatible models. For best results, we recommend using `gpt-5` or newer.
`model`	`string`	❌	Specifies the LLM model to use. Defaults to `"gpt-4o-mini-2024-07-18"`. IMPORTANT: This action is only compatible with `gpt-4o-mini-2024-07-18` and later and `gpt-4o-2024-08-06` and later.
`strict`	`string`	❌	Enforces schema adherence. Defaults to `false`; can be either `true` or `false`.
`output_schema_name`	`string`	❌	LLM-facing name for the schema (defaults to `extracted_value`)
`output_schema_description`	`string`	❌	Description of the schema for the LLM.
`reasoning_effort`	`string`	❌	Optional reasoning effort argument. Can be set to one of “minimal” (only `gpt-5` models), `"low"`, `"medium"`, or `"high"`. Must be left empty for non-reasoning models such as `gpt-4.1`.
`enable_web_search`	`boolean`	❌	Enables the live web search tool, allowing the model to retrieve current information from the web and include it in the model’s context rather than relying only on the model’s base knowledge. When neither `web_search_allowed_domains` nor `web_search_blocked_domains` is provided, there are no restrictions on what the model can search. Supported on all models except `gpt-5` models with `reasoning_effort` set to `minimal`. Defaults to `false`.
`web_search_allowed_domains`	`List[str]`	❌	Optional. Limit live web search results to these domains (up to 100), e.g. `["openai.com"]`. Omit the `http(s)` prefix; subdomains are included. Providing this enables web search automatically.
`web_search_blocked_domains`	`List[str]`	❌	Optional. Exclude these domains (up to 100) from live web search results, e.g. `["reddit.com"]`. Omit the `http(s)` prefix; subdomains are included. Providing this enables web search automatically.

️ `additionalProperties: false` must always be set in objects.

additionalProperties controls whether it is allowable for an object to contain additional keys / values that were not defined in the JSON Schema.

Output

Field	Type	Description
`generated_output`	`object`	Structured data matching the provided schema.

If the model refuses the request or the generation is incomplete, generated_output may be returned as null.

Usage Examples

Examples illustrate extraction, classification, and more. Include request schemas for easy implementation.

Example 1: Topic Classification (Existing Example, Expanded)

Classify a research abstract into predefined topics.

YAML - Compound Action

1 - action:
2     action_name: mw.generate_structured_value_action
3     input_args:
4       payload: data.research_paper_abstract
5       system_prompt: '''Given a research paper abstract and a list of topic options, output up to 5 topics that accurately apply to the paper.'''
6       output_schema: >-
7         {
8             "type": "object",
9             "properties": {
10                 "topic_tags": {
11                     "type": "array",
12                     "items": {
13                       "type": "string",
14                       "enum": data.topic_tag_options  # e.g., ["AI", "ML", "NLP"]
15                     }
16                 }
17             },
18             "required": ["topic_tags"],
19             "additionalProperties": false
20         }
21       strict: true
22     output_key: classified_topics

Conversational Process

1 payload: data.research_paper_abstract
2 system_prompt: '''Given a research paper abstract and a list of topic options, output up to 5 topics that accurately apply to the paper.'''
3 output_schema: >-
4     {
5         "type": "object",
6         "properties": {
7             "topic_tags": {
8                 "type": "array",
9                 "items": {
10                     "type": "string",
11                     "enum": data.topic_tag_options  # e.g., ["AI", "ML", "NLP"]
12                 }
13             }
14         },
15         "required": ["topic_tags"],
16         "additionalProperties": false
17     }
18 strict: true

Expected Output

1 generated_output: {
2   "topic_tags": ["LLM Capabilities", "Reinforcement Learning (RL)", "Reasoning"]
3 }

Example 2: Entity Extraction

Extract named entities like names, dates, and locations from text.

YAML - Compound Action

1 - action:
2     action_name: mw.generate_structured_value_action
3     input_args:
4       payload: data.user_message  # e.g., "John Doe will arrive in New York on October 15, 2025."
5       system_prompt: '''Extract entities such as persons, locations, and dates from the text.'''
6       output_schema: >-
7         {
8             "type": "object",
9             "properties": {
10                 "persons": {"type": "array", "items": {"type": "string"}},
11                 "locations": {"type": "array", "items": {"type": "string"}},
12                 "dates": {"type": "array", "items": {"type": "string"}}
13             },
14             "required": ["persons", "locations", "dates"],
15             "additionalProperties": false
16         }
17       reasoning_effort: '''low'''
18 			model: '''gpt-5-2025-08-07'''
19     output_key: extracted_entities

Conversational Process

1 payload: data.user_message  # e.g., "John Doe will arrive in New York on October 15, 2025."
2 system_prompt: '''Extract entities such as persons, locations, and dates from the text.'''
3 output_schema: >-
4     {
5         "type": "object",
6         "properties": {
7             "persons": {"type": "array", "items": {"type": "string"}},
8             "locations": {"type": "array", "items": {"type": "string"}},
9             "dates": {"type": "array", "items": {"type": "string"}}
10         },
11         "required": ["persons", "locations", "dates"],
12         "additionalProperties": false
13     }
14 reasoning_effort: '''low'''
15 model: '''gpt-5-2025-08-07'''

Expected Output

1 generated_output: {
2   "persons": ["John Doe"],
3   "locations": ["New York"],
4   "dates": ["October 15, 2025"]
5 }

Example 3: Sentiment Classification

Classify text sentiment with confidence scores.

YAML - Compound Action

1 - action:
2     action_name: mw.generate_structured_value_action
3     input_args:
4       payload: data.customer_review
5       system_prompt: '''Analyze the sentiment of the review and output the category with a confidence score.'''
6       output_schema: >-
7         {
8             "type": "object",
9             "properties": {
10                 "sentiment": {
11                     "type": "string",
12                     "enum": ["positive", "negative", "neutral"]
13                 },
14                 "confidence": {"type": "number"}
15             },
16             "required": ["sentiment", "confidence"],
17             "additionalProperties": false
18         }
19     output_key: sentiment_analysis

Conversational Process

1 payload: data.customer_review
2 system_prompt: '''Analyze the sentiment of the review and output the category with a confidence score.'''
3 output_schema: >-
4     {
5         "type": "object",
6         "properties": {
7             "sentiment": {
8                 "type": "string",
9                 "enum": ["positive", "negative", "neutral"]
10             },
11             "confidence": {"type": "number"}
12         },
13         "required": ["sentiment", "confidence"],
14         "additionalProperties": false
15     }

Expected Output

1 generated_output: {
2   "sentiment": "positive",
3   "confidence": 0.85
4 }

Example 4: Structured Extraction from an Image

Extract structured fields from an uploaded image, such as a receipt. The image parameter maps to a File object collected via a File Slot.

YAML - Compound Action

1 - action:
2     action_name: mw.generate_structured_value_action
3     input_args:
4       payload: '''Extract the merchant, total amount, and date from this receipt.'''
5       image: data.receipt
6       system_prompt: '''Extract structured expense details from the provided receipt image.'''
7       output_schema: >-
8         {
9             "type": "object",
10             "properties": {
11                 "merchant": {"type": "string"},
12                 "total_amount": {"type": "number"},
13                 "date": {"type": "string"}
14             },
15             "required": ["merchant", "total_amount", "date"],
16             "additionalProperties": false
17         }
18       model: '''gpt-4o'''
19     output_key: receipt_details

Conversational Process

1 payload: '''Extract the merchant, total amount, and date from this receipt.'''
2 image: data.receipt
3 system_prompt: '''Extract structured expense details from the provided receipt image.'''
4 output_schema: >-
5     {
6         "type": "object",
7         "properties": {
8             "merchant": {"type": "string"},
9             "total_amount": {"type": "number"},
10             "date": {"type": "string"}
11         },
12         "required": ["merchant", "total_amount", "date"],
13         "additionalProperties": false
14     }
15 model: '''gpt-4o'''

Expected Output

1 generated_output: {
2   "merchant": "Blue Bottle Coffee",
3   "total_amount": 12.50,
4   "date": "2025-10-15"
5 }

Example 5: Structured Extraction from a Document

Extract structured fields from an uploaded document, such as a PDF invoice. The file parameter maps to a File object collected via a File Slot.

YAML - Compound Action

1 - action:
2     action_name: mw.generate_structured_value_action
3     input_args:
4       payload: '''Extract the invoice number, vendor, and amount due from this invoice.'''
5       file: data.invoice_document
6       system_prompt: '''Extract structured invoice details from the attached document.'''
7       output_schema: >-
8         {
9             "type": "object",
10             "properties": {
11                 "invoice_number": {"type": "string"},
12                 "vendor": {"type": "string"},
13                 "amount_due": {"type": "number"}
14             },
15             "required": ["invoice_number", "vendor", "amount_due"],
16             "additionalProperties": false
17         }
18       model: '''gpt-4o'''
19     output_key: invoice_details

Conversational Process

1 payload: '''Extract the invoice number, vendor, and amount due from this invoice.'''
2 file: data.invoice_document
3 system_prompt: '''Extract structured invoice details from the attached document.'''
4 output_schema: >-
5     {
6         "type": "object",
7         "properties": {
8             "invoice_number": {"type": "string"},
9             "vendor": {"type": "string"},
10             "amount_due": {"type": "number"}
11         },
12         "required": ["invoice_number", "vendor", "amount_due"],
13         "additionalProperties": false
14     }
15 model: '''gpt-4o'''

Expected Output

1 generated_output: {
2   "invoice_number": "INV-2025-0042",
3   "vendor": "Acme Supplies Inc.",
4   "amount_due": 1450.00
5 }

Model Reference

Model	Capabilities					OpenAI Direct		Azure OpenAI
Model	Context	Max Output	Reasoning Effort	Live Search	Image & File Input	US	EU	US	EU	CA	AU	Gov
gpt-5.5-2026-04-23	400K	128K	✓	✓	✓	✓	✓	—	—	—	—	—
gpt-5.5	400K	128K	✓	✓	✓	✓	✓	—	—	—	—	—
gpt-5.4-2026-03-05	400K	128K	✓	✓	✓	✓	✓	✓	—	—	—	—
gpt-5.4	400K	128K	✓	✓	✓	✓	✓	—	—	—	—	—
gpt-5.4-mini-2026-03-17	400K	128K	✓	✓	✓	✓	✓	✓	—	—	—	—
gpt-5.4-mini	400K	128K	✓	✓	✓	✓	✓	—	—	—	—	—
gpt-5.4-nano-2026-03-17	400K	128K	✓	✓	✓	✓	✓	✓	—	—	—	—
gpt-5.4-nano	400K	128K	✓	✓	✓	✓	✓	—	—	—	—	—
gpt-5.2-2025-12-11	400K	128K	✓	✓	✓	✓	✓	✓	—	—	—	—
gpt-5.2	400K	128K	✓	✓	✓	✓	✓	—	—	—	—	—
gpt-5.1-2025-11-13	400K	128K	✓	✓	✓	✓	✓	✓	✓	—	—	✓
gpt-5.1-chat-latest	400K	128K	✓	✓	✓	✓	✓	—	—	—	—	—
gpt-5.1	400K	128K	✓	✓	✓	✓	✓	—	—	—	—	—
gpt-5-2025-08-07	400K	128K	✓	✓	✓	✓	—	✓	✓	—	—	—
gpt-5	400K	128K	✓	✓	✓	✓	✓	—	—	—	—	—
gpt-5-mini-2025-08-07	400K	128K	✓	✓	✓	✓	—	✓	✓	—	—	—
gpt-5-mini	400K	128K	✓	✓	✓	✓	✓	—	—	—	—	—
gpt-5-nano-2025-08-07	400K	128K	✓	✓	✓	✓	—	✓	✓	—	—	—
gpt-5-nano	400K	128K	✓	✓	✓	✓	✓	—	—	—	—	—
o4-mini-2025-04-16	200K	100K	✓	✓	✓	✓	✓	—	—	—	—	—
o4-mini	1M	100K	✓	✓	✓	✓	✓	—	—	—	—	—
gpt-4.1-2025-04-14	1M	32K	—	✓	✓	✓	✓	✓	✓	✓	✓	✓
gpt-4.1	1M	32K	—	✓	✓	✓	✓	—	—	—	—	—
gpt-4.1-mini-2025-04-14	1M	32K	—	✓	✓	✓	✓	✓	✓	✓	✓	✓
gpt-4.1-mini	1M	32K	—	✓	✓	✓	✓	—	—	—	—	—
gpt-4.1-nano-2025-04-14	1M	32K	—	✓	✓	✓	✓	✓	✓	✓	✓	✓
gpt-4.1-nano	1M	32K	—	✓	✓	✓	✓	—	—	—	—	—
o3-2025-04-16	200K	100K	—	✓	✓	✓	✓	—	—	—	—	—
o3	200K	100K	—	✓	✓	✓	✓	—	—	—	—	—
o3-mini-2025-01-31	200K	100K	—	✓	—	✓	✓	—	—	—	—	—
o3-mini	200K	100K	—	✓	—	✓	✓	—	—	—	—	—
o1-2024-12-17	200K	100K	—	✓	✓	✓	✓	—	—	—	—	—
o1	200K	100K	—	✓	✓	✓	✓	—	—	—	—	—
gpt-4o-2024-11-20	128K	16K	—	✓	✓	✓	✓	—	—	—	—	—
gpt-4o	128K	16K	—	✓	✓	✓	✓	—	—	—	—	—
gpt-4o-mini-2024-07-18	128K	16K	—	✓	✓	✓	✓	✓	✓	✓	✓	✓
gpt-4o-mini	128K	16K	—	✓	✓	✓	✓	—	—	—	—	—

Live Search

Live Search is enabled per call with the enable_web_search argument. It is available on all models except gpt-5 models with reasoning_effort set to minimal.

Image Input Limitations

Images must be supplied as a File object via a File Slot, which enforces a maximum file size of 5MB — files exceeding this limit will fail to upload. Supported image formats: PNG, JPEG/JPG, WEBP, and non-animated GIF. For best results, we recommend using gpt-5 or newer.

File Input Limitations

Files must be supplied as a File object via a File Slot, which enforces a maximum file size of 5MB — files exceeding this limit will fail to upload.

The combined size of all images and files in a single action call must stay under ~11 MB of raw file content (16 MiB encoded, enforced by the platform). As a rule of thumb, keep individual files under ~5 MB and prefer fewer, smaller attachments.

For PDFs, both the text and page images are extracted by vision-capable models such as gpt-4o and later. For non-PDF documents, only the extracted text is passed to the model — embedded charts and images are not included. Note that large PDFs also consume model context — every page contributes both text and image tokens — so very long documents may exceed the model’s context window even when they fit the size limit. For best results, we recommend using gpt-5 or newer.

Supported file types

The file parameter accepts the following file types:

Category	Extensions	MIME types
PDF files	`.pdf`	`application/pdf`
Spreadsheets	Excel sheets (`.xla`, `.xlb`, `.xlc`, `.xlm`, `.xls`, `.xlsx`, `.xlt`, `.xlw`)	`application/vnd.openxmlformats-officedocument.spreadsheetml.sheet`, `application/vnd.ms-excel`
Spreadsheets	CSV / TSV / IIF (`.csv`, `.tsv`, `.iif`), Google Sheets	`text/csv`, `application/csv`, `text/tsv`, `text/x-iif`, `application/x-iif`, `application/vnd.google-apps.spreadsheet`
Rich documents	Word / ODT / RTF docs (`.doc`, `.docx`, `.dot`, `.odt`, `.rtf`), Pages, Google Docs	`application/vnd.openxmlformats-officedocument.wordprocessingml.document`, `application/msword`, `application/rtf`, `text/rtf`, `application/vnd.oasis.opendocument.text`, `application/vnd.apple.pages`, `application/vnd.google-apps.document`, `application/vnd.apple.iwork`
Presentations	PowerPoint slides (`.pot`, `.ppa`, `.pps`, `.ppt`, `.pptx`, `.pwz`, `.wiz`), Keynote, Google Slides	`application/vnd.openxmlformats-officedocument.presentationml.presentation`, `application/vnd.ms-powerpoint`, `application/vnd.apple.keynote`, `application/vnd.google-apps.presentation`, `application/vnd.apple.iwork`
Text and code	Text / code formats (`.asm`, `.bat`, `.c`, `.cc`, `.conf`, `.cpp`, `.css`, `.cxx`, `.def`, `.dic`, `.eml`, `.h`, `.hh`, `.htm`, `.html`, `.ics`, `.ifb`, `.in`, `.js`, `.json`, `.ksh`, `.list`, `.log`, `.markdown`, `.md`, `.mht`, `.mhtml`, `.mime`, `.mjs`, `.nws`, `.pl`, `.py`, `.rst`, `.s`, `.sql`, `.srt`, `.text`, `.txt`, `.vcf`, `.vtt`, `.xml`)	application/javascript, application/typescript, text/xml, text/x-shellscript, text/x-rst, text/x-makefile, text/x-lisp, text/x-asm, text/vbscript, text/css, message/rfc822, application/x-sql, application/x-scala, application/x-rust, application/x-powershell, text/x-diff, text/x-patch, application/x-patch, text/plain, text/markdown, text/x-java, text/x-script.python, text/x-python, text/x-c, text/x-c++, text/x-golang, text/html, text/x-php, application/x-php, application/x-httpd-php, application/x-httpd-php-source, text/x-ruby, text/x-sh, text/x-bash, application/x-bash, text/x-zsh, text/x-tex, text/x-csharp, application/json, text/x-typescript, text/javascript, text/x-go, text/x-rust, text/x-scala, text/x-kotlin, text/x-swift, text/x-lua, text/x-r, text/x-R, text/x-julia, text/x-perl, text/x-objectivec, text/x-objectivec++, text/x-erlang, text/x-elixir, text/x-haskell, text/x-clojure, text/x-groovy, text/x-dart, text/x-awk, application/x-awk, text/jsx, text/tsx, text/x-handlebars, text/x-mustache, text/x-ejs, text/x-jinja2, text/x-liquid, text/x-erb, text/x-twig, text/x-pug, text/x-jade, text/x-tmpl, text/x-cmake, text/x-dockerfile, text/x-gradle, text/x-ini, text/x-properties, text/x-protobuf, application/x-protobuf, text/x-sql, text/x-sass, text/x-scss, text/x-less, text/x-hcl, text/x-terraform, application/x-terraform, text/x-toml, application/x-toml, application/graphql, application/x-graphql, text/x-graphql, application/x-ndjson, application/json5, application/x-json5, text/x-yaml, application/toml, application/x-yaml, application/yaml, text/x-astro, text/srt, application/x-subrip, text/x-subrip, text/vtt, text/x-vcard, text/calendar

Security: the lethal trifecta

Features like web search and file input make LLM Actions far more powerful, but they also expand what an attacker can attempt through prompt injection — malicious instructions hidden inside content the model reads (a web page returned by search, an uploaded PDF, a spreadsheet cell, or an API response). Because a large language model cannot reliably tell the difference between the instructions you wrote and instructions embedded in the data it is processing, any untrusted content should be treated as potentially adversarial.

The lethal trifecta (a term coined by security researcher Simon Willison) describes the specific combination of capabilities that turns prompt injection from a nuisance into a data-exfiltration risk. The danger arises when a single flow has all three of the following at once:

Access to private data — sensitive input such as a user’s records, internal documents, or data pulled from connected systems into payload / user_input.
Exposure to untrusted content — anything the model reads that you don’t fully control, including enable_web_search results, uploaded file and image inputs, and responses from external APIs.
The ability to communicate externally — a way to send data out of the environment, such as a subsequent HTTP action, an outbound notification, or even a URL the model can cause to be fetched.

When all three are present, injected instructions in the untrusted content can direct the model to take the private data and exfiltrate it through the outbound channel — without the user ever seeing it happen.

Avoid combining all three capabilities in one flow

The most reliable mitigation is to break the trifecta: design your Compound Actions and Conversational Processes so that a flow handling private data and untrusted content does not also have an unrestricted way to send data externally.

Additional safeguards:

Treat file, image, and web content as untrusted. Never feed model output derived from untrusted content directly into a downstream action that transmits data (HTTP calls, notifications, or dynamically constructed URLs).
Scope web search. Use web_search_allowed_domains to restrict searches to sources you trust, or web_search_blocked_domains to exclude risky ones.
Constrain outputs. Prefer generate_structured_value_action with a strict output_schema so the model can only return the specific fields you expect, rather than free-form text or arbitrary URLs.
Keep a human in the loop for any action that sends sensitive data outside the environment.

Overview

Security note

generate_text_action

Description

Input Parameters

Combining and listing image & file inputs

Output

Usage Examples

Example 1: Text Summarization

Example 2: Content Generation

Example 3: Step-by-Step Reasoning

Example 4: Image OCR

Example 5: Document Q&A

Example 6: Comparing Multiple Documents

generate_structured_value_action

Description

Input Parameters

️ additionalProperties: false must always be set in objects.

Output

Usage Examples

Example 1: Topic Classification (Existing Example, Expanded)

Expected Output

Example 2: Entity Extraction

Expected Output

Example 3: Sentiment Classification

Expected Output

Example 4: Structured Extraction from an Image

Expected Output

Example 5: Structured Extraction from a Document

Expected Output

Model Reference

Live Search

Image Input Limitations

File Input Limitations

Supported file types

Security: the lethal trifecta

Avoid combining all three capabilities in one flow

️ `additionalProperties: false` must always be set in objects.