User Ingestion Filters Guide

Overview

Control which user records and fields are imported at the source level - before processors run. This reduces data transfer, improves performance, and ensures only relevant users are ingested.

Table of Contents

1. Quick Reference
2. How to Configure
3. Generic Source Filter
4. Source-Specific Filters
5. Best Practices
6. Common Scenarios
7. Troubleshooting

Quick Reference

How to Configure

Navigation: Import Users → Connectors → Configure Selected Sources → Advanced Mode → “Filters and Attribute List”

Filters: Define which records to import from the source (applied before data transfer) Attribute List: Specify which fields to import (unlisted fields are not transferred)

Generic Source Filter

Works with any integration source. Use when source doesn’t have a dedicated filter type.

Syntax by Source:

Important: Syntax varies by source. Check your source system’s API documentation for filter/query syntax (search for “filter users” or “query parameters” in API docs).

Source-Specific Filters

Microsoft Graph (Azure AD)

Import users from specific Azure AD groups with OData filtering.

Example: Import active IT users:

Azure AD Group ID: a1b2c3d4-e5f6-7890-abcd-ef1234567890
OData Filter: accountEnabled eq true
Attributes: mail, displayName, givenName, surname, jobTitle, department

Note: For multiple groups, create multiple filter sources. See Microsoft Graph API docs for OData filter operators.

Salesforce

Import from User or Contact table with SOQL filtering.

Examples:

Active users:  IsActive = true
Specific accounts:  AccountId IN ('001...', '002...')
Exclude system users:  IsActive = true AND Username NOT LIKE '%@system.com'

Note: Default attributes: Id, Email, LastName, FirstName, Name. See Salesforce SOQL documentation for filter operators.

Exchange Online

Import users with PowerShell filtering.

Examples:

User mailboxes:  RecipientType -eq 'UserMailbox'
Specific database:  Database -eq 'DB01'
With size limit:  RecipientType -eq 'UserMailbox' -and ProhibitSendQuota -lt '50GB'

Note: Uses PowerShell comparison operators (-eq, -ne, -and, -or). See Exchange Online PowerShell documentation.

Other Source Filters

Google Workspace (GdriveSourceFilter / GdriveNGSourceFilter)

• Customer ID: Google Workspace Customer ID from Admin console
• Custom Schemas: Custom schema names to import (e.g., EmployeeData, LocationInfo)

Microsoft Teams (MSTeamsSourceFilter)

• Team ID: Azure AD ID of the Team to import users from

GitHub (GithubSourceFilter)

• Organization Name: GitHub organization to import from

Confluence (ConfluenceSourceFilter)

• Group ID: Override default groups (confluence-users, confluence-administrators)

SharePoint Online (SharepointOnlineSourceFilter)

• Sites: List of sites to import from (use site/subsite format for subsites)

SharePoint OneDrive (SharepointOneDriveNGSourceFilter)

• Azure AD Group ID: Group containing users
• OData Filter Query: OData v4.0 filter syntax (see Microsoft Graph API docs)
• Attributes: Fields to import

Cherwell (CherwellSourceFilter)

• Business Object ID: Cherwell user business object ID
• User Fields: List of field IDs to import

FreshDesk (FreshDeskSourceFilter)

• Agent ingestion: Include FreshDesk agents (default: false, requires bot admin privileges)

Best Practices

1. Filter at Source Level - Apply broad filtering at source before processors run (don’t import all users then filter out 80%)

2. Specify Only Required Attributes - List only needed fields to reduce data transfer (5 fields vs 50+ unused fields)

3. Use Source-Specific Filters - Prefer dedicated filters (e.g., MSGraphSourceFilter) over generic for better validation

4. Test with Small Data Sets - Start with restrictive filter, verify results, then broaden as needed

5. Document Complex Filters - Add comments in config for multi-condition filters explaining purpose

Common Scenarios

Import Active Users Only

• Okta: status eq "ACTIVE"
• Azure AD: accountEnabled eq true
• Salesforce: IsActive = true

Import Specific Departments

• Generic: department IN ["Engineering", "Sales", "Support"]
• Salesforce: Department IN ('Engineering', 'Sales', 'Support')

Multi-Group Import (Azure AD)

1. Add first Microsoft Graph Filter with Group ID 1
2. Click “Add Another Filter”
3. Add second Microsoft Graph Filter with Group ID 2

Minimal Attribute Import

• Specify only: email, firstName, lastName, department, manager
• Result: 5 fields transferred instead of 50+

Troubleshooting

No users imported

• Filter too restrictive → Start with no filter, verify import, then add incrementally
• Incorrect syntax → Check source API docs for correct query format
• Invalid group/table → Verify ID/name exists in source system

Missing fields in user profile

• Field not in Attributes list → Add field name and re-run ingestion
• Check exact field name in source system API (case-sensitive)

Filter syntax error

• OData uses: eq, ne, and, or (lowercase)
• SOQL uses: =, !=, AND, OR (uppercase)
• PowerShell uses: -eq, -ne, -and, -or
• Verify field names match source exactly

Duplicate users from multiple filters

• Expected: Multiple filters create union of users
• To fix: Remove redundant filters or use single filter with OR logic

Filter Syntax by Source

OData v4.0 (Azure AD, Microsoft Graph, SharePoint OneDrive)

• Documentation: Microsoft Graph API - List Users / OData query parameters
• Operators: eq, ne, and, or, not, lt, le, gt, ge (lowercase)
• Example: (accountEnabled eq true) and (userType eq 'Member')

SOQL (Salesforce)

• Documentation: Salesforce SOQL and SOSL Reference
• Operators: =, !=, AND, OR, NOT, <, >, <=, >= (uppercase)
• Example: IsActive = true AND Department IN ('IT', 'Engineering')

PowerShell (Exchange Online)

• Documentation: Exchange Online PowerShell - Get-Recipient / Get-Mailbox
• Operators: -eq, -ne, -and, -or, -not, -lt, -gt, -le, -ge
• Example: RecipientType -eq 'UserMailbox' -and Database -eq 'DB01'

Okta (Generic Source Filter)

• Documentation: Okta API - List Users with Search
• Standard: OData-like filter syntax
• Example: status eq "ACTIVE"

ServiceNow (Generic Source Filter)

• Documentation: ServiceNow REST API - Encoded Query Strings
• Standard: ServiceNow encoded query operators
• Example: active=true^user_name!=admin

Limitations

• Maximum 20 filters per source - Filters are OR’d together (union of results)
• Unlisted attributes not transferred - Only specified fields are imported
• Performance considerations - More restrictive filters and fewer attributes = faster ingestion