User Ingestion Filters Guide

Overview

Control which user records and fields are imported at the source level - before processors run. This reduces data transfer, improves performance, and ensures only relevant users are ingested.

---

Table of Contents

1. Quick Reference
2. How to Configure
3. Generic Source Filter
4. Source-Specific Filters
5. Best Practices
6. Common Scenarios
7. Troubleshooting

---

Quick Reference

---

How to Configure

Navigation: Import Users → Connectors → Configure Selected Sources → Advanced Mode → "Filters and Attribute List"

Filters: Define which records to import from the source (applied before data transfer) Attribute List: Specify which fields to import (unlisted fields are not transferred)

---

Generic Source Filter

Works with any integration source. Use when source doesn't have a dedicated filter type.

Syntax by Source:

Important: Syntax varies by source. Check your source system's API documentation for filter/query syntax (search for "filter users" or "query parameters" in API docs).

---

Source-Specific Filters

Microsoft Graph (Azure AD)

Import users from specific Azure AD groups with OData filtering.

Example: Import active IT users:

Azure AD Group ID: a1b2c3d4-e5f6-7890-abcd-ef1234567890
OData Filter: accountEnabled eq true
Attributes: mail, displayName, givenName, surname, jobTitle, department

Note: For multiple groups, create multiple filter sources. See Microsoft Graph API docs for OData filter operators.

---

Salesforce

Import from User or Contact table with SOQL filtering.

Examples:

Active users:  IsActive = true
Specific accounts:  AccountId IN ('001...', '002...')
Exclude system users:  IsActive = true AND Username NOT LIKE '%@system.com'

Note: Default attributes: Id, Email, LastName, FirstName, Name. See Salesforce SOQL documentation for filter operators.

---

Exchange Online

Import users with PowerShell filtering.

Examples:

User mailboxes:  RecipientType -eq 'UserMailbox'
Specific database:  Database -eq 'DB01'
With size limit:  RecipientType -eq 'UserMailbox' -and ProhibitSendQuota -lt '50GB'

Note: Uses PowerShell comparison operators (-eq, -ne, -and, -or). See Exchange Online PowerShell documentation.

---

Other Source Filters

Google Workspace (GdriveSourceFilter / GdriveNGSourceFilter)

• Customer ID: Google Workspace Customer ID from Admin console
• Custom Schemas: Custom schema names to import (e.g., EmployeeData, LocationInfo)

Microsoft Teams (MSTeamsSourceFilter)

• Team ID: Azure AD ID of the Team to import users from

GitHub (GithubSourceFilter)

• Organization Name: GitHub organization to import from

Confluence (ConfluenceSourceFilter)

• Group ID: Override default groups (confluence-users, confluence-administrators)

SharePoint Online (SharepointOnlineSourceFilter)

• Sites: List of sites to import from (use site/subsite format for subsites)

SharePoint OneDrive (SharepointOneDriveNGSourceFilter)

• Azure AD Group ID: Group containing users
• OData Filter Query: OData v4.0 filter syntax (see Microsoft Graph API docs)
• Attributes: Fields to import

Cherwell (CherwellSourceFilter)

• Business Object ID: Cherwell user business object ID
• User Fields: List of field IDs to import

FreshDesk (FreshDeskSourceFilter)

• Agent ingestion: Include FreshDesk agents (default: false, requires bot admin privileges)

---

Best Practices

1. Filter at Source Level - Apply broad filtering at source before processors run (don't import all users then filter out 80%)

2. Specify Only Required Attributes - List only needed fields to reduce data transfer (5 fields vs 50+ unused fields)

3. Use Source-Specific Filters - Prefer dedicated filters (e.g., MSGraphSourceFilter) over generic for better validation

4. Test with Small Data Sets - Start with restrictive filter, verify results, then broaden as needed

5. Document Complex Filters - Add comments in config for multi-condition filters explaining purpose

---

Common Scenarios

Import Active Users Only

• Okta: status eq "ACTIVE"
• Azure AD: accountEnabled eq true
• Salesforce: IsActive = true

Import Specific Departments

• Generic: department IN ["Engineering", "Sales", "Support"]
• Salesforce: Department IN ('Engineering', 'Sales', 'Support')

Multi-Group Import (Azure AD)

1. Add first Microsoft Graph Filter with Group ID 1
2. Click "Add Another Filter"
3. Add second Microsoft Graph Filter with Group ID 2

Minimal Attribute Import

• Specify only: email, firstName, lastName, department, manager
• Result: 5 fields transferred instead of 50+

---

Troubleshooting

No users imported

• Filter too restrictive → Start with no filter, verify import, then add incrementally
• Incorrect syntax → Check source API docs for correct query format
• Invalid group/table → Verify ID/name exists in source system

Missing fields in user profile

• Field not in Attributes list → Add field name and re-run ingestion
• Check exact field name in source system API (case-sensitive)

Filter syntax error

• OData uses: eq, ne, and, or (lowercase)
• SOQL uses: =, !=, AND, OR (uppercase)
• PowerShell uses: -eq, -ne, -and, -or
• Verify field names match source exactly

Duplicate users from multiple filters

• Expected: Multiple filters create union of users
• To fix: Remove redundant filters or use single filter with OR logic

---

Filter Syntax by Source

OData v4.0 (Azure AD, Microsoft Graph, SharePoint OneDrive)

• Documentation: Microsoft Graph API - List Users / OData query parameters
• Operators: eq, ne, and, or, not, lt, le, gt, ge (lowercase)
• Example: (accountEnabled eq true) and (userType eq 'Member')

SOQL (Salesforce)

• Documentation: Salesforce SOQL and SOSL Reference
• Operators: =, !=, AND, OR, NOT, <, >, <=, >= (uppercase)
• Example: IsActive = true AND Department IN ('IT', 'Engineering')

PowerShell (Exchange Online)

• Documentation: Exchange Online PowerShell - Get-Recipient / Get-Mailbox
• Operators: -eq, -ne, -and, -or, -not, -lt, -gt, -le, -ge
• Example: RecipientType -eq 'UserMailbox' -and Database -eq 'DB01'

Okta (Generic Source Filter)

• Documentation: Okta API - List Users with Search
• Standard: OData-like filter syntax
• Example: status eq "ACTIVE"

ServiceNow (Generic Source Filter)

• Documentation: ServiceNow REST API - Encoded Query Strings
• Standard: ServiceNow encoded query operators
• Example: active=true^user_name!=admin

---

Limitations

• Maximum 20 filters per source - Filters are OR'd together (union of results)
• Unlisted attributes not transferred - Only specified fields are imported
• Performance considerations - More restrictive filters and fewer attributes = faster ingestion