User Ingestion Processors Guide
Processors are data transformation tools that help clean, filter, and enrich user data during identity ingestion
Processors are optional data transformation steps in your User Import Flow. They clean, filter, and enhance user data during ingestion from source systems (Okta, Active Directory, Workday, etc.).
Note: Most processors operate on individual source data before merging, unless specifically stated otherwise. Processors execute in the order listed.
Processors to Apply: Add transformation processors that run during ingestion. Processors execute in the order listed.
Filter and Attribute List: Control which records and fields are imported at the source level before any processors run.
Processor Name: User Filter Processor
Excludes users from ingestion when a specified field matches any value in your exclusion list. This is a simple, single-field filter that performs exact matching.
Use Cases: Exclude terminated employees, contractors, test accounts, or specific departments based on a single field value.
Configuration:
Examples:
Processor Name: Filter Rule Post Processor
Excludes users from ingestion based on complex conditional logic. Unlike the simple Field Value filter, this processor allows you to combine multiple field conditions using AND/OR logic, perform date comparisons, and apply sophisticated filtering rules.
Use Cases: Apply multi-condition filtering (e.g., “active AND hired after date”), date-based filtering, or any logic requiring multiple field comparisons.
Configuration:
Examples:
💡 Tip: See Rule Syntax Reference for complete syntax.
Processor Name: DSL First Match Dedupe Processor
When the same user appears multiple times (identified by your Index Key, typically email), this processor evaluates all duplicate records and keeps only the first one that matches your filter condition. All other duplicates are discarded. This operates across all sources after they’re merged together.
Use Cases: Multiple integrations provide overlapping users, need to select which source’s data to prioritize, ensure each user appears only once in final roster.
⚠️ Note: Can be attached to any source - operates on merged data from all sources after ingestion.
Configuration:
Common Rules:
⚠️ Important: Always set Lowercase to
truewhen usingemail_addras Index Key.
Processor Name: User Timezone Processor
Automatically infers and populates the user’s timezone field by analyzing their location information (city, state, country). The processor uses geographic data to determine the most likely timezone for each user’s location.
Use Cases: Source system doesn’t provide timezone field, need consistent timezone data for time-based notifications and scheduling.
Configuration: No configuration needed - just add the processor. It automatically reads from standard location fields.
Processor Name: User Password Meta Info Processor
Fills in missing password date information using your organization’s password policy configuration. This processor operates on two fields in the user record: password_last_changed and password_expires.
What Fields It Uses:
password_meta_info.password_last_changed (date), password_meta_info.password_expires (date)password_expiry_in_days settingHow It Works:
Configuration:
Use Cases:
Processor Name: User Geocode Processor
Enriches user records with geographic coordinates (latitude/longitude) by geocoding their location information. The processor constructs a location query from specified fields, sends it to a geocoding service, and adds the resulting coordinates to the user’s geocodes field.
What Fields It Uses:
country_code, state, city)geocodes field with latitude/longitude dataUse Cases:
⚠️ Important: Attach to the source that contains the location fields you want to geocode.
Performance Note: Makes external API calls for geocoding - may slow ingestion for large user sets.
Configuration:
Processor Name: Unified Resolve Manager Processor
Establishes manager-employee relationships by resolving manager email addresses to internal user IDs. This processor builds an index of all users (email → ID), then replaces each user’s manager_email field value with the corresponding manager’s internal ID, enabling proper organizational hierarchy.
What Fields It Uses:
manager_email (manager’s email address)email_addr (all users’ emails)manager_email value with manager’s internal identifierHow It Works:
manager_email in the indexUse Cases:
⚠️ Note: Can be attached to any source - operates on all users after merge. Add AFTER deduplication to ensure manager links resolve correctly.
Configuration: No configuration needed - just add the processor.
Add filter processors before enrichment (like geocoding) to reduce processing time.
If using both processors, always apply deduplication first.
When deduplicating by email, always set Lowercase to true.
Add the geocode processor to the source that has location fields (country_code, state, city).
Goal: Exclude terminated and inactive users from Okta.
Steps:
employment_status, Filter List: Terminated, InactiveGoal: Use both Okta and Workday, preferring records with employee IDs.
Okta Source:
Workday Source:
worker_type, Filter List: Contractor, TempEither Source (Deduplication):
email_addrrecord.employee_id != ""trueGoal: Keep only active, full-time employees with company email addresses.
Steps:
Goal: Establish manager relationships when source provides manager emails.
Steps:
Note: Add AFTER any deduplication processors.
Solution:
Check:
true for email-based deduplicationCheck:
Check:
"value" not value["value1", "value2"]Direct field names, no prefix needed.
Uses record. prefix to access fields.
Processor Limits:
Rule Constraints:
Performance Considerations: