Data Mapper / JSON Bender Reference
Data Mapper / JSON Bender Reference
Data Mapper / JSON Bender Reference
This document highlights the theory of data mapping language used at Moveworks. To view some common examples visit this document
The Moveworks Data Mapping Language is a concise, domain-specific language crafted to facilitate the transformation and manipulation of data within the Moveworks platform. It offers a suite of operators and functions that enable users to efficiently map, aggregate, and format data according to their requirements. MW Data Mappers’ design prioritizes ease of use, empowering users to implement complex data transformations with minimal coding.
In the Moveworks Data Mapper language, paired with MwDSL, accessing and refining payload data is straightforward and user-friendly. For example, to neatly extract and clean the display_value from a ServiceNow ticket:
Example Payload:
Simplified with MW Data Mappers:
Outcome:
A core concept in the MW Data Mappers is the BenderDefinition, a function for specific operations, like the simple_eval used here. While simple_eval is implicit for its simplicity and frequent use, other functions may require explicit calls. This guide will dive deeper into these functions, making data manipulation with MW Data Mappers intuitive and effective.
The Data Mapper syntax unlocks a suite of operations for transforming data. These operations range from simple evaluations to complex list manipulations and conditional logic.
simple_eval for pulling values and eval for evaluating expressions.object), arrays (array), and manipulating structure with FLATTEN and MERGE.MAP, SORT, COALESCE, and FILTER.CONDITIONAL) and lookup operations (LOOKUP) for dynamic data handling.CONCAT for joining strings, RENDER for template rendering, and STRIP_HTML for removing HTML tags.Description: Facilitates the direct extraction or computation of values from the response object in your data. It provides straightforward access to specific fields and supports the evaluation of expressions involving those fields, such as arithmetic operations.
Output Type: The output type depends on the evaluated expression and can be string, number, boolean, etc.
Parameters:
DSL)Given the example payload:
MW Data Mapping:
Expected Result:
In this example, SIMPLE_EVAL is used in two ways:
display_value of the number field, returning the ticket number as a trimmed string.value of both the state and impact fields from strings to numbers and then adding them together.Description: Performs dynamic DSL expression evaluation, facilitating complex computations and logical operations. It can incorporate custom arguments to modify the evaluation context, allowing for refined data manipulation.
Output Type: Varies based on the expression, including string, number, boolean, etc.
Parameters:
DSL)BenderDefinition)Given the example payload:
MW Data Mapper Mapping:
Expected Result:
This example showcases EVAL performing an arithmetic operation by adding the numeric values of state and impact fields, demonstrating basic computation capabilities.
MW Data Mapping:
Expected Result:
In this scenario, EVAL utilizes custom arguments (x and y) for the calculation. x is the numeric value of state, and y is double the numeric value of impact. This illustrates how custom arguments can be used to perform more complex calculations.
MW Data Mapping:
Expected Result:
Here, by providing a custom context (x and y as constants) and adding fields from the payload response, EVAL does not disregards the original payload values, showcasing its ability to add to the parent context for specific calculations or additions.
{}Description: Constructs a JSON object, facilitating the organization of data into a structured format. It inherently supports nesting and allows for the specification of complex paths as keys, enabling the creation of deeply nested objects directly within mappers.
Output Type: object
Parameters:
map<string, BenderDefinition>)Given the example payload:
MW Data Mapping:
Expected Result:
In this example, the OBJECT functionality is implicitly used to create a structured JSON object named ticket_info, which includes nested information about the ticket. It demonstrates how to organize flat payload data into a more hierarchical structure, enhancing readability and accessibility. Additionally, a direct field mapping for action_on_reject is shown, illustrating how simple fields and complex objects can coexist within a single mapper configuration.
[]Description: The ARRAY function is utilized to construct an array, allowing for the aggregation of multiple data elements, including static values, dynamic content derived from the payload, and nested arrays.
Output Type: array
Parameters:
BenderDefinition, enabling the inclusion of both static and dynamically evaluated content. (Type: BenderDefinition list)Given the example payload:
For demonstration purposes, let’s create a mapper that generates an array based on the provided test case structure, adapted to the example payload:
MW Data Mapping:
Expected Result:
In this example, the ARRAY functionality is implicitly utilized to create an array named example_array. This array includes:
display_value of sys_created_by, dynamically pulled from the payload."Fixed Value".value of state, showcasing that dynamic content based on payload data can be directly included.display_value of impact, demonstrating how arrays can be nested within one another for complex data structures.This illustrates the versatility of the ARRAY function in combining static values, dynamically derived data, and nested arrays to construct complex data structures tailored to specific requirements.
Description: The FLATTEN operator is designed to streamline nested structures by expanding any array-type elements into a parent list, while non-list elements are simply copied over. This is particularly useful for consolidating data from various nested sources into a single, uniform array.
Output Type: array
Parameters:
BenderDefinition array)Given the example payload:
To demonstrate the FLATTEN functionality, let’s create a mapper based on the structure of the provided test case, adapted to the example payload:
MW Data Mapping:
Expected Result:
In this example, the FLATTEN function is used to combine elements from various fields of the payload into a single array. It takes a mix of individual elements and arrays:
display_value of sys_updated_on is included as an individual element.display_value**s of number and state is expanded into the parent list.display_value of sys_created_by and impact are added as individual elements.This demonstrates how FLATTEN effectively merges nested arrays and standalone elements into a cohesive list, simplifying the structure for easier data manipulation and access.
Description: The MERGE function combines multiple objects into a single object. If there are any key conflicts between the objects, the value from the object that appears later in the sequence takes precedence. This allows for the creation of composite objects from disparate sources, with a clear resolution strategy for overlapping keys.
Output Type: object
Parameters:
BenderDefinition, enabling the inclusion of both statically defined objects and dynamically generated ones. (Type: BenderDefinition)Given the example payload:
To illustrate the MERGE functionality, let’s create a mapper that combines several objects, some of which share common keys, based on the structure of the provided test case:
MW Data Mapping:
Expected Result:
In this example, the MERGE function is used to combine three objects:
ticket_number and created_by derived from the payload.ticket_number, which overrides the value from the first object due to its later position in the list, and adds a new key status.impact.This demonstrates how MERGE effectively consolidates multiple objects into a single composite object, with later values taking precedence in the case of key conflicts, allowing for dynamic object construction and key-value updates.
Description:**A**pplies a specified transformation to each element within an array or each entry in an object. When processing an object, it converts the object into an array of entries, each containing key and value fields. The transformation is defined by a converter, which can utilize additional context about the current loop iteration.
Output Type: array
Iterator Object loop
With the loop object you can reference the index of the iterator. loop has two keys index0 and index1. Respectively, you can reference the index of the loop on a 0-base or 1-base using these arguments.
Parameters:
{key, value} objects for processing. (Type: BenderDefinition)context. (Type: BenderDefinition)LoopContextKeys)Given the example payload:
MW Data Mapping:
Expected Result:
This example demonstrates using MAP to append the string ” processed” to each specified item from the payload, showcasing a simple transformation applied to a manually specified array of values.
Assuming we want to transform the display_value entries of our payload into a list that describes each field:
MW Data Mapping:
Expected Result:
This mapping constructs an object from specified fields and then uses MAP to append ” description” to each value, converting the object’s values into a list.
Description: Organizes elements within an list or an object according to a specified key. For objects, it first converts the object into an array of its values. The key parameter defines how to derive a sort key from each element, which determines the sort order. Additionally, a desc parameter can specify whether the sorting should be in descending order.
Output Type: array
Parameters:
BenderDefinition)BenderDefinition)BenderDefinition, STRING ‘“true”’ or ‘“false”’)LoopContextKeys)Given the example payload:
Let’s say we want to sort the display_value fields of the payload’s objects. First, we would need to transform the payload into a suitable array format that SORT can operate on.
MW Data Mapping:
Expected Result:
This example aims to illustrate sorting a collection of the display_value fields from the payload.
Given a modified payload with object values:
MW Data Mapping:
Expected Result:
This example illustrates sorting the values of an object, converting the object into an array of its values, and then sorting those values in ascending order.
Description: Scans through the elements of items and returns the first element for which the condition evaluates to a truthy value. If the condition is not specified, it returns the first element that is inherently truthy according to standard truthiness evaluation in most programming languages.
Output Type: Varies based on the input items
Parameters:
BenderDefinition)BenderDefinition)LoopContextKeys)Given the provided example payload, let’s demonstrate how the COALESCE operator can be applied in this context.
Given the example payload:
display_valueMW Data Mapping:
Expected Result:
This example demonstrates using COALESCE to find the first non-empty display_value across various fields in the payload. Given all display_value fields are non-empty, it returns the first one, “Cancel all future Tasks”.
Given a payload where we are looking for the first truthy value in an array:
MW Data Mapping:
Expected Result:
In this example, COALESCE scans through the list values and returns the first truthy value it encounters, which is 12. This is because all other values (0, '', null, false, {}, []) are considered falsy in most programming contexts, leaving 12 as the first truthy value.
Given a payload where no elements meet the truthiness criteria:
MW Data Mapping:
Expected Result:
In this scenario, since none of the elements in the list values are truthy, COALESCE returns null, indicating that no suitable element was found.
Given a payload and a condition to find the first item greater than 4:
MW Data Mapping:
Expected Result:
This example demonstrates using COALESCE with a condition (item > 4). It evaluates each element against the condition and returns the first one that satisfies it, which is 6. Note that the string "fail - tests early exit" is ignored in the condition evaluation because it does not meet the numerical comparison.
Description: Evaluates each element within items against a specified condition, returning a listlist of all elements for which the condition evaluates to a truthy value. If the condition is omitted, FILTER defaults to using a truthiness filter, returning all inherently truthy elements.
Output Type: list
Parameters:
BenderDefinition)BenderDefinition)LoopContextKeys)Given a payload:
MW Data Mapping:
Expected Result:
This example demonstrates the default behavior of FILTER when no condition is specified. It returns all truthy values from the values array, effectively filtering out 0, "", false, [], {}, and null.
Given a payload:
MW Data Mapping:
Expected Result:
In this scenario, FILTER is applied with a condition (item > 3), returning an array of values that are greater than 3.
Given a payload that is not directly related to the test cases but demonstrates filtering with objects:
MW Data Mapping:
Expected Result:
This additional example showcases how FILTER can be applied to an object, considering it as an array of its values, and using a condition to filter based on a specific property (age in this case). Note that the original keys (john, jane) are not preserved in the output, as the object is treated as an array of values for filtering purposes.
Description: The CONDITIONAL operator evaluates a given condition and returns the result of on_pass if the condition is truthy, or on_fail if the condition is falsy. This operator allows for branching logic within the data transformation process, making it versatile for conditional data handling.
Parameters:
Given a payload:
MW Data Mapping:
Expected Result:
This example demonstrates a basic conditional check where the val is true, leading to the on_pass value being returned.
Given a payload:
MW Data Mapping:
Expected Result:
This example demonstrates the on_fail path being taken when the condition evaluates to false.
Let’s apply a conditional operation to the provided example payload, using the state value to determine an action:
Given the example payload:
MW Data Mapping:
Expected Result:
In this scenario, we’re checking if the state value is '7', which corresponds to a “Closed” state. Since the condition is met, the on_pass value is returned, indicating that no further action is required.
Description: The LOOKUP operator searches for a specified key within a provided mapping and returns the corresponding output. If the key is not found, an optional default value can be returned instead.
Parameters:
key is not found in the mapping.Given a payload:
MW Data Mapping:
Expected Result:
This example demonstrates a lookup operation where the key i has the value "a", leading to the lookup of "a" in the mapping and returning the corresponding value associated with "value1" in the payload.
Given a payload:
MW Data Mapping (Same as Example 1):
Expected Result:
In this scenario, the key i with value "b" matches directly to a constant "value2" defined in the mapping.
MW Data Mapping:
Expected Result:
In this example, the LOOKUP operation uses the state.value ("7") as the key to find a matching entry in the provided mapping. Since there is a match, it returns the corresponding descriptive status, "Closed".
Let’s assume we’re looking up the impact based on its value to provide a more detailed explanation, but the impact value is not in our mapping.
MW Data Mapping:
Expected Result:
For this scenario, since the impact.value is "4" and does not match any keys in the mapping, the LOOKUP operation returns the default value, "Impact not specified".
Description: Combines a list of strings into a final composite string, with an optional separator inserted between elements.
Parameters:
Given the example payload:
display_value Fields with SeparatorLet’s concatenate the display_value fields of various properties in the payload, using a specific separator.
MW Data Mapping:
Expected Result:
This example demonstrates using CONCAT to join the display_value fields of various properties with a separator " | ", creating a single, readable string that combines all the specified values.
Description: The RENDER operator utilizes a Mustache template to generate a string output based on provided args. If args are omitted, the root object is used by default.
Parameters:
If you are accessing an array index, you will need to access it with dot notation . instead bracket notation []. For example, if want to access the first element of a given array called first_list, you would need to access the array element as first_list.0 since the Render function does not support bracket notation.
Given the example payload:
MW Data Mapping:
Expected Result:
This example demonstrates using RENDER with a Mustache template to generate a summary string directly from the root object (payload), as no args are specified.
MW Data Mapping:
Expected Result:
In this example, RENDER is used with a Mustache template and custom args to generate a more detailed summary. The args include direct references to payload values and a LOOKUP operation to provide a more descriptive impact level.
Description: The STRIP_HTML operator processes a given HTML string (text), stripping away HTML tags and returning the plain text content.
Parameters:
Let’s assume we have a payload where one of the fields contains HTML content, similar to the structure of the provided payloads:
MW Data Mapping:
Expected Result:
This example demonstrates the use of STRIP_HTML to convert an HTML-formatted description into plain text, improving readability and ensuring the content is suitable for contexts where HTML rendering is not available or desired.