Dataweave Tips & Guidelines

Dataweave Tips & Guidelines – Here are few tips and best practices which could be useful to consider while writing dataweave. There are plenty of operators in dataweave which can make your life easy while writing complex dataweave transformation. Obviously the list below is not an exhaustive list, but we have tried to cover few important aspects while writing dataweave.

Dataweave Tips

Consider the payload below for reference

{
	"name": "Mulesy Mulesy",
	"age": 30,
	"profession": "Mulesoft",
	"date": "2020-05-29T10:02:45-04:00",
	"comment": "Special characters. "
}

Dealing with special characters in dataweave

If you have some special character (e.g. “, $ etc) in your payload here is how you can deal them in dataweave using “\”.

%dw 2.0
output application/json
---
{
    name: payload.name,
    age: payload.age,
    profession: payload.profession,
    date: payload.date,
    comment: payload.comment ++ "\$\$ handling special character \$\$"
}

Pattern matching in Dataweave

if you want to validate a complex pattern in your payload, you can use power of regex in dataweave.

E.g. if you want to evaluate whether your payload.name has any number in it, here is how you can use regex. You can use regex for any complex pattern evaluation.

%dw 2.0
output application/json
---
{
    name: payload.name,
    age: payload.age,
    profession: payload.profession,
    date: payload.date,
    comment: payload.comment ++ "\$\$ handling special character \$\$",
    "regex_number_check" : payload.name contains /\d+/
}

Using static java methods of available libraries (e.g. apache) directly in Dataweave

For this you will need to add the dependency of the library in the pom file. In the script below we are using the split method of the apache StringUtils class directly into the dataweave

%dw 2.0
output application/json
java!org::apache::commons::lang3::StringUtils
---
{
    name: payload.name,
    age: payload.age,
    profession: payload.profession,
    date: payload.date,
    comment: payload.comment,
    other_Value: StringUtils::split("aa-bb-cc-dd","-")
}

Result

{
    "name": "Mulesy Mulesy",
    "age": 30,
    "profession": "Mulesoft",
    "date": "2020-05-29T10:02:45-04:00",
    "comment": "Special characters. ",
    "other_Value": [
        "aa",
        "bb",
        "cc",
        "dd"
    ]
}

Creating dynamic elements or keys in an Object

if you want to add a dynamic key or a key value pair dynamically, into an object. Here is how you can write it.

%dw 2.0
output application/json
var dynamic_element = {"dynamic_element": "dynamic element"}
var dynamic_key = "dynamic_key"
---
{
    name: payload.name,
    age: payload.age,
    profession: payload.profession,
    date: payload.date,
    comment: payload.comment ++ "\$\$ handling special character \$\$",
    "regex_number_check" : payload.name contains /\d+/,
    (dynamic_element),
    (dynamic_key) : "Dynaimc key"
}

Creating comments in Dataweave

Comments are very helpful if you are doing some complex transformation and wants to mention the outcome of the complex steps . Here is how you can use comments in dataweave scripts.

%dw 2.0
output application/json
var dynamic_element = {"dynamic_element": "dynamic element"}
var dynamic_key = "dynamic_key"
---
{
    name: payload.name, // single line comment
    age: payload.age,
    profession: payload.profession,
    date: payload.date,
    comment: payload.comment ++ "\$\$ handling special character \$\$",
    "regex_number_check" : payload.name contains /\d+/,
    /*
    * dynamic key below
    * dynamic element below
    * */
    (dynamic_element),
    (dynamic_key) : "Dynamic key"
}

TimeZone handling in the dataweave

If you want to convert the time to a different timezone, you can do that as below. You can get the value of various timezones using java!java::util::TimeZone::getAvailableIDs()

%dw 2.0
output application/json
---
{
    name: payload.name, // single line comment
    age: payload.age,
    profession: payload.profession,
    date: (now() as DateTime >> 'America/Chicago' ) as String,
    comment: payload.comment
}

Adding year, month, day, hour, minutes or second to a date time

Here is how you can add or subtract date time parameters from a date time.

%dw 2.0
output application/json
---
{
    name: payload.name, // single line comment
    age: payload.age,
    profession: payload.profession,
    date_1: payload.date as DateTime + |PT1S| + |PT1M| + |PT1H| + |P1D| + |P1M| + |P1Y|,
    date_2: payload.date as DateTime - |PT1S| - |PT1M| - |PT1H| - |P1D| - |P1M| - |P1Y|,
    comment: payload.comment
}

null check in dataweave

Various operator (e.g sizeOf , “to” etc) breaks and result in null pointer exception if the input to these operator is null. It is always useful to define default value using default operator to a variable

Consider the payload below

<employees>
	<employee department="HR">HR Mulesy Mulesy</user>
	<employee department="OPS">OPS Mulesy Mulesy</user>
</employees>

isEmpty(payload.employee) will result true
payload.employee == null will result true
payload.employee !=null will result false

Selectors use case in dataweave

Consider the payload below

<employees>
	<employee department="HR">HR Mulesy Mulesy</user>
	<employee department="OPS">OPS Mulesy Mulesy</user>
</employees>

payload.employees.*employee will result list of all employee : [ “HR Mulesy Mulesy “, “OPS Mulesy Mulesy”]
payload.employees.&employee will result key and value of all employee : {employee: “HR Mulesy Mulesy” , employee : “OPS Mulesy Mulesy” }
payload.employee? will result: false
payload.employee! will result: an error

Relational operator “~=” in dataweave

If you need to compare the value of two different types, this operator can come very handy. E.g. In the script below, we are trying to compare numeric 30 with string “30”. “==” operator would compare the types also and would result false.

%dw 2.0
output application/json 
---
{
    name: payload.name,
    age: payload.age,
    profession: payload.profession,
    date: payload.date,
    comment: payload.comment,
    operator1 : 30 ~= "30",
    operator2 : 30 == "30"
}

Iteration and aggregation over a collection in dataweave

Sometimes we need iterate through a collection and accumulate the value for every iteration. You can use reduce function in dataweave. Details here https://mulesy.com/reduce-function/

Conditional check for an element’s inclusion in object

if you ever need to create an output where the element/s occurrence depends on a condition, here is how you can do that in dataweave.

%dw 2.0
output application/json
var condition = 1
---
{
    name: payload.name, // single line comment
    age: payload.age,
    profession: payload.profession,
    date:payload.date,
    (comment: payload.comment) if (condition !=1)
}

Add/update a key/s of an object in dataweave

If you want to add or update the value of key of an object. Here is how you can write the Dataweave . If the key is not present in the payload, then it would add a new key to the object otherwise it would update the existing key’s value

%dw 2.0
output application/json
---
payload ++ {
    location: "India", // single line comment
}

Remove key/s of an object in Dataweave

%dw 2.0
output application/json 
---
payload -- ["date","comment"]

Prepend, Append, and Remove Operators for Arrays

If you need to add, remove any elements in array; you can use ++, — operator to do the same

Formatting in dataweave

You can format a string to a particular format in dataweave. First convert the input to a desired type and then to a String and then use {format: “#,###.###”}. E.g.

%dw 2.0
output application/json
---
{
    name: payload.name,
    age: payload.age,
    profession: payload.profession,
    date: payload.date,
    comment: payload.comment,
    number: 423424.978 as String {format: "#,###.0000"},
    format1: now() as String {format:'yyyy-MM-dd-HH:mm'},
    formtat2: 123131.3 as String {format :"0.0000" },
    format3 : now() as Date as String {format: "YYYY-MM-dd'T00:00:00Z'"}
}

Functions and it’s various use cases

We should always use functions to create a reusable piece of functionality/method. Functions can also be created to repeat or call a feature recursively. Function are defined using “fun” attribute. Function can accept any arguments e.g. object, collection, simple type, functions, lambdas. Functions and lambdas (anonymous functions) can be passed as values or be assigned to variables. Here are few ways to define the functions.

Sample payload

{
	"name": "Mulesy Mulesy",
	"age": 30,
	"profession": "Mulesoft",
	"date": "2020-05-29T10:02:45-04:00",
	"comment": "Special characters. ",
	"CHILD_Element1" : {
		"element1" : "element1",
		"child_element_2" : {
			"ELEMENT2"  : "element2"
		}
	}
}

Code Snippet.

%dw 2.0
fun fun1(user) = {
	firstName: user.name,
	age: user.age
}
var var1 = (user, age) -> {
	firstName: user.name,
	age: age
}
fun fun2(user) = do {
	var age_present = (user.age) !=null
---
if(age_present == true) {
	firstName: user.name,
	age: user.age,
	id: "1"
}
	else {
	firstName: user.name,
	age: "default age"
}
	
}
fun fun3(element, func) =
    if (element is Object)
        element mapObject (value, key) -> {
	(func(key)) : fun3( value, func)
}
    else element
output application/json
---
{
	name: payload.name,
	age: payload.age,
	profession: payload.profession,
	date: payload.date,
	comment: payload.comment,
	funusage_1: fun1(payload),
	variable_1: var1(payload, 30),
	funusage_2: fun2(payload),
	funusage_3: fun3(payload, (key) -> lower(key))
}

Transformation Result

{
	"name": "Mulesy Mulesy",
	"age": 30,
	"profession": "Mulesoft",
	"date": "2020-05-29T10:02:45-04:00",
	"comment": "Special characters. ",
	"funusage_1": {
		"firstName": "Mulesy Mulesy",
		"age": 30
	},
	"variable_1": {
		"firstName": "Mulesy Mulesy",
		"age": 30
	},
	"funusage_2": {
		"firstName": "Mulesy Mulesy",
		"age": 30,
		"id": "1"
	},
	"funusage_3": {
		"name": "Mulesy Mulesy",
		"age": 30,
		"profession": "Mulesoft",
		"date": "2020-05-29T10:02:45-04:00",
		"comment": "Special characters. ",
		"child_element1": {
			"element1": "element1",
			"child_element_2": {
				"element2": "element2"
			}
		}
	}
}

Function fun1 takes object as input and outputs an object with keys firstName and age. This is a simple function.

Function fun2  takes object as input and uses “do” attribute to create a variable in function. it assigns the variable “age_present” to boolean true or false depending upon the value of age attribute in payload passed as input. it then conditionally outputs the result depending upon the variable value. This function explains how we can create variable and use them in functions

Function fun3 is basically to achieve the recursion type of feature. Here were are changing the element’s key to lower case recursively. It accepts a payload  and a function as input parameter. Here we are trying to change all the keys of an object to lower case recursively. For recursion type of logic, you would always need to have a if else block to exit from the recursion when you have reached to end of it.

Best practices or Guidelines

Use Custom Dataweave modules

use custom dataweave modules for re-usability across the organization. Create a file under src/main/resources/dwmodules/common.dwl

Use the above global function as below in dataweave script

Use global variable in DataWeave

It optimizes and simplifies your DataWeave code. Since the global variables are evaluated once hence it helps in improving the performance also.

%dw 2.0
output application/json
import  * from dwmodules::common
var currentdate = getCurrentDateTimeCST()
---
{
    name: payload.name,
    age: payload.age,
    profession: payload.profession,
    date: currentdate,
    comment: payload.comment
}

Use local variable in dataweave.

It simplifies your code and improves performance in case you want to further evaluate again and again. This could be useful inside map operators, where you have to iterate over a changing collection again.

%dw 2.0
output application/json 
import *from dwmodules::common
---
using (localvar1 = "pp", localvar2 = "123") {
    name: payload.name,
    age: payload.age,
    profession: payload.profession,
    date: getCurrentDateTimeCST(),
    comment: payload.comment
}

Create dwl files for large resuable transformations.

Create dwl files instead of inine dataweave for reusable and long dataweave scripts. You can create a folder like src/main/resources/dwl refer all the *.dwl files from there. Make sure to include that in artifact.json under “exportedResources” element.

Use named parameters for key, index and value in operators which accept lambda 

Always use named parameters in operator e.g. map/mapobject/filter/pluck etc for any reference to key, value and index.  e.g. use “map (employee, employee index) – > {} “ instead of “map()- > {}”.  Don’t reference using $, $$, $$$ . This would also help in correlating the elements in different maps

Use dataweave functions in DataWeave

if you are doing a repetitive task in a particular dataweave scripts, then it’s better to create functions either global or local and refer them in the dataweave. Use global dataweave functions if the same function is used across the multiple projects.

Please do let us know if you have any specific guidelines related to dataweave that one should follow. Thanks and happy reading!