What is Runtime Data Mapping?
runtime data mapping is the process of transferring a piece of data from one place to another. runtime data mapping allows you to move data between action properties, your runtime configuration, and any of the Flux variable managers.
When Do I Need Runtime Data Mapping?
Generally, you'll only need runtime data mapping to pass data into action properties that cannot be easily set using Variable Substitution. In practice, this usually means that you'll only use runtime data mapping for collections of data that are generated on-the-fly during workflow executions where the size is not known beforehand – this might be something like the collection of filenames returned from a file trigger, or the results of a database query returned from a database action.
Because variable substitution provides some visibility improvements over runtime data mapping, we recommend using variable substitution when possible; however, while variable substitution is suitable for gathering most of your dynamic data within an action, runtime data mapping provides the simplest solution for passing collections of data through your workflow.
What Is a Variable Manager?
Flux's variable managers are the storage depots for all of the data used in your workflows. Flux provides four such managers that you can use in your runtime data maps to send and receive data (for a more detailed discussion of the variable managers, see Variable Managers):
Action Property Variable Managers - Each trigger and action within your workflow has its own action property variable manager, which stores all of the property settings for the action. These are the same settings that you'll supply while building your workflow and configuring your triggers and actions – for example, the "Collection" of a For Each Collection Action, or the "Simple File Source" of a File Copy Action.
In a runtime data map, you'll use this variable manager to pass new data into your action properties – most often, to set one of the previously mentioned properties when you are working with collections of data and dynamically setting up file transfers.
Runtime Configuration - This variable manager comes from the Runtime Configuration for your engine. Every workflow running in the same namespace will share the same runtime configuration variable manager, and workflows will also inherit runtime variables from their parents in the runtime configuration tree.
Runtime variables are specified either by the name of the variable you want to use (like myvariable), which will allow Flux to dynamically search back in the tree for the closest node containing that variable name; however, you can also retrieve a variable from a different namespace by specify the fully-qualified variable name (like /mynamespace/myvariable).
Runtime variables are very rarely used in runtime data mapping as Variable Substitution can typically handle all runtime variable usage requirements. Variable substitution provides some visibility improvement over runtime data mapping, so we strongly suggest using variable substitution when possible.
Flow Chart Variables - Flow chart variables are variables that are created at workflow design time. These are static variables that are defined to allow several actions in a workflow to share the same core set of data.
As with runtime configuration variables, we recommend using variable substitution whenever possible when working with flow chart variables.
Flow Context -The flow context is the variable manager that will most frequently hold dynamic data generated at runtime – in particular, the flow context stores the result of an action after it executes.
Most runtime data maps will transfer data from the flow context variable manager into an action property variable manager.
In the Designer
Using the designer, it is simple to specify runtime data mapping. Begin by either right clicking on the flow that you wish to map variables between and selecting "Edit runtime data map", or by clicking the button next to the "Runtime data map" field in a node's action properties. The window shown in the image below will appear:
Using the runtime data map editing tool is fairly straightforward. To specify a new entry, begin by clicking the "+" button. You will need to input a source variable manager and a destination variable manager to complete the map. The action variable manager is unavailable as a source variable manager, as the action will have already completed before this mapping could take place. Runtime configuration variables will be unavailable as a target in mapping, as they are specified before the engine is started and unable to be changed after. The "Type" specifies the variable manager from which to retrieve a variable, and the "name" is the name of the variable you want to retrieve.
For example, suppose you wanted to use map the results of a file exists trigger to the collection input of a for each collection element action. The type of the source variable manager would be set to "Flow Context" (where the results from nodes are put) and with the name input as RESULT.url_string_matches, RESULT.url_object_matches, RESULT.filename_matches, or RESULT.fileinfo_matches depending on which collection you want to use. The target type would be set to "Action" with "Collection" selected from the drop down box for the name. All possible entries for the type of an "Action" target will always be listed in a drop down box. Refer to the screenshot below for the proper configuration:
Mapping variables in code is a bit more difficult than in the designer. The basic progression for runtime data mapping is; create a Map object -> make variable manager entries of your choice -> link them in the map -> set runtime configuration for a node or flow using the map. The variable manager entries different for each manager and are modeled with the following classes:
- Action properties, only able to be the target in a data map entry, are specified using only the "ActionPropertyTarget" class. This class is manufactured from the flowchart you are currently working on using the makeActionPropertyTarget(String target) method. The target would be the name of property you want to map to, with each word in beginning with an uppercase letter. You will be given an error is the property does not exist for either the target node or the node you are editing the runtime map on; such as when specifying "Collection" when the target or current node is not a for each collection element action. Making this class would look similar to this:ActionPropertyTarget myTarget = myFlowChart.makeActionPropertyTarget("Simple File Source");Flow chart variables are specified through one of two methods depending on whether the entry to be mapped is a target or source - FlowChartTarget and FlowChartSource. Instances for both can be made from the flowchart object you are currently working on. They can be made using the following code:FlowChartSource mySource = myFlowchart.makeFlowChartSource("list");
- FlowChartTarget myTarget = myFlowchart.makeFlowChartTarget("newList");Flow context mapping is similar to flowchart variables as they can be both retrieved from as a source, and mapped to as a destination. The setup for the flow context is almost exactly the same as flow chart sources and destinations with a slight modification in the names of the classes and of the methods to instantiate the classes. Consider the following code:FlowContextSource myFCSource = myFlowchart.makeFlowContextSource("RESULT.filename_matches");
- FlowContextTarget myFCTarget = myFlowChart.makeFlowContextTarget("filename_list");The runtime configuration variable manager is only available as the source of a flow and cannot be written to. Once again, the flowchart you are working on serves as a factory for making instances of the class that represents a link in the runtime data map - RuntimeConfigurationSource. The following code demonstrates making this class:
RuntimeConfigurationSource myRCSource = myFlowchart.makeRuntimeConfigurationSource("variable");
Now that all the proper classes have been introduced, it is possible to create a working runtime map. First you will need to create a blank Map object and two corresponding links (a source and target) for each entry you want to add to the data map. With your Map object configured, you can link it to a particular node or flow by using the setRuntimeDataMap(Map) method. The following code demonstrates using the runtime data map from start to finish. In the code, the results from a file exists trigger are mapped to the collection of a for each collection element action, and then passed on to a console action.import flux.*;
Runtime Data Map Properties for Triggers and Actions
Triggers and actions contain static, constant properties that are defined when a flow chart is created. These constant properties configure the behavior of different triggers and actions. Documentation on these properties is available in each action or trigger’s entry in the Javadocs.
For example, the "Time Expression" property on the Timer Trigger trigger configures how often the Timer Trigger fires. Normally, this property is set when the flow chart is created and initialized using the methodTimerTrigger.setTimeExpression() or by setting the JavaBean property called "Time Expression" on the Timer Trigger.
However, properties on triggers and actions can also be set dynamically at runtime. This feature makes it possible to configure an action with data that is generated at runtime.
For example, suppose you have a flow chart that contains a file trigger that monitors all ".xml" files on a remote FTP server or on the local file system. Further suppose that in a subsequent action, the flow chart must move the actual XML file that caused the file trigger to fire to a new location. This setup requires a trigger and action – a File Exist Trigger to locate the files to be moved based on a file criteria, which would be “.xml” in this case, and a FileMoveAction to actually move the files.
In your File Move Action, you must set a dynamic property so that the file that will actually be moved is the file that actually caused the file trigger to fire. To configure the trigger and action properties that are set at runtime, use a runtime data map. A runtime data map describes what trigger and action properties should be configured with data derived at runtime.
The following example shows how to configure a File Move Action to use the actual file name that was found by a File Exist Trigger.
The above source code first creates a runtime data map and then populates it with a single entry. This entry maps a flow context variable called RESULT.url_matches, which is placed into the flow context by the File Exist Trigger after it fires, to an action property called "Simple File Source", which is a property on the File Move Action.
When this flow chart runs and the File Move Action is activated, just before it executes, data is copied from theRESULT.url_matches flow context variable into the "Simple File Source" property on the File Move Action.
This runtime data mapping allows the File Move Action to actually move the file that was found by the File Exist Trigger. At the time that the flow chart is created, the exact file name that will be found by the File Exist Trigger is not known, because the File Exist Trigger is configured to watch for all "*.xml" files. But at runtime, when an actual XML file is found, say, "foo.xml", then the File Move Action is configured, on the fly, to move the file called "foo.xml".
Note that the above runtime data map example will not work correctly if "fileMoveAction" is declared to have a join point, because after a join point, the flow context variable managers on inbound flows are deleted. In the above example, the source of the runtime data map will not exist, because after the join point is satisfied, the source flow context is destroyed.
The kind of data that can be mapped at runtime includes:
- Source Data** flux.runtimedatamap.FlowContextSource. Data that is copied out of the flow context at runtime.
- flux.runtimedatamap.RuntimeConfigurationSource. Data that is copied out of the runtime configuration at runtime. The runtime configuration is described in Section 20.
- flux.runtimedatamap.FlowChartSource. Data that is retrieved from the current flow charts variable manager.
- Target Data** flux.runtimedatamap.ActionPropertyTarget. Data that is copied into a trigger or action property at runtime.
- flux.runtimedatamap.FlowContextTarget. Data that is copied into the flow context at runtime.
- flux.runtimedatamap.FlowChartTarget. Data that is copied into the current flow chart’s variable manager.
These runtime data maps can be defined on any action or any flow. The data mapping for a flow chart is performed at runtime while the flow chart executes.