There are two configuration components for the Flux engine:
- Static. Static configuration options are described in greater detail in the Engine Configuration chapter.
- Runtime. Runtime configuration options typically determine how flow charts run (for example, limiting the number of flow charts that can run at once) or contain data that the flow charts will use while running. Unlike the static configuration, the runtime configuration is dynamically reloaded, so runtime configuration settings can change while the engine is running. Runtime configuration options are applied to nodes, or branches, on the flow chart namespace tree, allowing you to define runtime configuration properties on a per-namespace level.
Every engine is controlled by both its static (non-runtime) configuration and its runtime configuration.
The runtime configuration is option. To enable the runtime configuration, you must first specify the configuration file location, then set the properties you would like to use.
Setting the Runtime Configuration File
To use a runtime configuration file, you can simply add the following property to your engine configuration:
RUNTIME_CONFIGURATION_FILE=/path/to/myfile.config
Refreshing the Runtime Configuration File
If you make changes to the runtime configuration file, Flux can automatically and dynamically pick up your changes while the engine is running (without requiring an engine restart). This refresh is performed at a specific interval, which you can adjust by setting the RUNTIME_CONFIGURATION_FILE_REFRESH_FREQUENCY like so:
RUNTIME_CONFIGURATION_FILE_REFRESH_FREQUENCY=+30s
This property is a time expression. The default value is +m (every one minute).
Setting Properties in the Runtime Configuration
The Flux runtime configuration is a standard Java properties file. This means that each line of the file is expected to contain a single key / value pair.
In the runtime configuration file, the key is expected to be a namespace corresponding with your flow chart namespace (see #Tree of Configuration Properties below for more information) plus the name of the property you want to set. The value of each pair is the value should be the intended value of the property you are setting.
There are two types of properties available:
- Runtime Configuration Property. These properties configure how flow charts are executed on the engine. Runtime configuration properties may include concurrency throttle settings, priority values, FIFO settings, and more. See the #List of Runtime Configuration Properties for a complete list of available properties of this type.
- Runtime Variables. These are custom variables that you can use to provide data to your flow charts. Typically, this would be used when a single value (for example, a username or password) is used in several flow charts. Using runtime variables allows to specify variable values in a single location, rather than re-entering them across several flow charts. You can use variable substitution or runtime data mapping to access the runtime variable values inside a flow chart.
Runtime configuration properties are set like so:
/namespace/<PROPERTY TYPE>=<value>
Runtime variables, similarly, are set like:
/namespace/<VARIABLE NAME>=<value>
Runtime configuration and variable properties can be set on any namespace, including the root ("/"). These settings are applied to all flow charts on the namespace, as as well as all flow charts on child namespaces.
A sample runtime configuration might look like:
/MAIL_SERVER_VARIABLE=mymailserver.com
/CONCURRENCY_THROTTLE=8
/namespace/CONCURRENCY_THROTTLE=5
Runtime configuration namespaces
Runtime configuration properties can only be applied to a namespace, not directly to a particular flow chart or flow chart name. This means, for example, that a flow chart with the full namespace (including flow chart name) of:
/MyNamespace/MySubNamespace/MyFlowChartName
Will pick up any properties set in the runtime configuration for namespaces:
/MyNamespace
And:
/MyNamespace/MySubNamespace
But a property set on the full name of:
/MyNamespace/MySubNamespace/MyFlowChartName
Will be ignored. Keep this in mind when naming your flow charts and designing the runtime configuration.
List of Runtime Configuration Properties
The following properties can be set in the runtime configuration. All of the properties listed can be specified across the entire engine (by setting them on the root namespace), or assigned to an individual namespace.
Property Name |
Description |
---|---|
CONCURRENCY_THROTTLE |
The total number of flow charts that can run concurrently within this namespace. For example, a concurrency throttle of 5 would mean that 5 flow charts in this namespace should be allowed to run at once. Note that there is one exception to this: if a flow chart contains a split (or, in general, any concurrently executing actions), each action will count toward the total concurrency. For example, if a flow chart contains 5 actions running at once, and the namespace has a concurrency throttle of 5, that flow chart will use all of the available concurrency slots and no other flow charts will be allowed to run in the same namespace until the actions complete. |
CONCURRENCY_THROTTLE_CLUSTER_WIDE |
Identical to the CONCURRENCY_THROTTLE setting, except that this property specified the total number of flow charts in this namespace that can be allowed to run across the entire cluster. For example, if engines A and B are running in a cluster, the CONCURRENCY_THROTTLE_CLUSTER_WIDE for each engine is set to 8, engine A is running 5 flow charts, and engine B is running 3 flow charts, then neither engine will accept any new flow charts until the concurrency slots are available again. |
DEFAULT_FLOW_CHART_ERROR_HANDLER |
The error handler that flow charts in this namespace should use. This property should be set as a path to the error handler file. The path can be absolute, like "/path/to/myfile.ffc", or it can be relative to the Java home directory (that is, the directly where the JVM was started, usually the directory containing the script to launch the Flux engine). A relative path may look like "folder/myfile.ffc". |
FIFO_SCHEDULING_ENABLED |
Normally, flow charts that are eligible for execution are scheduled for execution based on their priority and their "next execution date". Sometimes, however, it is desirable to schedule flow charts to run in a first-in-first-out (FIFO) order. For example, suppose 10,000 trucking manifests need to be generated, and the manifest flow charts that were scheduled first should be, generally speaking, processed first. Flux normally processes such flow charts in this manner by default. However, the existence of a trigger or an error handler in a flow chart can cause a flow chart to "go to the back of the queue". In these cases, it may be that the trucking manifest flow chart that was scheduled first may end up running behind the ones that were originally submitted afterwards. |
INTERNAL_LOGGER_DEBUG_ENABLED |
If this property is set to true, action and trigger properties and variables for flow charts in this namespace will be logged in greater detail to the log file and audit trail. As the name implies, this property can only be used if the internal logger is enabled. |
LISTENER_CLASSPATH |
Specifies the listener class path for flow charts in this namespace. This property specifies the listener classpath for flow charts, which is used to load listener classes in Java Actions and Dynamic Java Actions. |
PRIORITY |
Specifies the priority for flow charts in this namespace. Priorities are used to give precedence to certain flow charts when there are more flow charts available to run than concurrency slots open to run them. |
RUN_AS_USER |
The username of a user whose permissions will be used to run this flow chart. This can only be used when using the built-in Flux security mechanism. |
Tree of Configuration Properties
The runtime configuration is set up as a tree of nodes (following the normal tree structure). Runtime configuration settings are applied on a per-namespace level, so the runtime configuration tree will mimic the flow chart namespace tree. A property set on a branch in the runtime configuration tree is applied to the branch of the same name in the flow chart namespace tree.
Runtime configuration namespaces are not case-sensitive. For example, the namespace "/MyNamespace" is equivalent to setting "/mynamespace" in the runtime configuration.
Each configuration property consists of a key and a value. The key must be a string. The value must be a persistent variable.
Although the runtime configuration tree can contain any kind of property, it includes pre-defined properties: concurrency throttle, concurrency throttle cluster-wide, default flow chart error handler, and priority.
Flow chart names are hierarchical. They form a tree of jobs. For example, the two flow charts, "/lightweights/job 1" and "/heavyweights/job 2", form a tree with two branches off the root, "/lightweights" and "/heavyweights".
You can annotate this tree of jobs using properties in the runtime configuration. For example, to set a concurrency throttle for heavyweight jobs, set an appropriate concurrency throttle in the "/heavyweights" branch of the runtime configuration tree. To set a default flow chart error handler for lightweight jobs, set an appropriate error handler in the "/lightweights" branch of the runtime configuration tree.
Properties set on a child namespace will override any properties set on a parent. For example, if a flow chart runs in the namespace "/parent/child/", it will first check the namespace "/parent/child/" for any runtime properties. Properties for the "/parent" namespace would only be applied if they were not set in "/parent/child".
In summary, the tree of configuration properties mirrors the tree of jobs. A job’s behavior can be further configured based on the branches in the runtime configuration tree that correspond to that job’s fully qualified name.
Runtime Configuration Trees on Externally Secured Engines
If your engine has external security enabled, the runtime configuration tree must contain the name of the group for the properties you are trying to set, as well as the namespaces you are setting them on. The group name should be included at the base of the configuration tree, preceding the namespace.
When using an external security provider (such as LDAP), all users and flow charts are automatically assigned to the "default" group. Therefore, your runtime configuration might look something like:
/default/mynamespace/CONCURRENCY_THROTTLE=2
Repeat this as necessary for each namespace and property combination you want to set.
Runtime Configuration Properties for Namespaces that contain a space
If your namespace contains a space character, like:
/Folder A/MyFlowChart
You will need to make sure the space is escaped in your runtime configuration by adding a backslash ('\') character before the space in the configuration. For example, to set a runtime configuration property for a flow chart with the name and namespace above, you could add the following line to your runtime configuration file:
/Folder\ A/CONCURRENCY_THROTTLE=0
Memory Usage
It is assumed that your entire runtime configuration tree can be stored in memory. A very large runtime configuration can cause memory problems in Flux, so take care to ensure that you are not setting any unnecessary or extraneous data in the tree.
Storing Encrypted Passwords in the Runtime Configuration
If you have a password value that is used in several flow charts across the engine, it often makes sense to store the password in one central location where all of the flow charts can easily access it.
In this case, you would normally not want to store the password in plain text in the runtime configuration either (where any user with access to the runtime configuration file or the logs might access it). To help with this, Flux allows you to store an encrypted value in the runtime configuration, protecting your password values and ensuring that they are not available in plain text anywhere on the system.
To store an encrypted password in the runtime configuration and access it within your flow charts, follow these steps:
- Follow the Command Line Interface for generating an encrypted password. This will create an encrypted password which should look something like Gc4f0oRkPJ4=.
- Edit your runtime configuration to store the encrypted password. The encrypted password doesn't require any special syntax and can be set like any other variable:
/MyEncryptedPassword=Gc4f0oRkPJ4=
- Use runtime substitution to access the value in the password field on your action:
${runtime MyEncryptedPassword}
That's it! This technique can be used for any action in Flux that requires a password, including file actions and triggers that use a password for authentication to a remote host.
The Runtime Configuration File and Java API
When using the runtime configuration file, it is automatically refreshed from the file system to pick up any changes that might be made. For this reason, it is important that you do not make any changes to the runtime configuration using the API if you are using a runtime configuration file (the engine has no way of knowing whether the file or API changes are the "correct" version, so the automatic file refresh assumes that it is correct and will overwrite any changes you have made in the API).
If you are using the runtime configuration file, any changes to the runtime configuration should be performed in the file alone. If you are using the API, the runtime configuration should be managed entirely that way, avoiding the runtime configuration file altogether.
Comments
Please sign in to leave a comment.