Flux models jobs using traditional flow charts. Flux’s flow charts are as expressive and powerful as a finite state machine. By using a flow chart to model your job, your job can be arbitrarily complex. For example, your job may not contain any timing elements at all. It may simply execute actions A, B, and C in order.
On the other hand, if your application requirements dictate that you execute actions A, B, and C in order, and then wait for a certain time to occur, followed by actions D, E, and F, then a flow chart is powerful enough to model that job.
Flow Chart Namespaces
In Flux, flow charts are stored and organized using a hierarchical namespace system (called the "namespace tree"). Just like a file system on a server, namespaces in Flux begin at a root level ("/"), followed by a series of 0 or more "folders" or "branches", then finally a unique name for the flow chart itself, to identify the flow chart among other flow charts on the same branch.
A typical flow chart namespace might look something like:
This namespace begins with the root ("/"), followed by the name of a branch (branch names should end with the "/" character, to indicate a branch in the namespace tree rather than a specific name), another branch below that one, and finally the flow chart name.
Flow charts can also be stored directly in the root of the namespace tree:
When you create a flow chart in Flux, you'll be asked to give it a name that conforms to the namespace system. Your namespaces can be arbitrarily simple or complex; Flux doesn't place any restrictions on flow chart namespaces, except that every namespace must begin at the root ("/") level.
Flow charts can also be stored at any level on the tree. For example, you might have several flow charts running in Flux with a namespace tree that looks like:
These namespaces are not only used to organize flow charts according to their branches, but also control how a particular flow chart will relate to other concepts in Flux like the runtime configuration. For this reason, we suggest storing flow charts that share many common properties or similar functionality under the same namespace to allow easy configuration.
For example, suppose that you have several flow charts that all use Flux's file transfer capabilities. Now suppose that within those flow charts, one group will use a particular FTP server "ServerA", and another will use a different server, "ServerB". When setting up the flow charts, you would just place all file transfer flow charts under some namespace (say "FTP"), then you could add namespaces below that for "ServerA" and "ServerB". You might end up with a namespace tree looking something like:
You could then use the runtime configuration to simplify the configuration of these flow charts, and provide a central, common location for setting shared properties across those groups of flow charts.
Occasionally while using Flux, you may be prompted to input a namespace in order to search for or operate on a flow chart or set of flow charts. If Flux prompts you for a namespace, note that you can either use the specific namespace of a certain flow chart ("/FolderA/MyFlowChartName"), or you can enter a branch on the tree ("/FolderA/") to get all of the flow charts running under that branch. In these cases, the term "namespace" can be used interchangeably to refer to a particular flow chart name and its full namespace, or simply to a branch on the namespace tree.
Flow Chart Properties
The following properties can be set on any flow chart in Flux.
The date when this flow chart (or flow chart run) is expected to be complete by. A flux.audittrial.server.DeadlineExceededEvent is published to the audit trail if this flow chart does not complete execution by the deadline.
If the flow chart does not contain a run (that is, one action marked as a start of run and another marked as an end of run), the deadline will apply across the entire lifetime of the flow chart. If the flow chart does contain a run, the deadline will be re-evaluated for each run of the flow chart.
A deadline date cannot be set if a deadline time expression is set.
Deadline Time Expression
A time expression specifying the time frame that the flow chart (or flow chart run) is expected to be completed in. For example, a deadline time expression of "+7m" means the flow chart (or each run in the flow chart) is expected to finish within seven minutes.
If the flow chart does not contain a run (that is, one action marked as a start of run and another marked as an end of run), the actual deadline date is calculated when the flow chart is first exported to the engine by applying the time expression to the date and time that the flow chart was exported. If the flow chart does contain a run, the deadline date for each run within the flow chart is calculated at the start of that run.
A deadline time expression cannot be set if a deadline date is set.
A relative time expression which specifies how soon before the deadline the engine will publish the event flux.audittrail.server.DeadlineApproachingEvent to the audit trail. This event allows you to use the audit trail to view any flow charts that are approaching their deadline. The time expression should be relative to the date and time of the actual deadline (for example, a time expression of "-7m" would mean "seven minutes before the date and time of the deadline).
The deadline window defaults to "-1H" (one hour before the deadline) if this property is not defined.
Listener Class Path
If your flow chart contains Java Actions or Dynamic Java Actions, the listener class path allows you to dynamically load your action listeners without requiring a JVM restart (and even allows you to dynamically install new or updated versions of your listeners without restarting the JVM).
The listener class path can be set directly in the flow chart using this property, or more broadly on the runtime configuration tree. If the listener is set using this property, it will take precedence over any class path set in the runtime configuration.
The listener class path can be used to load JAR files on the file system. For example, assume that you have a file database_listener.jar located on the directory C:\action_listeners on Windows (or /action_listeners on Unix). To allow the flow chart to dynamically load any listener classes in the JAR, you could just set the listener class path to:
The listener class path will also accept a path to a directory rather than a particular JAR. If a directory is specified, any JAR files within that directory will be automatically included in the listener class path.
You can even mix and match JAR file and directory paths in the listener class path. If, for example, we have multiple JAR files in the directory /reports_listeners that we want to include in our listener class path, we can modify the listener class path above to:
Each item in the list is separated by a ":" or ";" character, depending on the operating system on which the code runs (note that on Windows, a letter followed by either ":" or ";", like "C:", is interpreted as a drive letter). Paths to JAR files and directories can contain the "/" or "\" path separator characters.
You can also specify an HTTP URL path to a JAR file in the listener class path (note, though, that URL paths cannot be mixed and matched with local JAR or directory paths).
For example, ifdatabase_listener.jar was located at the URL http://localhost:8080/jars/database_listener.jar, you could use the following listener class path setting:
We strongly recommend that you do not use the listener class path setting if you are running Flux within an application server. The application you use (probably packaged as a WAR or EAR file) should be structurally sound within itself and should not require outside classes to run.
The listener class path is only designed to load a few listener classes for use with Java Actions or Dynamic Java Actions. If you need to load hundreds or thousands of classes, or if you need to load dependent JARs for your listeners, you should use the standard JVM class loading techniques rather than the listener class path.
The listener class path also cannot be used to reload a JAR file if it also appeared on the JVM class path. Take care to ensure that any classes that need dynamically reloaded are not included on the JVM class path.
Every flow chart is assigned a priority. Higher priority flow charts generally run before lower priority flow charts. Flow chart priorities make it possible to specify that important or time-critical flow charts need to run before other, less important, flow charts. Flow chart priorities allow you to specify that if two flow charts could otherwise both execute at the same time but concurrency throttles will allow only one of those two flow charts to run, the flow chart with the higher priority runs first.
If a flow chart has a priority explicitly set, it will override any priority set in the runtime configuration. If a null flow chart priority is used, it will allow the runtime configuration priority to take precedence.
The highest priority is 1. Lower priorities have values greater than 1, such as 10, 25, 500, etc.
If two different flow charts have different priorities, the flow chart with the priority closer to 1 runs first. If two different flow charts have the same priority, then the flow chart with the oldest timestamp runs first. Each running flow chart contains a timestamp that indicates the next time the flow chart requires attention from the Flux engine.
It is possible that higher priority flow charts will run so frequently as to prevent lower priority flow charts from running at an acceptable rate. This behavior is called starvation, that is, lower priority flow charts can be starved or prevented from running as frequently as you would like. By default, Flux permits starvation, because it often does not cause any serious problems, and it is sometimes desirable.
If starvation impacts you adversely, you can enable the Flux configuration property FAIRNESS_TIME_WINDOW. This property contains a time expression that indicates how frequently starved flow charts should have their priorities increased by 1 (the effective priority). (To increase a flow chart's effective priority, its numerical value is decremented by 1.) Eventually, such flow charts will reach an effective priority of 1 and be eligible for execution before all other flow charts. After firing, such flow charts revert to their original priorities and are again eligible to have their priorities increased if they starve. This anti-starvation behavior is called fairness.
Note that whenever a trigger fires internally, even if it does not fire out of the trigger and force execution of the flow chart to continue to the next action, that flow chart’s effective priority is reset to its original priority.
Also note that because starved flow charts have their priority increased by one (their priority value decremented by one), appropriate priorities should be given at start. If you have your highest priority flow chart set to one, and all others set to 500, the benefits of fairness may not be significant or even noticeable.
Flow chart priorities are stored in an engine’s runtime configuration. Each branch in the runtime configuration tree can contain a PRIORITY property, which specifies the priorities of all flow charts in that branch of the tree. If a PRIORITY property is not explicitly set, the engine searches higher branches in the runtime configuration until it finds a PRIORITY property. If no explicit PRIORITY property is found, even at the root of the tree, then the flow chart defaults to a priority of 10.
Changes made to PRIORITY properties in the runtime configuration tree are propagated to running flow charts in the engine.
Flux allows you to perform operations on a flow chart (or group of flow charts in a namespace) to influence the flow chart's behavior at runtime. For example, you might recover a failed flow chart, expedite a flow chart that is waiting to run, or remove a flow chart from the engine entirely.
Operations are performed from the Operations Console. To operate on a flow chart, navigate to the Flow Charts page of the console, select the flow chart or namespace you want to act on, then select the appropriate operation from the bottom of the grid.
The operations listed below can be applied to a single flow chart (acting on all flow contexts in that flow chart), or across an entire namespace (acting on all flow contexts for every flow chart in the namespace).
Advanced Users (Java knowledge required)
You can also perform these operations using APIs available from the Flux engine. For more information on using these APIs, refer to the Javadoc documentation for the flux.Engine and flux.Transactable interfaces.
For any flow contexts in the flow chart that are in the ERROR or FAILED sub-state, the recover operation will roll back the flow context to the beginning of the transaction that caused the error condition.
This will cause the flow context to begin executing again from the last successful transaction break it reached. See Transactions for more information about transaction breaks.
The recover operation does not affect any flow contexts that are not in the ERROR or FAILED sub-state.
For any flow contexts in the flow chart that are waiting on a trigger that can be expedited, the expedite operation will cause those triggers to fire as soon as possible.
The following triggers can be expedited:
- Business process trigger
- Timer trigger
- Delay trigger
- Manual trigger
If a flow context is running an action when the expedite is performed, it will continue running as normal until it reaches a trigger that can be expedited. At that point, it will expedite the trigger and continue running.
If the flow context is waiting on a trigger that cannot be expedited, it will ignore the expedite operation.
Expediting a timer trigger or delay trigger will not decrease the count for that trigger, or cause it to be rescheduled. Expediting is a one-time operation that does not impact normally scheduled firings.
Rolls the flow chart back to its last transaction break as soon as possible. If the flow chart has already started running an action or a trigger is in the process of firing, the flow chart must continue executing normally until it reaches the next transaction break, then roll back to the previous transaction break from there.
After the flow chart rolls back, it will begin running again immediately from the point of the transaction break.
Interrupt is typically used with the pause and / or remove operations, to roll back the work done by a flow chart before pausing or removing it. Typically, users will first interrupt, then pause or remove the flow chart – once the flow chart is able to roll back, it will do so, then apply the pause or remove operation (as opposed to the normal case, where the operation is simply performed after Flux commits the transaction at the next transaction break).
NOTE: Rolling back the flow chart will undo any work that Flux has performed in the database (including flow chart state changes), but note that it cannot undo work that was performed on the system (like file transfers or native process invocations). Take note of this when interrupting a flow chart whose actions might impact the file system or other software systems.
Removes executing flow contexts from the engine. It is not possible to interrupt a running action once it has started, so if any actions have already begun running when the remove operation is performed, the Flux engine will wait until the flow reaches a transaction break, then remove the flow chart at that point.
If you want Flux to roll back its work in the database before the flow chart is removed, you can first interrupt the flow chart, then remove it. This will roll back the work in the database before removing the flow chart, rather than committing it as normal.
Advanced Users (Java knowledge required)
Calling remove() using the current job ID from within a running job will fail, because at the next transaction break, the engine will see that the job was deleted and rollback the transaction. Rolling back the transaction will undo the work of the remove() method. A simple workaround is to fork a new thread from within the running job, have that thread call Engine.remove(), and wait for the thread to finish. This solution works because the new thread uses a new database connection to perform its work.
Removing "Stuck" Flow Charts
Advanced Topic: Database Administration
The flow chart removal method below requires direct access to (and manipulation of) the data in the Flux database tables. Some database administration knowledge may be required to successfully complete this operation.
In some extreme cases, a flow chart can become "stuck" executing and unable to be removed from the Operations Console. Typically, this occurs if there is a corruption or invalid data in the database that prevents Flux from successfully removing the flow.
To remove such a flow chart, you can delete it directly from the database. Every flow chart has at least one entry in the FLUX_READY database table. You can identify which flow chart a column belongs to using the NAMESPACE column in that table.
To delete a flow chart, just remove any entries for the flow chart from the FLUX_READY table. The Flux engine will then automatically clean up any associated data for the flow chart in any other tables, so no further manual action is necessary to remove the flow chart.
Note that if the flow chart contains an action that is still executing (in-memory), the action will continue running until the flow chart reaches a transaction break. Because Java does not provide a clean way to destroy a running action once it has started, Flux must wait for the action to complete before it is fully removed from the engine.
Pauses all flow contexts in the selected flow chart to prevent them from executing further. The flow context will remain in the PAUSED state until it is resumed, either manually by a user or through an API call.
Due to the relationship between the Flux Engine and the database, it is not possible for the engine to put a flow context in the PAUSED state until it reaches the next transaction break. This means that if you pause a flow context while an action is executing, the engine must finish the currently executing action before putting the flow context into the PAUSED state. In addition, if you pause the flow context in the middle of a transaction (that is, the middle of a series of actions that are not marked as transaction break) the flow context will not enter the PAUSED state until it reaches the next transaction break.
Because of this, all of the actions that occur between the time you pause the flow context and the next transaction break (including any actions that are currently executing at the time of the pause) will be executed in their entirety before the flow context enters the PAUSED state.
If you want to roll back to the last transaction break (undoing any work Flux performs in the database) when the flow chart is paused, you can first interrupt, then pause the flow chart. If the flow chart is interrupted before pausing, Flux will roll back the transaction to the last transaction break, then pause the flow chart at that point and await a user's input before resuming.
Resumes any flow contexts in the PAUSED state for the selected flow chart. This will allow the flow contexts to resume normal execution.
If the flow chart selected does not have any flow contexts in the PAUSED state, the resume operation is ignored.