Projects:ARRA:SlicerEM:Developer:WorkflowManager

Back to SlicerEM:Developer page

Summary

• EMSegmenter has a fairly complicated workflow
• Need a mechanism to validate user input and transition appropriately between steps of the workflow
• Should be dependent on Qt only (and not VTK)
• Current plan (subject to change) = use Qt's state machine implementation, with our own workflow manager on top (Qt Wizard doesn't quite work in harmony with the Qt state machine framework, and the UI itself can't be embedded in a pre-existing top-level)

Examples

• current software is publicly available on github
• examples of how to create a custom workflow step:
  • ctkExampleDerivedWorkflowWidgetStep.h and ctkExampleDerivedWorkflowWidgetStep.cpp: custom step created by deriving ctkWorkflowWidgetStep
  • ctkExampleWorkflowWidgetStepUsingSignalsAndSlots.h and ctkExampleWorkflowWidgetStepUsingSignalsAndSlots.cpp: custom step created by implementing the step's functionality in a class derived from QObject, and communicating with the workflow using its signal-slot mechanism
• examples of how to create a workflow:
  • ctkExampleUseOfWorkflowWidgetUsingDerivedSteps: builds a simple workflow using custom steps that derive ctkWorkflowWidgetStep
  • ctkExampleUseOfWorkflowWidgetUsingSignalsAndSlots: builds a simple workflow using custom steps that communicate with the workflow using its signal-slot mechanism

Workflow classes

• Core workflow functionality (no widgets):
  • ctkWorkflowStep: a step in the workflow
  • ctkWorkflow: the workflow
  • ctkWorkflowTransition: a transition in the workflow
• Workflows associated with widgets:
  • ctkWorkflowWidgetStep: a step in the workflow with a user interface
  • ctkWorkflowWidget: a workflow with a user interface
  • ctkWorkflowAbstractPagedWidget: a workflow with a user interface with many pages (abstract class)
  • ctkWorkflowStackedWidget: a workflow with a user interface with many pages, based on a QStackedWidget
  • ctkWorkflowTabWidget: a workflow with a user interface with many pages, based on a QTabWidget

Previous workflow manager implementation in KWWidgets

• Please see the documentation here, as it is already well described

Uses:

• KWWidgets state machines - incorporates states (ex. user interaction within a workflow step, validation of the user input within a workflow step), transitions (between states), and inputs (pushed onto a queue to trigger transitions)
• KWWidgets wizard workflow - provides additional functionality to manage workflow using a state machine (ex. bundles pairs of user interaction and validation states into a workflow "step", handles a navigation stack of steps encountered along the way that triggers updates of widgets and/or dialogs)

Transition from KWWidgets state machines to Qt state machines

• Core functionality in KWWidgets state machines and equivalent (if available) in Qt (In Progress):

KWWidgets functionality	Present in Qt?
State machine involve states, inputs and transitions	State machine involve states, events/signals and transitions
States, inputs and transitions are instantiated separately and added to the state machine	Only states are added to state machine, transitions are added to state, events/signals are implied by transition
Transitions require an input, origin and destination	Transitions can be targetless (stay in place but execute something / emit signal),
State machine runs until there are no more inputs in its queue	State machine runs its own even loop automatically, can be started, stopped/resumed and will run until FinalState is reached or app is quit.
Can define custom inputs	Can define custom events and transitions (by deriving QEvent and QAbstractTransition)
States have enter/leave events & callbacks, reimplementable Enter()/Leave()	States have enter/exit signals, reimplementable onEnter()/on Exit()
Transitions have start/end events & callbacks, reimplementable Start()/End()	Transitions have triggered signal, reimplementable onTransition()
Ordering of state and transition events: transition - start origin state - leave destination state - enter transition - end	... different: origin state - exit transition - triggered destination state - enter
Clusters of states (one level)	A state can have a parent state allowing for arbitrary level grouping; a transition can be shared by many states by assigning it to the parent, but can be redefined by a child (Limitation: entry into the parent's initial state trigger's the initial state's onEntry() function, but not the parent state's onEntry() function)
Can save history of encountered transitions	No but a HistoryState can be automatically maintained to keep track of the last known child state of a grouping state when it is exited. It can be referred to directly to get back in the group and resume.
No	Object properties can be set/restored automatically when entering/leaving a state and animated during transition
No	FinalState class exists, state emits finished signal (parent emits, if final state is in group, allowing for blackblox behavior)
No	Transition can be guarded, i.e. the transition's input (a QEvent) can have a payload that can be tested by a custom transition (say a StringEvent that only triggers when the payload is "Hello")
No	Parallel states can be used to avoid combinatorial explosion. When a parallel state group is entered, all its child states will be simultaneously entered.

• If some of these components are not available in Qt's state machine framework, it means some more refactoring/restructuring work

Transition from KWWidgets workflow manager to a workflow manager using Qt's state machines

• The four KWWidget workflow wizard classes:
  • vtkKWWizardStep: a wizard step
  • vtkKWWizardWorkflow : a wizard workflow
  • vtkKWWizardWidget : a wizard widget, embedding UI (buttons) and a wizard workflow engine
  • vtkKWWizardDialog : a wizard dialog, embedding a wizard widget in a toplevel/dialog window.

• The ctkWorkflowManager workflow classes (in progress):
• ctkWorkflowStep - defines a step, in terms of both the widgets shown and the criteria for successful transition to the next step
• ctkWorkflow - manages transitions between steps
• ctkWorkflowWidget - provides a clientArea onto which a step's widgets should be placed
  • ctkWorkflowWidget - clientArea is a QWidget
  • ctkWorkflowStackedWidget - client area is a QStackedWidget
  • ctkWorkflowTabWidget - client area is a QTabWidget

To define a new workflow step

• very similar to KWWidgets:
  • derive ctkWorkflowStep
  • implement validate(): evaluates user input and returns true if valid, false if not
  • implement showUserInterface(): shows or enables the widgets associated with each step
  • implement hideUserInterface(): hides or disables the widgets associated with each step

Overview of state diagram underlying ctkWorkflow

• processing step = handles user interaction, and should support additional processing (ex. image processing) as well
• validation step = evaluates the success of the processing step
• currently supported transitions: transition to the next step, transition to the previous step
• TODO:
  • automatic transition to a "finished" step

<graphviz>

digraph workflow { subgraph cluster_0 { style=filled; color=lightgrey; node [style=filled,color=white]; "Processing 1" -> "Validation 1" [label="validationTransition"];

               "Validation 1" -> "Processing 1" [label="validationFailedTransition"];

label = "ctkWorkflowStep* step1"; }

subgraph cluster_1 {

               style=filled;

color=lightgrey; node [style=filled,color=white]; "Processing 2" -> "Validation 2" [label="validationTransition"];

               "Validation 2" -> "Processing 2" [label="validationFailedTransition"];

label = "ctkWorkflowStep* step2"; }

       "Validation 1" -> "Processing 2" [label="validationSucceededTransition"];
       "Processing 2" -> "Processing 1" [label="goBackToSelfTransition"];

} </graphviz>

Signal/slot mechanism to transition to the next step

• ex. user enters parameter values, and then clicks "Next" to go to the next step

<graphviz> digraph signalsAndSlotsNext {

       "SIGNAL: pushButton->clicked()" -> "SLOT: workflow->triggerValidationTransition()"
       "SLOT: workflow->triggerValidationTransition()" -> "posts ValidationTransition event" -> "SIGNAL: validationState->onEntry()" -> "SLOT: workflow->attemptToGoToNextStep()" -> "step1->validate()"

"step1->validate()" -> "posts ValidationSucceededTransition event" [label="true"]

       "posts ValidationSucceededTransition event" -> "SIGNAL: validationSucceededTransition->triggered()" -> "SLOT: workflow->showAndHideUserInterface()"
       "step1->validate()" -> "posts ValidationFailedTransition event" [label="false"]

       "SLOT: workflow->showAndHideUserInterface()" -> "step1->hideUserInterface()"
       "SLOT: workflow->showAndHideUserInterface()" -> "step2->showUserInterface()"

} </graphviz>

Signal/slot mechanism to trigger transition to the previous step

• ex. user clicks "Back" to go to the previous step

<graphviz> digraph signalsAndSlotsBack {

       "SIGNAL: pushButton->clicked()" -> "SLOT: workflow->triggerGoBackToSelfTransition()"
       "SLOT: workflow->triggerGoBackToSelfTransition()" -> "posts GoBackToSelfTransition event" -> "SIGNAL: goBackToSelfTransition->triggered()" -> "SLOT: workflow->showAndHideUserInterface()"
       "SLOT: workflow->showAndHideUserInterface()" -> "step2->hideUserInterface()"
       "SLOT: workflow->showAndHideUserInterface()" -> "step1->showUserInterface()

} </graphviz>

GUI implementation in ctkWorkflowWidget

• workflowWidget->addWidget(QWidget* widget): adds a widget to the clientArea
• workflowWidget->showWidget(QWidget* widget): shows the clientArea (i.e. for ctkWorkflowStackedWidget and ctkWorkflowTabWidget, ensures that the correct "page" is shown
• TODO:
  • plus qDialog as well
  • in Qt Designer - could have one .ui file for the entire stacked widget, or, for widgets representing complicated steps, could have a separate .ui

Concepts to keep in mind

• Undo / redo and forward/back transitions
• Automatic transition to the last state ("GoToSelf" in KWWidgets)

Additional ideas and questions

• Image processing throughout - need to deal with failures in image processing that are unrelated to the GUI
• Would be a good idea to make the workflow manager as general as possible for CTK - ex. use for management of IGT workflows in Slicer, where you may be coordinating several modules (ex. calibration module -> registration module -> tracking module), and may need to notify other components of current state (ex. over OpenIGTLink) (ex see this IJ paper)
  • would need to make show/hide user interface functions more generic
  • Ex. make a workflow step more generic: instead of an "interaction" state and a "validation" state, could think about a "do stuff" state and a "validation" state, where "do stuff" can be user interaction, image processing, etc.
  • In validation states, could also have "borderline" success, where you may want to warn the user and allow them to choose whether to go ahead to the next step or to redo the current step with different inputs
• May even like to provide the option to save the MRML tree at the end of each step (to restore state if there is a crash, for example
• Automatic validation at each user input, to enable the "next" button once the user's input is valid
• Logging state transitions, user actions, inputs and results thoughout (logging states?)
• Design / output of state machines in a graphical format
• Replaying workflows
• Branching workflows