User:Lantiga

Ideas for Slicer3

vmtk

• figure out an official way to use vmtk within slicer and provide concrete examples and several modules:
  • vessel enhancement
  • vessel segmentation
  • geometric analysis
  • meshing for CFD
  • visualization of CFD data
• running modules in a localized space is vital: one solution has been implemented in Boston, but a more general approach would have a broader impact (see below)
• output fiducials would be useful (see below) but not vital for vmtk integration, can be circumvented using world points
• multistage modules would be useful (see below) but not essential
• packaging is tricky, but can be postponed for now
• future developments: vmtk through Tcl, or vmtk through Python through Tcl

Modules in localized space

• the rationale is to give command line modules a chance to run in a space where images are oriented as X Y Z (i.e. fully specified by a vtkImageData) but still have correct information available as to image origin, spacing and up/down, left/right, front/back consistency.
• the solution implemented in Boston consists in computing the transform from the localized space (the XYZ one) to Slicer's RAS for the first image in the CLI. All other CLI arguments (images, surfaces, fiducials and ROIs) are transformed using that transform before being written to command line. Upon completion, Slicer reads the results (which have been generated in localized space) and puts them in the scene as children of the RAStoLocal transform.
• nicer (more general) solution: there should be a localization mechanism by which Slicer3 creates a miniscene relative to a transformable node (image, model, transform, etc) and runs the module using that miniscene. This involves that all inputs, fiducials, ROIs, etc are first transformed to that miniscene root transform node. The outputs would also be reimported as children of the miniscene.
  • if the CLI module is MRML-aware, Slicer3 will let the module deal with the mini scene
  • if the CLI module is not MRML-aware, Slicer3 will take care of hardening the transforms up to the miniscene's root on the images, surfaces, fiducials, etc, exporting the hardened nodes (which are now direct descendants of the miniscene root), reimporting the results and transferring the mini scene contents to the main scene.

Hardening transforms

• a user should be able to harden transforms on nodes. In the MRML tree, if a user moves a node and changes its parent (say, from a parent trasform to root), this can have two different meanings:
  • the basis changes, the data doesn't (the way it works now): the effect is that the node changes its position and orientation in the 3D window.
  • the basis doesn't change, the data does: the effect is now that the position and orientation of the node are unchanged, but the node is now child of a different transform (this is what I'd call hardening). This is essential for non-MRML aware command line modules: e.g. if an input image is a child of some transform, it should be hardened (i.e. the IJKToRAS should be composed with the parent transform) before it's written to command line.

Output fiducials

• currently outputs can be data (images, models) and transforms. However, I also see a need for output fiducials (and ROIs). For example, a module could run in two stages: 1) identification of feature points; 2) operation performed on the feature points. Between 1) and 2) the user should be able to manipulate the fiducials (here I'm thinking at centerline computation: 1) could identify possible endpoints, 2) could trace centerlines between user-selected pairs of enpoints). Therefore, 1) should spit fiducials (with text) as output.
• output fiducials with text to be placed in scene: probably either through a vtkMRMLFiducialStorageNode that outputs a vtp file or through an XML data structure sent to output as a file or on stdout; the solution of XML description in a XML file is probably the best.

Python

• write some sample modules;
• extend scripted modules to Python?

DBPs

• • vascular (vmtk): segmentation, analysis, meshing, CFD for atherosclerosis, cerebral aneurysms, AAAs.
  • segmentation and registration in PKD (polycystic kidney disease);
  • functional renal imaging;
  • functional brain imaging;

Multistage modules

Below is a thread on the mailing list on multistage modules. The discussion took place in Boston.

It's sorted according to positive time.

Luca

Hi everyone,

 I've got a couple of thoughts regarding command line modules.

Currently Slicer processes command line modules by looking in the modules directory, choosing files that could either be executables or shared libraries, peeking into them to see if they are well behaved and later querying them for their XML description. Another possible design (somewhat closer to what Paraview uses) could have been to have executables (or shared libraries) accompanied by XML files containing the description of both the executable (or the shared library) and the GUI parameters. The disadvantage would have been to have two files per plugin instead of one. The advantages would be avoiding peeking into executables or shared libraries (the XML file could contain that information already), possibly also speeding up module discovery times. Another advantage would be making it easier to add other kinds of command line modules (I'm thinking about Python classes) without messing too much with the module discovery phase. This solution would be come with an additional advantage: multi- staged command line modules. Very often a module has the need of doing some processing, asking the user for info, then processing, then asking for info, and so on. This is now possible in Slicer by breaking up a program in several command line modules, but it would be neat to have a tabbed or wizard GUI that guides you through the process. What I'm thinking is that a single command line module plugin could be made up of several executables called at each step of a wizard built using the XML file. Separating XML descriptions and executables (or shared libraries) would make this easier, since one XML file could contain a reference to several executables, while the current design of asking the executable for its XML description would make this more complicated. Just to clarify: it's not my intention to spread confusion and mess with a solid and already well established design, but I figured I would share this thoughts. Thanks and see you soon Luca

Hi Luca -

Steve

That's a very interesting idea.... it seems to me that could coexist what the current paradigm nicely.

There's another way of creating modules currently supported in slicer3, which we haven't pushed much because it's somewhat experimental. I'm referring to the ScriptedModule approach used by the Editor module - it's analogous to the CommandLineModule in that modules are discovered at startup, but instead of defining the GUI indirectly with XML, the ScriptedModule can use the tcl API to create it's own GUI and interact with MRML directly. But I like what you are proposing too because it can retain more of the flexibility to run as either command line tools or as module.

It'll be fun to discuss these options when we get together. -Steve

Bill

Interesting ideas, Luca!

I have found myself wanting a bit more interactivity in the command-line modules, too. My ambitions are not as lofty as Luca's, however. It would be nice to have an input parameter be a 2-way parameter, and to have another special execution mode run from the command line so that the module could accept the input parameter values, compute some value that derived from these inputs, set the value of one or more of the 2-way parameters, and then return to Slicer and have Slicer display the computed values. This could be mediated by one or more new XML command-line options like <cmd>update-params</cmd> or some such.

Another thing that I have wanted is some more flexibility in how the GUI is built for command-line modules. I am working on an application that would really benefit from having sliders in the GUI, but if I read the Execution Model docs correctly, that only works for built-in modules, not for command line modules.

Finally, and this is a biggy -- I wonder if it would be possible to implement the interface between Slicer3 and the command line modules so that parameters and data could be passed in memory instead of in temporary files on disk? This might be possible if pipes are substituted for disk files; the API for reading and writing to pipes is pretty similar to file I/O, and most systems implement pipes as shared memory transfer. Just a thought.

-Bill Leue

Raul

Hi,

I want to join to the great ideas of Luca and Bill to extend the interactivity of the command-line-module. In the same direction, it would be nice to be able to have more control (for projects that required some advanced customization) before and after the module execution in the Slicer execution environment. For example, for tensors, it would be great to have the chance to set up MRML nodes according to the outputs to assigned a mask and a DWI to the tensor MRML node. This would equate to be able to prescribe two virtual methods, BeforeUpdate and AfterUpdate, from the XML description that would generate the appropriate code in the Command Line Interface part. I'm thinking something like: <code, method=AfterUpdate> <method> //output1Node is a vtkMRMLDiffusionTensorNode output1Node->SetAndObserveDiffusionWeightedVolume(inputNode->GetID()); </method>

/Raul

Specific themes

• Transforms and CLI:
  • the rationale is to give command line modules a chance not to worry about transforms but still have correct information available as to spacing and up/down, left/right, front/back consistency
  • the image passed to a command line module should be in normalized orientation space, i.e.: real origin, real spacing,
  • reference node specified in XML (either tag of the reference node or tag of child nodes to the reference - the latter allows many references in the same command line). Images in normalized orientation space.
  • the reference transform should be just a rigid transformation bringing the axes to XYZ (by looking at projections, but not necessarily). Therefore, from the node transform, a RASToLocalMatrix should be extracted, containing only the cosines that bring the axes to XYZ.
  • for each XML node, specify what's its reference node. The default is patient space (identity matrix), but in general it could be another node. So the current node matrix should be multiplied by the inverse of the reference node matrix before being written out. Correspondingly, when the command line module has executed, all results have to be multiplied by the corresponding reference node matrix. A reference node could be the node itself, meaning that the node transform wil be multiplied by the inverse of its reference transform.
  • find ways to harden transforms for specific MRML nodes
  • in the end: the writer of a command line module or Python scripted module etc. has to be put in the condition of not knowing anything about transforms, and still deal with legal vtkImageData objects (with their own spacing and right-handedness)

• Multi-stage CLI:
  • wizard or tabbed interface with content generated one page per command line module, and user control is returned from one page to the next.
  • one XML that puts command line modules together. This way command line modules can be either used alone or in a wizard.
  • change the way CLI are specified: XML files are discovered first, each XML describes how's the module is going to be (executable, shared library, Python class, etc). This saves time for module discovery (at least on OSX), and (as suggested by Raoul) it makes it easy for a developer to know how that CLI is going to behave and to develop against it.

• Fix vtkITK CMakeLists.txt (generate vtkITKTCL.so separately from vtkITK.so).

• XML module GUI:
  • give possibility to build a GUI from XML in ScriptedModules (either from Tcl or Python) or even everywhere in Slicer3, even from C++.

• Python:
  • try wrapping VTK. What happens when Python through Tcl is used and ShallowCopy to a true VTK module is called?

• Annotation widget
  • do it for command line modules (like: put this string at this point in the scene)