Difference between revisions of "Slicer3:Pipeline Integration"

From NAMIC Wiki
Jump to: navigation, search
Line 1: Line 1:
−
# The main features of the project
+
LONI Pipeline website: http://pipeline.loni.ucla.edu
−
## http://pipeline.loni.ucla.edu
 
−
# The main outstanding questions that must be answered before you can complete the project ::
 
−
## Pipeline -> Slicer3
 
−
### Once the API for submission to Slicer3 is released, we can integrate it into the Pipeline.
 
−
## Slicer3 -> Pipeline
 
−
### We have made considerable progress in making Pipeline more stable, robust and scalable. Detailed progress is listed below ::
 
−
#### GUI : A new and improved graphical interface is now available. The gui is now much simpler in design and easier to use. The list of improvements/ new features is as follows :
 
  
−
- Completely new data model/xml
+
With the release of Pipeline v4.0 in December '07, we've developed a stable and robust framework for continuing development into the future. With regards to the Slicer 3 (currently in Beta) integration, the LONI Pipeline has a complete [http://cms.loni.ucla.edu/pipeline/default.aspx?id=2134#Running%20from%20the%20command%20line command line interface] that can be used to run workflows with dynamically bound values in addition to it's GUI interface.
−
- Significantly easier parameter editing
+
<code>
−
- Cleaner drag and drop of modules
+
  Usage: pipeline [OPTIONS] PIPEFILE [paramID1=value ... paramIDN=valueN]
−
- Easier list processing
+
    -execute            Validates then executes the specified workflow
−
- Stronger type checking (with feedback to help the user)
+
    -validate          Just validates the specified workflow then exits
−
- Variable can be used on parameter bindings
+
    -display            Displays all the required/enabled parameters in the
−
- Personal Library of workflows and moduels
+
                          workflow
−
- Copy/Paste of single modules and groups of modules (and still retain connections)
+
    -sync              Program will not exit until workflow finishes executing
−
- Undo/Redo on most operations
+
    -credentials=FILE  Specify an XML file containing credentials to a Pipeline
−
- Workflows can be layed out horizontally now
+
                          server that will be needed for execution or validation.
−
- Faster and easier zooming
+
                          This is not necessary if you're username and password
−
- Scalable GUI functionality - is able to handle hundreds of modules in a single workflow
+
                          are saved in your list of connections in the graphical
 +
                          client. Format is the following:
 +
                          <?xml version="1.0" encoding="UTF-8"?>
 +
                          <connections>
 +
                            <conn address="cranium.loni.ucla.edu"
 +
                                  user="jane"
 +
                                  password="secret" />
 +
                          </connections>
 +
    -help              Prints out LONI Pipeline help. (this text)
 +
    -version            Prints out the LONI Pipeline version
 +
</code>
  
−
# ## ### #### DataModel : The data model, associated XML descriptions have been simplified greatly. The data model has been decoupled from the execution and the GUI, this component has been completed
+
Using information from the [http://wiki.na-mic.org/Wiki/index.php/Slicer3:Execution_Model_Documentation#Executable_plugins_with_command_line_options Slicer3 execution model] we will add support for the --xml flag to the Pipeline command line so Slicer can use the Pipeline from it's GUI. This will occur after the release of the final version of Slicer3.
−
#### Translation : The translation portion of the software is responsible for accomodating logic that separates GUI/Execution vs Execution. This component has been completed, it takes the workflow and converts it into a flattened graph (it loses the property of module groups at this stage) and constructs the commands using the data model
 
−
#### Validation : The validation portion conducts sanity checks on the workflow. It allows the user to fix errors prior to "actual" execution and informs the user of all steps needed to make the workflow run
 
−
#### Execution : The execution portion now is an independent component that is agnostic of the data model and can be invoked on the command line directly. Self-sufficient workflows (with previously bound parameters), can be now invoked and executed from the commandline, bypassing the GUI entirely. Workflows or pipes can now be executed both locally and on the LONI grid. The scalability has dramatically improved from 30 inputs to thousands of inputs per workflow. The changes in the execution model are ::
 
  
−
- Agnostic of the data model
+
'''Developers'''<nowiki>: </nowiki>
−
- Minimal synchronization to express sequentiality and/or parallelism
 
−
- Local execution and LONI grid execution supported
 
−
- Detailed logs to keep track of progress
 
−
- Periodic status messages passed to GUI for immediate user feedback
 
−
- Scalability of execution allows running > 1000 inputs per workflow
 
  
−
# To start a pipeline on the commandline with parameters:
+
* Celia Cheung
−
 
+
* Arash Payan
−
java -classpath pipeline.client.jar ui.cli.Main -execute prebound_workflow
+
* Jia Wei Tam
−
 
 
−
'''Team Members'''<nowiki>: </nowiki>
 
−
 
 
−
* Jagadeeswaran Rajendiran - UCLA
 

Revision as of 20:42, 4 February 2008

Home < Slicer3:Pipeline Integration

LONI Pipeline website: http://pipeline.loni.ucla.edu

With the release of Pipeline v4.0 in December '07, we've developed a stable and robust framework for continuing development into the future. With regards to the Slicer 3 (currently in Beta) integration, the LONI Pipeline has a complete command line interface that can be used to run workflows with dynamically bound values in addition to it's GUI interface. 

 Usage: pipeline [OPTIONS] PIPEFILE [paramID1=value ... paramIDN=valueN]
   -execute            Validates then executes the specified workflow
   -validate           Just validates the specified workflow then exits
   -display            Displays all the required/enabled parameters in the
                          workflow
   -sync               Program will not exit until workflow finishes executing
   -credentials=FILE   Specify an XML file containing credentials to a Pipeline
                          server that will be needed for execution or validation.
                          This is not necessary if you're username and password
                          are saved in your list of connections in the graphical
                          client. Format is the following:
                          <?xml version="1.0" encoding="UTF-8"?>
                          <connections>
                            <conn address="cranium.loni.ucla.edu"
                                  user="jane"
                                  password="secret" />
                          </connections>
   -help               Prints out LONI Pipeline help. (this text)
   -version            Prints out the LONI Pipeline version

Using information from the Slicer3 execution model we will add support for the --xml flag to the Pipeline command line so Slicer can use the Pipeline from it's GUI. This will occur after the release of the final version of Slicer3.

Developers:

  • Celia Cheung
  • Arash Payan
  • Jia Wei Tam
Retrieved from "https://www.na-mic.org/w/index.php?title=Slicer3:Pipeline_Integration&oldid=21800"