Difference between revisions of "AHM2010:Non-interactive-tools"
From NAMIC Wiki
| Line 14: | Line 14: | ||
== Use cases == | == Use cases == | ||
* 3DSlicer, Teem, CLI Modules, MRML, ... | * 3DSlicer, Teem, CLI Modules, MRML, ... | ||
| − | |||
| − | |||
| − | |||
= CDash / CTest = | = CDash / CTest = | ||
| Line 39: | Line 36: | ||
* Nightly testing on linux, Mac, windows | * Nightly testing on linux, Mac, windows | ||
| − | == | + | = XNAT = |
| − | |||
| − | = | + | == Summary == |
| + | == Features added == | ||
* XNAT 1.4 RC3 available | * XNAT 1.4 RC3 available | ||
* More robust QC modeling via xnat:qcManualAssessorData. | * More robust QC modeling via xnat:qcManualAssessorData. | ||
| Line 52: | Line 49: | ||
* Prebuilt XNAT images using VMWare and Virtualbox | * Prebuilt XNAT images using VMWare and Virtualbox | ||
* xnatfs provides WebDav access to XNAT instances | * xnatfs provides WebDav access to XNAT instances | ||
| + | |||
| + | == Use cases == | ||
= Grid Wizard Engine = | = Grid Wizard Engine = | ||
== Summary == | == Summary == | ||
| + | GWE is an open source application scheduler that can run multiple (thousands of) commands simultaneously, on multiple clusters of computers, by a single command. It works without requiring any code modification or scripting. It can be used by itself, or as part of a web-based portal environment. It is written in Java and works well with clusters based on Sun Grid Engine and Condor. | ||
== Features added == | == Features added == | ||
* Interactive experiments' results browser. | * Interactive experiments' results browser. | ||
| + | ** Next generation: Record set explorer | ||
* CSV and generated commands dumps and preview. | * CSV and generated commands dumps and preview. | ||
* Support for LSF and Lava cluster DRMs. | * Support for LSF and Lava cluster DRMs. | ||
| Line 65: | Line 66: | ||
* Function to extract experiments' parameters from text files. | * Function to extract experiments' parameters from text files. | ||
* Operations to delete, pause and resume experiments. | * Operations to delete, pause and resume experiments. | ||
| − | |||
| − | |||
== Use cases == | == Use cases == | ||
| + | [[Image:GWE-Slicer3 Parameter Explorer Prototype 2009-02-23.png|thumb|right|300px|Screenshot of the prototype implemented in slicer3. The prototype script reads the automatically generated GWE parameter file to determine each valid combination of variables. A preprocessing routine creates images of each volume dataset. Using the sliders, the user can scroll through the preview images in real time. To explore a particular volume in more detail, the user can click the load button to bring the entire volume in to slicer.]] | ||
== Related funding == | == Related funding == | ||
Revision as of 17:50, 5 January 2010
Home < AHM2010:Non-interactive-toolsBack to AHM 2010
CMake / CPack
Summary
- Release candidate 2.9
- 1200 downloads per day
- Users: KDE, Blizzard, NVidia, ...
Features added
- Library support: Qt, R, ...
- Superbuild: single run to download and install multiple libraries and executables
Use cases
- 3DSlicer, Teem, CLI Modules, MRML, ...
CDash / CTest
Summary
- CDash rewritten
- Php, Mysql
- 100 projects freely hosted on cdash.org
- Over 1,000 submissions to ITK's dashboard on ITK's 10th birthday
Features added
- CDash
- New statistics generated
- Compression
- CTest
- Parallel test (ctest -j)
- Dependencies
Use cases
- Over 100 tests added to Slicer
- Increasing code coverage
- Nightly testing on linux, Mac, windows
XNAT
Summary
Features added
- XNAT 1.4 RC3 available
- More robust QC modeling via xnat:qcManualAssessorData.
- New 'columns' variable for REST based listings.
- LDAP support.
- Improved memory footprint and efficiency.
- Refactored Search UI in the listing engine.
- Prebuilt XNAT images using VMWare and Virtualbox
- xnatfs provides WebDav access to XNAT instances
Use cases
Grid Wizard Engine
Summary
GWE is an open source application scheduler that can run multiple (thousands of) commands simultaneously, on multiple clusters of computers, by a single command. It works without requiring any code modification or scripting. It can be used by itself, or as part of a web-based portal environment. It is written in Java and works well with clusters based on Sun Grid Engine and Condor.
Features added
- Interactive experiments' results browser.
- Next generation: Record set explorer
- CSV and generated commands dumps and preview.
- Support for LSF and Lava cluster DRMs.
- Ability to override auto-discovered DRMs with configured ones.
- Ability to override default database installation location.
- Function to extract experiments' parameters from text files.
- Operations to delete, pause and resume experiments.
Use cases
Screenshot of the prototype implemented in slicer3. The prototype script reads the automatically generated GWE parameter file to determine each valid combination of variables. A preprocessing routine creates images of each volume dataset. Using the sliders, the user can scroll through the preview images in real time. To explore a particular volume in more detail, the user can click the load button to bring the entire volume in to slicer.
Related funding
BatchMake
Summary
- Scripted description of a processing pipeline that can be applied to a large collection of images or to a large collection of parameter values.
Features added
Use cases
- UNC SBIR, see below
- Registration module testing
Related funding
- UNC, Phase 2 SBIR with Kitware: High throughput murine image analysis
- Slicer modules executed on a grid when mouse MRI are uploaded to a MIDAS server
- DTI analysis, atlas-based segmentation, brain morphormetry
Renewal
Centralized Testing
- CDash operates as a master for CTest slaves
- slaves set their availability and resources
- CDash server allocates tests to the slaves
- Ensures testing of important configurations
- More efficient use of available resources
- Test combinations can change nightly to cover complex combinations
Integrated project tracking
- SVN, CMake, CPack, CTest, CDash, Wiki, and BugTracker will be extended to report to a central server
- Generates a comprehensive timeline of a project.
- Example use cases
- Commit a code change and a report is sent to the server indicating
- files changed, by whom, on what date,
- changes to the dashboard that resulted from the commit,
- wiki page changes made by that author around that date/time,
- bug associated with that commit
- Commit a code change and a report is sent to the server indicating
iPython for distributed computing
- Python chosen as a favored scripting language for Slicer
- VTK and Qt are well supported by python
- ITK v4 will have python as a target language
- Considered perl (biologists), tcl (comp sci), BatchMake, JavaScript, ...
- iPython contains extensions to python for threaded, grid, and distributed computing
- Reduced setup and learning curve
- Broad support
