CTSC Slicer XNE

UXP & functionality planning for Slicer's XNE client

Workflow & Implementation approaches

All Workflow Steps:

• Select XNAT Enterprise server
• Authentication dialog presented
• XNAT login performed and session opened;
• Response is checked for error codes
• XNAT queried for projects
• response is checked for http error codes
• xml parsed to get list of projects
• GUI populated
• One project selected
• XNAT queried for subjects and experiments in that project
• GUI populated
• User performs search for data
• User selects one scan and "download"
• Scan is loaded into Slicer

This workflow is detailed below. Note: Slicer uses the curl_easy library for client calls to RESTful web services. So the unix commandline curl calls below will need to be packaged up appropriately (including whatever basic authentication header information is required).

1. Select XNAT Enterprise Server

This sets the current URI handler to be the XNE_HttpHandler for subsequent REST-based client calls to XNE web services.

2. Authentication Dialog Presented

• Enter user name and password

3. Process Dialog Input

• Save the username (XNE_UserName) and password (XNE_Password)
• Authenticate with $XNE_Svr and create a session:

curl -d POST $XNE_Svr/REST/JSESSION -u $XNE_UserName:$XNE_Password

Comment (DM): Will the username/password be encrypted in this exchange?

The response from this request is a session ID that should be stored in $JSessionID; that ID can be used in all subsequent queries so authentication doesn't have to be performed at each transaction.

4. Query XNAT for Projects Available to User

• XNAT queried for appropriate projects

curl $XNE_Srv/REST/projects?format=xml --cookie JSESSIONID=$JSessionID

• response is checked for http error codes
  • if error is found, report to user
  • else xml is parsed to get list of scans subjects

Comment (DM): You'll need to continue drilling down to get to the 'scan' level. The typical hierarchy is project>subject>session>scan. I'm pretty sure you'll want to give the user the opportunity to drill down the hierarchy.

Question: Eventually, it would be nice to get the same info that the web GUI presents on its project view. Can we get a set of queries from Tim that give us the info there? including:

• Project Name
• Project ID
• PI
• Description
• Permissions

Comment (DM): Most of this information is retrieved using this call: http://host/REST/projects/PROJECT_ID?format=xml. Permissions info can be retrieved like this: http://host/REST/projects/PROJECT_ID/users?format=xml

5. User Selects One Project from GUI and clicks "Search Button"

• XNAT-E notebook panel is displayed with the "Choose Project" panel raised.
• GUI is populated with list of projects the user is permitted to see.
• User selects a project (only one for now)

Comment (DM): The checkbox interfaces implies that more than one project can be selected. Perhaps use a button instead or just click on the project title

Mockup:

Question: Would be nice to search across multiple projects at once. Ask Tim for help on formulating a single query to search for subjects, experiments, etc., for projects {P1, P2,...,PN}.

Comment (DM): Right now, there's a way to do this for experiments: http://HOST/experiments?format=xml&project=PROJECT1,PROJECT2. At this point, that only works for experiments. In the future, we'll add subjects as well. Let us know if that's high priority. Right now, you'd have to iterate through each selected project. You could also do more complicated searches like this using the search API.

6. Query XNAT for Subjects in Selected Project

• XNAT queried for subjects in the selected project

curl $XNE_Svr/REST/projects/$projectID/subjects?format=xml --cookie JSESSIONID=$JSessionID

• response is checked for http error codes
  • if error is found, report to user
  • else xml is parsed to get list of scans
• "Browse & Download" panel is automatically raised
• Selected Project Name and Project ID are shown
• subject menu in "Browse & Download" panel is populated with list of subjects.

Comment (DM): There may be hundreds of subjects in a project, so the dropdown may be overwhelming. Is there a better widget to use when there are many entries? Would you ever want the user to select more than one subject?

• "none" is selected in subject menu by default.
• experiment menu in "Browse & Download" panel is disabled until a subject is selected.
• Resource Filters panel is closed by default.

Mockup:

7. User Selects One Subject, Configures Filters to Narrow Search & Selects Search

Initial view presents a search that only requires SubjectID to be selected in order to populate the list of resources, but each additional selection (corresponding to the basic XNE data model) can be used to focus the search. Eventually add funcationality that allows the inclusion/exclusion of "shared subjects".

7a. Initial Presentation

• Subject: { none } default selection
• Experiments: { none } (disabled)
  • Scans: on (disabled)
  • Reconstructions: on (disabled)
  • Imaging Assessors: on (disabled)

query: curl $XNE_Svr/REST/projects/$ProjectID/subjects?format=xml --cookie JSESSIONID=$JSessionID
parse response into list of SubjectIDs
update GUI

Mockup (user has opened the Resource Filters panel which is collapsed by default):

7b. Just a Subject ID is selected; Filter state & GUI are updated

User Selects a subject, experiments menu is populated with all experiments for that subject and "all" is selected by default. All Resource Filters default to "all". This view provides a list of all resources for the selected SubjectID across all experiments.

• Subject: $SubjectID
• Experiments: { all } by default
  • Scans: on by default
  • Reconstructions: on by default
  • Assessors: on by default

query: curl $XNE_Svr/REST/projects/$ProjectID/subjects/$SubjectID/experiments?format=xml --cookie JSESSIONID=$JSessionID
parse response into list of ExperimentIDs
for each id in ExperimentIDs
{
 query: curl $XNE_Svr/REST/projects/$ProjectID/subjects/subjects/$SubjectID/experiments/$id/scans?format=xml --cookie JSESSIONID=$JSessionID
 parse response and add to list of ScanIDs
 query: curl $XNE_Svr/REST/projects/$ProjectID/subjects/subjects/$SubjectID/experiments/$id/reconstructions?format=xml --cookie JSESSIONID=$JSessionID
 parse response and add to list of ReconstructionIDs
 query: curl $XNE_Svr/REST/projects/$ProjectID/subjects/subjects/$SubjectID/experiments/$id/assessors?format=xml --cookie JSESSIONID=$JSessionID
 parse response and add to list of AssessorIDs
}
combine all IDs into resource list
update GUI

Resource list presented in GUI might need to be constructed like one of these to be unique:

• ExperimentID:Scan:ID or
• ExperiementID:Recon:ID or
• ExperimentID:Assr:ID

Then the resource list could be a std::map<std::string, std::vector<std::string> > that maps a uniquely named resource to a vector of strings that contains the following info, to help query building:

• projectID
• subjectID
• experimentID
• scanID (or NULL)
• reconstructionID (or NULL)
• assessorID (or NULL)

Question: Do we need anything else here?

Comment (DM): There are often multiple catalogs at the scan level, including DICOM, SNAPSHOTS, NIFTI, etc. You'll want to handle that.

Mockup:

7c. Both SubjectID && Experiment ID have been selected; Filter state & GUI are updated

User selects an experiment and Scan, Reconstruction, and Imaging Assessor menus are populated.

• Subject: $SubjectIDs
• Experiments: $ExperimentID(s)
  • Scans: on (focused for $SubjectID && $ExperimentID(s) )
  • Reconstructions: on (focused for $SubjectID && $ExperimentID(s) )
  • Assessors: on (focused for $SubjectID && $ExperimentID(s) )

for each $id in $ExperimentID(s)
{
 query: curl $XNE_Svr/REST/projects/$ProjectID/subjects/subjects/$SubjectID/experiments/$id/scans?format=xml --cookie JSESSIONID=$JSessionID
 parse response and add to list of ScanIDs
 query: curl $XNE_Svr/REST/projects/$ProjectID/subjects/subjects/$SubjectID/experiments/$id/reconstructions?format=xml --cookie JSESSIONID=$JSessionID
 parse response and add to list of ReconstructionIDs
 query: curl $XNE_Svr/REST/projects/$ProjectID/subjects/subjects/$SubjectID/experiments/$id/assessors?format=xml --cookie JSESSIONID=$JSessionID
 parse response and add to list of AssessorIDs
}
combine all IDs into resource list
update GUI

Mockup:

7d. Filter is adjusted to refine search; Resource is selected;

• Subject: $SubjectIDs
• Experiments: $ExperimentID
  • Scans: on
  • Reconstructions: off
  • Assessors: off

 query: curl $XNE_Svr/REST/projects/$ProjectID/subjects/subjects/$SubjectID/experiments/$id/scans?format=xml --cookie JSESSIONID=$JSessionID
 parse response and add to list of ScanIDs
combine all ScanIDs into resource list
update GUI

Mockup:

8. User Clicks Download

Questions:

• Once we get this far, presume we don't know the format of the resource (DICOM, Analyze, NRRD, etc.) so we can't just use the following to retrieve a list of files:

curl $XNE_Svr/projects/$ProjectID/subjects/$SubjectID/experiments/$ExpID/scans/$ScanID/resources/DICOM/files/format=xml --cookie JSESSIONID=$JSessionID

• What is the recommended way to download a resource? In response, we want query to return a list of URIs which Slicer uses to populate a vtkMRMLVolumeArchetypeStorageNode:URI and vtkMRMLVolumeArchetypeStorageNode::URIListMembers

9. Scan is Loaded into Slicer

vtkMRMLVolumeArchetypeStorageNode::ReadData( vtkMRMLVolumeArchetypeStorage *node) is called, which calls vtkMRMLStorageNode::StageReadData (*node) to download data using the XNE_HttpHandler.

Question for Nicole: Does the approach here seem to fit with the multi-file volume code you've written?

We can also ask for a .zip (or .xar?) file and use something like the code in Base/GUI/Tcl/Loader.tcl to load the archive from cache.

Questions

• We'll need an XNETagTable to hold metadata for all resources coming from and going up to $XNE_Svr:
  • We'll need to decide what metadata to put on resources downloaded from $XNE_Srv (stored in each vtkMRMLStorableNode's UserTagTable)
  • need to know what metadata is put to stuff going back up to XNE
  • need to translate this metadata into appropriate URIs to PUT or POST back to $XNE_Svr.
• XNE web services allows us to DL multiple resources using one URI (see here); should we implement this in a first pass, or later?
• Custom tags are present in XNE -- how can we query for all custom tags and their values for a selected $XNE_Svr? This would allow us to populate an "advanced" search widget.

Things to Build

• Authentication dialog (try using existing Base/GUI/vtkSlicerXNATPermissionPrompterWidget code) (wjp)
• extend FetchMI infrastructure to admit XNE web services (wjp)
• Improve/generalize http error checking
  • bring to superclass vtkFetchMIParser (right now, just looking for error string in html title in vtkFetchMIParserXND)
  • recognize and report specific http error codes via vtkFetchMINode::ErrorMessage and vtkFetchMINode::RaiseErrorEvent
• Extend FetchMI's GUI to include an XNE notebook, switch notebooks when serverType changes. (wjp)
• Build vtkFetchMIQueryBuilder superclass (wjp)
• Derive vtkFetchMIQueryBuilderXND class and move XND query formulation into here. (wjp)
• Add vtkFetchMIQueryBuilderXNE class
• Add vtkFetchMIWriterXNE class
• Add vtkFetchMIParserXNE class
• Add vtkFetchMIWebServicesClientXNE class to encapsulate
• Add vtkXNETagTable in Libs/RemoteIO
• Add vtkXNEHandler in Libs/RemoteIO