Difference between revisions of "CTSC IGT, BWH"

From NAMIC Wiki
Jump to: navigation, search
Line 72: Line 72:
 
==Target Data Management Process (Step 1.) Option B. (web services API) '''CURRENTLY BEING TESTED!'''==
 
==Target Data Management Process (Step 1.) Option B. (web services API) '''CURRENTLY BEING TESTED!'''==
  
−
* Create new project on XNAT instance using web GUI
+
'''1. Create new project on XNAT instance using web GUI'''
 +
 
 
* Manage project using web GUI: Configure settings to automatically place data into the archive (no pre-archive)
 
* Manage project using web GUI: Configure settings to automatically place data into the archive (no pre-archive)
−
* Batch Anonymize ('''Download and install the command-line tools''': http://nrg.wustl.edu/projects/DICOM/DicomBrowser-cli.html)
 
−
* Batch Create Metadata
 
−
* Batch Upload using curl or XNATRestClient (See here to '''download XNATRestClient''' in xnat_tools.zip from here: http://nrg.wikispaces.com/XNAT+REST+API+Usage)
 
  
 
<br>
 
<br>
 
<br>
 
<br>
−
'''1. Authenticate''' with server and create new session; use the response as a sessionID ($JSessionID) to use in subsequent queries
+
'''2. Batch Anonymize
 +
*  Run CLI Tool for batch anonymization (See here for HowTo: http://nrg.wustl.edu/projects/DICOM/DicomBrowser/batch-anon.html)
 +
* Also See here for new (possibly buggy) command line options: http://nrg.wustl.edu/projects/DICOM/DicomBrowser-cli.html
 +
 
 +
For next steps, use curl or XNATRestClient (See here to '''download XNATRestClient''' in xnat_tools.zip from here: http://nrg.wikispaces.com/XNAT+REST+API+Usage)
 +
 
 +
<br>
 +
<br>
 +
'''3. Authenticate''' with server and create new session; use the response as a sessionID ($JSessionID) to use in subsequent queries
 
  curl -d POST $XNE_Svr/REST/JSESSION -u $XNE_UserName:$XNE_Password  
 
  curl -d POST $XNE_Svr/REST/JSESSION -u $XNE_UserName:$XNE_Password  
 
  or, use the XNATRestClient
 
  or, use the XNATRestClient
Line 87: Line 93:
 
<br>
 
<br>
 
<br>
 
<br>
−
'''2. Create subject'''
+
'''4. Create subject'''
 
   XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT /REST/projects/$ProjectID/subjects/s0001  
 
   XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT /REST/projects/$ProjectID/subjects/s0001  
 
  (This will create a subject called 'S0001' within the project $ProjectID)
 
  (This will create a subject called 'S0001' within the project $ProjectID)
Line 93: Line 99:
 
A script can be written to automatically create all subjects for the project.  
 
A script can be written to automatically create all subjects for the project.  
  
−
'''2a. specify the demographics of a subject already created, or create with demographic specification'''
+
'''4a. specify the demographics of a subject already created, or create with demographic specification'''
  
−
'''2.a.1''' No demographics are applied to each subject by default. To edit the demographics (like gender or handedness) of a subject '''already created''' using XML Path shortcuts.
+
'''4.a.1''' No demographics are applied to each subject by default. To edit the demographics (like gender or handedness) of a subject '''already created''' using XML Path shortcuts.
  
 
  xnat:subjectData/demographics[@xsi:type=xnat:demographicData]/gender = male
 
  xnat:subjectData/demographics[@xsi:type=xnat:demographicData]/gender = male
Line 106: Line 112:
 
All XML Path shortcuts that can be specified on commandline for projects, subject, experiments are listed here: http://nrg.wikispaces.com/XNAT+REST+XML+Path+Shortcuts
 
All XML Path shortcuts that can be specified on commandline for projects, subject, experiments are listed here: http://nrg.wikispaces.com/XNAT+REST+XML+Path+Shortcuts
  
−
'''2.a.2''' Alternatively, specify the demographics '''during subject creation''' by generating and uploading an xml file with the subject:
+
'''4.a.2''' Alternatively, specify the demographics '''during subject creation''' by generating and uploading an xml file with the subject:
 
   
 
   
 
  XNATRestClient -host $XNE_srv -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/s0002" - local ./$ProjectID_s0002.xml
 
  XNATRestClient -host $XNE_srv -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/s0002" - local ./$ProjectID_s0002.xml
Line 125: Line 131:
 
  </xnat:Subject>
 
  </xnat:Subject>
  
−
'''2.b (optional check) Query the server to see what subjects have been created:'''
+
'''4.b (optional check) Query the server to see what subjects have been created:'''
 
  XNATRestClient -host $XNE_Svr -user_session $JSessionID -m GET -remote /REST/projects/$ProjectID/subjects
 
  XNATRestClient -host $XNE_Svr -user_session $JSessionID -m GET -remote /REST/projects/$ProjectID/subjects
  
−
'''2.c Create experiments (collections of image data) you'd like to have for each subject'''
+
'''4.c Create experiments (collections of image data) you'd like to have for each subject'''
  
 
  XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/MRExperiment?xnat:mrSessionData/date=01/02/09"
 
  XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/MRExperiment?xnat:mrSessionData/date=01/02/09"
Line 134: Line 140:
 
  XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/PETExperiment1?xnat:petSessionData/date=01/02/09"
 
  XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/PETExperiment1?xnat:petSessionData/date=01/02/09"
  
−
'''2.d (optional check) Query the server to see what experiments have been created:'''
+
'''4.d (optional check) Query the server to see what experiments have been created:'''
 
  XNATRestClient -host $XNE_Svr -user_session $JSessionID -m GET -remote /REST/projects/$ProjectID/subjects/s0001/experiments?format=xml
 
  XNATRestClient -host $XNE_Svr -user_session $JSessionID -m GET -remote /REST/projects/$ProjectID/subjects/s0001/experiments?format=xml
  
Line 140: Line 146:
 
<br>
 
<br>
 
<br>
 
<br>
−
'''3. Batch Anonymize ''' scan data
+
'''5. Batch Anonymize ''' scan data
  
 
* Run CLI Tool for batch anonymization (See here for HowTo:  http://nrg.wustl.edu/projects/DICOM/DicomBrowser/batch-anon.html)
 
* Run CLI Tool for batch anonymization (See here for HowTo:  http://nrg.wustl.edu/projects/DICOM/DicomBrowser/batch-anon.html)
Line 146: Line 152:
 
* Also See here for new (possibly buggy) command line options: http://nrg.wustl.edu/projects/DICOM/DicomBrowser-cli.html
 
* Also See here for new (possibly buggy) command line options: http://nrg.wustl.edu/projects/DICOM/DicomBrowser-cli.html
  
 +
<br>
 +
<br>
 +
'''6. Create metadata from DICOM headers'''
  
 
<br>
 
<br>
 
<br>
 
<br>
−
'''4. Create uris for scans, reconstructions and upload them.'''
+
'''7. Create uris for scans, reconstructions and upload them.'''
  
 
Note: when uploading images, it is good form to define the format of the images (DICOM, ANALYZE, etc) and the content type of the
 
Note: when uploading images, it is good form to define the format of the images (DICOM, ANALYZE, etc) and the content type of the
Line 184: Line 193:
 
<br>
 
<br>
 
<br>
 
<br>
−
'''5. Confirm''' data is uploaded & represented properly with web GUI
+
'''8. Confirm''' data is uploaded & represented properly with web GUI
  
 
==Target Query Formulation (Step 2.)==
 
==Target Query Formulation (Step 2.)==

Revision as of 19:42, 17 August 2009

Home < CTSC IGT, BWH

Back to CTSC Imaging Informatics Initiative


Mission

Mark Anderson at Surgical Planning and Channing labs currently manages data for many investigators, pulling data from PACS into the research environment. There is interest in setting up a parallel channel by which the data are also enrolled into an XNAT database and accessed from client, and comparing its ease of use with the existing infrastucture. To explore XNAT as a possible long-term informatics solution for the NCIGT project, Mark will be uploading retrospective data for a number of NCIGT efforts (and PIs):

  • NCIGT_Brain_Function (SS/AG)
    • Key Investigators:
    • Brief Description:
  • NCIGT_Tumor_Resection (HK/AG)
    • Key Investigators:
    • Brief Description:
  • NCIGT_Prostate (HE/CT)
    • Key Investigators:
    • Brief Description:
  • NCIGT_Prostate_Fully_Segmented (HE/CT)
    • Key Investigators:
    • Brief Description:
  • NCIGT_Brain_Biopsy (FT)
    • Key Investigators:
    • Brief Description:

Use-Case Goals

Step 1. Data Management

  • Anonymize, apply DICOM metadata and upload retrospective datasets; confirm appropriate organization and naming scheme via web GUI.

Step 2. Query & Retrieval

  • Make specific queries using XNAT web services,
  • Download data conforming to specific naming convention and directory structure, using XNAT web services

Each effort listed above will have different requirements for being able to query, retrieve and use data collections. Brief description of how retrospective data will be used within the NCIGT is described below:

  • NCIGT_Brain_Function:
  • NCIGT_Tumor_Resection:
  • NCIGT_Prostate:
  • NCIGT_Prostate_Fully_Segmented:
  • NCIGT_Brain_Biopsy:

Step 3. Disseminating & Sharing

  • In addition to NCIGT mandate to share data, each effort listed above will have different requirements for being able to make data available to collaborating and other interested groups.

Outcome Metrics

Step 1. Data Management

Step 2. Query & Retrieval

Step 3. Dissemination & Sharing

Fundamental Requirements

Participants

  • Mark Anderson
  • Tina Kapur

Data

Workflows

Current Data Management Process

Target Data Management Process (Step 1.) Option A.

  • Create new project using web GUI
  • Manage project: Configure settings to automatically place data into the archive (no pre-archive)
  • Create a subject template
  • Create a spreadsheet conforming to subject template
  • Upload spreadsheet to create subjects
  • Run CLI Tool for batch anonymization (See here for HowTo: http://nrg.wustl.edu/projects/DICOM/DicomBrowser/batch-anon.html)
  • Need pointer for script to do batch upload & apply DICOM metadata.
  • Confirm data is uploaded & represented properly with web GUI

Target Data Management Process (Step 1.) Option B. (web services API) CURRENTLY BEING TESTED!

1. Create new project on XNAT instance using web GUI

  • Manage project using web GUI: Configure settings to automatically place data into the archive (no pre-archive)



2. Batch Anonymize

For next steps, use curl or XNATRestClient (See here to download XNATRestClient in xnat_tools.zip from here: http://nrg.wikispaces.com/XNAT+REST+API+Usage)



3. Authenticate with server and create new session; use the response as a sessionID ($JSessionID) to use in subsequent queries 

curl -d POST $XNE_Svr/REST/JSESSION -u $XNE_UserName:$XNE_Password 
or, use the XNATRestClient
XNATRestClient -host $XNE_Svr -u $XNE_UserName -p $XNE_Password -m POST -remote /REST/JSESSION



4. Create subject 

 XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT /REST/projects/$ProjectID/subjects/s0001 
(This will create a subject called 'S0001' within the project $ProjectID)

A script can be written to automatically create all subjects for the project.

4a. specify the demographics of a subject already created, or create with demographic specification

4.a.1 No demographics are applied to each subject by default. To edit the demographics (like gender or handedness) of a subject already created using XML Path shortcuts. 

xnat:subjectData/demographics[@xsi:type=xnat:demographicData]/gender = male
xnat:subjectData/demographics[@xsi:type=xnat:demographicData]/handedness = left

The entire command looks like this (Append XML path shortcuts and separate each by an &. Note that querystring parameters must be separated from the actual URI by a ?): 

XNATRestClient -host $XNE_svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/s0001?xnat:subjectData/demographics[@xsi:type=xnat:demographicData]/gender=male&xnat:subjectData/demographics[@xsi:type=xnat:demographicData]/handedness=left"

All XML Path shortcuts that can be specified on commandline for projects, subject, experiments are listed here: http://nrg.wikispaces.com/XNAT+REST+XML+Path+Shortcuts

4.a.2 Alternatively, specify the demographics during subject creation by generating and uploading an xml file with the subject: 

XNATRestClient -host $XNE_srv -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/s0002" - local ./$ProjectID_s0002.xml

The XML file you create and post looks like this: 

<xnat:Subject ID="s0002" project="$ProjectID" group="control" label="1" src="12"  xmlns:xnat="http://nrg.wustl.edu/xnat" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <xnat:demographics xsi:type="xnat:demographicData">
       <xnat:dob>1990-09-08</xnat:dob>
       <xnat:gender>female</xnat:gender>
       <xnat:handedness>right</xnat:handedness>
       <xnat:education>12</xnat:education>
       <xnat:race>12</xnat:race>
       <xnat:ethnicity>12</xnat:ethnicity>
       <xnat:weight>12.0</xnat:weight>
       <xnat:height>12.0</xnat:height>
   </xnat:demographics>
</xnat:Subject>

4.b (optional check) Query the server to see what subjects have been created: 

XNATRestClient -host $XNE_Svr -user_session $JSessionID -m GET -remote /REST/projects/$ProjectID/subjects

4.c Create experiments (collections of image data) you'd like to have for each subject 

XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/MRExperiment?xnat:mrSessionData/date=01/02/09"
XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/CTExperiment1?xnat:ctSessionData/date=01/02/09"
XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/PETExperiment1?xnat:petSessionData/date=01/02/09"

4.d (optional check) Query the server to see what experiments have been created: 

XNATRestClient -host $XNE_Svr -user_session $JSessionID -m GET -remote /REST/projects/$ProjectID/subjects/s0001/experiments?format=xml




5. Batch Anonymize scan data



6. Create metadata from DICOM headers



7. Create uris for scans, reconstructions and upload them.

Note: when uploading images, it is good form to define the format of the images (DICOM, ANALYZE, etc) and the content type of the data. This will not translate any information in the DICOM header into metadata on the scan. 

//create SCAN1
XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/$ExperimentID/scans/SCAN1?xnat:mrScanData/type=T1"

/upload SCAN1 files...
XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/$ExperimentID/scans/SCAN1/files/1232132.dcm?format=DICOM&content=T1_RAW" -local /data/subject1/session1/RAW/SCAN1/1232132.dcm


//create SCAN2
XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/$ExperimentID/scans/SCAN2?xnat:mrScanData/type=T2"

//upload SCAN2 files...
XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/$ExperimentID/scans/SCAN2/files/1232133.dcm?format=DICOM&content=T2_RAW" -local /data/subject1/session1/RAW/SCAN2/1232133.dcm


//create reconstruction 1
XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/$ExperimentID/reconstructions/session1_recon_0343?xnat:reconstructedImageData/type=T1_RECON"

//upload reconstruction 1 files...
XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/$ExperimentID/reconstructions/session1_recon_0343/files/0343.nfti?format=NIFTI" -local /data/subject1/session1/RECON/T1_0343/0343.nfti


//create reconstruction 2
XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/$ExperimentID/reconstructions/session1_recon_0344?xnat:reconstructedImageData/type=T2_RECON"

//upload reconstruction 2 files...
XNATRestClient -host $XNE_Svr -user_session $JSessionID -m PUT -remote "/REST/projects/$ProjectID/subjects/$SubjectID/experiments/$ExperimentID/reconstructions/session1_recon_0344/files/0344.nfti?format=NIFTI" -local /data/subject1/session1/RECON/T1_0344/0344.nfti



8. Confirm data is uploaded & represented properly with web GUI

Target Query Formulation (Step 2.)

Target Processing Workflow (Step 3.)

Other Information

Retrieved from "https://www.na-mic.org/w/index.php?title=CTSC_IGT,_BWH&oldid=41659"