Difference between revisions of "2016 Summer Project Week/Segmentation Editor and Terminology"

From NAMIC Wiki
Jump to: navigation, search
 
(18 intermediate revisions by 2 users not shown)
Line 3: Line 3:
 
Image:PW-Summer2016.png|[[2016_Summer_Project_Week#Projects|Projects List]]
 
Image:PW-Summer2016.png|[[2016_Summer_Project_Week#Projects|Projects List]]
 
Image:TerminologyInEditor.jpg|Slicer 4.5 Editor with terminology
 
Image:TerminologyInEditor.jpg|Slicer 4.5 Editor with terminology
 +
Image:Terminology_web_app.jpg|Component of the dcmqi web app for populating terminology
 +
Image:Terminology_web_app-color.jpg|Component of the dcmqi web app for populating terminology, with color picker widget
 
</gallery>
 
</gallery>
  
Line 9: Line 11:
 
* Csaba Pinter (Queens)
 
* Csaba Pinter (Queens)
 
* Andrey Fedorov (BWH)
 
* Andrey Fedorov (BWH)
 +
* Marco Nolden (DKFZ)
 +
* Andras Lasso (Queen's)
 +
* Christian Herz (BWH)
  
 
==Project Description==
 
==Project Description==
<div style="margin: 20px;">
+
 
<div style="width: 27%; float: left; padding-right: 3%;">
+
This project is focussed on linking various types of information related to segmented structures. The new Segments editor is being extended to associate segments with standard terminolgy describing anatomical regions, expanding the color selection to index into the SNO-MED terminology hierarchy.
<h3>Objective</h3>
+
 
 +
{| class="wikitable"
 +
! style="text-align: left; width:27%" |  Objective
 +
! style="text-align: left; width:27%" |  Approach and Plan
 +
! style="text-align: left; width:27%" |  Progress and Next Steps
 +
|- style="vertical-align:top;"
 +
|
 +
<!-- Objective bullet points -->
 
* Build on the work done in the [[2016_Summer_Project_Week/Segmentations_Integration|Segmentations Integration]] project to incoporate terminology information from [http://wiki.slicer.org/slicerWiki/index.php/Documentation/4.5/Extensions/Reporting Reporting module] as ported to the Editor module as documented in [http://www.na-mic.org/Bug/view.php?id=4047 issue 4047] and committed as [http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&revision=24742 svn 24742].
 
* Build on the work done in the [[2016_Summer_Project_Week/Segmentations_Integration|Segmentations Integration]] project to incoporate terminology information from [http://wiki.slicer.org/slicerWiki/index.php/Documentation/4.5/Extensions/Reporting Reporting module] as ported to the Editor module as documented in [http://www.na-mic.org/Bug/view.php?id=4047 issue 4047] and committed as [http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&revision=24742 svn 24742].
</div>
+
|
<div style="width: 27%; float: left; padding-right: 3%;">
+
<!-- Approach and Plan bullet points -->
<h3>Approach, Plan</h3>
 
 
* The initial plan was for the Segment Editor to incoporate the display of terminology, ported from the Editor module. The next stage is to add support for editing the terminologies and adding new ones.
 
* The initial plan was for the Segment Editor to incoporate the display of terminology, ported from the Editor module. The next stage is to add support for editing the terminologies and adding new ones.
* Review overview sildes [https://www.slicer.org/slicerWiki/index.php/File:20160526_Segmentations.pptx | Segmentations]
+
* Review overview slides [https://www.slicer.org/slicerWiki/index.php/File:20160526_Segmentations.pptx Segmentations]
* Source code review: [https://subversion.assembla.com/svn/slicerrt/trunk/SlicerRt/src/ | SlicerRT], with a focus on containers (vtkSegment, vtkSegmentation, vtkMRMLSegmentationNode), and widget GUI (Segmentations Widgets qMRMLSegmentsTableView)
+
* Source code review: [https://subversion.assembla.com/svn/slicerrt/trunk/SlicerRt/src/ SlicerRT] ([https://github.com/SlicerRt/SlicerRT/ github mirror]), with a focus on containers (vtkSegment, vtkSegmentation, vtkMRMLSegmentationNode), and widget GUI (Segmentations Widgets qMRMLSegmentsTableView)
* Compare to web app for populating SEG metadata: http://qiicr.org/dcmq
+
* Compare to web app for populating SEG metadata: http://qiicr.org/dcmqi ([https://github.com/QIICR/dcmqi/tree/gh-pages github source])
</div>
+
** To view in slicer's webkit
<div style="width: 27%; float: left; padding-right: 3%;">
+
  view = qt.QWebView()
<h3>Progress</h3>
+
  view.setUrl(qt.QUrl('http://qiicr.org/dcmqi/'))
 +
  view.show()
 +
* From Andrey: disconnect segment ID from the color ID and from the terminology associated with it - it should be possible to have more than one distinctive segment that has the same color and the same terminology, but are differentiated by segment ID and segment label.
 +
|
 +
<!-- Progress and Next steps bullet points (fill out at the end of project week -->
 
* Tue Jun 21: Csaba and Nicole met and discussed ways to incorporate the terminology information into the segment editor
 
* Tue Jun 21: Csaba and Nicole met and discussed ways to incorporate the terminology information into the segment editor
 
** Each segmentation node has a custom temporary color table that's built up from color selected for each segment.
 
** Each segmentation node has a custom temporary color table that's built up from color selected for each segment.
Line 34: Line 49:
 
*** Bring up the CTK color picker by default on adding a new segment to set/edit the default color/terminology (Csaba)
 
*** Bring up the CTK color picker by default on adding a new segment to set/edit the default color/terminology (Csaba)
 
*** Extend/fix the CTK color picker:
 
*** Extend/fix the CTK color picker:
**** Show the custom color node by default not the Generic Anatomy one (Nicole)
+
**** Show the custom color node by default not the Generic Anatomy one (Nicole - fixed in Colors module)
 
**** How to best show if adding a new color or editing an old one?
 
**** How to best show if adding a new color or editing an old one?
 
**** Fix the bug where you can pick a new color for a segment, but if you click on a different color in the color picker's table it still sets the color for the entry that was clicked in the editor GUI (disable selection highlight?) (Nicole)
 
**** Fix the bug where you can pick a new color for a segment, but if you click on a different color in the color picker's table it still sets the color for the entry that was clicked in the editor GUI (disable selection highlight?) (Nicole)
 +
***** Not a bug, allows you to copy colors, but do set the first selection
 
**** Add display of the terminology associated with the color node displayed (Nicole)
 
**** Add display of the terminology associated with the color node displayed (Nicole)
 
**** Add terminology editing/look up to set a new terminology (Nicole - post project week)
 
**** Add terminology editing/look up to set a new terminology (Nicole - post project week)
 
** Consider loading a segmentation node from disk and rebuilding the custom terminology - will each segment need to have a the terminology category, property, region and modifiers set?
 
** Consider loading a segmentation node from disk and rebuilding the custom terminology - will each segment need to have a the terminology category, property, region and modifiers set?
 
** Add tool tips showing terminology, as in Colors module, see [https://github.com/Slicer/Slicer/blob/master/Libs/MRML/Widgets/qMRMLColorModel.cxx#L340 qMRMLColorModel] (Csaba)
 
** Add tool tips showing terminology, as in Colors module, see [https://github.com/Slicer/Slicer/blob/master/Libs/MRML/Widgets/qMRMLColorModel.cxx#L340 qMRMLColorModel] (Csaba)
</div>
+
* Met with Andrey and he filled in use cases below
</div>
+
* Fixed color node selection in Colors module, providing UI to set it from other modules
 +
* Next Steps:
 +
** Continue working on CTK widget update/integration
 +
|}
 +
 
 +
=== Use cases ===
 +
 
 +
* Category = Segmented Property Category
 +
* Type = Segmented Property Type (+Modifier, optional)
 +
* Anatomic region (+Modifier, optional)
 +
 
 +
==== Loading segmentation from DICOM ====
 +
Assumptions:
 +
* can expect that Category and Type will be initialized, if not - fall back to default codes (Tissue/Tissue codes)
 +
* anatomic region may or may not be initialized (example: can specify anatomic location of the tumor, but may not be present)
 +
* colors may or may not be specified
 +
Consequences:
 +
* need to be able to handle these various combinations
 +
* if colors are not specified, would be nice to suggest some good default color choices (to optimize differentiation of the structures? lookup of default colors may not be easy for medium-large dictionaries)
 +
* it is not clear that it is a good idea to restrict the choices of terminology (or colors) if a user wants to modify segmentation loaded from DICOM - so it might be helpful to choose from existing color maps / terminology dictionaries for a DICOM-loaded segmentation
 +
 
 +
==== Creating segmentation from scratch ====
 +
* choose from predefined dictionaries (combinations of Type/Category/Anatomy customized for a specific use case)
 +
* choose from predefined color tables, or perhaps color assignment strategies (e.g., high contrast, similar shades of color for the segments corresponding to the same kind of structure)
 +
 
 +
=== Workflow ===
 +
 
 +
To add a new segment:
 +
* user clicks "add segment"
 +
* terminology and color are populated to some default value
 +
* user has an option to change the color and terminology separately TerminologyWidget (naming to be confirmed)
 +
* user has the option to choose the dictionary in the TerminologyWidget
 +
* the purpose of the dictionary is to define specific combinations of category/type/anatomy for the specific use case (GenericAnatomy LUT is one example, and we will also need a larger set of codes defined by the standard)
 +
 
 +
=== Implementation approach ===
 +
 
 +
Terminology browser widget:
 +
* Slicer widget vs CTK widget: consensus seems to be that CTK widget makes sense
 +
* Qt widget vs web app: considering that basic web app already exists, it may make sense to experiment reusing it; Marco notes that it may be futile to experiment with web app integration in Slicer using the current Qt 4
 +
** Marco confirmed that qiicr.org/dcmqi renders correctly in Qt 5.6 (used in MITK); it does not render in Slicer Qt 4.x webkit
 +
* We will continue with prototyping in the web app (add support for color picker and dictionary selection)
 +
** web app color picker issue https://github.com/QIICR/dcmqi/issues/21

Latest revision as of 09:06, 25 June 2016

Home < 2016 Summer Project Week < Segmentation Editor and Terminology

Key Investigators

  • Nicole Aucoin (BWH)
  • Csaba Pinter (Queens)
  • Andrey Fedorov (BWH)
  • Marco Nolden (DKFZ)
  • Andras Lasso (Queen's)
  • Christian Herz (BWH)

Project Description

This project is focussed on linking various types of information related to segmented structures. The new Segments editor is being extended to associate segments with standard terminolgy describing anatomical regions, expanding the color selection to index into the SNO-MED terminology hierarchy.

Objective Approach and Plan Progress and Next Steps
  • The initial plan was for the Segment Editor to incoporate the display of terminology, ported from the Editor module. The next stage is to add support for editing the terminologies and adding new ones.
  • Review overview slides Segmentations
  • Source code review: SlicerRT (github mirror), with a focus on containers (vtkSegment, vtkSegmentation, vtkMRMLSegmentationNode), and widget GUI (Segmentations Widgets qMRMLSegmentsTableView)
  • Compare to web app for populating SEG metadata: http://qiicr.org/dcmqi (github source)
    • To view in slicer's webkit
 view = qt.QWebView()
 view.setUrl(qt.QUrl('http://qiicr.org/dcmqi/'))
 view.show()
  • From Andrey: disconnect segment ID from the color ID and from the terminology associated with it - it should be possible to have more than one distinctive segment that has the same color and the same terminology, but are differentiated by segment ID and segment label.
  • Tue Jun 21: Csaba and Nicole met and discussed ways to incorporate the terminology information into the segment editor
    • Each segmentation node has a custom temporary color table that's built up from color selected for each segment.
      • Add a node reference to the segmentation node pointing to the color node that is used to build the custom color table (generic anatomy colors has terminology associated with it). This will help on load as the segment has the color index and the custom color table will be rebuilt from the reference color node. (Csaba)
      • When creating the new temp color node, also create a new terminology associated with it (Nicole)
    • Adding a new segment
      • A color RGB is set on the display node, that's used to rebuild the custom color table on load
      • Consolidate to the segment editor (Csaba)
      • Bring up the CTK color picker by default on adding a new segment to set/edit the default color/terminology (Csaba)
      • Extend/fix the CTK color picker:
        • Show the custom color node by default not the Generic Anatomy one (Nicole - fixed in Colors module)
        • How to best show if adding a new color or editing an old one?
        • Fix the bug where you can pick a new color for a segment, but if you click on a different color in the color picker's table it still sets the color for the entry that was clicked in the editor GUI (disable selection highlight?) (Nicole)
          • Not a bug, allows you to copy colors, but do set the first selection
        • Add display of the terminology associated with the color node displayed (Nicole)
        • Add terminology editing/look up to set a new terminology (Nicole - post project week)
    • Consider loading a segmentation node from disk and rebuilding the custom terminology - will each segment need to have a the terminology category, property, region and modifiers set?
    • Add tool tips showing terminology, as in Colors module, see qMRMLColorModel (Csaba)
  • Met with Andrey and he filled in use cases below
  • Fixed color node selection in Colors module, providing UI to set it from other modules
  • Next Steps:
    • Continue working on CTK widget update/integration

Use cases

  • Category = Segmented Property Category
  • Type = Segmented Property Type (+Modifier, optional)
  • Anatomic region (+Modifier, optional)

Loading segmentation from DICOM

Assumptions:

  • can expect that Category and Type will be initialized, if not - fall back to default codes (Tissue/Tissue codes)
  • anatomic region may or may not be initialized (example: can specify anatomic location of the tumor, but may not be present)
  • colors may or may not be specified

Consequences:

  • need to be able to handle these various combinations
  • if colors are not specified, would be nice to suggest some good default color choices (to optimize differentiation of the structures? lookup of default colors may not be easy for medium-large dictionaries)
  • it is not clear that it is a good idea to restrict the choices of terminology (or colors) if a user wants to modify segmentation loaded from DICOM - so it might be helpful to choose from existing color maps / terminology dictionaries for a DICOM-loaded segmentation

Creating segmentation from scratch

  • choose from predefined dictionaries (combinations of Type/Category/Anatomy customized for a specific use case)
  • choose from predefined color tables, or perhaps color assignment strategies (e.g., high contrast, similar shades of color for the segments corresponding to the same kind of structure)

Workflow

To add a new segment:

  • user clicks "add segment"
  • terminology and color are populated to some default value
  • user has an option to change the color and terminology separately TerminologyWidget (naming to be confirmed)
  • user has the option to choose the dictionary in the TerminologyWidget
  • the purpose of the dictionary is to define specific combinations of category/type/anatomy for the specific use case (GenericAnatomy LUT is one example, and we will also need a larger set of codes defined by the standard)

Implementation approach

Terminology browser widget:

  • Slicer widget vs CTK widget: consensus seems to be that CTK widget makes sense
  • Qt widget vs web app: considering that basic web app already exists, it may make sense to experiment reusing it; Marco notes that it may be futile to experiment with web app integration in Slicer using the current Qt 4
    • Marco confirmed that qiicr.org/dcmqi renders correctly in Qt 5.6 (used in MITK); it does not render in Slicer Qt 4.x webkit
  • We will continue with prototyping in the web app (add support for color picker and dictionary selection)