Projects:ARRA:miAnnotation

Back to Slicer ARRA home page

Aim

Medical images often contain a wealth of information, such as anatomy and pathology, not explicitly accessible. One way to address this issue is via image annotation and markup. We propose to create a comprehensive framework for annotation and markup within 3D Slicer, enabling users to capture structured information easily. Furthermore, we will develop schemas for saving and recovering this information into and from XNAT, allowing queries of larger data sets of medical scans. This tool will provide clinicians with a relatively simple way to capture information latent in medical scans, and also to select micro-cohorts of medical scans for studying diseases.

Research Plan

3D Slicer currently provides very basic technology for annotating images. This limits users in their ability to properly capture semantic information contained in images and data sets. We propose to address this issue by expanding Slicer's mark up and annotation capabilities. New features will include:

• a rich set of geometric objects for improved visual differentiation between annotations
• markers for measuring anatomical characteristics, such as the volume of an annotated region, to provide patient specific information difficult to extract from visual inspection
• entry fields beyond free-text, such as graphics and external data, to capture comprehensive information and support for emerging domain specific ontologiesand
• a full integration of these capabilities with the mrml tree to support Scenesnapshots, load, save both to disk and XNAT.

We will implement these features by developing two different modules. The first module, called Marker Module, creates different types of markers based on current ITK technology. The user defines the appearance of the marker by specifying its color, size, and shape, such as points and 3D boxes. The user also labels each marker with tags and specifies its function, such as measuring the volume of a region.

The Annotation Module, the second module, provides the interface for annotating images with these markers. Users place the markers on the image and further specify the semantic information through free text, plots, and references to ontology and internet. The annotations are shown both in 3D and 2D viewers. The module also allows annotating entire scenes by linking annotations across images, as well as within an image. All annotations are stored in a database targeted towards medical imaging, called XNAT. The structure of the database is automatically defined by the tags of the markers. Thus, users can query across large image data sets by looking for specific tag values.

Both modules are accompanied by training materials and documentation to ensure usability.

Design of Module

• Current GUI outline - Screenhot in Slicer
• Suggested MRML outline
• Wendy's Mouse Mode sketches https://www.slicer.org/wiki/Slicer4:UIDesign:MouseModeSketches

Documentation

• Documentation of the module
• Using the module with corresponding data ( sample data )

Key Personnel

10% Kilian Pohl (UPenn)
95% Daniel Haehn (UPenn)
5% Dominique Belhachemi (UPenn)

Past Contributors

Yong Zhang (IBM)

Events

• 05/18/10 Presentation at the Department of Healthcare Informatics, IBM, Almaden
• 12/16/09 - 12/17/09 Annotation Brain Storming
• 01/06/10 Break out session at AHM 2010
• 01/19/10 Biweekly Annotation Tcon from 4:00 - 5:00 pm EST (next meeting 2/9): Tel: (218) 862-1115 Conference ID: 348265#, Desktop Sharing
• 08/30/10 Budget Year 1 Review meeting

Progress

• 09/10/11
  • work on Projects:ARRA:miAnnotation:Feedback
• 09/03/11
  • work on Projects:ARRA:miAnnotation:Feedback
• 08/26/11
  • 'Feedback by Ron' Session: Projects:ARRA:miAnnotation:Feedback
  • work on the requested changes (see status updates on page above)
• 08/19/11
  • work on lightbox support (not supported by vtkWidgets yet, wrote test for Kitware)
  • work on undo/redo mrml support which stopped working
• 08/12/11
  • work on new Qt widgets to enable (row-based) editing of point, line and text properties
  • fixed locking issue
  • fixed jump slices behavior
• 08/05/11
  • new property dialog in the works..
• 07/29/11
  • preparation to be able to easily set properties for a complete widget (see http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&revision=17587 for details)
  • facelift draft for the property dialog by Ron: http://www.na-mic.org/Bug/view.php?id=1302
  • new default colors for annotations
• 07/22/11
  • switched color dialogs to the slicer custom ones
  • 2D ROI Annotations
  • fixes on mouse-modes and hierarchies
  • simplified module GUI
  • tweaks to placement icons
• 07/15/11
  • new place-mode: integrated in the toolbar
  • Fixed bug: Stop widgets from disappearing when panning, zooming the sliceViews.
  • Cancel placement now uses events on the interaction Node and works properly again.
• 07/08/11
  • deactivated bidimensional widget in 3D until it is properly realized as a 3D widget/representation
  • other fixes
• 07/01/11
  • • several performance and stability updates
    • presentation and user-feedback during the NA-MIC project week
• 06/24/11
  • • the annotation module is now called 'Annotations'
    • re-structured module panels
• 06/17/11
  • • added selected vs. highlighted mode when choosing annotations
    • optimized mouse-modes
    • upgraded screenshot functionality to support scale factor
• 06/10/11
  • • made all annotations transformable
    • annotation widgets now available in python scripted modules
    • several hierarchy discussions and fixes
• 06/03/11
  • fixed flickering when changing fiducial properties
  • added scalefactor to screenshot dialog
  • optimized event handling and performance
  • rulers: added infrastructure to change properties
  • roi: added infrastructure to change properties
• 05/27/11
  • fixed a huge blocker where the fiducial coordinates between 2D and 3D were not stable. Now they get properly synchronized in a robuste way.
  • added backup methods to be able to cancel annotation property settings which then restores the node
• 05/20/11
  • change mouse cursor when in place modes to the specific icons
  • place-once as the default mode in the annotation module
    • added advanced panel to activate persistence place when using the annotaton module
  • work on the keypress event to place fiducials
• 05/13/11
  • MRML: add required modified events for certain display properties
  • add opacity slider to text properties in the property dialog
  • add a spacer to the property dialog to have a consistent design for text, point and line properties
  • add label position to the display node
  • add utility methods to get the text, point, line display nodes for a given mrml node id
  • handle the main annotation types when setting them active in the selection node
  • hook up the text opacity, the point material properties. Grey out the line tab if no line display node. Add the point values to the table in the points tab
  • removed the Save panel from the module GUI and moved the buttons for report and screenshot in the edit toolbar (this might change at one point)
  • added a storage node for the annotation screen snapshot node to properly save the screenshots
  • cleanup of annotation snapshot nodes
  • use the text opacity, the point material properties
  • add value constraints for several sliders in the property dialog
  • MRML: add new method SetControlPoint(id, coords) without the visible and selected flags
  • property dialog now complete for text, point, line! (not for hierarchies yet)
• 05/06/11
  • fiducials are now shown the same way in 2D and 3D
  • the property dialog is connected to configure point colors and glyph types for fiducials
  • it is possible to add text to a fiducial in 3D
  • meeting with Ron to discuss further development and usability issues
• 04/29/11
  • fixed the flickering when placing or interacting with fiducials and rulers
  • removed some obsolete render calls to enhance performance
• 04/22/11
  • worked together with Nicole
    • finally fixed the crash on Slicer exit
    • investigated why MRMLScene import works fine with the annotations but MRMLScene load does not
      • first success: import and load mrml scenes now both load the annotations in 2D and 3D
• 04/15/11
  • Maintenance
• 04/08/11
  • Maintenance
• 04/01/11
  • Maintenance
• 03/25/11
  • Maintenance
• 03/18/11
  • Work on MRML Load/Save issues (Nicole)
  • Maintenance
• 03/11/11
  • Work on hierarchies (Nicole)
  • Maintenance
• 03/04/11
  • Maintenance
• 02/25/11
  • Maintenance
• 02/18/11
  • Maintenance
  • Communication with Daniel Rubin from Stanford to plan AIM integration
    • provided Annotation MRML examples
• 02/11/11
  • Maintenance
• 02/04/11
  • Clarified how to add Node Selectors for Annotations to the GUI of 3rd Party 3D Slicer Modules
    • Kitware provided necessary infrastructure
    • Proof of Concept in the new CropVolume module with ROI Annotations (Andriy)
  • Clearing up the Annotation Storage Node Tests (Nicole)
• 01/28/11
  • Presented Annotation module and SceneView module at NAC EAB Meeting
  • Smaller bugs and maintenance
  • Meeting with Nicole and Alex concerning the final API of hierarchies in Slicer4
• 01/21/11
  • Re-activated ROI annotation with backwards compatibility to the old vtkMRMLROINode
    • Tested successfully with a scene from Slicer3
    • Provided documentation to relevant people
  • Rulers now support the measurement tape behavior
  • Changed the UPenn logo
  • Several smaller fixes
  • provided Wendy with Screenshots for NAC Poster
    • 

      The current state of the Annotation module.
    • 

      The report dialog.
  • Found displayableManager inconsistency between 2D and 3D and the MRML Node Added Event
  • MRML backwards compatibility fix for all Annotations
    • new tag "visible", old tag "visibility"
• 01/14/11
  • Attended NA-MIC project week
  • Enabled picking when placing annotations in 3D.
  • Fixed display of ruler intersections.
  • Added new icons.
  • Use only three seeds to place bidimensional widgets which results in more stability.
  • It is also now possible to add annotations to the toplevel hierarchy after another hierarchy was created by clicking in an empty space of the treeView.
  • Re-added the ROI functionality, so now we have 5 widgets
  • Ruler: enabled display of ticks (measuring tape mode) in 2D and 3D
• 01/07/11
  • re-factored the annotation module in terms of changes in the Slicer4 environment
    • ready for the new developer release
  • worked on the outsourced SceneView functionality to make SceneViews survive round-trips (mrml load and save)
• 12/31/10
  • Maintenance
  • Documentation
• 12/24/10
  • Maintenance
  • Received the first bug reports of potential users
    • Updated bugtracker
• 12/17/10
  • Extended Report functionality
    • now includes screenshots
    • shows the hierarchies
  • Sorting of Annotations now possible
  • Widgets:
    • added the 3D representation of rulers
    • removed the vtkInteractorRecorder hack by placing the widgets programmatically
  • Outsourced the former integrated scene snapshot functionality
    • Wrote a new module called SceneViews
    • vtkMRMLSceneViewNode replaces the old snapshot node and includes thumbnail, description..
      • is not part of the displayable framework anymore
  • Added confirmation questions prior to removing annotations
  • Worked on MRML save/load
  • Updated the priority list: http://www.na-mic.org/Wiki/index.php/Projects:ARRA:miAnnotation:PriorityList
  • Added a section for known issues: http://wiki.na-mic.org/Wiki/index.php/Projects:ARRA:miAnnotation#Known_issues
  • Annotation module Lines of Code (without .ui files and no blank lines): 24820
• 12/10/10
  • Changed some things in the MRML architecture to enhance encapsulation
    • Adding new Annotations should be easier now
  • Lock selected, toggle visibility of selected and delete selected implemented
  • Report functionality works again
    • Added support to print and export as a HTML report
• 12/03/10
  • Added icons to the hierarchy tree
  • Configured a proxy model to display only vtkMRMLAnnotationHierarchyNodes and vtkMRMLAnnotationNodes in the hierarchy tree
  • Smaller layout fixes
  • Added 'full layout' thumbnail functionality to Annotation snapshots
  • Removed the old table and replaced fully by the treeView
  • Over 1000 of lines of code were removed
    • Hierarchies are now usable
    • 

      Screenshot of an example Annotation using the new features
• 11/26/10
  • Added functionality to the module logic to be able to place fiducials programmatically
    • This will be used if one presses the 'p' key on a slice view
  • Re-activated the text annotation
    • Used vtkCaptionWidget instead vtkTextWidget to be able to have an anchorpoint in 3D space
  • The scene, camera and view nodes are now hidden from the hierarchy tree
    • The toplevel annotation hierarchy is the new root and also hidden
  • Added functionality to show Annotation snapshot nodes in the hierarchy tree
    • This required adding the original snapshot node to the displayable framework (as a workaround)
    • Later, we will add thumbnail and description directly to vtkMRMLNode, so that a screenshot can be attached to every MRML node
  • Overall code polishing in terms of code conventions
    • including the destruction of the frozen seedWidget on Slicer exit
• 11/19/10
  • Small fixes to the Annotation snapshot functionality
  • New functionality: reset view to when the annotation was placed
  • Derived qMRMLAnnotationTreeWidget and qMRMLSceneAnnotationModel to support hierarchies in the module
    • the widgets let us customize the look and feel of the treeWidget
    • in this context, fixed the creation of hierarchies to a specific order in the scene (which is need for the widget)
    • the Annotation hierarchy nodes and the Annotations get already displayed in the treeWidget
  • Fixed stack smashing in connection with a recently introduced compiler flag
• 11/12/10
  • All widgets are now restricted to the current renderer which makes the interaction more robust.
  • Fixed funky behavior after fiducials were placed.
  • New functionality: Annotation snapshots (screenshot + description + scene snapshot) implemented.
    • 

      The new annotation snapshot functionality (part 1)
      

      The new annotation snapshot functionality (part 2)
• 11/05/10
  • Widget Design Fiesta http://www.na-mic.org/Wiki/index.php/WidgetDesign2010
  • Widget intersections works nearly perfectly
    • Unfortunately, only for centered datasets
  • Restrict ruler placement to the current renderer works now. This means, the ruler does not disappear when one point moves outside the sliceview.
• 10/29/10
  • Lightbox view works for Fiducials
    • Other widgets have to be refined to be used in a lightbox view due to inconsistent VTK API. The VTK Widget Fiesta should target this.
  • Widget intersections in the works, not perfectly working yet.
• 10/22/10
  • Priority list for the December release of Slicer4: http://www.na-mic.org/Wiki/index.php/Projects:ARRA:miAnnotation:PriorityList
• 10/15/10
  • Adopting the old TCL logic to enable 2D widgets on reformats and lightbox views
    • Still working on the lightbox view (very close)
• 10/08/10
  • Code-cleanup of the logic class - any old and clumsy code was removed.
  • Introducing the StringHolder design pattern [(c) Steve] to always show measurement correctly in the GUI.
  • Widget creation now works on Unix aswell.
  • Added the RegisterNodes method to register node classes when the scene gets attached to the logic class.
  • Hierarchy nodes get added when an annotation is created (code workflow to be optimized).
  • Configured a nightly build system to build and test the annotation code every night on a SPL workstation.
• 10/01/10
  • The hierarchy changes will replace the table in the GUI with a tree widget
    • Added a tree widget to the GUI additionally to the table to display both during the transition stage
  • Fixed the follower bug, which sometimes dragged the second controlpoint of a widget behind when the other controlpoint was moved.
• 9/24/10
  • Wrote performance test for vtkWidgets in a standalone VTK mode
    • compared performance in Slicer against standalone VTK
    • rendering of text labels was found to be the performance hog, disabling these labels increases performance inside and outside Slicer dramatically
  • Meeting between Wendy, Nicole, Steve and Daniel to review the Annotation module status
    • Notes: Projects:ARRA:SlicerAM:Meeting20100921
  • Annotation hierarchy in the works
• 9/17/10
  • 2D placement of Rulers, Bidimensional and Fiducials works now
    • The widgets get synchronized along the slice viewers and the 3D view
  • Added extra layer on top of vtkWidgets and vtkWidgetRepresentations to add some extra functionality like calculating distances of measurement correctly in 2D
    • bidimensional measurement and rulers now show the correct distance also in 2D
  • Fixed possible null pointer exception in vtkMRMLSliceLogic
  • Refactored GetSliceOffSet from vtkMRMLSliceLogic to vtkMRMLSliceNode
• 9/10/10
  • 2D placement of Annotations in the works
    • change of the underlying architecture to use one displayableManager for 2D and 3D resulting in less code duplication
    • to perform the changes, all annotations but fiducials were disabled for now
• 9/3/10
  • Annotation module status review and planning meeting took place in 1249 Boylston
    • Agenda and Notes: http://wiki.na-mic.org/Wiki/index.php/Projects:ARRA:SlicerAM:Meeting20100831
      • includes new generated class diagram
  • Property dialog updated (see screenshot)
    • 

      The new annotation property dialog.
    • first MRML-Widget synchronization works, the text tab in the property dialog works well with the text widget
  • new icons for bidimensional measurement and ROI by Wendy
  • updated ctkColorPickerButton to include the ability to disable the setting of the color hexcode as the buttontext to avoid the workaround we had in the annotation module
• 08/27/10
  • added a work-around to initialize renderer properly resulting in improved display of annotations in 3D viewer
  • cleaned up the code (more then 1000 lines of code where obsolete)
  • updating property dialog, which includes revisiting the displayableManagers to support MRML and widget updates
• 08/20/10
  • fiducials and sticky node work now
  • ruler and spline work now
    • spline widget causes problems b/c after one spline was placed, no interaction with 3D scene possible
  • ROI works now
  • all annotations now can be placed to examine widget behavior
  • started working on a new interface for the property dialog
  • consolidated experts for 2D slice displayableManagers
  • started working on synchronizing the widgets with MRML display nodes
  • fixed GUI problem: the table did not expand even if there was free space in the module frame. now it works.
• 08/13/10
  • Wendy, Nicole and Daniel discussed how to integrate mouse modes of Slicer into Annotation module and vice versa. Solution is a widget which appears in the Annotation module panel but can also be shown outside of it as a toplevel window in Slicer (maybe with keybinding), see here: http://wiki.na-mic.org/Wiki/index.php/File:Widget.png
  • current status (workflow, user places seeds and widget appears):
    • angle annotations work (note: the widget did not support programmatical placement. a trick is used to place the widget where the user has clicked)
    • text annotations work
  • modified vtkMRMLInteractionNode to have a customTag for place modes not only from annotation module but all modules
• 08/06/10
  • Enhanced the workflow for creating text annotations. This approach should serve as a general framework for all annotations (sequence diagram available http://wiki.na-mic.org/Wiki/index.php/File:TextAnnotationWorkflow.png)
  • modified vtkMRMLInteractionNode to support the place modes for the annotations
    • meet with Wendy next week to discuss integration
  • added a click counter class to support the creation of widgets with multiple clicks (very generalized to support all annotations)
    • plan is to place seeds before the actual widget gets created
• 07/29/10
  • Created an extra layer called vtkMRMLAnnotationDisplayableManager which serves as a superclass for all specific Annotation displayableManagers. F.e. the vtkMRMLAnnotationTextDisplayableManager is now really lightweight.
  • Coordinated with Wendy to add additions interaction modes to vtkMRMLInteractionNode to support different interaction types for different annotations. This will happen under the hood, so the user sees only the "place" mode.
  • Got together with members of the 3D & Advanced Imaging Lab at UPENN to investigate how commercial products realize annotations (see http://wiki.na-mic.org/Wiki/index.php/Projects:ARRA:SlicerAM:vtkWidgets#How_do_they_do_it.3F).
  • Interactive placement of text annotations is now possible. This should be the standard way of placing annotations for all available types.
  • Added a test for the creation of a text annotation and its widget without a displayableManager (vtkMRMLAnnotationTextNodeAndWidgetTest1)
  • Removed initial values from AnnotationNodes due to different seg-fault errors when properties where not initialized. To cut down confusion, no initializations are now necessary.
• 07/22/10
  • Meeting of Nicole, Steve and Daniel to talk about architecture
    • biggest change is the introduction of a hierarchy for annotation
      • more information and discussion during the tcon:Meeting / Weekly meeting
  • fixed AnnotationTextDisplayableManager to support multiple text annotations
  • added a test for the AnnotationTextDisplayableManager
  • property dialog for annotations gets displayed again, nevertheless the connection to the MRML display nodes needs further work
• 07/15/10
  • add text nodes using the new displayable manager architecture is now possible
    • nevertheless, full functionality is not restored yet
  • added fiducial displayable manager
  • new Slicer bug tracker category: Annotations
    • filed open bugs and feature requests - please add all new requests there
• 07/08/10
  • moved tests to Testing/Cxx
  • created a superclass for all displayable managers (vtkMRMLAnnotationDisplayableManager)
  • created vtkMRMLAnnotationTextDisplayableManager based on the old qMRMLAnnotationTextManager
  • working on the add text node functionality with the new displayable manager
  • fixed linking errors appearing on the Slicer Dashboard
• 07/01/10
  • re-factored module from miAnnotation to Annotation
  • re-integrating functionality without KWWidgets
    • MRMLDisplayableManager was created by Kitware guys and includes base functionality for the re-integration
    • starting with adding text nodes
      • UML diagram for involved classes: MiAnnotation_Documentation#Development (needs to be checked for logic/manager/mrml pattern consistency)
  • added option to set helptext for the module in consistency with qSlicerWidget
• 06/24/10
  • Compiling without KWWidgets on Mac OS is now possible.
  • Starting to include the functionality of KWWidgets in pure QT code.
  • Daniel Haehn took over project from Yong Zhang.
• 06/18/10
  • Tutorial is created.
  • GetAnnotationMeasurement is modified to return multiple measurements.
  • ROI annotation is modified to display extension measurements.
  • Bidimensional annotation is modified to display bi-line measurements.
• 06/11/10
  • Documentation page is created.
  • Bugs for property dialog are fixed.
  • Constructions of Bidimensional and Spline annotations are modified using proper control point initialization.
  • Bugs for showing values for some annotation types are fixed.
• 06/04/10
  • vtkSlicerAnnotationROIManager is implemented.
  • vtkSlicerAnnotationTextManager is implemented.
  • vtkSlicerAnnotationSplineManager is implemented.
  • vtkSlicerAnnotationBidimensionalManager is implemented.
• 05/28/10
  • vtkMRMLAnnotationROINode is added;
  • vtkMRMLAnnotationBiDimensionalNode is added;
  • vtkMRMLAnnotationSplineNode is added.
  • Logic and ModuleWidget are modified to include new Nodes
• 05/21/10
  • ROI, Text, BiDimensional, and Spline Annotations are initialized.
  • fixed BUG: HTML report cannot be saved properly.
• 05/14/10
  • fixed BUG: When opening display properties from Sticky note -> reduce frame and enlarge it again by pressing triangle next to title : value and coordinates appear
  • fixed BUG: fiducial manager is added.
  • fixed BUG: AnnotationMouseModePick/Place are updated.
• 05/07/10
  • fixed BUG: Lock does not lock the position of a widget in the 3D Viewer
  • fixed BUG: changing coordinates to AngleNode does not have any effects on the Angle Widget.
  • fixed BUG: fix coloring for annotation selection
• 04/30/10
  • fixed BUG: changing control points cannot be done when property dialog is open.
  • fixed BUG: annotation is not highlighted when selected.
  • fixed BUG: starting from the 2nd annotation, changing the location of the left point for ruler or center point for angle does not update the table properly.
  • fixed Bug : changing the center point of angle node does not update table properly
• 04/23/10
  • Created a ruler widget manager
• 04/16/10
  • Created the new annotation type sticky note, which allows you to enter text independent from the viewer
• 04/09/10
  • Modified source code to be compliant with VTK 5.6
  • Grid out your buttons (Create Frame)
  • Fixed BUG: When changing line color in Display Properties it changes the Text Color
  • Added script for testing color setting
  • Fixed BUG: Label of line color changes in property window when defining a unselected color

• 04/02/10
  • Fixed BUG: Nothing happens when Property Window is open and you press the edit button for another annotation
  • Restructured Annotation MRML
  • Fixed Bug: Resetting of Lines and Points does not work correctly GetNumberOfLines > 0

• 03/26/10
  • The property restoration is now implemented properly.
  • Property Modification is re-designed to simplify the operation.
  • Removed bug: ruler/angle disappear after adding fiducials
  • Removed BUG: in property dialog, when editing the text, the character is entered from right to left.
  • include lock annotation into option button (use same logic as for new (De)select All button)
  • For each annotation type provide list outlining which display property can be changed, which cannot , and which one you do not know
  • Resolved BUG: When saving a report with a file name that already exists then the following message appears in the command shell "Error: cannot make directory" and directory with files is not updated
  • Resolved BUG: Annotation Properties: (Un)selected color is initially white even though widget in 3D viewer is different
  • Resolved BUG: Select Color window does not show the correct color
  • Resolved BUG: Selected and Unselected Color are switched
  • Define Tooltips for fiducial, angle, ruler,report, save mrml button
  • In table: Add proper tool tip for visualization and edit button
  • Resolved BUG: when saving report with out ending html to the file name - icons will not be saved under Linux
  • Created Test Module for testing miAnnotation logic
  • In collaboration with Nicole Aucoin figured out line display bug in 3D Viewer
  • Use the Kitware slider and colorPicker under Display Property
  • Fixed BUG: Slicer crashes when creating an annotation while Display properties is open, and when closing the property window and then pressing edit for the new annotation

• 03/19/10
  • In the property window display properties are shown via collapsed frame by default
  • List all attributes for text/point/line and implement those that can be modified
  • Fixed BUG: When pressing the property button warning messages pop up in my command shell
  • Annotation Properties: For Angle and Ruler Widget also shows now values of AnnotationDisplayNode and AnnotationControlPointsNode
  • Change design of html report
  • Implemented text edition and update for ruler and angle widgets.
  • Changed the default mouse mode icon to MousePick.
  • Modified Report Dialog to avoid horizontal slider when screen shot is included in the report.
  • Removed multiple warning messages.
  • Fixed Bug in Report: empty text fields are displayed without boundaries
  • Ruler Distance is now displayed just in mm
  • After talking to BWH changed name of tool from Reporting to miAnnotation
• 03/12/10
  • Created icons for:
    • edit conditioning for all annotation types (done),
    • mouse mode icons, pause, play, cancel, ok/done, (Un)lock (done)
    • new the annotation types poly line, spline with control points, volume information extractor, and sticky note (done)
    • Modify fiducial icon just to show one star and change save annotation icon (done)
    • Modify save annotation and save scene icons (done)
    • Modify move up/down delete icons (done)
  • Write 5 page, double column, AMIA paper about Reporting
• 03/05/10
  • Removed Warning Messages when compiling
  • Fixed Bug: When entering text in property box (under linux) it is entered from right to left instead left to right
  • Fixed Bug: Annotation Property: funny behavior with selecting set of digits in a numerical value and typing in a numeric value - only the first digit gets change.
  • Fixed Bug: Value of Angle node is not update in table
  • Fixed Bug: Updating the value in the table slows down the operations
  • Changed the default ruler from 1mm to 500mm.
  • Angle Widget is added by manually clicking three points on the viewer. The angle value is now shown in the table. The behavior of the angle button is in the "toggle down" state until all three control points are set
  • When selecting annotation in table, color of annotation in 3D Viewer changes to selected color
  • Revised "add ruler" button to click-style instead of toggle-style
  • When creating a new annotation, set this entry to selected and all other table entries to unselected
  • (De) Select All button: change as discussed with Wendy Plesniak, BWH, to option button with proper logic
  • Setting color/selected color for individual Ruler and Angle widgets in property dialog
  • Fixed Bug: hiding/unhiding ruler and angle annotations
  • Change display property for each annotation individually
  • Lots of warning messages when compiling code
• 02/26/10
  • Created pop up window showing screen shot
  • Manual editing of coordinates via "Property Window"
  • Resolved column width problem in table (original problem, second screen shot problem)
  • Fixed bug where hide icon changed size in table
  • Include icons when saving html report in a directory with the same name as the html file
  • Reorganized GUI based on feedback from Wendy Plesniak, BWH
  • Include screen shots (if any) in the report (see an example)
  • Fixed the following bug: After resizing the 3D Window taking a screen shot only displays the window with the original size (see screen shot)
  • Fixed Bug: Value update of ruler widget is slow
  • Text defined in property box now shows up in table
  • Integrated vtkMRMLAnnotationAngleNode with AngleWidget
  • Added new buttons to create frame based on feedback from BWH
• 02/19/10
  • Implement a general structure for vtkMRMLAnnotation*Node
  • Work together with Kitware to resolve QT Table layout problem
  • Allow defining long text annotations for each widget
  • Improve user interaction through pop up windows when trying to modify an existing annotation
  • Creating and removing ruler now works correctly
  • Use correct label format to display measurement values in Annotation Property window
  • Created a "Save annotation" Window
  • Fixed changing icon size in table (see original screen shot )
  • Fixed slicer crashing closing the property window
  • Renamed and resized columns in Table
• 02/12/10 removed bugs related to table buttons, linked button to angle widget and ruler widget
• 02/05/10 implemented "Save Annotation Report" button and "Move up/down" buttons; added extra column in the report table for visibility; modified annotation property dialog; implemented button-click in the table.
• 01/29/10 implemented functionality for "Annotation Property Modification" button, "Create Report" button, and integrated new icons for buttons.
• 01/22/10 Set up the source control for the reporting module; implemented functionality for "Select All" button, "Visible" button, and "Delete" button in the GUI; created new icons for buttons
• 01/15/10 Organized biweekly Tcon
• 01/08/10 Participated in all week All Hands Meeting in Utah; Expanded QT interface with functionality to set seeds in images; Defined a final MRML Structure for storing annotation.
• 01/01/10 Generated a GUI interface via QT for the annotation module. Reviewed all VTK Widgets related to annotation.
• 12/25/09 Created project and organized break out session at AHM meeting. Coordinated efforts with BWH team in preparation of the meeting.
• 12/18/09 Held workshop to integrate AMI into 3D Slicer. Hired new person to implement AMI in Slicer.
• 12/11/09 Summary of tools demoed at RSNA
• 12/04/09 Visited RSNA to review annotation tools by GE, Siemens & Phillips. Connected to caBIG AIM project to see how we can make use of their data scheme
• 11/29/09 Created GUI for MarkUp module
• 11/20/09 Design MRML Structure of Annotation and MarkUp Module
• 11/13/09 Organize Annotation Brain Storming Session
• 11/06/09 Designed User Interface , Meet with Julien Finet and Jean-Christophe Fillion-Robin from Kitware to discuss integration of Qt in 3D Slicer
• 10/30/09 Participated in Qt-Tcon, Interviewed candidate at Almaden, coordinated efforts with Nicole Aucoin
• 10/23/09 Organized onsite interview , got in contact with Steve Pieper to discuss next steps, installed Slicer3
• 10/17/09 Started interviewing postdoc as well as solving several HR issues for hiring personal

Feature Request

VTK Widgets

• For further detail go here

QT Related

• Submit bugs and feature request via Mantis
• Feature requests that may open discussions can be added here

Daniel

All features/issues are listed here: http://www.na-mic.org/Wiki/index.php/Projects:ARRA:miAnnotation:PriorityList

Known issues

• MRML
  • After loading of scene, the annotations are not displayed in the hierarchy tree anymore
  • Load scene does not display 3D widgets, Import scene does
  • After importing or load, the user has to click on the widget to enable the connection between tree entry and widget
• Annotations
  • Property dialog does not work yet and might crash
  • Pick mode does not work correctly, if placement was paused and resumed (the first picked coordinate gets lost)
  • Screenshot imagedata does not get saved on Scene save
• GUI
  • if moving an annotation up or down inside a hierarchy, the tree gets confused
  • Drag and drop does not work
  • after creating hierarchies, the selected item does not reflect the hierarchy which new annotations get added to
  • save buttons not implemented
  • widget intersections do not work
  • Slicer4 crashes on exit after annotations were placed and the scene was not closed manually prior exiting Slicer

Feedback by Users

• Ron

1. I sometime like the fiducials to be spheres, not crosses. In Slicer 3 we had a whole list of options. Can I have them for Slicer4? Can I also change their size?
2. The ruler has different colors in the 3D view and in the slice views. That is wrong.
3. The properties popup require a lot of clicking, including double click. Usability will need to be optimized.
4. No drag and drop in the module.
5. text annotation: arrow in the slice view, but not in the 3d view (see attached)
6. text is not in the box in the slice view
7. the font is butt-ugly. In addition, part of the letter n is cut off by the t. The space between the capital V and the e is too big
8. Slicer 3 let you jump to a fiducial location by right clicking in the list in the module.
9. Also, The ruler does have issues with visibility, when the slice is rendered as well. See attached screensnapshot: little dots to the left, no dots to the right.

• Isaiah

1) need "selected" parameter for each fiducial, separate from visibility. It's used as boolean inclusion criteria in several modules - in my case DTI FiducialSeeding but also some other ones like segmentation and for CLI modules.
2) need to set properties for groups of fiducials at once. This is tricky because the hierarchy is generic, but ability to set all parameters for a group of fiducials in one place is important.
3) meta visibility control is helpful but maybe not imperative, ie show/hide a whole group but still maintain the individual visibility settings.
4) flicker of fiducials in 2d-view slice whenever other fiducial is moved across that slice. I have two coplanar fiducials in coronal slice, and move a third back/forth across the axial slice. Each time the third crosses the coronal there is a noticeable flicker of the two fiducials in-plane.
5) positioning with decimal precision rather than integer?
6) the persistent-drag mouse interaction in both 2d and 3d is not ideal (at least to me).
7) deleting a hierarchy with ~30 fiducials incurs a noticeable delay around 7s. I compiled with the standard (debug mode?) instructions on x86, ubuntu 10.04.
8) there are some strange mouse interactions when placing in the 3d view after clicking the "cancel" or "accept"  buttons. for example, the first time I create a list it works fine: one click to place each. However, if I stop, delete, and start adding again then it doesn't work the same way: the first click makes a fiducial, but subsequent clicks only occasionally place new fidus. I think the problem is that the interactor is selecting the new first fiducial even if the mouse is very far away - can click on the other side of the screen and drag that fiducial, even after pressing "stop" again.
8a) minor style: is it possible to use stop-sign or something else rather than the current x-in-circle for "cancel"?... my brain sees that as the fiducial button 1/2 the time :)
9) I'm confused by the edit properties display (window) for fiducials. What is the purpose for each tab: text, points, lines? points seems to be a place for a list, but x/y/z is empty and position is instead displayed under "Measurement value" on the text tab.

Meeting with Ron 05/06/11

Nicole and Daniel met with Ron to discuss usability issues and future development.

• Ron would like the 3d sphere to be the default setting for the glyphs. (H)
• Change the mouse cursor when in place mode, use the images from the mouse modes toolbar icons. (H)
• Text annotations
  • line breaks for long text, automatic (H) - breaks are done, not automatic yet
  • show some of the text (first 20chars?) in the fiducial text box with ... to denote more visible in the Qt interface (M)
  • sometimes don't want an arrow head (M)
  • consolidate the text and fiducials into one conceptual package allowing turning on/off the text box, detaching it, leader, etc. (L)
  • get the text working in 2d slice windows (H)
• Set one time place as the default in the annotation module (add a persistent radio button?) (H)
• Rulers
  • expose the tick mark options in the GUI (need to support mice, humans, galaxies) (L)
  • use the NeuroDTI example data set from Ron's drop box (H)
  • 3d tumor model when measuring it's diameter, the measurement text is hidden inside of it, add an option to position the text at and end or in the middle (H) label position parameter exposed
  • toggle label visibility (M)
• Bidim
  • does RECIST need to be orthogonal? Yes
  • add the word orthogonal to the tooltip (done)
  • make a new icon with a quarter circle between two arms of the axes and use it (L)
• Reformat widget
  • estimated time to implementation? (email sent to Will S.)
• Save
  • Ron vetoed the Save panel in the Annotation module, all saving is to be done via the centralised widget (H)
  • take out the save/load buttons, move the report buttons to another part of the GUI (H)
  • add the screen snapshot functionality (especially the over sampling) to the scene views pop up window and bring it up from the shortcut box in the lower left of the GUI (this may need some infrastructure changes) (H)
• Workflows
  • use the DTI Tumor scene as a sample (M)
  • when Ron created the views, each scene view is a workflow end point
  • once the annotation functionality is all there, Nicole will meet with Ron and add the annotations to the scene view and save a new scene, and also document the process for usability improvements and testing (M)
• ROIs
  • see the Slicer3 crop volume module for a good initialization method: make the ROI node and then update it by clicking in the 2d slice (M)
  • 2d annotation ROI -> label map (H)
  • match handle colours of the widget to 2d elements in the GUI (H)
  • implement the fit ROI to volume functionality (L)
    • once it is included in volume rendering in Slicer 4 please take a look at this functionality - if it is easy to import than it has high priority
• Scene views (M)
  • test the Neuro DTI file on Slicer4
  • create a new Slicer4 Annotation Testing mrml file and upload it to the drop box

Display Properties of Widgets

• Unavailable / Unknown:
  • Set text colors for Ruler and Angle Widget
  • Get/Set glyph scale / point size for point display node
  • Get/Set text opacity/ambient/diffuse/specular from text display node
  • Color of "angle line" cannot be defined
• Available:
  • Get/Set text colors from display node
  • Get/Set text scale from display node
  • Get/Set point colors from point display node
  • Get/Set point opacity/ambient/diffuse/specular from point display node
  • Get/Set line colors from line display node
  • Get/Set line width/opacity/ambient/diffuse/specular from line display node
  • Get text format for angle and ruler