Difference between revisions of "2016 Winter Project Week/Projects/TrackedUltrasoundStandardization"

From NAMIC Wiki
Jump to: navigation, search
 
(40 intermediate revisions by 7 users not shown)
Line 1: Line 1:
−
__NOTOC__
 
 
<gallery>
 
<gallery>
 
Image:PW-MIT2016.png|link=2016_Winter_Project_Week#Projects|[[2016_Winter_Project_Week#Projects|Projects List]]
 
Image:PW-MIT2016.png|link=2016_Winter_Project_Week#Projects|[[2016_Winter_Project_Week#Projects|Projects List]]
Line 5: Line 4:
  
 
==Key Investigators==
 
==Key Investigators==
 +
[[Image:UltrasoundToolGuidance3dOnly.gif|right]]
 
* Andras Lasso, Tamas Ungi (PerkLab, Queen's University)
 
* Andras Lasso, Tamas Ungi (PerkLab, Queen's University)
 
* Christian Askeland, Ingerid Reinertsen, Ole Vegard Solberg (CustusX, IGT research, SINTEF)
 
* Christian Askeland, Ingerid Reinertsen, Ole Vegard Solberg (CustusX, IGT research, SINTEF)
Line 15: Line 15:
 
==Project Description==
 
==Project Description==
 
{| class="wikitable"
 
{| class="wikitable"
 +
! style="text-align: left; width:27%" |  Objective
 +
! style="text-align: left; width:27%" |  Approach and Plan
 +
! style="text-align: left; width:27%" |  Progress and Next Steps
 
|- style="vertical-align:top;"
 
|- style="vertical-align:top;"
−
! style="text-align: left; width:27%" | Objective
+
|
 
* Establish a common software platform for tracked ultrasound and image-guided interventions by converging existing IGT toolkits:
 
* Establish a common software platform for tracked ultrasound and image-guided interventions by converging existing IGT toolkits:
 
** [http://www.slicer.org 3D Slicer], [http://www.slicerigt.org SlicerIGT extension]
 
** [http://www.slicer.org 3D Slicer], [http://www.slicerigt.org SlicerIGT extension]
 
** [http://www.plustoolkit.org Plus toolkit]
 
** [http://www.plustoolkit.org Plus toolkit]
 
** [http://custusx.org/ CustusX]
 
** [http://custusx.org/ CustusX]
−
** [http://www.bic.mni.mcgill.ca/ServicesSoftwareVisualization/IBIS IBIS]
+
** [http://www.bic.mni.mcgill.ca/~sdrouin/Ibis IBIS]
 
** [http://www.imaging.robarts.ca/petergrp/Technology Echelon]
 
** [http://www.imaging.robarts.ca/petergrp/Technology Echelon]
 
** [http://mitk.org/wiki/IGT MITK-IGT]
 
** [http://mitk.org/wiki/IGT MITK-IGT]
−
! style="text-align: left; width:27%" | Approach, Plan
+
|
 
* Pre-Project Week online meetings and emails
 
* Pre-Project Week online meetings and emails
 +
** Notes below
 +
** SINTEF design document [https://github.com/SINTEFMedtek/CustusX/wiki/OpenIGTLink-US-messaging-proposal]
 
* Coordination with the CommonTK effort
 
* Coordination with the CommonTK effort
 
* Survey of current implementations and use cases
 
* Survey of current implementations and use cases
 
* Development of long-term development practices to encourage collaboration
 
* Development of long-term development practices to encourage collaboration
−
!! style="text-align: left; width:27%" | Progress
+
|
−
*
+
* IGSIO [https://github.com/igsio]
 +
* Preparation meeting minutes [[2016_Winter_Project_Week:TrackedUltrasoundStandardization/PreparationMeetings|[0]]]
 +
* Live meeting minutes [https://docs.google.com/document/d/1jOHkFQJ5-GPyT3b9XHBJ5rnVkNOQQ2Q6G6evTfnl-CU/edit?usp=sharing]
 +
|}
 +
 
 +
== OpenIGTLink Revisions ==
 +
v3 proposal document [http://openigtlink.org/protocols/v3_proposal.html]
 +
=== Tracked ultrasound OpenIGTLink client ===
 +
* Move out part of OpenIGTLinkIF
 +
* Only use OpenIGTLink and VTK
 +
 
 +
====Commands====
 +
=====SetDeviceParameters=====
 +
<b>Message</b>
 +
<Command>
 +
  <Parameter Name=”Depth” Value=”45” />
 +
  <Parameter Name=”Gain” Value=”35” />
 +
</Command>
  
−
|}
+
<b>Response 1</b>
 +
<Command>
 +
  <Result>SetDeviceParameters: success</Result>
 +
  <Parameter Name=”Depth” Value=”45” />
 +
  <Parameter Name=”Gain” Value=”35” />
 +
  ...
 +
</Command>
  
−
<h3>Preparation meetings</h3>
+
<b>Response 2</b>
 +
<Command>
 +
  <Result>SetDeviceParameters: failure</Result> <!-- see command status error code -->
 +
</Command>
  
−
<h4>Christian's list of potential topics of collaboration</h4>
+
=====GetDeviceParameters=====
 +
<b>Message</b>
 +
<Command>
 +
  <Parameter Name=”Depth” />
 +
  <Parameter Name=”Gain” />
 +
  ...
 +
</Command>
  
−
''Enhance connection of CustusX client to PLUS server''
+
<b>Response</b>
 +
<Command>
 +
  <Result>GetDeviceParameters: success</Result>
 +
  <Parameter Name=”Depth” Value=”45” />
 +
  <Parameter Name=”Gain” Value=”35” />
 +
  ...
 +
</Command>
  
−
Background: CustusX has used PLUS to connect to an Ultrasonix scanner with an attached tracking device from Ascension (Sonix GPS), as part of the EU RASimAs project. CustusX also has tracking support from the IGSTK toolkit, and own direct support for a number of video/US devices. We would like to add PLUS to this family, poosibly replacing some of our older components.
+
=====SubscribeDeviceParameters=====
 +
This will cause the server to send a GetDeviceParameters reply every time any of the subscribed parameters change.
  
−
Topics:
+
<b>Message</b>
−
* Tracking of BrainLab devices.
+
<Command>
−
* Connection to BK medical scanners.
+
  <Parameter Name=”Depth” Subscribe=”TRUE” />
−
* Enhance transfer of US probe metadata. Might include extending the OpenIGTLink standard.
+
  <Parameter Name=”Gain” Subscribe=”FALSE” />
−
* Use of PLUS US reconstruction algorithms in CustusX as an alternative to CX internal algorithms, and enable comparison of algorithms.
+
  ...
 +
</Command>
  
−
''Standardization of IGT software''
+
<b>Response</b>
 +
<Command>
 +
  <Result>SubscribeDeviceParameters: success</Result>
 +
  <Parameter Name=”Depth” Subscribed="TRUE" />
 +
  <Parameter Name=”Gain” Subscribed="FALSE" />
 +
  ...
 +
</Command>
  
−
Background:
+
=====GetCapabilities=====
−
* Emails, major players converge towards IGTLink. Several groups are using PLUS and/or Slicer, but also open-sourcing their own software and want to be compatible with each other.
+
SINTEF will specify and implement.
−
* Data should be usable by several systems in order to ease comparison in e.g. studies.
 
−
* Our group need to communicate models and volumes with Slicer, and would like to use PLUS.
 
  
−
There are several areas where standardization would be beneficial:
+
<b>Message</b>
−
* messaging between applications.
+
<Command></Command>
−
* file formats.
 
−
* algorithms/shared libraries.
 
−
It is important to build as much as possible on existing material and de facto standards.
 
  
−
Topics:
+
<b>Response</b>
−
* Messaging: OpenIGTLink is the de facto standard for messaging. Is the standard complete? What do we need? US-probe data, polydata colors, volume transfer functions /LUTs.
+
<Command>
−
* File Formats:  Models, Volumes, Video streams, US metadata, scenegraph. Some of these objects are already somewhat standardized, but not all.
+
  <Result>GetCapabilities: success</Result>
−
* Algorithms: A library of IGT algorithms would be welcome. It might be included into an existing library (VTK, CTK, …) or emerge as a new one.
+
  <UltrasoundCapabilities>
 +
    <Probes>
 +
      <Probe name="L14-5/38"/>
 +
      <Probe name="C5-2/42"/>
 +
    </Probes>
 +
    <ImagingModes>
 +
      <ImagingMode Name=”b-mode+angio”>B-Mode,Angio</ImagingMode>
 +
    </ImagingModes>
 +
    <Streams>
 +
      <Stream name="B-Mode">
 +
        <Parameters>
 +
          <Parameter name="depth" min="5" max="220" step="5"/>
 +
        </Parameters>
 +
      </Stream>
 +
      <Stream name="Angio">
 +
        <Parameters>
 +
          <Parameter name="depth" min="5" max="220" step="5"/>
 +
        </Parameters>
 +
      </Stream>
 +
    </Streams>
 +
    <Presets>
 +
      <Preset name="Vascular small object B+Angio">
 +
        <Probe Name=”L14-5/38” />
 +
        <Parameter Name=”Depth” Value=”50” />
 +
      </Preset>
 +
    </Presets>
 +
  </UltrasoundCapabilities>
 +
</Command>
 +
===Tracking===
 +
Tracked ultrasound communication will use the TDATA message.
  
−
<h4>Meeting 2015-11-23</h4>
+
Use 1 bit of the Reserved byte to indicate valid/invalid status
  
−
Participants: Andras Lasso, Tamas Ungi, Steve Pieper, Junichi Tokuda, Adam Rankin, Simon Drouin, Ole Vegard Solberg, Ingerid Reinertsen, Christian Askeland
+
Use metadata attributes for tool status information:
  
−
Notes:
+
====Attributes====
−
* Introduction of all present. Everyone seems to have a strong interest in IGT and systems integration.
+
<ToolStatus>
−
* Junichi: Funded OpenIGTLink grant scope is well aligned with this standardization effort.
+
  <Tool Id=”Reference” Status=”WARNING” Description=”Marker 1 on tool X is damaged” />
−
* Christian: project manager for CustusX, Plus is used for data acquisition, 3D Slicer used for pre/post operative processing
+
</ToolStatus>
−
* Ingrid: research scientist, IGT and CustusX user, uses Slicer for pre-processing, segmentation, registration, post-processing; clinicians use Slicer for tumor segmentation, becoming a routine
 
−
* Simon: IBIS has been developed for 10 years, next step is integration with Plus, plan is to move features to 3D Slicer/SlicerIGT/Plus and use that as a platform (grant application submitted for getting funding)
 
−
* Walkthrough of (some of) Christian’s topics
 
−
** CustusX - BrainLab connection through PLUS: BrainLab already supports OpenIGTLink, connection either through Plus or directly to the application should be straightforward.
 
−
** CustusX - BK connection through PLUS: Partially implemented in PLUS. NOTE: Must buy additional license from BK to access continuous image streaming (GRAB ON/OFF command). Without that the maximum acquisition rate is limited to a few FPS. GRAB ON/OFF interface implementation is partially complete in Plus. If a BK machine is available that supports this command then implementation in Plus requires a couple of days.
 
−
** 4D (3D+t) US acquisition support: Robarts has access to a private Philips API, SINTEF has access to a private GE API. Both have converter to OpenIGTLink (Philips converter is implemented in Plus, GE is external).
 
−
** Ultrasound scanner two-way communication support in OpenIGTLink (data query, device control - depth, imaging mode): several of those present (Adam, Christian, ..) and others need this, and several slightly different implementations exist (Plus, CustusX, MUSiiC). Junichi agrees that this should be standardized into OpenIGTLink. We need a backward and forward compatible solution, possibly XML in STRING OpenIGTLink message. '''Needs further discussion'''
 
−
** Shared file formats: All IGT data (pose tracking, 2D+t, 3D, 3D+t images) should be stored in a common format. '''Needs further discussion'''
 
−
*** Steve suggests DICOM. Some work already in progress on using DICOM as interchange format between BrainLab and Slicer (BrainLab already uses DICOM as its internal data representation). Most of the required structures are present, possibly excluding tracking positions. This can be (with time and work) added to the standard. General agreement on DICOM, but more discussion needed. Real-time streaming to DICOM file may be problematic (DICOM header has to be prepended to image data; if image data is several GB then it may take long time, not convenient for intra-operative use).
 
−
*** Can also discuss with Michael Onken/DCMTK when in Boston.
 
−
* General agreement that standardization and cooperation is needed, and that more discussions are needed before Boston.
 
  
−
TODO:
+
* Status
−
* Contact others if they were interested to join these efforts (MUSiiC, MITK-IGT, Nifty-IGT) - Andras
+
** "OK"
−
* Ask Tina about possibility for Amigo visit (BK ultrasound, BrainLab integration) - Andras
+
** "WARNING"
−
* Talk to BrainLab engineers about tracked ultrasound (and general IGT) information storage in DICOM format - Steve
+
** "ERROR"
 +
* Description: human-readable string
  
−
'''Next meeting''': December 7., 0900 EST (1500 Trondheim time)
+
===Meta Data===
 +
====Header V1====
 +
Bytes
 +
0  2                      14                                      34              42              50              58
 +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 +
| V | TYPE                  | DEVICE_NAME                          | TIME_STAMP    | BODY_SIZE    | CRC64        |
 +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  
−
<h4>Meeting 2015-12-07</h4>
+
====Header V2====
 +
<nowiki>#</nowiki>Bytes HEADER_FIELD
 +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
 +
0 V 2 TYPE 14 DEV_NAME 34 TS 42 BODY_SIZE 50 CRC64 58 EXT_HEADER_SIZE 60 METADATA_SIZE 64 MSG_ID 68 RV 72 CONTENT… | METADATA… |
 +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
 +
* V = VERSION
 +
* DEV_NAME = DEVICE_NAME
 +
* RV = RESERVED
  
−
Date: December 7., 0900 EST (1500 Trondheim/Heidelberg time)
+
=====EXT_HEADER=====
 +
v1 header (58 bytes) + new header (EXT_HEADER_SIZE bytes)
  
−
Participants: Andras Lasso, Tamas Ungi, Steve Pieper, Junichi Tokuda, Adam Rankin, Simon Drouin, Ole Vegard Solberg, Christian Askeland, Alfred Franz, Thomas Kirchner, Janek Groehl
+
=====CONTENT (=v1 body)=====
 +
BODY_SIZE - EXT_HEADER_SIZE - METADATA_SIZE
  
−
'''Topics of interests'''
+
=====METADATA=====
−
* Tracking and imaging data stream (image includes distance maps, various color images, etc):
+
<nowiki>#</nowiki>Bytes HEADER_FIELD
−
** Storage file format
+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
−
*** MetaImage (mha/mhd), NRRD [http://perk-software.cs.queensu.ca/plus/doc/nightly/user/FileSequenceMetafile.html with custom fields]
+
|INDEX_COUNT (uint16) | KEY_SIZE (uint16) | VALUE_ENCODING (uint16) | VALUE_SIZE (uint32) | … | KEY | VALUE | … |
−
**** Very simple, flexible, human-readable file format
+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
−
**** Lossless compression, special image types (4D, color, etc.) are supported
 
−
**** Basic support is available in all research software, C++ and Matlab libraries are available for reading/writing
 
−
**** Reading/writing is fully supported by [http://wwww.plustoolkit.org Plus], [http://www.slicer.org 3DSlicer]/[http://www.slicerigt.org SlicerIGT]/[https://www.slicer.org/slicerWiki/index.php/Documentation/Nightly/Extensions/Sequences Sequences]
 
−
**** Limitations: only one image stream per file, no per-frame fields (simulated by adding per-file custom fields; standard libraries cannot associate them with frames); numbers stored as text, some precision is lost
 
−
*** DICOM: Ultrasound image IOD, Ultrasound multi-frame image IOD, Ehanced ultrasound volume IOD
 
−
**** [http://dicom.nema.org/Dicom/2013/output/chtml/part17/sect_PP.2.html#sect_PP.2.2 use cases], [http://dicom.nema.org/medical/dicom/2014c/output/chtml/part17/sect_PP.3.html overview]
 
−
**** Standard exists for tracked 2D, 3D ultrasound sequences (frame of reference relative to probe, patient, other)
 
−
**** Physiological waveforms (cardiac, respiratory)
 
−
**** Physical units, real world values (speed, etc), color calibration
 
−
**** Custom per-frame fields can be specified (could be added to standard)
 
−
**** Multiple image regions (biplane, etc)
 
−
**** Limitation: no standard for storing navigation data (only probe and image position) - to be confirmed with David Clunie
 
−
** Storage in memory for visualization and processing
 
−
*** Plus: rotating buffer for image frames (vtkImageData + custom fields), tool tracking data (pose, status); TrackedFrameList (list of image data with string field list)
 
−
*** 3D Slicer: sequences of data nodes (vtkMRMLScalarVolume, vtkTransform)
 
−
** Real-time transfer: colors, volume transfer functions / LUTs?
 
−
* Ultrasound metadata: imaging parameters (depth, dynamic range, etc.), image geometry (clipping rectangle, clipping fan, image spacing, axis directions, transducer center position), freeze state, probe
 
−
* Surgical microscope metada: focus, zoom
 
−
* Calibration info: from/to coordinate system name, linear transform
 
−
** File storage: In Plus: XML, CoordinateDefinitions element
 
−
** In memory: vtkMatrix4x4, TransformRepository
 
−
** Real-time transfer: application receives as TRANSFORM message, set by OpenIGTLink command
 
−
* Meshes, point cloud
 
−
** File storage: vtkPolyData, VTK file formats are OK but there is no coordinate system information; DICOM?
 
−
** Memory: VTK class
 
−
** Real-time transfer: is anything missing in OpenIGTLink?
 
−
* Dependency, data provenance
 
−
** Data represenation for segmentation, keeping relationship to source structures in DICOM
 
−
** Make sure there is a way to store DICOM UIDs, use definitions that are already specified in DICOM
 
−
button press
 
−
** File storage: currently in XML in Plus
 
−
** In memory storage: XML data elements
 
−
** Real-time transfer: MUSiiC custom OpenIGTLink messages have some info
 
−
* Scene:
 
−
** File storage: Slicer uses MRML - alternatives?
 
−
** Memory: MRML library
 
−
** Real-time transfer?
 
−
* Common algorithms:
 
−
** Pivot calibration (with robust point matching, error rejection, error metrics, etc.)
 
−
** Landmark registration (with automatic landmark detection, etc.)
 
−
** Volume reconstruction
 
−
** Scan conversion, brightness conversion
 
−
** Bone surface detection
 
−
** Ultrasound simulation
 
−
** US/MRI registration
 
−
** Custom reader/writer classes for tracked image data (supporting compressed streaming, compressed seeking, all necessary metadata, etc.)
 
  
−
'''Current implementations'''
+
=====VALUE_ENCODING=====
−
* Tracked ultrasound storage in file:
+
As specified at http://www.iana.org/assignments/character-sets/character-sets.xhtml
−
** Plus: sequence metafile
 
−
** CustusX:
 
−
*** Saving during recording: a new folder is created, each image frame is written to a separate file (optionally compressed), separate file stores tracking data and metadata
 
−
*** Sequence metafile (MetaImage format with Plus custom fields) is also used
 
−
*** Application state is saved to an XML file (similar to MRML scene) that links to other files (sequence metafile, VTK polydata, etc)
 
−
*** The entire procedure is also recorded to a separate, simple, compressed file
 
−
** IBIS:
 
−
*** Sequence of MINC files: each frame is a separate MINC file, contains both tracking transform and calibration transform that was current when the frame was acquired. All provenance information is kept in the header of each file (up to the original data source, DICOM, etc.)
 
−
** MITK-IGT:
 
−
*** There is not really a standard, images can be saved as a 3D NRRD image using ITK, tracking data can be saved as xml or csv
 
−
*** Sometimes sequence metafile is used
 
−
* Tracked ultrasound storage in memory:
 
−
** Plus: TrackedFrameList class, list of PlusVideoFrame, PlusVideoFrame is a VTK class with vtkImageData (with zero origin and unit spacing), timestamp, and custom string fields (with convenience function to store/retrieve transforms, numerical values, etc.)
 
−
** CustusX:
 
−
*** Uses VTK as main backend (with ITK converters for algorithms)
 
−
*** Custom image class that contains vtkImageData (with zero position, unit spacing), additional with orientation, in vtk image data position is 0
 
−
*** all transforms in eigen
 
−
*** sequences of video and ultrasound in a class
 
−
** IBIS:
 
−
*** vtkImageData data and separate transform vtkMatrix4x4 for individual frame and separate for registration
 
−
*** Everything is kept in memory, no streaming writing to disk
 
−
** MITK-IGT:
 
−
*** OpenCV mat image for image acquisition and pre-processing
 
−
*** MITK image sequence could be used as well
 
−
*** Calibration is a property of the MITK image
 
−
*** Tracking data: tracking data set is a separate structure
 
−
* Tracked ultrasound streaming:
 
−
** Plus:
 
−
*** Transforms: receive TRANSFORM or TDATA, send TRANSFORM (TDATA implementation is in progress)
 
−
*** Images: IMAGE, USMESSAGE (MUSiiC ultrasound message), TRACKEDFRAME (Plus tracked frame: image data + string fields)
 
−
** CustusX:
 
−
*** OpenIGTLink: has been used for 4 years, Plus for about half year
 
−
*** Image: both 3D and 4D US, using own image message type, non-standard message
 
−
*** Tracking: TRANSFORM
 
−
** IBIS: everything is in the same process, no IPC
 
−
** MITK: TDATA or TRANSFORM messages, depending on the application
 
  
−
==Plans==
+
=====BODY=====
 +
Contains extended header data, message content, and metadata.
  
−
* We focus on real-time data transfer standardization first, then file storage, in-memory representation, reader/writer and processing software library
+
==US over OpenIGTLink==
 +
===Parameters===
 +
<b>Read-write</b>
 +
* Depth (mm): (“40”, “45”, “50”, …)
 +
* ImagingMode (string): (“bmode”, “bmode+rf”, “bmode+angio”, “rf+angio”, ...)
 +
* Probe (string): probe name (“L14-5/38”)
 +
* Frequency (MHz)
 +
* DynRange (db)
 +
* Gain (%)
 +
* Power (%)
 +
* Zoom (%)
 +
* SoundVelocity (m/s)
 +
* TGC (space-separated string, each entry -1.0 to 1.0)
 +
* AcquisitionState (string): “FREEZE” or “RUN”
  
−
'''Next meeting''': December 14., 0900 EST (1500 Trondheim/Heidelberg time)
+
<b>Read-only</b>
 +
* ClipRectangleOrigin="27 27"
 +
* ClipRectangleSize="766 562"
 +
* FanAnglesDeg=”-30 30” (for 3D: FanAnglesDeg=”-30 30 -15 15”)
 +
* FanOriginPixel=”240 10”
 +
* FanRadiusStartPixel=”30”
 +
* FanRadiusStopPixel=”500”
 +
* Encoding: how to interpret values ("BRIGHTNESS", "RF_REAL", "RF_IQ_LINE", "RF_I_LINE_Q_LINE", "RGB_COLOR")
  
−
'''Homework before next meeting''': Add to this wiki page the description of all data to be transferred in real-time using OpenIGTLink
+
== Implementation plan ==
 +
=== Tasks ===
 +
* Clean up the vtkMRMLIGTLConnectorNode to be logic, MRML and Slicer free
 +
* Creating a new converter base class which does not depend on MRML
 +
* Implementation of the new Command Message
 +
=== Github Fork ===
 +
* OpenIGTLinkIF fork [https://github.com/drouin-simon/OpenIGTLinkIF]
 +
* OpenIGTLink fork [https://github.com/lassoan/OpenIGTLink]
 +
=== PLUS ===
 +
* Interson probe settings [https://www.assembla.com/spaces/plus/tickets/869-interson-probe-settings-/details]
 +
* Interson class cleanup [https://www.assembla.com/spaces/plus/tickets/897-clean-up-interson-video-source-code/details]
 +
* Get/Set TGC support for US devices [https://www.assembla.com/spaces/plus/tickets/1056-add-tgc-support-to-ultrasonix-interface/details]
 +
* Get transform over OpenIGTLink [https://www.assembla.com/spaces/plus/tickets/1054-add-get_transform-command-to-plus-server-to-support-querying-of-transforms/details]
 +
* Sonix transition to imaging parameters [https://www.assembla.com/spaces/plus/tickets/1058-convert-sonixvideo-source-to-use-new-imaging-parameters-design/details]
  
−
'''Data necessary to be transferred in real time'''
+
=== CustusX ===
 +
* Example CustusX class [https://github.com/SINTEFMedtek/CustusX/blob/master/source/resource/OpenIGTLinkUtilities/cxIGTLinkConversionImage.h]
  
−
* MITK-IGT group:
+
==Presentations==
−
** For image data:
+
* [[Media:NAMIC - IGT - Adam Rankin.pptx|Adam Rankin, Robarts Research Institute]]
−
*** Option to add meta data for different modalities (e.g: ultrasound- and photoacoustic imaging):
+
* [[Media:PerkLabTrackedUs.pdf|Andras Lasso, PerkLab, Queen's University]]
−
*** Additional metadata for PAI:
+
* [[Media:MITKTrackedProbes.pdf|Thomas Kirchner, CAI group, German Cancer Research Center]]
−
**** Wavelength (for multispectral image shots)
 
−
**** Laser pulse energy (for normalisation of signals)
 
−
**** Adding specific message types based on IMGMETA could be of use [optional/custom fields?]
 
−
** For tracking data:
 
−
*** Consistent format
 
−
*** TDATA message suits our needs
 
−
* IBIS group:
 
−
** Image data:
 
−
*** RGB + Depth images
 
−
*** synchronized stereo images
 
−
*** Meta-data for augmented reality: zoom, focus setting of microscope/endoscope/camera
 
−
*** Meta-data for US: depth, mode (b-mode, doppler, power doppler), image mask parameters
 
−
** Tracking data:
 
−
*** TDATA + Tracker tool state
 
−
*** Meta-data for augmented reality: calibration parameters:
 
−
**** Camera model type: Zhang, Heikkila, Tsai.
 
−
**** focal length
 
−
**** Image center
 
−
**** distortion coefficients (depends on model type)
 
−
*** Meta-data for US : calibration matrix ( tool to image transform )
 

Latest revision as of 18:40, 8 January 2016

Home < 2016 Winter Project Week < Projects < TrackedUltrasoundStandardization

Key Investigators

UltrasoundToolGuidance3dOnly.gif
  • Andras Lasso, Tamas Ungi (PerkLab, Queen's University)
  • Christian Askeland, Ingerid Reinertsen, Ole Vegard Solberg (CustusX, IGT research, SINTEF)
  • Simon Drouin (Mcgill University, Montreal, Canada)
  • Junichi Tokuda (BWH)
  • Steve Pieper (Isomics)
  • Adam Rankin (VASST Laboratory, Western University, Canada)
  • Thomas Kirchner, Janek Gröhl (MITK, DKFZ, Heidelberg, Germany)

Project Description

Objective Approach and Plan Progress and Next Steps
  • Pre-Project Week online meetings and emails
    • Notes below
    • SINTEF design document [1]
  • Coordination with the CommonTK effort
  • Survey of current implementations and use cases
  • Development of long-term development practices to encourage collaboration
  • IGSIO [2]
  • Preparation meeting minutes [0]
  • Live meeting minutes [3]

OpenIGTLink Revisions

v3 proposal document [4]

Tracked ultrasound OpenIGTLink client

  • Move out part of OpenIGTLinkIF
  • Only use OpenIGTLink and VTK

Commands

SetDeviceParameters

Message 

<Command>
  <Parameter Name=”Depth” Value=”45” />
  <Parameter Name=”Gain” Value=”35” />
</Command>

Response 1 

<Command>
  <Result>SetDeviceParameters: success</Result>
  <Parameter Name=”Depth” Value=”45” />
  <Parameter Name=”Gain” Value=”35” />
  ...
</Command>

Response 2 

<Command>
  <Result>SetDeviceParameters: failure</Result> 
</Command>
GetDeviceParameters

Message 

<Command>
  <Parameter Name=”Depth” />
  <Parameter Name=”Gain” />
  ...
</Command>

Response 

<Command>
  <Result>GetDeviceParameters: success</Result>
  <Parameter Name=”Depth” Value=”45” />
  <Parameter Name=”Gain” Value=”35” />
  ...
</Command>
SubscribeDeviceParameters

This will cause the server to send a GetDeviceParameters reply every time any of the subscribed parameters change.

Message 

<Command>
  <Parameter Name=”Depth” Subscribe=”TRUE” />
  <Parameter Name=”Gain” Subscribe=”FALSE” />
  ...
</Command>

Response 

<Command>
  <Result>SubscribeDeviceParameters: success</Result>
  <Parameter Name=”Depth” Subscribed="TRUE" />
  <Parameter Name=”Gain” Subscribed="FALSE" />
  ...
</Command>
GetCapabilities

SINTEF will specify and implement.

Message 

<Command></Command>

Response 

<Command>
  <Result>GetCapabilities: success</Result>
  <UltrasoundCapabilities>
    <Probes>
      <Probe name="L14-5/38"/>
      <Probe name="C5-2/42"/>
    </Probes>
    <ImagingModes>
      <ImagingMode Name=”b-mode+angio”>B-Mode,Angio</ImagingMode>
    </ImagingModes>
    <Streams>
      <Stream name="B-Mode">
        <Parameters>
          <Parameter name="depth" min="5" max="220" step="5"/>
        </Parameters>
      </Stream>
      <Stream name="Angio">
        <Parameters>
          <Parameter name="depth" min="5" max="220" step="5"/>
        </Parameters>
      </Stream>
    </Streams>
    <Presets>
      <Preset name="Vascular small object B+Angio">
        <Probe Name=”L14-5/38” />
        <Parameter Name=”Depth” Value=”50” />
      </Preset>
    </Presets>
  </UltrasoundCapabilities>
</Command>

Tracking

Tracked ultrasound communication will use the TDATA message.

Use 1 bit of the Reserved byte to indicate valid/invalid status

Use metadata attributes for tool status information:

Attributes

<ToolStatus>
  <Tool Id=”Reference” Status=”WARNING” Description=”Marker 1 on tool X is damaged” />
</ToolStatus>
  • Status
    • "OK"
    • "WARNING"
    • "ERROR"
  • Description: human-readable string

Meta Data

Header V1

Bytes
0   2                       14                                      34              42              50              58
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| V | TYPE                  | DEVICE_NAME                           | TIME_STAMP    | BODY_SIZE     | CRC64         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Header V2

#Bytes HEADER_FIELD 

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
0 V 2 TYPE 14 DEV_NAME 34 TS 42 BODY_SIZE 50 CRC64 58 EXT_HEADER_SIZE 60 METADATA_SIZE 64 MSG_ID 68 RV 72 CONTENT… | METADATA… |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
  • V = VERSION
  • DEV_NAME = DEVICE_NAME
  • RV = RESERVED
EXT_HEADER

v1 header (58 bytes) + new header (EXT_HEADER_SIZE bytes)

CONTENT (=v1 body)

BODY_SIZE - EXT_HEADER_SIZE - METADATA_SIZE

METADATA

#Bytes HEADER_FIELD 

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|INDEX_COUNT (uint16) | KEY_SIZE (uint16) | VALUE_ENCODING (uint16) | VALUE_SIZE (uint32) | … | KEY | VALUE | … |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
VALUE_ENCODING

As specified at http://www.iana.org/assignments/character-sets/character-sets.xhtml

BODY

Contains extended header data, message content, and metadata.

US over OpenIGTLink

Parameters

Read-write

  • Depth (mm): (“40”, “45”, “50”, …)
  • ImagingMode (string): (“bmode”, “bmode+rf”, “bmode+angio”, “rf+angio”, ...)
  • Probe (string): probe name (“L14-5/38”)
  • Frequency (MHz)
  • DynRange (db)
  • Gain (%)
  • Power (%)
  • Zoom (%)
  • SoundVelocity (m/s)
  • TGC (space-separated string, each entry -1.0 to 1.0)
  • AcquisitionState (string): “FREEZE” or “RUN”

Read-only

  • ClipRectangleOrigin="27 27"
  • ClipRectangleSize="766 562"
  • FanAnglesDeg=”-30 30” (for 3D: FanAnglesDeg=”-30 30 -15 15”)
  • FanOriginPixel=”240 10”
  • FanRadiusStartPixel=”30”
  • FanRadiusStopPixel=”500”
  • Encoding: how to interpret values ("BRIGHTNESS", "RF_REAL", "RF_IQ_LINE", "RF_I_LINE_Q_LINE", "RGB_COLOR")

Implementation plan

Tasks

  • Clean up the vtkMRMLIGTLConnectorNode to be logic, MRML and Slicer free
  • Creating a new converter base class which does not depend on MRML
  • Implementation of the new Command Message

Github Fork

  • OpenIGTLinkIF fork [5]
  • OpenIGTLink fork [6]

PLUS

  • Interson probe settings [7]
  • Interson class cleanup [8]
  • Get/Set TGC support for US devices [9]
  • Get transform over OpenIGTLink [10]
  • Sonix transition to imaging parameters [11]

CustusX

  • Example CustusX class [12]

Presentations

Retrieved from "https://www.na-mic.org/w/index.php?title=2016_Winter_Project_Week/Projects/TrackedUltrasoundStandardization&oldid=92079"