OpenIGTLink/Discussion/NewCommands

What's new in OpenIGTLink Protocol version 2 and Library Version 2

• Better support for IGS Systems
  • New IGS specific message types
• Matlab interface support
  • Remote Matlab command execution
  • Matlab interface library
  • IGS support in Matlab
• Other new messages
  • Associative Array message
  • NIfTI support ?

Timeline for V.2 Protocol Release

• Events:
• June 20 - June 26: NA-MIC Summer Project Week in Boston
  • June 22, 10:30 - : OpenIGTLink Update Presentation by Junichi Tokuda
  • June 22, 16:00 - : OpenIGTLink User Group Meeting
  • Release v.2.0!!
• Notes:
  • Because of the software release schedule, we will develop protocol for IGS and protocol for others separately.

Week	4/11 - 4/17	4/18 - 4/24	4/25 - 5/1	5/2 - 5/8	5/9 - 5/15	5/16 - 5/22	5/23 - 5/29	5/30 - 6/5	6/6 - 6/12	6/13 - 6/19	6/20 - 6/26
Events											NA-MIC Project Week
Protocol Draft [IGS]	Review	Review, Freeze
Protocol Draft [other]					Draft	Review	Review	Review			Review, Freeze
IGS Dev (Alexander)	Dev	Dev	Dev	Test	Test	Test
Library (Junichi)		Dev (IGS)	Dev (IGS)	Dev (IGS)	Test (IGS)	Test (IGS)	Dev (other)	Dev (other)	Test (other)	Test (other)
Matlab IF (Junichi)					Dev	Dev	Dev	Test
3D Slicer (OIGTL IF) (Junichi, Haiying)						Dev			Test	Test
3D Slicer (IGS Module) (Haiying)						Dev	Dev	Dev	Test	Test

Common GET_LIST message ?

A common GET_LIST message could be defined with the data type, like IMAGE, in the device name field. The answer could be a LIST message with ids (each 20 bytes, so they can be used as device names) of available data. Afterwards a GET_xy message with the id could be used to query the details.

Well, I don't think this will help much in practice.

Example: an application wants to show a list of available images before querying them from the server. Ids are not very specific, so an additional meta data message must be defined. So we have a) GET_LIST and b) for each id: GET_IMGMETA. It would be easier to get all the meta data at once.

Next example: all ficucial points shall be queried. A two-step mechanism would result in 1+n queries, whereas getting all points at once would end in only 1 query. Even if the user would like to get only 1 specific point, how could he know which Id to use?

Another reason is that sending all the data at once is much more efficient.

However, there might be use cases, where single items are useful. If the item size is constant, it makes no difference if only one or several items are transmitted. The number is bodySize/itemSize. Using this definion, single items can still be pushed over the network as it's currently done in most OpenIGTLink applications.

Comments

Junichi: I agree that generalized GET_LIST message is not necessary in many case. I would suggest to design GET_IMGMETA to allow requesting either list of metadata or metadata for a specific image. If there is a way to request meta data for a specific image, we can provide two-step approach (GET_LIST->GET_IMGMETA) in the future.

Alexander: Sure, the Id can be entered in device name field. If device name field is empty, all image meta data is returned. If Id is entered, only one image meta data is returned.

GET messages

Get messages are not well defined yet. The following things should be fixed:

* If 0 items are available, the bodysize of the answer is 0.
* If an error occurs while processing the request, a STATUS message shall be returned.

Comments

Junichi: The STATUS message defines status code 4 "Not found (file, configuration, device etc)", which can be used to tell 0 items are available. The problem is that it is not possible to specify the device type in the STATUS message. The requesting host (the host that issues GET_* message) may not be able to identify which GET_* message is associated with the received STATUS message, because the OpenIGTLink message allows having different devices with the same device name. One possible solution is to use "Status name" field in STATUS message for specifying device name. For example:

• Host A requests image (device type: "IMAGE", device name: "diffusion 1") to host B by sending GET_IMAGE message.
• Host B receives the GET_IMAGE message, but it does not have such image.
• Host B sends STATUS message with device type "STATUS", device name "diffusion 1", status code 4, and status message "NO IMAGE".

Fortunately, maximum length of the status name field is 20, longer than the maximum length of device name.

Alexander: Well ok, if 0 items are available, a STATUS msg can be returned, but I would leave the error name as it is already defined - as error name. Imagine you have a GET message with type, device name and some other parameters. You send several of these messages with equal type and device name, but other parameters. You have to associate the STATUS messages... We can avoid this by defining the following:

• A GET message shall be answered by exactly one answer message.
• The answer messages shall be returned in the same sequence as the GET messages were sent.

However, the STATUS message has a device name field. I would write into that field the type name of the GET message, e.g. IMAGE, because this is the "toplevel" information. What do you think?

Image data

I would like to introduce a GET_IMGMETA message. This get-message has no parameters. The answer is IMGMETA:

Data	Type	Description
Name/Description	char[64]	Name or description of the image
Id	char[20]	Id to query the IMAGE and COLORT
Modality	char[32]	String which specifies the modality
Patient name	char[64]	Name of the patient
Patient id	char[64]	Id of the patient
Timestamp	64 bit unsigned	Scan time, see OpenIGTLink/Timestamp
RI, RJ, RK	16 bit unsigned	Number of pixels in each direction (same as in IMAGE)
S	8 bit unsigned	Scalar type (e.g. 3:uint8, 5:uint16, same as in IMAGE)

* More than one item can be transmitted. The number is bodySize/itemSize.
* To get the IMAGE, GET_IMAGE is used with the Id in the device name field.
* To get the COLORTABLE, GET_COLORT is used with the Id in the device name field. See Standard OpenIGTLink Protocol description -- Version 2 draft for details.

Comments

Junichi:

• I would define time stamp as 64 bit unsigned int so that we can also specify time. (this is also used in the OpenIGTLink header. see OpenIGTLink/Timestamp.)
• If a device name is specified in GET_IMGMETA, only one set of meta data for the image with specified device name is returned. (This allows two-step approach (GET_LIST->GET_IMGMETA), while supporting multiple sets of meta data in a single IMGMETA message.)
• I would call the first field "Image description"

Alexander:

• Agree to your first point. I have adapted the table above.
• Agree to your second point.
• Regarding your third point: let's call it "Name or description".

Point or fiducal data

Currently only transformations can be sent with OpenIGTLink, but especially points like fiducals or trajectories consist of more than a transformation.

GET_POINT and GET_TRAJ are used to get the point data. They have no parameters. The answer is a POINT or TRAJECTORY message:

POINT

Data	Type	Description
Name	char[64]	Name or description of the point
Group name	char[32]	Can be "Labeled Point", "Landmark", "Fiducal", ...
R,G,B,A	8 bit unsigned	Color in RGBA
X,Y,Z	32 bit float	Coordinate of the point
Radius	32 bit float	Radius of the point, can be 0.
Owner image	char[20]	Id of the owner image/sliceset. Points from different slicesets can be sent if slicesets are fused.

TRAJECTORY

Data	Type	Description
Name	char[64]	Name or description of the trajectory.
Group name	char[32]	Can be "Trajectory", ...
Type	8 bit unsigned	1: trajectory with only entry point, 2: trajectory with only target point, 3: trajectory with entry and target point
R,G,B,A	8 bit unsigned	Color in RGBA
X1,Y1,Z1	32 bit float	Entry point of the trajectory
X2,Y2,Z2	32 bit float	Target point of a trajectory
Diameter	32 bit float	Diameter of trajectory, can b 0
Owner image	char[20]	Id of the owner image/sliceset. Trajectories from different slicesets can be sent if slicesets are fused.

* More than one item can be transmitted if device name field is empty. The number is bodySize/itemSize.

Comments

Junichi:

• For diameter, I think It's OK to allow value greater than 0 for single point.
• Do we really need group name?
• Is it possible to have TRAJECTORY type independently? I know POINTS and TRAJECTORY are very similar, but a bit confusing for those who new to the protocol.

Alexander:

• Yes, points can have a diameter/radius.
• Group name: Yes, I think so. With the group name it is possible to distinguish e.g. between pre-op and intra-on landmarks, etc.
• If you really think it would be better to have two queries, I'll accept that. In the past we already had two messages. I was not sure if it is better to combine these messages or not. However, I have apapted the table above. If anyone else likes to have one message, please comment!

Start / stop push

Currently there is no way to start or stop a pushing request. The message START_PUSH and STOP_PUSH shall be used for that purpose. The device name field shall contain the data type to be pushed. Everytime this data changes, new messages shall be sent until the pushing is stopped. The body of START_PUSH and STOP_PUSH can contain data specific arguments.

START_PUSH

Data	Type	Description
Max Resolution	32 bit unsigned	Maximum update time in ms. Use 0 for as fast as possible. E.g. if data changes with 20 Hz, 20 messages per second will be sent.
Data specific arguments	char[bodysize - 4]	Can be everything, depends on the specific data type.

STOP_PUSH

Data	Type	Description
Data specific arguments	char[bodysize - 4]	Can be everything, depends on the specific data type.

Comments

Junichi:

• possible fields in START_PUSH message:
  • device type (TRACKINGDATA, TRANSFORM, IMAGE)
  • maximum time resolution (Hz)

Alexander:

• Yes, the device name field of the header could be e.g. TRACKINGDATA.
• Good idea about the maximum resolution!

I have added two tables above.

Tracking data

In version 1 of the protocol, transformation matrices can be transmitted. But there are some additional requirements.

• Specifing that something is not valid/visible anymore (could be done by 0-matrix)
• Specifing which data is taken at the same time / part of the same camera frame (can be done with the timestamp field of the header)
• Specifing the type of data, e.g. instrument, instrument without a direction, tracker, ... (could be done by a meta-data query)
• Sending all tracking data each frame, e.g. 5 instruments/trackers at 60 Hz result in 300 message per second, which is very much. This should be reduced.

All the points above can be handles with a new message, let's call it TRACKINGDATA:

Data	Type	Description
Name	char[20]	Name (=Id) of the instrument/tracker
Type	8 bit unsigned	1: tracker, 2: instrument with tip and handle, 3: instrument only with tip defined
Matrix	32 bit float	12 values like in TRANSFORM message

* All tracking data from one frame is included.
* Invisible/unavailable trackers/instruments are not included.
* Easy to develop. Sample pseudo code: while(true) { recv(trackingdata); updateView(trackingdata); }

This message cannot be queried with a regular GET message, but with START_PUSH and STOP_PUSH messages. The device name field consists of the string "TRACKINGDATA".

Usually the tracking data will be sent using the standard coordinate system, which is also used for POINT, IMAGE, ... But this does only work after patient registration. Therefore the body of START_PUSH has an optional field for specifing the coordinate system "CAMERA". To switch back to the standard coordinate system, one has to send STOP_PUSH and afterwards START_PUSH without explicitly specifing the camera coordinate system.

Comments

Junichi:

• How about adding 8- or 16-bit status field in TRACKINGDATA? This will allow us to indicate that coordinate system is not registered. I would like to keep START_PUSH message simple....

Alexander:

• What status types can be specified?
• In the case the coordinate system is not valid, a STATUS message should be returned.
• Well, I understand that you would like to keep it as simple as possible. We really like to specifiy the coordinate system like "Camera" or "Patient". We could also specifiy a SET_COORD message, but I think this would be overkill. I still vote for a data specific argument field in START_PUSH. Maybe other future data types can also use this field?
• Due to consistency, I think we should add this field to STOP_PUSH, too.
• We would like to allow only one TRACKINGDATA push for each client at a time, so a second START_PUSH will stop the first and start the second. A STOP_PUSH will stop the push regardless of the arguments in the body. What do you think about that?