StandardsInterfaces

We are intreated in getting feedback on creating standards and interfaces for microscopy image analysis in ITK

Key Investigators

• Harvard - Sean Megason

Objective

There are now several groups using ITK to develop filters for microscopy image analysis. However there are a number of possibilities for the input and output formats for these filters. For filters from different groups to be interchangeable, it is important that these filters use a defined interface. The goal of this project is to solicit opinions from both groups working on microscopy image analysis as well as those with experience designing interfaces for image processing in other domains.

Approach, Plan

If you have opinions, suggestions, or interfaces you are already using please talk to Sean and come to the Microscopy Image Analysis breakout session on Wed afternoon.

Issues

Specifically we would like to define functional groups of filters (e.g. denoising, seeding, segmentation, tracking, lineaging, post-processing, quantitation, classification) to allow for filter swapping within a group and then to define input/output types between these groups. ImageToImage filters are a sufficient interface for some filter groups but are problematic for others due to changes in dimension (tracking, lineaging), excess size (seeding), and differences with the "natural" output of an algorithm (e.g. a segmentation algorithm may output a mesh rather than image).

Progress

I met with Luis Ibanez and Jim Miller to discuss how related issues have been handled in ITK previously. They suggested itk::SpatialObject and itk::LabelObject. LabelObject seems like a good intermediate between an image based representation and an object(cell) based representation, and allows for either perspective depending on which is more convenient. LabelObject uses run-length enconding of objects in an image which results in a lot of compression. LabelObject also allows for objects to be attributed with values (e.g. cell size, intensity, velocity). One downside is that there are not currently many filters in ITK that use LabelObjects. In our research, we (Kishore) have already been working with LabelObjects as the output to segmentation filters. This seems like a promising approach. We need to standardize the representation of cells in xy, xyt, xyz, xyzt, xyzt+division using LabelObjects and the associated cellular attributes, and complete a LabelObjectToGoFigure filter.

We also discussed standards for image formats at the Microscopy Breakout session. We broke this issue down into 3 components:

• File Types: e.g. jpg, png, tif, lsm. The problem of needing to read a bunch of different file types has been around a long time. This problem is still not ideally solved (there is no BSD licensed library that will read all microscopy formats), but there are enough workarounds that this issue is now more of an inconvenience than a road block. In practice most groups only use 1 maybe 2 types of microscopes and have solved their problem locally. A global solution would make it easier for groups to work together and would thus be preferable. However, current file types do not meet the needs of advanced imaging applications anyway (see "meta" file types below) so efforts should be aimed at both creating BSD readers for existing file types as well as meta file types.
• Metadata: metadata refers to all the nonimage data that is part of an imaging experiment. Metadata include image acquisition parameters (objective, laser power, PMT gain, filters, pixel dwell time) much of which is microscope specific. It can also include information about how the specimen was labeled and and its stage and orientation for imaging. It can even include a full description of the experiment (genotype, drugs...). Current solutions are to store the data in XML files, databases, and image headers. Metadata is very heterogeneous and user-specific and thus needs a flexible format.
• Meta image formats: For