Project Week 25/Multimodal:

Back to Projects List

Key Investigators

• Guido Gerig (NYU Tandon School of Engineering, USA)
• Sungmin Hong (NYU Tandon School of Engineering, USA)

Project Description

• Project details can downloaded here as pptx or pdf.
• Download a Slicer tutorial.

Objective	Approach and Plan	Progress and Next Steps
3D/4D Ophthalmology Image Anaylsis Framework: Continuation of initial January 2017 project by Sungmin Hong: Read 4D hyperspectral data Viewer and interactor for 3D hi-res image data and 4D hyperspectral data Co-registration between 3D hi-res data and 4D hyperspectral data Cell segmentation in 3D hi-res data Statistics of cells (possibly location, size, distribution) Plot of spectra of selected cells or a region-of-interest	Review existing modules in 3D Slicer Review existing modules for 4D data viewer, such as, multi-volume viewer extension Try existing segmentation modules in Slicer to see if they can work on SIM data Review existing modules for cell statistics after segmentation Implementation/Integration Implement/integrate 4D hyperspectral data viewer to show image and spectral information. Integrate registration functionality for co-registration between 3D hi-res data and 4D hyperspectral data at image level Explore most recent "Segment Editor" features for cell and organelle segmentation User Manual Create an user manual to comprehend a overview of an extension	Manual Slicer tutorial explaining multi-volume conversion and 3D/4D registration. Registration Basic registration algorithms in Slicer worked well for linear registration between 3D SIM and cropped and dimension reduced 4D LSM data. Automatic detection of corresponding subregions between hires 3D SIM and low-res 4D LSM data may be implemented via autocorrelation. Segmentation GrowCut segmentation worked well for 3D segmentation of organelles of different contrast, e.g. dark Lipofuscin and bright Melanolipofuscin as required by clinical researchers. Cell segmentation/outlining and combination with organelle segmentation can be solved with the Segment Editor features. Hyperspectral Analysis Module to convert 4D LSM data to a series of 3D data compatible to MultiVolume Explorer was implemented at Jan. 2017 project week. With such series of 3D data, MultiVolumeExplorer offers very basic qualitative analysis hyperspectral data by plotting a spectral curve per voxel. Quantitative label statistics by combining 3D segmentation labels with 4D hyperspectral data so far solved via external python script, will need to be added to the multivolume explorer in the future.

Illustrations

Background and References

[1] Thomas Ach, Sungmin Hong, Rainer Heintzmann, Jost Hillenkamp, Kenneth R. Sloan, Neel Dey, Guido Gerig, R. Theodore Smith, Christine A. Curcio, Katharina Bermond, High-resolution and hyperspectral imaging of autofluorescent retinal pigment epithelium (RPE) granules, ARVO 2017 Annual Meeting Abstracts, No. 3382, abstract

[2] Tong Y, Ben Ami T, Hong S, Heintzmann R, Gerig G, Ablonczy Z, Curcio CA, Ach T, Smith RT., HYPERSPECTRAL AUTOFLUORESCENCE IMAGING OF DRUSEN AND RETINAL PIGMENT EPITHELIUM IN DONOR EYES WITH AGE-RELATED MACULAR DEGENERATION, Retina. 2016 Dec;36 Suppl 1:S127-S136. doi: 10.1097/IAE.0000000000001325., PMID: 28005671