Difference between revisions of "2017 Winter Project Week/HyperspectralOpht"

From NAMIC Wiki
Jump to: navigation, search
 
(28 intermediate revisions by the same user not shown)
Line 2: Line 2:
 
<gallery>
 
<gallery>
 
Image:PW-Winter2017.png|link=2017_Winter_Project_Week#Projects|[[2017_Winter_Project_Week#Projects|Projects List]]
 
Image:PW-Winter2017.png|link=2017_Winter_Project_Week#Projects|[[2017_Winter_Project_Week#Projects|Projects List]]
 +
File:4DLSMData.png|link=File:4DLSMData.png|4D hyperspectral data
 +
File:SIM_LSM_ROI.png|link=File:SIM_LSM_ROI.png|3D hrSIM image data and 4D LSM data
 +
File:Segmentation_Cells.png|link=File:Segmentation_Cells.png|Cell segmentation on 3D SIM data. The segmentation mask is overlayed on 4D LSM data for spectral analysis.
 +
File:LSM_SIM_Zoomed.png|link=File:LSM_SIM_Zoomed.png|Individual granules (Lipofuscin (bright) and Melano-lipofuscin (dark)) in a zoomed view of LSM and SIM.
 +
File:MultiVolExp Zoomed Plot.png|link=File:MultiVolExp Zoomed Plot.png|Hyperspectral Information of pre-segmented individual granule (LF).
 +
File:Tutorial HOA.pdf|link=File:Tutorial HOA.pdf|Tutorial on Hyperspectral Analysis with 3D Slicer
 
<!-- Use the "Upload file" link on the left and then add a line to this list like "File:MyAlgorithmScreenshot.png" -->
 
<!-- Use the "Upload file" link on the left and then add a line to this list like "File:MyAlgorithmScreenshot.png" -->
 
</gallery>
 
</gallery>
Line 7: Line 13:
 
==Key Investigators==
 
==Key Investigators==
 
<!-- Add a bulleted list of investigators and their institutions here -->
 
<!-- Add a bulleted list of investigators and their institutions here -->
−
* Sungmin Hong (NYU)
+
* Sungmin Hong (New York University)
−
* Guido Gerig (NYU)
+
* Guido Gerig (New York University)
  
 
==Project Description==
 
==Project Description==
 +
 +
This project aims to offer a tool which makes use of 3D/4D ophthalmology data in different modalities to extract information which can compensate each other for richer analysis.
 +
3D high resolution data (high resolution structured illumination microscopy, SIM) displays individual cells with sharp boundaries which are hard to be localized in 4D hyperspectral data (confocal multispectral laser scanning microscopy, LSM) because of low resolution.
 +
The tool that we want to provide to users should offer image processing modules, such as, co-registration between SIM and LSM data, segmentation on SIM and mapping the segmentation label from SIM to LSM to analyze spectral information of each granule. The examination of spectral characteristics and statistics of granules, related with age and disease progress might reveal metabolism in human retina physiology.
 +
 
{| class="wikitable"
 
{| class="wikitable"
 
! style="text-align: left; width:27%" |  Objective
 
! style="text-align: left; width:27%" |  Objective
Line 18: Line 29:
 
|
 
|
 
<!-- Objective bullet points -->
 
<!-- Objective bullet points -->
−
* 3D/4D Ophthalmology Image Anaylsis Framework
+
 
−
**
+
3D/4D Ophthalmology Image Anaylsis Framework
 +
 
 +
* Read 4D hyperspectral data
 +
* Viewer and interactor for 3D hi-res image data and 4D hyperspectral data
 +
* Co-registration between 3D hi-res data and 4D hyperspectral data
 +
* Cell segmentation in 3D hi-res data
 +
* Statistics of cells (possibly location, size, distribution)
 +
* Plot of spectra of selected cells or a region-of-interest
 
|
 
|
 +
 
<!-- Approach and Plan bullet points -->
 
<!-- Approach and Plan bullet points -->
 +
* Review existing modules in 3D Slicer
 +
** Review existing modules for 4D data viewer, such as, multi-volume viewer extension
 +
** Try existing segmentation modules in Slicer to see if they can work on SIM data
 +
** Review existing modules for cell statistics after segmentation
 +
 +
* Implementation/Integration
 +
** Implement/integrate 4D hyperspectral data viewer to show image and spectral information.
 +
** Integrate registration functionality for co-registration between 3D hi-res data and 4D hyperspectral data at image level
 +
** Integrate user-initialized level set segmentation for cell segmentation or EM segmentation module
 +
** Implement a viewer and an interactor for cell statistics
 +
 +
* User Manual
 +
** Create an user manual to comprehend a overview of an extension
 +
** Guide users to different extensions in a algorithmic flow chart if there are any desired functions (registration, segmentation, or etc.) which are already implemented in existing modules.
 +
 
|
 
|
 
<!-- Progress and Next steps bullet points (fill out at the end of project week -->
 
<!-- Progress and Next steps bullet points (fill out at the end of project week -->
−
*
+
* Hyperspectral Analysis
 +
** Implemented a module to convert 4D LSM data to a series of 3D data compatible to MultiVolume Explorer
 +
** With a converted series of 3D data, MultiVolume explorer offered a basic analysis tool for hyperspectral data.
 +
** Label statistics or segmentation need to be added in the future
 +
 
 +
* Registration
 +
** Basic registration algorithms in Slicer worked well on linear registration between 3D SIM and a cropped and dimension reduced 4D LSM data.
 +
** Detecting corresponding regions of 3D SIM in 4D LSM data needs to be developed in the future.
 +
 
 +
* Segmentation
 +
** Watershed segmentation on 3D hi-res image data (WASP) was not successful.
 +
** Editor/Segmentation Editor worked good on slice-by-slice segmentation
 +
** Will investigate more on 3D segmentation capability of Slicer with possible collaboration with other groups.
 +
 
 +
 
 +
 
 
|}
 
|}
  
 
==Background and References==
 
==Background and References==
 
<!-- Use this space for information that may help people better understand your project, like links to papers, source code, or data -->
 
<!-- Use this space for information that may help people better understand your project, like links to papers, source code, or data -->
 +
 +
==Acknowledgement==
 +
We would like to thank Dr. Thomas Ach (University Hospital Würzburg, Germany) for the significant contribution on the data and the project.

Latest revision as of 16:05, 13 January 2017

Home < 2017 Winter Project Week < HyperspectralOpht

  • 4D hyperspectral data

  • 3D hrSIM image data and 4D LSM data

  • Cell segmentation on 3D SIM data. The segmentation mask is overlayed on 4D LSM data for spectral analysis.

  • Individual granules (Lipofuscin (bright) and Melano-lipofuscin (dark)) in a zoomed view of LSM and SIM.

  • Hyperspectral Information of pre-segmented individual granule (LF).

  • Tutorial on Hyperspectral Analysis with 3D Slicer

Key Investigators

  • Sungmin Hong (New York University)
  • Guido Gerig (New York University)

Project Description

This project aims to offer a tool which makes use of 3D/4D ophthalmology data in different modalities to extract information which can compensate each other for richer analysis. 3D high resolution data (high resolution structured illumination microscopy, SIM) displays individual cells with sharp boundaries which are hard to be localized in 4D hyperspectral data (confocal multispectral laser scanning microscopy, LSM) because of low resolution. The tool that we want to provide to users should offer image processing modules, such as, co-registration between SIM and LSM data, segmentation on SIM and mapping the segmentation label from SIM to LSM to analyze spectral information of each granule. The examination of spectral characteristics and statistics of granules, related with age and disease progress might reveal metabolism in human retina physiology.

Objective Approach and Plan Progress and Next Steps

3D/4D Ophthalmology Image Anaylsis Framework

  • Read 4D hyperspectral data
  • Viewer and interactor for 3D hi-res image data and 4D hyperspectral data
  • Co-registration between 3D hi-res data and 4D hyperspectral data
  • Cell segmentation in 3D hi-res data
  • Statistics of cells (possibly location, size, distribution)
  • Plot of spectra of selected cells or a region-of-interest
  • Review existing modules in 3D Slicer
    • Review existing modules for 4D data viewer, such as, multi-volume viewer extension
    • Try existing segmentation modules in Slicer to see if they can work on SIM data
    • Review existing modules for cell statistics after segmentation
  • Implementation/Integration
    • Implement/integrate 4D hyperspectral data viewer to show image and spectral information.
    • Integrate registration functionality for co-registration between 3D hi-res data and 4D hyperspectral data at image level
    • Integrate user-initialized level set segmentation for cell segmentation or EM segmentation module
    • Implement a viewer and an interactor for cell statistics
  • User Manual
    • Create an user manual to comprehend a overview of an extension
    • Guide users to different extensions in a algorithmic flow chart if there are any desired functions (registration, segmentation, or etc.) which are already implemented in existing modules.
  • Hyperspectral Analysis
    • Implemented a module to convert 4D LSM data to a series of 3D data compatible to MultiVolume Explorer
    • With a converted series of 3D data, MultiVolume explorer offered a basic analysis tool for hyperspectral data.
    • Label statistics or segmentation need to be added in the future
  • Registration
    • Basic registration algorithms in Slicer worked well on linear registration between 3D SIM and a cropped and dimension reduced 4D LSM data.
    • Detecting corresponding regions of 3D SIM in 4D LSM data needs to be developed in the future.
  • Segmentation
    • Watershed segmentation on 3D hi-res image data (WASP) was not successful.
    • Editor/Segmentation Editor worked good on slice-by-slice segmentation
    • Will investigate more on 3D segmentation capability of Slicer with possible collaboration with other groups.


Background and References

Acknowledgement

We would like to thank Dr. Thomas Ach (University Hospital Würzburg, Germany) for the significant contribution on the data and the project.

Retrieved from "https://www.na-mic.org/w/index.php?title=2017_Winter_Project_Week/HyperspectralOpht&oldid=95130"