2010 Summer Project Week Microscopy extensions for ITK

• 

  Back to Projects List

Key Investigators

• Gaetan lehmann: INRA
• Alex Gouaillard: CoSMo Software / A*STAR
• Luis Ibanez: Kitware Inc.

Introduction

Fluorescence microscopy is a very common image acquisition modality, used in many fields including medical imaging and biological research. Depending on the techniques used, the images produced can be in 2D or 3D, time lapse, and contain several channels. ITK provides a lot of useful tools to analyze the images produce in fluorescence microscopy. The N-dimensionality of ITK is especially useful when dealing with 3D data sets. However, it contains very few methods to correct the aberrations introduced in the image during the acquisition. We propose to implement some tools dedicated to the restoration of images in fluorescence microscopy. Because no algorithm exists to perfectly correct those aberrations, we propose to implement several algorithms for each kind of problem, including the most common algorithms, which are well know and widely used, and some of the state of the art algorithms.

Bold and Tag'ed with a #, working prototype available.

Bold and tag'ed with a @, prototype expected on or before august

preliminary work

• fft convolution # ( Insight Journal publication )

Most of the deconvolution methods are based on multiple deconvolution or on FFT transforms. This preliminary work will be very useful to implement the deconvolution filters.

• noise simulation # ( Insight Journal publication )

Being able to simulate the noise behavior is very important because it let the user compare the result of a restoration method with the known unblurred object.

Deconvolution

• non-blind deconvolution

This is the most common deconvolution type. With all those methods, it is supposed that the point spread function is known. Most of those methods, for practical reasons, also assume that the point spread function (PSF) is spatially invariant in the image. Some of the methods are focused on the computation efficiency, at the cost of the restoration quality - often the linear algorithms -, and some on the restoration quality at the cost of the computation complexity - usually the iterative algorithms. The fluorescence noise follows a Poisson distribution, but is often considered as gaussian noise to simplify the computations. This assumption is generally valid when the SNR is high - most of the time for wide field microscopy. Because of the noise amplification produce by the deconvolution process, most of the methods are regularized. In fact, the only unregularized methods are iterative, and are effectively regularized by stopping the iteration process before the convergence.

• Space invariant PSF - Linear algorithms

All the proposed linear algorithms are made with the assumption of an additive gaussian noise. Other algorithms combined with wavelet transformed produce better results.

1. Regularized Linear Least Squares (truncated SVD) #
2. Maximum A Posteriori Linear Least Squares #
3. Wiener #
4. Tikhonov Miller #

• Space Invariant - Iterative Algorithms

1. van Cittert #: No noise assumption. A single convolution is required by iteration.
2. Jansson-van Cittert #: This is a constrained version of the van Cittert algorithm. The pixel values are constrained in a range of values.
3. Landweber #: The noise is assumed to follow a gaussian additive model. The algorithm requires two convolutions by iteration and the step size is fixed, which makes it converge slowly. It can be optionally constrained to avoid negative values as expected in fluorescence microscopy.
4. Richardson-Lucy #: The noise is assumed to follow a poisson model. The algorithm requires two convolutions by iteration and the step size is fixed, which makes it converge slowly. The non negativity property is ensured without additional constraint. This algorithm is also known as Maximum Likelihood Maximization Expectation.
5. Damped Richardson-Lucy # This is a simple regularized version of the Richardson-Lucy algorithm. It is widely used in astronomy.
6. Tikhonov-Miller regularized Richardson-Lucy #: This is a regularized version of the Richardson-Lucy algorithm using the Tikhonov-Miller usual penalization. Ref: Dey et al. 3D Microscopy Deconvolution using Richardson-Lucy Algorithm with Total Variation Regularization. hal.archives-ouvertes.fr (2004)
7. Total variation regularized Richardson-Lucy: This is a regularized version of the Richardson-Lucy algorithm which also minimize the Total Variation of the image to avoid the noise amplification.
8. Maximum Entropy regularized Richardson-Lucy #: This is a regularized version of the Richardson lucy algoritm by imposing a maximum entropy constraint on the unblurred image.
9. Good regularized Richardson-Lucy: This is a regularize Richardson-Lucy algorithm which also minimize the Good's roughness.
10. Iterative Constrained Tikhonov-Miller @: The noise is assumed to follow an additive gaussian model, and the process is regularized using the usual penalization. The non negativity is enforced at each iteration and the convergence is accelerated with a conjugate gradient acceleration.
11. Verveer's Maximum A Posteriori (MAPGG, MAPGE, MAPPG, MAPPE, MAPGR and MAPPR): The noise is assumed to follow either a gaussian or a poisson model. Three different a priori can be imposed on the unblurred image: a gaussian model, a maximum entroy or a Good's roughness. The process is accelerated with a Newton acceleration.
12. Markov Random Field MAP: The noise is assumed to follow a poisson model. The unblurred image is assumed to be a realization of a Markov random field. The problem is solved with a split gradient technique which is known to converge slowly, but the result quality is enhance compared to other state of the art algorithms.
13. Generalized Vector-Valued Total Variation: The noise is assumed to follow an additive gaussian model. The process also minimize the total variation and is accelerated. The code is available for matlab with a BSD license.

• space variant psf

1. Depth-variant ML-EM

• PSF

The PSF is required for all the non-blind deconvolution methods. It is possible to get it either by measuring it with beads, are with a model. One way are the other both have their advantages and inconvenience and should be available for the final user.

1. PSF simulation
2. measurement / Evaluation of psf

• Regularization Parameters Estimation

1. Generalized Cross Validation (GCV): A good estimation can be computed for the Tikhonov-Miller regularization with the generalized cross validation method.

Usability

On the implementation side, several goal should be reached.

1. observability #: The iterative deconvolution algorithm are usually quite long to run - several minutes or tens of minutes on todays computers. As a consequence, it is important that the process can be observed by the user so that he can validate visually the intermediate results. The various measurements made should also be available during the process. If the result is not the expected one, or if the expected result is reach sooner than expected, the user should be able to interrupt the process cleanly.
2. continue a completed process to increase the number of iterations #: The iterative filter should provide a mechanism to restart a completed deconvolution in order to increase the number of iterations.
3. preconditionning #: Preconditionning is a usual technique where the blurred image and/or the PSF are transformed at the beginning at the deconvolution in order to improve the deconvolution quality or to decrease the number of iterations needed to reach the convergence. The usual precoditionning methods include the gaussian blurring, the Wiener fieltering and various denoising methods (mostly wavelet based).
4. multithreaded calculators: ITK does not currently provide any base class to easily implement multithreading in the calculators. We propose to implement such a base class and to use it in the calculators required in the other tasks.
5. FFTW wisdom integration #: FFTW provides a mechanism to store the optimized plans on the disk. We propose to use this capability in ITK FFTW filters and to make it's usage transparent for the user.

Objective

The objectives of the project week is to share with the community the filters to come to avoid multiple groups working on the same algorithms. Persons interested can contact gaetan or alex directly.

Approach, Plan

Transition as many filters as possible to the Insight Journal.

Progress

• Update the denoising and FFT aricles in IJ
• prepare the tests and papers for the blind deconvolution algorithms
• discuss with another team funded for deconvolution through A2D2
• attend the microscopy workshop and synchronized effort

Delivery Mechanism

This work will be (eventually) delivered to the NA-MIC Kit as an

1. ITK Module

References

[01] Ref: Preza et al. Regularized method for reconstruction of three-dimensional microscopic objects from optical sections. Journal of the Optical Society of America (1992) vol. 9 (2) pp. 219-228
[02] Ref: Preza et al. Image reconstruction for 3-D light microscopy with a regularized linear method incorporating a smoothness prior. Presented at the IS&T/SPIE Symposium on Electronic Imaging (1993) vol. 1905 (129-139)
[03] Ref: Richardson. Bayesian-Based Iterative Method of Image Restoration. Journal of the Optical Society of America (1917-1983) (1972) vol. 62 pp. 55-+
[04] Ref: Lucy. An iterative technique for the rectification of observed distributions. Astronomical Journal (1974) vol. 79 pp. 745-+
[05] Ref: White. Image Restoration Using the Damped Richardson-Lucy Iteration. Proceedings (1994) vol. 61 pp. 292-+
[06] Ref: Dey et al. 3D Microscopy Deconvolution using Richardson-Lucy Algorithm with Total Variation Regularization. hal.archives-ouvertes.fr (2004)
[07] Ref: Monvel et al. Image Restoration for Confocal Microscopy: Improving the Limits of Deconvolution, with Application to the Visualization of the Mammalian Hearing Organ. Biophysical Journal (2001)
[08] Ref: Verveer et Jovin. Acceleration of the ICTM image restoration algorithm. Journal of Microscopy (1997) (188) pp. 191–195
[09] Ref: Verveer. Computational and Optical Methods for Improving Resolution and Signal Quality in Fluorescence Microscopy. adsabs.harvard.edu (1998)
[10] Ref: Vicidomini et al. Application of the split-gradient method to 3D image deconvolution in fluorescence microscopy. Journal of Microscopy (2009) vol. 234 (1) pp. 47-61
[11] Ref: P. Rodriguez, B. Wohlberg, "Efficient Minimization Method for a Generalized Total Variation Functional" IEEE Transactions on Image Processing, 2009, 18:2(322-332)
[12] Ref: Preza et Conchello. Depth-variant maximum-likelihood restoration for three-dimensional fluorescence microscopy. Journal of the Optical Society of America A (2004) vol. 21 (9) pp. 1593-1601
[13] Ref: Agard. Optical sectioning microscopy: cellular architecture in three dimensions. Annual review of Biophysics and Bioengineering (1984) vol. 13 pp. 191-219
[14] Ref: Gibson et Lanni. Experimental test of an analytical model of aberration in an oil-immersion objective lens used in three-dimensional light microscopy. J. Opt. Soc. Am. A (1992) vol. 9 (1) pp. 154-166
[15] Ref: Haeberle. Focusing of light through a stratified medium: a practical approach for computing microscope point spread functions. Part I: Conventional microscopy. Optics Communications (2003) vol. 216 (1-3) pp. 55 - 63
[16] Ref: Haeberle. Focusing of light through a stratified medium: a practical approach for computing microscope point spread functions: Part II: confocal and multiphoton microscopy. Optics Communications (2004) vol. 235 (1-3) pp. 1 - 10
[17] Ref: Dey et al. 3D Microscopy Deconvolution using Richardson-Lucy Algorithm with Total Variation Regularization. hal.archives-ouvertes.fr (2004)
[18] Ref: Van Kempen et al. A quantitative comparison of image restoration methods for confocal microscopy. Journal of Microscopy (1997) vol. 185 (3) pp. 354-365