Microscopy and Analysis

Inquiries about the use of the book should be directed to INTECH rights and permissions department ([email protected]). The publisher accepts no responsibility for any damage or injury to persons or property resulting from the use of any materials, instructions, methods or ideas contained in the book.

Quantum Image-Forming Theory for Calculation of Resolution Limit in Laser Microscopy

Introduction

One typical example of laser microscopy is confocal fluorescence microscopy, which is widely used as an optical imaging technique. Although we only consider laser microscopy here, our theory can be applied to any type of optical microscopy.

Theoretical framework

Model description
Transmission linear confocal microscopy
Reflection linear confocal microscopy
Coherent nonlinear microscopy
Incoherent microscopy

Distribution of the wavenumber vector (3-D pupil function) of the signal focused by the signal collection objective. With the sample stage displacement (x′, y′, z′), in transmission microscopy, the electric field distribution of the SRG signal ESRG (x′, y′, z′; xa, ya, za) in the detection space is written as.

Figure 2. Double-sided Feynman diagram and energy-level diagram for the χ (1) -derived optical process

Optical resolution limit

Diagram technique

For a right-pointing arrow in a Feynman diagram or an up-pointing arrow in an energy-level diagram, the wavenumber vector of the excitation light corresponds to +kex. For a left-pointing arrow in a Feynman diagram or a down-pointing arrow in an energy-level diagram, the wavenumber vector of the excitation light corresponds to −kex.

Figure 11. The relation between the phase-matching condition and the CTF for the transmission fluorescence confocal microscopy

Discussion

Theorem

As an example to see the difference between reflection and transmission microscopy, CTF of transmission and reflection CARS micro‐. If there is no a priori information on the object (sample), the resolution limit (the maximum value of the frequency cutoff) is determined by the diagram describing the optical process.

Figure 12. Calculation samples of the CTF for (a) CARS, (b) SRG, (c) SRL, and (d) THG microscopy.

Conclusions

As long as the optical process described by a particular diagram is used to visualize χ(i)(x, y, z), the resolution limit calculated from the diagram cannot be exceeded, no matter how well the optical apparatus is designed. Interestingly, in incoherent process microscopy, the vacuum field plays a role as part of the excitation light and contribution.

Author details

Any microscopy application, including SIM and stimulated emission depletion (STED) microscopy [16], that does not have a priori information on the object must follow this theorem. In SRS microscopy, the transmission type mainly observes the imaginary part of the nonlinear susceptibility, while the reflection type can detect the real part.

We have constructed a theoretical framework to handle the imaging of all types of microscopy using the two-sided Feynman diagrams and energy level diagrams that describe optical processes. Kinetic model of development and aging of artificial skin based on analysis of microscopy data.

Kinetic Model of Development and Aging of Artificial Skin Based on Analysis of Microscopy Data

Morphology of the fully developed epidermis of artificial skin
Properties of artificial skin epidermis revealed by confocal microscopy
Simulation of electron-beam penetration into skin
Kinetic model of epidermis formation and aging
Effect of corneal screening on irradiation of the viable epidermis
Conclusions

A significant shrinkage of the stable epidermis (basal, spinosal and granular layers) during this time was reported in reference [14]. The biological reason for these changes in the thickness of the combined basal, spinosal, and granular layers is not clear.

Figure 1. Representative histological sections showing morphology and differentiation in the EpiDermFT TM skin model 17–20 days after the seeding of keratinocytes onto the dermal substrate

Automatic Interpretation of Melanocytic Images in Confocal Laser Scanning Microscopy

Malignant melanoma and benign nevi

People with early stage melanomas are treated by surgical removal of the skin lesion. Therefore, the early and reliable recognition of melanomas at an early stage is of particular importance [6].

Confocal laser scanning microscopy

In addition to the reflected light from the focal point, the optical system of the microscope also projects scattered light from sample points outside the focal light (coming from places above or below the focal point) and thus contributes to the composition of the image. The brightness of the resulting pixel corresponds to the relative intensity of the reflected light.

Figure 2. Principle of the confocal (left) and laser scanning (right) microscopy.

Interpretation of confocal laser scanning microscopic images

This makes the appearance of the tissue in a CLSM image so different from conventional histological views. Layers from the level of the arachnoid keratinocytes (polygonal cells) to the level of the basal cells (dermo-epidermal junction) are used for diagnosis.

Analysis of tissue structures at different scales

Then the architectural structure information is accumulated along the subbands (from coarse to fine). For the discrimination of the CLSM images, the CART (Classification and Regression Trees) algorithm is used [18].

Figure 7. Shape of the Laplacian of Gaussian convolutional filter kernel.

Biological motivation for neural networks

The dendrites are cytoplasmic extensions of the cell body with many branches that allow the cell to receive signals from other neurons. These changes in the cross-membrane potential are transmitted as a wave of successive depolarization and repolarization processes along the cell's axon.

Artificial neural networks

Fully connected neural networks are useful where individual features in a data set are not very informative. Traditional feed forward neural networks consist of layers where each neuron is connected to every other neuron in the layers above and below it.

Figure 12. Structure of a feed forward artificial neural network.

Convolutional neural networks

Such an area in the input image is called the local receptive field for the corresponding hidden neuron. Pooling layers simplify the information in the output of the convolutional layer by generating a condensed feature map (this removes the positional information from the learned features, meaning the learned features are position-invariant).

Figure 13. The structure of a typical seven-layer convolutional neural network.

Deep learning analysis of a CLSM image dataset

Input into the neural network
Keras

For example, a convolution layer can be added to a network using the add function: model.add(Convolution2D(…)). The network is built in this way until the desired structure is defined, and then it is translated using the object model's translation function.

Figure 14 describes the convolutional layer and max-pooling layer in more detail. The input into the convolutional neural network is a vector x∈ R 1×m , and the input layer has one neuron per feature

Results

Multiresolution analysis
Convolutional deep learning neural network
Transfer learning

Microscopy and Analysis 70 . common nevi, 296 malignant melanomas) in the training set and 285 cases (132 benign common nevi, 153 malignant melanomas) in the test set. The distribution of the classes in the whole data set and in the training and test set.

Table 1. Classification results for features based on multiresolution analysis.

Discussion

In the case of the CLSM image classification task presented here, all that was needed was a labeled dataset of previous observations. Contemporary in vivo confocal microscopy of the living human cornea using white light and laser scanning techniques: a major review.

Super‐Resolution Confocal Microscopy Through Pixel Reassignment

Super resolution by pixel reassignment

Principle of pixel reassignment
Computational realization of pixel reassignment 1. Image scanning microscopy

Figure 1(a) shows the principle of pixel reassignment in terms of excitation and detection point spread function (PSF) [7]. The excitation PSF (PSFex, indicated by a blue line) represents the distribution of the corresponding excitation focus.

Figure 2. Super‐resolution image scanning microscopy (ISM) with computational pixel reassignment

Compared to the wide‐field images, the multifocal‐excited, pinholed, scaled, and

Optical realization of pixel reassignment
Conclusion

Rescan confocal microscopy (RCM) is another optical realization of the pixel reallocation technique, proposed by Luca et al. The camera is in the exposure state for optical sum of the fluorescent focus during rescan.

Figure 5. Resolution doubling of MSIM by imaging antibody‐labeled microtubules in human osteosarcoma (U2OS) cells

Acknowledgements

However, cross talk of the different fluorophores always occurs due to the broad and overlapping excitation and emission bands of fluorophores. Linear spectral unmixing analysis is implemented for the spectral differentiation of the live cells stained with different dyes.

Second Harmonic Generation Microscopy: A Tool for Quantitative Analysis of Tissues

Principles of second harmonic generation

Nonlinear optical microscopy refers to all microscopic techniques based on nonlinear optics in which light-matter interactions violate the principle of linear superposition. These coefficients contain the non-linear optical properties of the material and the sum of zero for inversion symmetry [4].

SHG microscopy of biological samples

According to the non-linear nature of harmonic generation, the intensity of the SHG signal depends on the square of the intensity of the excitation laser and itself appears limiting to the focus of the microscope objective. In general, the SHG directivity depends on the distribution of the induced dipoles in the focal length where the nonlinear process takes place [33].

Figure 2. SHG image of starch grains. These are plant polysaccharides with a convenient radial structure

Imaging ocular tissues with SHG microscopy

In those cases the organization presents contributions to the pivotal moment by changing the coherent SHG process. To our knowledge, Han and colleagues were the first to show SHG imaging of the sclera [ 36 ].

Figure 6. Comparison of collagen distributions in SHG images of a human cornea (a) and a piece of human sclera (b).

Measurement of collagen organization in ocular tissues

The distribution of the spatial frequencies on the 2D-FFT is also generally fitted by an ellipse, and the ratio of its axes is used as a parameter to quantify the collagen organization. The former ranges between 0 and 1, and its value increases with the order of the structure (ie, the more aligned the fibers are, the higher the DoI).

Figure 8. SHG images of a human cornea (upper row) and bovine sclera (bottom row) and their respective spatially‐

Adaptive optics SHG imaging

Nevertheless, the resolution of the 2D-FFT is limited by the noise of the SHG image and image filtering is often required. It can be observed how AO improves the quality of the images at both locations.

Figure 10. Comparison of SHG images before (upper panels) and after (bottom panels) using AO for two different lo‐

Polarization‐sensitive SHG microscopy

Abnormalities of stromal structure in the cornea of bullous keratopathy identified by second harmonic generation imaging microscopy. Precise motionless polarization control in second harmonic generation microscopy using a liquid crystal modulator at infinity.

Figure 12. SHG images of a bovine sclera recorded for three different incident polarization states

Nonlinear Microscopy Techniques: Principles and Biomedical Applications

Principles of nonlinear microscopy techniques

The usual linear susceptibility χ(1) contributes to the single-photon absorption and reflection of the light in tissues. The two-photon excitation band is not exactly half of the one-photon excitation band because the selection rules are different.

Figure 1. Energy diagrams of linear and nonlinear optical process. Solid lines represent electronic and vibrational states of molecules, while dashed lines denote virtual states

Nonlinear optical technique implementation

The LSM780 scan head has a gallium arsenide phosphide (GaAsP) spectral detector with 32 line elements and 2 adjacent PMTs. The LSM780 scan head has a gallium arsenide phosphide (GaAsP) spectral detector with 32 line elements and 2 adjacent PMTs.

Figure 2. Schematic setup of a multimodal NLO microscope using a femtosecond laser source (Mai‐Tai) and picosec‐

Biomedical applications

Cancer detection
Osteogenesis imperfecta

The information revealed by each modality can be directly compared, providing a better understanding of the tissue. For example, merged SHG/THG signals can be used to distinguish the epithelial/stromal interface.

Figure 3A exemplifies this combination, where NLO techniques were applied to differentiate between normal and malignant (fibroadenoma and invasive lobular carcinoma (ILC)) human breast tissue

Analysis methods used as diagnostic tools

Ratio between collagen and elastic tissue (SAAID)
Tumor‐associated collagen signatures (TACS)
Fast Fourier transform (FFT) analysis
Gray level co‐occurrence matrix (GLCM) analysis

For example, the energy parameter is the squared sum of the GLCM pixel values. Using fresh biopsies, one could detect a marked decrease in collagen fiber network density in the 3D representations of SHG images from severe OI patient samples (Figure 8B (Type III) and Figure 8C (Type IV)), as compared to the 3D representation of SHG images taken from fresh biopsies of normal skin (Figure 8A).

Figure 7. Depicting several applications of different methods to analyze NLO signals. The panel shows representative TPEF (green) and SHG (red) images of (A, E, H, L) normal and (B, F, I, M) cancer ovary

Skin Wound Healing Revealed by Multimodal Optical Microscopies

Skin and wound healing phases

As wound maturation progresses, the tensile strength of the wound increases, eventually reaching 80% strength. Hypertrophic scars, on the other hand, are raised scars that remain within the boundaries of the wound and often resolve spontaneously.

Figure 1. (a) Representative image of the key players in the healing of a skin wound [3]

Current methodologies of wound diagnosis

Optical techniques used in skin observations

Macro‐optical imaging modalities
Micro‐optical modalities

Changes in SHG intensity explain the relative degradation and regeneration of collagen at the center and at the edge during wound healing. Changes in SHG intensity explain the relative degradation and regeneration of collagen at the center and at the edge during wound healing.

Figure 2. (a1) Visible color image of fingertips and (a2) color‐coded blood perfusion map [51]

Conclusion

Webb, "In vivo confocal scanning laser microscopy of human skin II: advances in instrumentation and comparison with histology," J. Jain, "Dynamic imaging of collagen and its modulation in tumors in vivo using second harmonic generation."

Automated Identification and Measurement of

Haematopoietic Stem Cells in 3D Intravital Microscopy Data

Intravital microscopy of mouse bone marrow
Application and challenges of bone marrow intravital microscopy image analysis
Image analysis

Image acquisition and processing
Image segmentation
Parameter optimisation

Parameter optimisation

Multi-resolution parameter α 1. α for HSCs
Distance to neighbour d parameter 1. d for HSCs

Machine learning

Decision tree classifier

The choice of scale parameter α depends on the physical and textural structure of the objects. The optimization of the MDN threshold depends on the correct detection of HSC edges.