although most of these measurements were based on manual inspection and intervention , with the advent of fluorescence microscopy , many studies also involved quantitative imaging of living cells either using video or ccd cameras ( inoue , 1981 ; allen and allen , 1983 ) . in the early years of live - cell microscopy , methods for segmentation and tracking of cells
( berg and brown , 1972 ; berns and berns , 1982 ) were rapidly developed and adapted from other areas .
nowadays , techniques for fully automated analysis and time  space visualization of time series from living cells involve either segmentation and tracking of individual structures , or continuous motion estimation ( for an overview , see fig .
1 ) 
 
  . for tracking a large number of small particles that move individually and independently from each other ,
single particle tracking approaches are most appropriate ( qian et al . , 1991 ) .
once the images have been acquired by microscopy and preprocessed to improve the signal - to - noise ratio , they can be directly visualized by methods like volume rendering . for multiple objects in motion ,
single particle tracking , in which a particle is tracked over different time - steps , is the most direct method used .
surface rendering is obtained after segmentation of contours in each individual section and gives rise to volumetric measurements such as volume and surface area .
measurements of concentration changes for segmented areas in frap or fluorescence loss in photobleaching experiments give rise to estimates of kinetic parameters such as diffusion and binding coefficients .
the estimation of flow of gray values is an approach to quantify mobility in continuous space .
all these processes lead to accurate estimates of quantitative parameters . for the determination of more complex movement ,
two independent approaches were initially developed , but recently have been merged . optical flow ( mitiche and bouthemy , 1996 ) methods
image registration ( terzopoulos et al . , 1991 ; lavallee and szeliski , 1995 ) aims at identifying and allocating certain objects in the real world as they appear in an internal computer model .
the main application of image registration in cell biology is the automated correction of rotational and translational movements over time ( rigid transformation ) .
this allows the identification of local dynamics , in particular when the movement is a result of the superposition of two or more independent dynamics .
registration also helps to identify global movements when local changes are artifacts and should be neglected .
the basic principle of single particle tracking is to find for each object in a given time frame its corresponding object in the next frame .
the correspondence is based on object features , nearest neighbor information , or other inter - object relationships .
object features can be dynamic criteria such as displacement and acceleration of an object as well as area / volume or mean gray value of the object .
optical flow has been defined as the motion flow ( i.e. , the motion vector field ) that is derived from two consecutive images in a time series ( jhne , 2002 ) .
however , due to high levels of noise , this assumption is usually distorted , and standard region - based matching techniques give unsatisfactory results ( anandan , 1989 ) .
a more reliable tracking approach involves fuzzy logic - based analysis of the tracking parameters ( tvarusko et al . , 1999 ) .
( apprehend and allocate ) certain objects in the real world as they appear in an internal computer model .
initially , only rigid transformations were used to superimpose the images , whereas nowadays , research is focused on the integration of local deformations .
a parametric image registration algorithm specifies the parameters of a transformation in a way that physically corresponding points at two consecutive time steps are brought together as close as possible .
such algorithms have been broadly studied in medical imaging and cell biology ( maintz and viergever , 1998 ; bornfleth et al . , 1999 ) .
although one class of algorithms operates on previously extracted surface points ( lavallee and szeliski , 1995 ) , other algorithms register the images directly based on the gray - value changes .
nonrigid deformations , i.e. , transformations others than rotation and translation , present an active body of research in computer vision .
nonrigid approaches differ with respect to the underlying motion model ( terzopoulos et al . , 1991 ; szeliski , 1996 ) .
most commonly , a cost or error function is defined and an optimization method is chosen that iteratively adjusts the parameters until an optimum has been achieved .
other approaches extract specific features ( e.g. , correspondence between points ) that serve as a basis for directly calculating the model parameters ( arun et al . , 1987 ; rohr , 1997 ) .
computer vision is a discipline that focuses on information extraction from the output of optical sensors , and on the representation of this information in an internal computer model ( faugeras , 1993 ) .
a computer vision framework for detecting and tracking diffraction images of linear structures in differential interference contrast microscopy was developed for measuring deflections of clamped microtubules with a freely moving second end ( danuser et al . , 2000 ) . based on measurements of thermal fluctuations
further , prior knowledge based on geometric and dynamic models of the scene can lead to restoration of information beyond the resolution limit of an imaging system ( danuser , 2001 ) .
this super - resolution concept was illustrated by the stereo reconstruction of a micropipette moving in close proximity to a stationary target object .
complex dynamic processes in cells should ideally be studied in three spatial dimensions over time . thereby , large and complex data sets typically consisting of 5,00010,000 single images are generated .
such data are virtually impossible to interpret without computational tools for visual inspection in space and time .
typically , 3-d images have been represented as stereoscopic pairs or as anaglyphs by pixel shift method ( white , 1995 ) . displaying time series as movies is still a widely used method for visual interpretation . for fast - moving objects such as trafficking vesicles imaged with high time resolution ,
time - lapse movies are very informative . however , for much slower nuclear processes or for processes with mixed kinetics that need to be observed over a longer period of time , the total number of time points for imaging are limited due to the photo toxicity of the light exposure during in vivo observation ( konig et al . , 1996 ) .
therefore , an interpolation between consecutive time steps is required to reconstruct intermediate time steps . as a  side effect ,
 additional information about the continuous development of the observed processes between the imaged time steps ( subpixel resolution in time ) is achieved , and quantitative information can be derived ( see next section ) .
although early studies explored 4-d data sets by simply browsing through an image gallery and highlighting interactively selected structures ( thomas et al . , 1996 ) ,
two commonly used rendering algorithms for displaying 3-d structures are volume rendering and surface rendering ( chen et al . , 1995 ; fig .
1 ) . volume rendering is a technique for visualizing complex 3-d data sets without explicit definition of surface geometry .
the classification step assigns a level of opacity , contrast , and color to each voxel in the 3-d volume ( e.g. , wright et al . ,
then , shading techniques are used to simulate both the object surface characteristics and the position and orientation of surfaces with respect to light sources and the observer . the colored , semitransparent volume
a ray is cast into the volume through each grid point on the projection plane . as the ray progresses through the volume
, it computes the color and opacity at evenly spaced sample locations , and finally yields a single pixel color .
although volume rendering techniques provide a satisfactory display of biological structures , this method is limited to pure visualization and does not deliver quantitative information .
in addition , the high anisotropy typical for live - cell imaging with low z - resolution limits the quality of this visualization technique .
these limitations are overcome by surface - rendering techniques , where the object surface is represented by polygons .
the polygonal surface is displayed by projecting all the polygons onto a plane that is perpendicular to a selected viewing direction .
the most commonly used method to triangulate the 3-d surface is the marching cube algorithm ( cline et al . , 1988 ) .
the 3-d structure is defined by a threshold value throughout the data set , constructing an isosurface .
the drawback of this method is that the surface of many biological structures can not be defined using a single intensity value , resulting in loss of relevant information .
a great advantage of the combination of segmentation and surface reconstruction is the immediate access to quantitative information that corresponds to visual data ( eils et al . , 1996 ;
these approaches were designed to deal particularly with the high degree of anisotropy typical for 4-d live - cell recordings and to directly estimate quantitative parameters , e.g. , the gray values in the segmented area of corresponding images can be measured to determine the amount and concentration of fluorescently labeled proteins in the segmented cellular compartments .
measuring concentration changes by frap and fluorescence loss in photobleaching have become standard methods to evaluate diffusion , binding , and trafficking in live cells ( for review see phair and misteli , 2001 ) .
these methods give direct access to kinetic parameters such as the diffusion coefficient of molecules ( axelrod et al .
2000 ) or exchange rates of molecules between different compartments ( hirschberg et al . , 1998 ; phair and misteli , 2001 ) . in combination with motion estimation techniques
, parameters such as the velocity of the mass center for individual objects or for each point on the object surface can be readily accessed .
further , local parameters such as acceleration , tension , or bending ( bookstein , 1989 ) can be estimated . during motion estimation ,
global quantities are estimated such as the parameters of rotation and translation ( germain et al . , 1999 ) .
the evolution of these eigenvalues can be used to characterize and analyze the observed motion .
statistical analysis of velocity histograms can be applied to compute peak velocities corresponding to the most frequently occurring velocity values ( uttenweiler et al . , 2000 ) .
an alternative technique for statistical analysis is the confinement tree analysis of the intensity image ( mattes et al . , 2001 ) .
for different threshold levels , objects ( confiners ) are segmented in the image . calculated for different levels ,
besides the estimation of global quantitative values ( e.g. , the global homogeneity of the motion ) , this approach allows the analysis and comparison of movements . a challenge for future
work is to better understand the biomechanical behavior of cellular structures , e.g. , cellular membranes , by fitting a biophysical model to the data  an approach already successfully implemented in various fields of medical image analysis ( ferrant et al . , 2001 ) .
in vivo images of gfp - tagged proteins combined with computational imaging has revealed the dynamic organization of various nuclear subcompartments in the interphase nucleus .
live - cell microscopy images of labeled pre - mrna splicing factors were examined for evidence of regulated dynamics by computational segmentation and tracking ( eils et al . , 2000 ) .
it was shown that the velocity and morphology of speckles , as well as budding events , were related to transcriptional activity .
1997 ) undergoes slow diffusional motion , and that this movement is confined to relatively small regions in the nucleus .
importantly , the constraint on diffusional motion is regulated throughout the cell cycle ( heun et al .
. a long - standing question has been whether nuclear compartments can also undergo directed , energy - dependent movements , thereby providing a potential mechanism of regulated gene expression .
computational imaging revealed that several nuclear subcompartments do undergo directional transport dependent on metabolic energy ( calapez et al .
, 2002 ; muratani et al . , 2002 ; platani et al . , 2002 ) .
the role of dynamic tension in actin polymerization in motile cells was investigated by analyzing polarized light images of the flow of the actin network and the motion of actin bundles and filopodia in crawling neurons ( oldenbourg et al . , 2000 ) . in a study on nuclear envelope breakdown , quantification and visualization of four - channel images with labeled chromatin , lamin - b receptor , nucleoporin , and tubulin ( beaudouin et al . , 2002 ) revealed that piercing of the nuclear envelope by spindle microtubules was the mechanism responsible for forming the initial hole during nuclear envelope breakdown . to further investigate stresses on the nuclear envelope during breakdown
stresses detected during hole formation were compared for the different grid vertices with respect to the positions of the hole , thus providing information about localized stresses during the tearing process ( mattes et al . , 2001 ) .
conversely , the effect of stress on the morphology of cells has been measured using an experimental approach of imposing known stresses on cells in solid - state culture .
changes in height , width , volume , and surface area of the cell are measured from 3-d confocal microscopy images , helping to understand the mechano - transduction response ( guilak , 1995 ) .
the positioning of chromosomes during the cell cycle was investigated in live mammalian cells with a combined experimental and computational approach .
in contrast to the random behavior predicted by a computer model of chromosome dynamics , a striking order of chromosomes was observed throughout mitosis ( gerlich et al . , 2003 ) .
further , strong similarities between daughter and mother cells were found for mitotic single chromosome positioning .
these results support the existence of an active mechanism that transmits chromosomal positions from one cell generation to the next .
computational imaging has been proven to be a powerful and integral part of cell biology .
computational imaging provides an important building block for the description of biological phenomena on a quantitative level , which is a prerequisite for mathematical models of dynamic structures and processes in the cell . in combination with models of biochemical processes and regulatory networks , computational imaging as part of the emerging field of systems biology ( kitano , 2002 ) will lead to the identification of novel principles of cellular regulation derived from the huge amount of experimental data that are currently generated .
 microscopy of cells has changed dramatically since its early days in the mid - seventeenth century . 
 image analysis has concurrently evolved from measurements of hand drawings and still photographs to computational methods that ( semi- ) automatically quantify objects , distances , concentrations , and velocities of cells and subcellular structures . 
 today 's imaging technologies generate a wealth of data that requires visualization and multi - dimensional and quantitative image analysis as prerequisites to turning qualitative data into quantitative values . 
 such quantitative data provide the basis for mathematical modeling of protein kinetics and biochemical signaling networks that , in turn , open the way toward a quantitative view of cell biology . here , we will review technologies for analyzing and reconstructing dynamic structures and processes in the living cell . 
 we will present live - cell studies that would have been impossible without computational imaging . 
 these applications illustrate the potential of computational imaging to enhance our knowledge of the dynamics of cellular structures and processes . 
Evolution of quantitative live-cell microscopy
Single particle tracking
Image registration
Computer vision
Visualization
Quantitative image analysis
Applications
Summary