Chapter 5

Some necessary mathematical preliminaries : What Frequency space is all about 1 (277)

▪ Some computations are more efficient in frequency domain rather then spatial domain.

▪ Image compression. Image degradation is bad. Speed not that big of an issue anymore.

▪ FFT Fast Fourier Transform alogorithm to compute the fourier transform.

▪ Start with 1D waveform and then expand It to 2D(image)

The Fourier Transforms of real functions 278

▪ Fitting functions with terms from fourier expansion.

▪ The greater the number of terms, the better the image quality.

▪ To get power of two or deal with edges, pad out image to larger size.

▪ Step functions – Add up sine waves to produce step.

▪ Possible to form any 1D function f(x) as a summation of a series of sine and cosine terms of increasing frequency.

▪ The Fourier transform of f(x) is F(u).

▪ The Fourier transform describes the amount of each frequency that must be added together to make f(x).

▪ The greater the number of terms, the better the fit.

▪ Repetitive or cyclical. Fit goes past edges…Padding. For this and to make an exact power of two.

▪ No info is lost.

▪ One point in a Fourier image contains info about the entire image. Frequencies are independent.

▪ Power spectrum – where power is the square of the magnitude. Everything’s scaled down.

▪ lower frequency represents shape, higher frequency represents details/edges.

▪ Superimpose frequency transforms, get sum of 3 individual transforms.

▪ Lower the frequency, the closer to the center.

▪ Frequency transforms rotate with features.

Orientation and Spacing 286

▪ Peak location measurement.

▪ periodic structures (in spatial) = peaks (in frequency)

▪ Periodic spatial structures = frequency peaks

▪ Radius based on spacing / direction corresponds to orientation.

▪ Direction based on orientation

▪ Most images aren’t periodic – means peaks with noisy backgrounds.

▪ Analog microscope optics to make frequency domain transformations. But you lose phase info (so you can’t construct it) and the computer does it quicker and better.

▪ Computer also allows you to isolate ROI’s and adjust contrast.

▪ The diffraction pattern is analogous to the frequency domain representation.

Preferred Orientation 290

▪ Reconstructing periodic structures that aren’t perfectly aligned.

▪ In frequency transform you can see repeating structures and angular variations.

▪ Retransform with just peaks and then mask with spatial image.

▪ arcs = variations in orientations – easily quantified.

▪ length of arcs and variation in brightness determines preferred orientation.

▪ method : select arc (its length determines angles), retransform just arc, use spatial domain as mask.

▪ Astigmatism – defocusing of image and loss of sharpness in one direction.

▪ Ellipse not circle because of bad microscope optics.

▪ correct focus – you should see a gradual drop off in frequencies.

▪ bad focus – asymmetric power spectrum

▪ due to camera limitations. Spacing of detectors. FT shows these limitations in resolution.

Isolating Periodic Noise 297

▪ Subtract unwanted part of image (ex. Periodic noise)

▪ Subtraction – get rid of periodic noise. Find out the frequency of the noise. Set amplitude of those frequencies to zero, then transform back to the spatial domain.

▪ Select portions of frequency transform based on frequency. Low (keep low)/high (keep high) pass filters. Draw a circle.

▪ equivalent to Gaussian (smoothing) or laplacian (sharpening)

▪ Arbritary filters cause ringing (oscillations near edges), so you have to shape the edges of filter.

▪ filter examples à

- linear interpolation/parzen window function

- parabolic cosine function/Welch and hanning window function

- Gaussian

- butterworth

▪ Tophat, annular, threshold.

▪ Ringing at edges – shape the edges of filter AND increase the distance over which the transition takes place.

Masks and Filters 303

▪ phase à where in image the frequency occurs.

▪ Noise peaks/spikes isolated with small, smoothed circles. Set magnitude of those locations to zero but don’t alter the phase info (because the phase info tells where f is in an image)

▪ Manually select regions on power spectrum.

▪ Compression dosent remove spikes. Compression gets rid of terms whose magnitudes are small. Reduces high f info.

▪ auto detection – peaks in the power spectrum that are narrow and rise significantly above the local background should be removed. Use rank based filter (ex top hat) to remove peaks.

▪ other simple method – select region that exhibits noise, clear rest of image, make inverse of this and multiply against original image.

▪ halftone dots are a common example.

▪ Auto or manually select regions – Smooth, level, threshold for peaks

Motion Blur 325

▪ Low light – requires longer exposure and results in image blur.

▪ Divide blurred image by defect.

- identify motion blur with a line on spatial.

-Take frequency transform of that line and divide it into transform of blurred image.

- Retransform to get sharp image.

▪ Draw motion vector based on speed, direction and exposure to define blur.

▪ Divide frequency transform of line into transform of blurred image.

▪ Creates noise – use Weiner Deconvolution to reduce.

▪ limitations – precision – bit depth.

Template matching and correlation 328

▪ Find stuff

- Shift target path to every location in the image

- Multiply values by overlaid pixels

- Total is stored at that position to form image showing where regions identical or similar to the target are located.

▪ Locate features w/in images. Vehicles, hurricanes, text in binary images.

▪ Parallax to calculate range

▪ Align serial images

▪ quicker and more efficient in the frequency space.

▪ fusion – locating matching points in two images, measure parallax, calculate range.

▪ pad edges with zeros or average.

Dyadic operations – 2 images

Deconvolution – division

Convolution – multiplication

Correlation – multiplication by the conjugate. Locating features in one image that appear in another.

Monadic operations

Filtering and masking

Conclusion

▪ Remove noise and image blur

▪ Apply large convolution kernels

▪ enhance periodic structures

▪ locate and define structures

▪ measure image for periodicity or preferred orientations

▪ can do this all in the spatial domain as well, but it is more efficient in frequency.

Thresholding 333

▪ Segmentation - dividing image into interest/foreground and background.

▪ Humans cant see twice at the same time, group. Computers look at individual pixels.

▪ Select features by choosing a brightness range. If its in the range – make it the foreground. If it’s not – set it to the background.

▪ Range determines foreground. Not in range, then it’s the background.

▪ Adjust using histogram, user selecting range

▪ Histograms with > 256 levels. Big. Scale down.

▪ Peaks = distinct structures. Peaks = phase. No location information.

▪ Histogram analysis by fitting Gaussian functions to histogram.

Multiband images 336

▪ ex. Color imaging or combining images that have undergone different processing operations.

▪ Segment multiple original images. Combine different bands (ex. Texture/brightness)

▪ See things in one band that you may not see in another

▪ Brightness threshold image, combine all with an AND

▪ Calculate hue from RGB then combine. Convert RGB pixel values to HIS b/c that’s what we’re used to.

▪ 3d hard to do. Mark pixel/region in image and see color values in color space. Combine 3 one d histograms/threshold levels.

Boundary Lines 367

▪ Threshold based on brightness instead of location

▪ Manually outline regions (mouse, stylus, touch senser) problems.

▪ Draw polygonal region outline. Humans tend to draw bigger.

▪ Boundary = step in brightness. Humans start computer off.

▪ Automatic edge following … problems with connectedness and forked edges. Hard to look ahead. Dosen’t know when to end.

1▪ faster Get boundary lines with thresholding and skeletonization.

2▪ user draws line near boundary and the algorithm moves the point onto the nearest darkest point. Active countours/snakes

Countours 370

▪ Contours = constant brightness.

▪ Provide boundary info, guaranteed to be continuous.

▪ Iso elevation contour lines on topographical maps.

▪ Lines mark constant elevation/brightness.

▪ Delineate features/structural meaning/similarity/ show minor variations in brightness.

▪ method

scan pixels once

compare each pixel and its neighbor above and to te left to contour value

mark pixel if the values bracket the test value.

▪ sub pixel sampling/measurement. Measuring locations of lines.

Image Representation 373

▪ RLE (Chord encoding) image as scan lines. Good with area and position measurements. Faxs. Line number, start position, length of line.

▪ Good with area and position, bad with perimeter and shape but boundary cords is good with this.

▪ can’t just inverse image…ambiguous, touching pixels

▪ Square pixel array better then rectangle.

▪ But diagonals are still farther away- solution – hexagonal – but cameras don’t take pictures like this.

▪ boundary from endpoint in series of chords – calculate perimeter or describe shape.

▪ chain code has shape info – corners, simplify shape outline.

Other segmentation methods 376

▪ Split and merge and Region growing

▪ Split and merge – top down , subdividing quadrants. Parent and children relationships in quadtree. May merge together stuff too much.

Depends on quality of test for homogeneity.

Looks at more then one region at a time.

Image property criteria for uniformity. Not uniform? Continue subdivide.

Complete segmentation in a finite iteration.

▪ Region growing – bottom up. Examine neighbor and add to growing region if they are similar. Good with small areas/ few regions.

▪ edge following. Snakes (deformable boundaries). Moving boundaries in a sequence of images.

Chapter 7

Boolean operators 383

▪ Boolean – combining images. Morphological, modify individual pixels within image.

▪ Combine images. Combine info from several different image planes.

▪ Colocalization plot…pixel brightness and location determines coordinate.

▪ Operators. Pixel by pixel.

- AND – both pixels on – on. Ex. Color thresholding channels. Quick and efficient.

- OR – either on – on

- Ex Or- either on, but not both on – on

- NOT – revere each pixel (only need 1 image)

▪ combining – order of operations important.

Combining Boolean Operations 387

▪ Multiple criteria for selecting foreground – combine with Boolean operators

▪ Ex. Multiband images – shows locations and concentration.

▪ Multiple image planes (ex diff colors or elements or diff processing operations (brightness and texture) use and to combine seperatly thresholded images. Complex specimen with many elements.

Masks 389

▪ Binary image to mask out grey scale images.

▪ Modify stored image by multiplying it.

▪ Blank out parts of grey scale images

- over lay/alpha channel

- multiply grey scale by binary

▪ Dilate labels to make them easier to see.

▪ Overlays. Combine portions of two or more images.

Erosion and Dilation 409

▪ Neighborhood operations.

▪ Morphological operations on binary images. Process greyscale images in the spatial domain.

▪ Adding or removing pixels from binary images depending on pattern of neighboring pixels.

▪ can change just part of image.

▪ Erosion – turn pixels off that were on.. Remove pixels from features. Remove pixels that shouldn’t be there.

Ex. Pixels that were selected because they fell into brightness range of image.

▪ Turn off pixel that is part of background (which is already off) – classical erosion. May cause shrinking/breaking up features.

▪ Also known as etching/plating or shrinking/growing because they cause a reduction or increase in the size of regions.

▪ Dilation – add pixels. Add layer of pixels around all features and regions. Merging and increase in dimension. Fills in small holes within features.

Opening and Closing 410

▪ Remove pixel noise from binary images à

EDO opening : erosion followed by dilation. Opens up gaps b/n just touching features, separate touching features. Remove pixel noise.

DEC closing : opposite , closes breaks in features.

▪ Control with neighbor pattern and number of iterations

▪ opening to separate touching features. After separation dilation grows features back towards their original size.

▪ when growing back- use logic – don’t turn pixel on if it belongs to a different feature.

Don’t turn on any pixel that was not on in the original image.

▪ coefficients – count pixels in neighborhood that are opposite, compare this number to a threshold value and only change the state of the central pixel if that test coefficient is exceeded.

Change coefficient value to alter the rate at which features grown and shrink.

Isotropy 413

▪ Erosion on circle will not shrink circle uniformly. Faster rate at 45 degree diagonal direction – erode toward a diamond shape.

Dilation will turn a circle to square. Squares will be unaffected.

▪ Use a coefficient of 1 instead of 0 and see opposite results. Erode to square, dilate to diamond

▪ No intermediate between results 0 and 1…..1.5 ratio (corner as 2, edge as 3) or alternate 0 and 1

▪ Do operations more then once – depth of operation

▪ Larger neighborhood – another solution to moderate anisotropy.

Euclidean Distance Map 425

▪ Avoids directional bias in morphological operations because of their restrictions to pixels on a square grid.

▪ EDM – make a greyscale image (EDM) from a binary image where every pixel within a feature is assigned a value that is its distance from the nearest background pixel.

Works on binary to make grey scale. Each pixel in foreground is given a brightness value equal to its straight line distance from the nearest point in the background (dist = each pixel in feature to nearest pixel in background)

▪ Inefficient. Alternative - restrict to 90/45 degree direction…just like using 4 or 8 neighborhood. Limited directions.

Watershed segmentation 429

▪ Separate touching features.

▪ Difficulty when features touch and cant be seperatly identified, counter or measured.

▪ Relies on fact that eroding a binary image will cause touching features to separate before they disappear

▪ Repeatedly erode image at each step, features that disappeared from the previous step are designated ultimate erode points (UEP’s) and saved as an image with the iteration number.

▪ Continue until the image is erased

▪ Grow back UEP’s image with dilation to their original boundaries except with lines of separation between touching features. Ignore if it causes a connection.

▪ Slow, requires lots of storage.

▪ EDM more efficient.

Brightness value of each pixel correspond to physical elevation features appear as a mountain peak.

Features that touch or overlap get two peaks.

Where they meet – watershed lines. Boundaries. Separation between features.

▪ Can’t have concave/ irregular particles

▪ Can’t handle too big of overlaps.

▪ Can’t handle holes in features. Confuses watershed and breaks the features up into many fragments. Fill holes before applying watershed.

Chapter 11

Volume imagine versus sections 555

▪ We’re visual, need to see things. 2d lacks topological properties.

▪ 2D sections aren’t sufficient.

▪ 3D imaging – directly by gathering a 3d set of information or indirectly by fathering a sequence of 2d (slice) images.

▪ Tomography – reconstructs internal structural information within an object by mathematically reconstructing it from a series of projections.

▪ Medical tomography – xray, mri (magnetic resoace imaging), pet (positron emission tomography), ultrasound (sound waves).

▪ other signals – bounce electrons, protons, sound, liquid.

Basics of Reconstruction 558

▪ Absorption tomography – based on physical processes that reduce intensity as radiation or particles pass through the samples in straight lines.

Ex. Xray absorbed according to the composition and density which they encounter.

▪ intensity – photons per second. Reduced according to linear attenuation coefficient (product of density/mass absorbent coefficient)

▪ phantom – test object w/ geometric shapes of known density.

▪ sonogram/radon transform – collection of 3d views corresponding to angle and position.

▪ filtered back projection – attenuation values of each view are projected back through the object space along each projection line.

▪ view – profile of intensity found by measuring a series of parallel lines.

▪ sonogram/radon transform – collection of “views”

▪ phantom – sample planar figure. Test object with geometrical shapes of known density.

▪ basic tomographic imaging principal

1 get views/sets of projections

2 frequency transform each

3. plot in 3D complex image

4. reconstruct.

▪ other method – filtered back projection – superimpose density or attenuation values from many vies.

Cons – too much weight in the center, blurred edges.

Fix by suppressing the low frequencies and enhancing the high f edges with a inverse filter.

Chapter 12

Sources of 3d data 591

▪ volume imaging by tomographic reconstruction – measure density and composition.

▪ cubic voxels are easier. Voxel – volument element.

▪ planes of pixels (define one plane at a time as an array of square pixels) vs an array of true voxels.

▪ slicing and reconstructing. Cut w/ microtome. Alignment problems. Distortion from cutting. Stretch to fit.

▪ fiducial marks.

▪ methods – tomographic reconstruction (measure density/composition)

- can produce a set of cubic voxels

- can use many different signals

- resolution varies

▪ computer graphics

▪ dissect sample into a series of planar sections, then pile up as a stack of voxels. Physical sectioning.

Serial sections 592

▪ serial section methods – align, structures as a guide. Use a binary ex-or. Auto

▪ fiducial marks – ex drill holes with a laser.

▪ microtome produces distortion. Stretch to connect. Estimate distortion/compression.

▪ calibrating depth. Varies. Hard to get accurate depth scale.

▪ using only some of the sections. Thin.

Optical sectioning 596

▪ physical sectioning destroys sample. Hard to control

▪ confocal scanning light microscope

light from a point source is focused on asingle point in the specimen and collected by a identical set of optics with a pinhole detector.

Shallow depth of field and high rejection of stray light.

▪ optical sectioning à isolating a single blane within a bulk sample.

- transmitted or -reflected light images.

Stereo Measurement 600

▪ human stereoscopic vision. Judge relative distance to objects. Our field of view overlaps.

▪ other ways to tell depth – shadowing, relative size, precedence, atmospheric effects and motion flow.

▪ sem good with stereoscopy because of its large depth of field.

▪ basic idea – measuring the relief of the surface which is the equivalent to what is done in analyzing topographic elevation.

▪ match two points in a stereo pair image by cross correlating between the two images looking for a similar pattern.

▪ match many points, get a new image where the pixel value is based on parallax so it represents elevation.

▪ median filter gets rid of bad points.

▪ second approach to matching stereo pairs – focus on discontinuities/interesting features, get a much shorter list of pairs.

Correlate resulting short list of points.

Interpolate straight lines segments between the points, create a complete display of elevation (range image) which is also a contour map of surface.

▪ displaying stereo pairs, keeping track of all the points.

▪ markers for computer.

3d Data Sets 604

▪ database of points. Used for measurement (just need coordinates and info about which points are connected) or reconstruction (will require you to interpolate additional points)

▪ most common way to store 3d data is as a series of 2d images (array of pixels with depth)

▪ use voxels (cubic)

▪ discussion of how voxels are arranged. Voxel stacking arrangements.

▪ symmetrical neighborhood with neighbors at uniform distances simplifies processing.

▪ cubic arrays are the most common 3d arrangement of voxels.

▪ adjusting to cubic of not already

1. interpolating additional planes of voxels between those that have been measured.

2. reduce the resolution within the plane by sampling every nth pixel.

▪ formats for storing 3d data sets.

1. stack of individual images

2. array of voxel values.

▪ the third dimension of 3d images makes file size significantly larger.

▪ compress with jpeg, rle, mpeg (high correlation)

▪ the more storage it takes, the quicker you get the info.

▪ difficulty of aligning images from sifferent sources.

Two methods

1 cross correlate

2 use specific features as fiducial marks.

Slicing the data set 606

▪ animate series of 2d images. Give user control of feedback. Allow change in orientation.

▪ slices – individual image planes

▪ different views

transaxial – perpendicular to the spine

sagittal – parallel to the spine

coronal – parallel to the spine and perpendicular to the “straight ahead” line of sight.

▪ extend the voxels in space (intersecting box picture)

▪ combine views using orthogonal planes.

▪ issues of showing animation in books. Flip books. Including cdroms, etc.

▪ multidimensional data, transitions, varying opacity.

▪ time as the third dimension.

Volumetric display 615

▪ sectioning the data set obscures much of the voxel array.

▪ a volumetric display is produced by ray tracing.

▪ ray tracing – uses an alogorithm to calculate the effect of light on the surface of objects.

▪ put light source behind voxel array, rays each each point on the display generates an image.

▪ density determined by the absorption of light.

▪ allows you to change direction of view easily.

▪ true ray traced images include refraction and reflection.

▪ shadows make for more realistic 3d images.

▪ rotation the array (changing the view direction) interactively with desktop computers.

Stereo viewing 618

▪ two side by images in a rotation can be viewed as a stereo pair.

▪ some can see it, some can’t. some have stereo vision. Some do not.

▪ show it directly on screen. Side by side.

▪ use red and green, glasses, only with greyscale images.

▪ removing artifacts. Same procedure as on 2d images.

▪ projecting the images, special polarized projectors.

▪ adjust depth by changing the viewing angle of the two images.

Color cube