Medical imagine processing
From 2D to 3D¶
Moving from 2D (two-dimensional) to 3D (three-dimensional), we move from pixels (picture elements) to voxels (volume elements). Pixels are more commonly known than voxels. Voxels are just the 3D counterpart of pixels, which are 2D. A 2D image is composed of many tiny squares. A 3D volume of images is composed of many tiny cubes. In 2D, we have one dimension for width, another dimension for height. In 3D, we get an additional measurement: depth.
Spartan demo #1¶
See the lungs, the heart and the remaining anatomy? This is a cross-sectional cut of someone’s upper torso. It looks kind of different, because medical images are usually not represented this way. But the different organs are completely recognisable. Here, each pixel houses a character. Some pixels house the same character, others don’t — it is from the similarities and differences in the characters housed that we make out the anatomy.
Demo #1: pixels at work¶
This is the same image represented in a more conventional way — each pixel is filled with a colour — some pixels share the same colour, others don’t — it is from the similarities and differences in colour that we make out the anatomy.
Pixels explained pixel-by-pixel¶
Back to the pixels housing characters. This image is in fact saved simply in text format, with file extension .txt. Completely rudimentary. Formats like jpg, gif, png etc would have been taken larger diskspace. We can in fact count how much diskspace we need for this text file. We have 86 rows of pixels. Each row consists of 147 columns of pixels, plus an invisible newline or carriage return marker at the end. Without these invisible newline markers, our image would have been a single row of 86 x 147 columns of pixels and we won’t be able to make out any anatomy. No, that’s not what we want. So we do need the invisible newline markers, which take up their small share of diskspace. The diskspace of the text file is therefore 86 x (147+1) = 12728 bytes.
Spartan demo #2¶
Here, again I have used the pixels-housing-characters format to show you an image. Guess what I’m trying to show you. Not easy. Because, this format can display images up to a certain point only. We need to resort to colours filling up pixels.
Demo #2: pixels at work¶
Voila, let there be colours. Here’s what I was trying to show you. Darling Pablo listening to me.
Radiographer, physicist, radiologist¶
The audience might feel attracted to any possible tips on how to decode CT, MRI, X-ray, ultrasound, PET images. But this talk offers nothing whatsoever of that sort; one would be really disappointed. Interpreting and diagnosing medical images are part of a radiologist’s remit. I’m no radiologist. I’m a physicist. My role is to ensure that features on images faithfully represent the inside of the patient’s body. The radiographer’s role is to position the patient in the best possible way, execute the best possible protocol and present the images to the radiologist. Together the radiologist, the physicist and the radiographer work hand in hand, striving towards the best diagnosis — either true positives or true negatives, never false positives or false negatives.
Slice whichever way you like¶
Apart from X-ray images, most modern-day medical images are in 3D. With a set of 3D data, we may make cross-sectional slices whichever way we like. Transverse, sagittal and coronal are common cuts. We can even make oblique cuts. Just an example — don’t bother to try imagining and getting the example right. Most of us do not think in 3D. Some people are able to; that gift is a truly great asset.
What’s in the pixel value?¶
The pixel value (or pixel intensity) may represent different quantities, depending on the imaging modality. Pixel values on photographs are a measure of visible light reflected by objects. Pixel values of CT (computed tomography) images are the Hounsfield Units (HU, or CT numbers) which measure linear attenuation of photons by the objects. Pixel values on PET (positron emission tomography) images measure radiotracer (18F, for example) uptake.
Noise and resolution shouldn’t be mixed up¶
See how the photo becomes ‘not so nice’ with the addition of noise and the degradation of resolution. Noise makes the photo grainy. Poor resolution blurs the photo.
Image contrast¶
Let us move from photos to medical images. See the adverse effects of adding noise, degrading contrast, and degrading resolution from 105x147 (105 rows of 147 columns of pixels) to 53x74 (53 rows of 74 columns of pixels).
Image "quality"¶
High resolution alone does not promise good image or display quality. Contrast, noise and resolution are the three quantities for gauging image quality. The sharper the contrast (less blur), the higher the image quality. The lower the noise, the higher the image quality. The smaller the pixel size, the higher the resolution and therefore the higher the image quality. Sharp contrast alone, however, does not imply good image quality — we must still check the noise level and the resolution. By the same token, low noise, or high resolution, alone cannot make a good image. For an image to be of high quality, we need a combination of all three factors: sharp contrast and low noise and high resolution. The implications of contrast, noise and resolution of medical images are not whether the pictures look nice or not. They may cause artefacts or misrepresentations suggesting features which are in fact absent in the body of the person scanned. They may also cause important features not to be visible. This would lead to wrong diagnosis.
Magnifying¶
For pixel graphics, magnifying the image without increasing the resolution would degrade display. We would see step artefacts where smooth surfaces become stepped or sawtoothed. Vector graphics, on the other hand, would not have this problem when magnified.
Vector graphics¶
Here’s an example. Zooming in pixel graphics will degrade the image. Zooming in vector graphics won’t
Behind the scenes¶
By the way, here is the making of the pair of hands which occasionally appears on this website: both hands are really mine. My right hand, which had a fracture when I was thrown off a mule like a projectile back in September 2010. We were enjoying a beautiful walk on a wonderful day when she broke into an unplanned gallop. I took the X-ray images and superimposed my bones over my favourite textile using GIMP — a fantastic open-source alternative to Photoshop.
Reducing noice, improving contrast, increasing resolution¶
Given an image, can we reduce the noise? Yes, denoising is a commonly available tool. We can denoise the image, usually at the expense of resolution. Most denoising algorithms inevitably smoothen. Given an image, can we improve the contrast? Yes, through edge detection and sharpening. Where pixel values jump, we can exaggerate the difference and make the gradient even higher. Given an image, can we increase the resolution? For pixel graphics, no! Even if we upsample, say, from 256×256 to 512×512, we won’t be able to recover information which was never there in the original 256×256 image.
Partial volume effect¶
Partial volume artefact: we cannot have any details within a pixel. A pixel can only take one pixel value, or one colour. It cannot be partly this colour, partly another colour. The bigger the size of the pixel, the lower the resolution, partial volume artefact becomes more pronounced.
Compare 4 images: part I¶
All four images on this slide have a common resolution. Low noise alone cannot give us a good image unless contrast is good (high). High contrast alone cannot give us a good image unless noise is low.
Compare 4 images: part II¶
All four images on this slide have a common contrast. Low noise alone cannot give us a good image unless resolution is good (high). High resolution alone cannot give us a good image unless noise is low.
Cannulation during CT scans¶
In an imaging clinic, nurses are often needed to administer cannulation on the patient. Contrast media (or contrast agents) are either injected or ingested into the body. These foreign compounds accumulate in specific tissues only, highlighting specific areas to help radiologists see.
Grayscale is not black-and-white¶
Fancy color-maps I¶
Fancy color-maps II¶
Color maps: a bad example¶
Delineating white matter from gray matter¶
Color maps: a good example¶
Color assignment¶
Digital photos vs medical images¶
Windowing¶
Windowing changes the appearance of an image. Different windowing can expose and hide certain features. Limiting the window so that the window is smaller than the full range of pixel values will expose more details within the window, but will lose details outside the window. Here, window refers to the range of displayed pixel values, not the geometric field of view.
Wide windows¶
Setting the window wider than the full range of pixel values will reduce contrast by not making full use of the full range of colours.
Narrow windows¶
We lose some features and gain other features when we zoom into a narrow window.
Inherent vs display contrasts¶
Inherent contrast is limited by the detection system. We can’t change the inherent contrast by windowing but we can tune the display to taste by windowing.
Guys we’ve been seeing¶
The two gentlemen starred in this slideshow...
Pablo and VIP-Man¶
Monsieur Pablo, the darling mule who listened like a horse. Who says donkeys are dumb? I took the picture, myself on his back. Mr Joseph Paul Jernigan, who kindly consented to the prison chaplain to donate his body for scientific research or medical use. The Visible Human Project sliced up his corpse at 1 mm intervals. I believe he did not envisage such a global fame prior to execution by lethal injection. He posed himself as my virtual patient and received countless radiotherapy shoots in my research. I’m against execution and I do not believe in punishing except as deterrent; I count Mr Jernigan one among the smiling species. I choose not to express this stand of mine by shying away from this precious data collected from a human body.
Quantities for imagine¶
Image quality quantified. Basic: spatial resolution, contrast, noise. Integrating: MTF (modulation transfer function) and SNR (signal-to-noise ratio). Unifying: DQE (detection quantum efficiency).
QA phantoms¶
Physicists should be able to recognise different QA phantoms, even without seeing the model or brand before. Phantoms with line pairs test spatial resolution. Phantoms with areas of varying sizes against a background of a different colour check contrast. Phantoms with areas of the same colour check uniformity.
Origin of noise¶
The origin of noise ...
Photoelectric or Compton?¶
Scatter causes noise. So what’s wrong with scatter? Compton scattering, for example, emits the photon at an angle away from the original. When detected, this photon would suggest an origin which is not true, causing noise. Photoelectric absorption, on the other hand, do not emit scatter photons.
Field size¶
Physicists needs to be able to estimate the file size of medical images, which is the total number of voxels multiplied bytes per pixel, plus the size of the metadata, which is present in all DICOM (and some NIfTI) files.
Matrix dimensions¶
This is how the total number of voxels is calculated.
Pitch¶
This is the pitch.
Bytes per pixel¶
Bytes-per-pixel depends on the colour depth. Deeper depths require more bytes per pixel and allow more shades on our palette.
Magic number 256¶
1 byte = 8 bits. One byte allows 256 colour depths, because the number of possible numbers in the range 0 to 1111 1111 is 256. This is the reason why we always see 256 and 512 if not 1024.
Metadata¶
The metadata.