Test Protocol for Intraoral Video Cameras

Revised: August 22, 2006 General Science Department
Clinical Research Associates

  1. Product Information
    1. Record the following:
    2. Brand nameof camera.
    3. Manufacturer's name, address, toll-free phone number, local phone number, fax number, web address, email, & contact names & titles.
    4. Cost of basic camera system & items included.
    5. Optional items & optional system configurations & their cost.
    6. Safety certifications shown on the camera (UL, CSA, CE, etc.).
  2. Technical Specifcations.
    1. Record all manufacturer's technical specifications that may relate to clinical performance (i.e. camera type, sensor size & resolution, auto-exposure & white balance controls, etc.)
  3. Dimensions.
    1. Measure &record the following:
    2. Dimensions of camera, system, & portable module if applicable.
    3. Weight of camera, system, & portable module if applicable.
    4. Length of camera handpiece cord.
    5. Length, diameter, & shape of camera handpiece.
    6. Width of camera head.
    7. Thickness of camera head from lens to back of head.
    8. Size & weight of foot pedal.
  4. Controls.
    1. Describe all controls provided, including the following if applicable:
    2. Power switches.
    3. Capture (freeze) & return to live.
    4. Focus.
    5. Print.
    6. Toggle between live & frozen image.
    7. Light source.
    8. Toggle between multiple view modes.
    9. Review of captured images controls.
    10. Camera handpiece cradle switch (or other remote interlocks) & if switch controls light, video or both.
  5. Infection Control.
    1. Barriers
      1. Record ease of placement, including any problems or challenges.
      2. Record amount of haze, distortion, glare, etc. caused by sheath. Document with photos, if possible.
      3. Record fit of sheath over lens & handpiece body.
      4. Record effectiveness of sheath, including pinholes, tears, incomplete coverage, etc.
      5. Record any interference with handpiece controls.
      6. Record ease of removal.
      7. Record use aspects of any other barriers provided.
      8. Rate sheaths on all characteristics relative to expectations & other sheaths evaluated.
      9. Record cost of barriers.
    2. Handpiece disinfection & sterilization.
      1. Record manufacturer's recommended disinfection procedures.
      2. Record manufacturer's recommended disinfection solutions.
    3. General disinfection.
      1. Record all controls or surfaces that are touched during routine use of camera & their options for disinfection.
    4. If camera claims ability to withstand immersion disinfection then perform following test:
      1. Place camera in Positioning Fixture, focus on Standard Image.
      2. Capture & print Standard Image, label as "baseline".
      3. Disinfect camera per manufacturer's directions (for example: 3.4% glutaraldehyde soak for 1 hour).
      4. Remove camera & rinse, clean, & dry per manufacturer's directions.
      5. Capture & print Standard Image, note any differences in performance or image quality.
      6. Repeat soaking, rinsing, & capturing of Standard Image as many times as possible to test endurance of camera.
      7. Label all prints with the camera name, date, & number of soakings completed.
  6. Clinical Performance.
    1. Rate clinical performance while capturing the following test shots:
      • Intraoral: distal of upper last molar (tooth #2).
      • Intraoral: buccal of upper last molar (tooth #2).
      • Intraoral: lower lingual cuspid to cuspid (teeth #22 - 27).
      • Extraoral: lower full arch, occlusal.
      • Extraoral: full face.
      • Extraoral: bitewing x-ray on view box
      1. Note if each image has acceptable & useful quality & diagnostic detail.
      2. Note amount of handpiece manipulation required to capture image (i.e. focus adjustment, lens change, etc.).
      3. Rate ease of intraoral access, especially in posterior regions while taking distal & buccal photos.
      4. Rate image quality in both resolution & color. Use a high quality video monitor (CRT or broadcast quality LCD). (An image of the lower anterior mucco-buccal fold is a good supplemental shot. Have the patient pull out their lower lip & capture from the incisal of the lower anteriors to the lower lip. Look for correct coloration in the soft & hard tissue, & fine detail such as plaque, calculus, & the fine blood vessels.)
      5. Compare resolution & color to a known high quality digital clinical camera (Fuji FinePix Pro S1 with macro lens & ring flash).
      6. Rate ease of focus & ability to achieve sharp focus.
      7. Note ease of manipulating controls & any problems encountered.
      8. Note ergonomics of handpiece, including cable drag & stiffness, location of buttons, location of focus control, diameter of gripped area of handpiece, etc.
    2. Perform video exams of patients, viewing all area of mouth & note clinical ease of use, including manipulation of controls, focusing, image quality, & image stability. Note ease of capturing desired images for later review with patient.
    3. Close-up capability
      1. Identify a small feature to be shown close-up (i.e. degraded margin of old restoration, crack, small caries, etc.).
      2. Capture image of area with greatest magnification possible while retaining sharp focus & clarity.
      3. Determine magnification by comparing actual size of area (defect) to that shown when displayed on a standard video monitor.
    4. Speed of Use.
      1. Record if camera activates when picked up, & what activates: light, video, or both.
      2. Note ease of sheath placement.
      3. Note ease of setting controls to proper settings.
      4. Note ease of access & obtaining desired views.
      5. Note any image quality problems that detract from ease of use.
      6. Note ease of capturing desired images for review with patient.
      7. Note ergonomics & "feel" of handpiece that improve ease of use.
      8. Rate overall speed of use based on these factors & comments from clinical evaluators.
    5. Distortion
      1. Note amount of distortion evident in both intraoral & extraoral shots.
      2. Rate distortion based on clinical images.
    6. Setup & maintenance
      1. Note steps required & difficulty of connecting camera to docking station or computer or video network in operatory.
      2. Note portion or portions of system that are designed to be transferred between operatories (portability).
      3. Note ease of initial setup of system, including software installation.
      4. Note ease of routine maintenance that may be required during use or between patients.
    7. Unique features
      1. Note all features that facilitate ease & speed of use (i.e. auto-focus, image memory, on-screen pointer, anti-fogging, etc.).
      2. Note any features or characteristics that impede ease of use.
  7. Comparison of Multiple Cameras.
    1. Perform following steps to facilitate comparison & ratings of image quality.
      1. Capture the same test images & save with the highest resolution possible.
      2. View images on the same high quality monitor to facilitate comparison.
      3. Group similar cameras & score according to image sharpness & natural color.
      4. If possible, compare live images on separate systems simultaneously.
  8. Technical Measurements.
    1. Angle of View & Field of View.
      1. Prepare a grid of 1 mm squares approximately 5 cm X 5 cm. Bold the vertical & horizontal center lines of the grid.
      2. Place the grid on a spacer block about 5 cm high.
      3. Drop a plumb line down to the horizontal center line of the grid, just to one side of the vertical center line. Tie the plumb line to a support (such as a lab stand).
      4. Position the camera lens above the center of the grid using a lab stand & clamp. The lens should be parallel to the grid below, & the plumb line should pass by the center of the lens (see figure 1).



        Figure 1: Setup for measuring angle of view, field of view, & magnification.

      5. Examine the camera monitor to determine if the angle of view differs from 90 degrees. If the horizontal center line of the grid does not pass through the center of the screen, then the angle of view is greater than or less than 90 degrees & must be calculated (see figure 2). Note: The vertical lines are not always perfectly centered either, but may be shifted left or right or rotated!



        Figure 2: Screen display during measurement of angle of view.

      6. Calculate the angle of view as follows:
        • θ = tan-1(d/h) + 90
        • θ = angle of view from the horizontal (degrees)
        • d = difference between center of grid & center of screen (in mm, as shown by grid)
        • h = height of camera lens above grid (mm)
        • [Note: if the center of the grid is below the center of the screen then θ = 90 - tan-1(d/h).]
      7. Calculate field of view from the same screen image as follows (see figure 3):
        • αhor = 2 x tan-1(w/2h)
        • αver = 2 x tan-1(H/2h)
        • αhor = horizontal field of view (degrees)
        • αver = vertical field of view (degrees)
        • w = width of screen (in mm, as shown by grid)
        • h = height of camera lens above grid (mm)
        • H = height of screen (in mm, as shown by grid)



        Figure 3: Screen display during measurement of field of view.

    2. Magnification.
      • Note: Magnification should always be measured for an object in sharp focus, but since many cameras have a variable focus &/or a large depth of field, the magnification can vary over a range of values. Therefore, it is necessary to decide beforehand how to standardize the magnification measurement. Examples: 1) Measure maximum magnification at the closest focus; 2) Measure the range of magnification across the depth of field at a fixed focal setting; etc. In any case, the focal setting (if applicable) & distance between the camera & object should be specified along with the magnification.
      1. Measure magnification at the same time, & using the same setup as the Angle of View test (see figure 1).
      2. Lay a ruler on the monitor screen & measure the apparent width (in mm) of one square in the middle of the screen (see figure 4).



        Figure 4: Taking the magnification measurement right from the monitor.

      3. Calculate the magnification as follows:
        • Mag = L/m
        • where
        • Mag = magnification (no units) for the specified focus setting & distance to the grid.
        • L = apparent length of 1 mm (mm)
        • m = actual length of 1 mm (mm)
    3. Depth of field: "Grid focus" method.
      • Note: Cameras with a variable focus may have a unique depth of field at any given focus setting, thus the focus setting must be specified along with the depth of field.
      1. Set up the camera & grid as for the Angle of View measurement, omitting the plumb line (see figure 5).
      2. If the camera has a variable focus then set it to the desired value & record it.
      3. Lower the camera towards the grid until the grid is noticeably out of focus.
      4. Slowly raise the camera until the grid is just perceived to be in focus.
      5. Record the distance (d, in mm) between the camera lens & the grid.
      6. Raise the camera away from the grid until the grid is noticeably out of focus.
      7. Slowly lower the camera until the grid is just perceived to be in focus.
      8. Record the distance (D, in mm) between the camera lens & the grid.



        Figure 5: Setup for measuring depth of field.

      9. Calculate the depth of field as follows:
        • δ = D - d
        • where
        • δ = depth of field (mm)
        • D = maximum distance in focus (mm)
        • d = minimum distance in focus (mm)
      10. Note: This method tends to exaggerate the actual depth of field defined by the distance in "sharp focus". This is due to human perception. The same is true clinically, where the eye & brain indicate that an image is acceptable because detail can be resolved, even though it is slightly out of focus. Another approach is to standardize the desired resolution (as explained in the next test), but this also has serious clinical drawbacks. Consequently, measuring depth of field is always subjective, depending on the observer's perception of what is in focus & acceptable & what is not.
    4. Depth of field: "Line Pairs" method.
      • Note: Cameras with a variable focus may have a unique depth of field at any given focus setting, thus the focus setting must be specified along with the depth of field.
      1. Clamp the camera into an adjustable lab stand.
      2. Place a line pair target (USAF 1951 target, or Edmund's Scientific Depth of Field target, or Welch Allyn starburst target, or equivalent) on the spacer block under the camera.
      3. Using the data from the Angle of View test, position the camera to look straight down at the target.
      4. Select the desired number of line pairs per millimeter that you want to resolve at all times. This decision is somewhat arbitrary since any given line pair may not represent the clinically useful range of the camera. For example: At the distance required to focus on a single tooth surface the camera may resolve 10 line pairs / mm, but at the distance required to look at the lower arch at the same focus setting it may only be able to resolve 6 line pairs / mm, even though the lower arch still appears to be in sharp focus to the human eye.
      5. If the camera has a variable focus then set it to the desired value & record it.
      6. Lower the camera towards the target until the selected line pairs can no longer be resolved.
      7. Slowly raise the camera until the line pairs are just able to be resolved.
      8. Record the distance (d, in mm) between the camera lens & the target.
      9. Raise the camera away from the target until the line pairs can no longer be resolved.
      10. Slowly lower the camera until the line pairs are just able to be resolved.
      11. Record the distance (D, in mm) between the camera lens & the target.
      12. Calculate the depth of field as follows:
        • δ = D - d
        • where
        • δ = depth of field (mm) for the specified line pair resolution
        • D = maximum distance where line pairs could be resolved (mm)
        • d = minimum distance where line pairs could be resolved (mm)
    5. Depth of field: "Clinical Focus" method.
      • Note: Cameras with a variable focus may have a unique depth of field at any given focus setting, thus the focus setting must be specified along with the depth of field.
      1. Set up the camera as for the Angle of View measurement, omitting the plumb line & grid (see figure 5).
      2. Place a model quadrant (with real teeth) on the table below the camera lens.
      3. Position the teeth under the lens with a visually distinct & interesting feature in view (i.e. stained pit or fissure, margin of restoration, etc.).
      4. If the camera has a variable focus then set it to the desired value & record it.
      5. Lower the camera towards the tooth until the feature is noticeably out of focus.
      6. Slowly raise the camera until the feature is just perceived to be in sharp focus.
      7. Record the distance (d, in mm) between the camera lens & the feature.
      8. Raise the camera away from the tooth until the feature is noticeably out of focus.
      9. Slowly lower the camera until the feature is just perceived to be in sharp focus.
      10. Record the distance (D, in mm) between the camera lens & the feature.



        Figure 5: Setup for measuring depth of field.

      11. Calculate the depth of field as follows:
        • δ = D - d
        • where
        • δ = depth of field (mm)
        • D = maximum distance in focus (mm)
        • d = minimum distance in focus (mm)
      12. Repeat measurements for each focal setting desired.
    6. Distortion.
      1. Place the 1mm grid on the countertop.
      2. Position the camera above the center of the grid using a lab stand & clamp.
      3. Using the data from the Angle of View test, position the camera so it is looking straight down at the grid.
      4. Adjust the distance between the camera & the grid, & the focus of the camera if possible, to bring the grid into focus.
      5. Capture & save the image of the grid.
      6. Examine the image to determine the level of distortion present. Compare grid image to other intra- & extraoral images to decide if distortion affects the clinical usefulness of the camera.
      • Note: Distortion may depend on focal setting, so measurements may need to be repeated at various focal settings.
    7. Resolution.
      • Note: Because the resolution will depend on the focal setting & the distance between the camera & the target, these parameters should be controlled & recorded. It is possible to achieve high resolution by moving closer to the target even though the line pairs are out of focus; it should be decided beforehand if such readings will be counted. Furthermore, the meaning of resolution measurements is not certain because clinically useful photos may have low resolution. For example: A photo of a patient's full face may be well lighted & in sharp focus, but only have a resolution of 0.5 line pairs per millimeter since the camera is so far away. The same camera may have a resolution of 30 line pairs per millimeter when at a distance corresponding to viewing the margin of a restoration. One possible way to standardize the measurement for comparing cameras is to have a predefined area completely filling the field of view. For example: Completely fill the screen with the occlusal view of a lower first molar, then lay the target on top of it & measure the maximum resolution.
      1. Place the USAF 1951 resolution target (black lines on transparent glass) in an adjustable lab stand.
      2. Position the target over an x-ray viewbox to provide backlighting through the glass.
      3. Clamp the camera in a lab stand & position it over the target. Adjust the angle as necessary using the data from the Angle of View test to ensure that the camera is looking straight down at the target (see figure 6).
      4. Deactivate the camera's built-in light source to reduce glare on the target.
      5. Adjust the camera &/or target up & down to get the maximum resolution (i.e. to resolve the three distinct lines of the highest group & number possible on the target).
      6. Record the group & number resolved & the equivalent line pairs per millimeter as indicated by the conversion chart provided with the target.
      7. Record the focal setting of the camera, if applicable, & the distance between the lens & the target.



        Figure 6: Setup for measuring resolution.

Appendix

Scoring & ratings for the August 2006 CRA Newsletter report used many of the criteria established below which help define how cameras differ, but final ratings were based subjectively on clinical expectations & relative performance of the cameras available. Consequently, grades were relatively high (fair, good, & excellent, but never poor) because all cameras worked well & produced useful intraoral images.

Scoring used for the March 2001 CRA Newsletter report follows. Scoring was done on a 7 point scale (7 = excellent, 5 = good, 3 = fair, 1 = poor). The criteria are listed under each heading.

  1. Portability:

    • Small modular unit with a single, simple connection = 7

    • -1 for heavy weight or large size
    • -1 for no handle on a heavy or large camera or module
    • -1 for each additional connection that must be made
    • -1 for each connection which is difficult or requires careful alignment
    • -6 for a non-portable unit, or one that makes no provisions to be portable
  2. Camera ready when handpiece picked up:

    • (Evaluate after main power has been turned on, & after patient file is selected on computer systems.)

    • Camera & light ready when handpiece lifted from cradle, & turns off when replaced = 7

    • -2 for each additional control that must be operated when picked up or sat down
    • -3 if same control must be operated when picked up & sat down, & requires reaching to base unit
  3. Ease of intraoral access:

    • (Evaluate while taking 6 test photos, with particular attention on distal & buccal of upper last molar.)

    • Camera easily positioned for all photos = 7

    • -1 for slight difficulty aligning photo or patient discomfort
    • -1 for heavy or stiff cord that makes positioning handpiece uncomfortable
    • -2 for significant difficulty aligning photo or patient discomfort
    • -3 for each test photo that cannot be aligned correctly
    • -4 for each test photo that cannot be taken
  4. Ease of focus & ability to achieve sharp focus:

    • (Evaluate for all test photos within the camera's focal range.)

    • All photos in sharp focus & no effort required = 7

    • -1 if manual focus adjustment is required
    • -1 if focus adjustment is difficult
    • -1 if focus adjustment has large range of motion (>100°)
    • -1 if focus control is not labeled
    • -1 if focus adjustment not provided, but images lack sharpness
    • -2 or more, if focus adjustement not provided, but images are significantly blurry
  5. Monitor image is sharp & has natural color:

    • (Evaluate for all test photos within the camera¿s focal range. If alternate lenses are available, use best image.)

    • All photos are sharp, with acceptable color, & pathology can be resolved = 7

    • -1 for slight blur or pixelization
    • -1 for slightly unnatural color
    • -2 for significant blur or pixelization
    • -2 for severely unnatural color
    • -3 or more, for image quality so poor that pathology cannot be resolved
    • -6 for image quality so poor that image is not useful
    • -1 for slight distortion
    • -2 for severe distortion
  6. All component power switches are obvious & easy to use:

    • (For the March 2001 Newsletter, only the camera power switch(es) was(were) considered.)

    • Obvious, easy switch = 7

    • -1 for each non-obvious switch (small, camouflaged, unlabeled, on side of unit, etc.)
    • -2 for each hidden switch (on back or side of unit)
    • -1 for each switch more than one
  7. Simple, fast, easy controls:

    • (Evaluate all commonly used controls, except power switches & focus controls which are graded separately.)

    • All controls are clearly labeled, obvious, simple, & easily operated = 7

    • -1 for each control that is not clearly labeled
    • -1 for each control that is difficult to operate
    • -1 if it is easy to cause accidental prints or press the wrong button (especially on foot pedal)
    • -1 for each control that requires reaching to operate (i.e. printer)
    • -1 for unstable capture (button on handpiece, capture on press rather than release, hard button, etc.)
    • -1 for unstable foot pedal
    • -2 for complicated controls (software menu systems, buttons with multiple functions that are not clear, etc.)
    • -2 for easily misplaced controls (remotes control units, etc)
  8. Ease of infection control:

    • Simple, easy, thorough infection control = 7

    • -1 if camera & cable cannot tolerate immersion disinfection
    • -1 for each additional component that must be touched (mouse, keyboard, printer, etc.)
    • -1 if sheath doesn't cover entire handpiece
    • -1 if sheath is not disposable
    • -1 for any feature that compromises or hampers infection control
  9. Minimal handpiece manipulation needed for 6 test photos:

    • The criterion for this test are meant to reduce the question "Can you pick up this camera & easily get any shot you want?" to a simple NO or YES (1 or 7 points). It is meant to identify the truly versatile cameras. The only allowance made is for cameras that just require focusing to achieve it.

    • All photos are acceptable & require no adjustment of controls = 7

    • -1 if manual focusing is required
    • -6 if lens change is required to get all test photos
    • -6 if all six test photos cannot be taken with acceptable quality
  10. Sheath ease of use:

    • Sheath slides on easily (1), sheath is easily removed (2), sheath fits well (3), sheath does not interfere with controls (4), & sheath does not slide around handpiece (5) = 7

    • -1 for each of the five criteria with a slight problem
    • -2 for each of the five criteria with a significant problem
    • -6 if sheath makes acceptable use of camera impossible

(OPTIONAL, IF CAMERA SYSTEM INCLUDES A PRINTER)

  1. Print image is sharp & has natural color:

    • (Evaluate for all test photos within the camera's focal range. If alternate lenses are available, use best image.)

    • All prints are sharp, with acceptable color, & pathology can be resolved = 7

    • -1 for slight blur or pixelization
    • -1 for slightly unnatural color
    • -2 for significant blur or pixelization
    • -2 for severely unnatural color
    • -3 or more, for image quality so poor that pathology cannot be resolved
    • -6 for image quality so poor that image is not useful
    • -1 for slight distortion
    • -2 for severe distortion