The FERET Verification Testing Protocol

The FERET tests are a set of benchmarks for face recognition algorithms. As such, they operate on single frame data, and return false alarm and verification rates.

Identification is evaluated seperately from verification; that is to say, algorithms that match a name to a face are evaluated apart from those that give back a confidence that a face matches a name. The results are generally displayed as ROC curves. The two methods of evaluation — giving the images with and without eye points — remind me of the two methods that ViPER implements for tracking objects (with and without first frame boxes).

To add ROC curves to ViPER's scripting environment would not be difficult, but it would be slow, and far better to add it to viper-pe. It will, unfortunately, require fundamental changes in the object filtering stage of Object Evaluations.

Link Reference

@misc{ rizvi99feret,
  author = "S. Rizvi and P. Phillips and H. Moon",
  title = "The feret verification testing protocol for face recognition algorithms",
  text = "S. Rizvi, P. J. Phillips, and H. Moon. The feret verification testing protocol
    for face recognition algorithms. Image and Vision Computing Journal, (to
    appear) 1999.",
  year = "1999",
  url = "citeseer.nj.nec.com/rizvi99feret.html" }

- posted by David @ 6:49 PM

Performance Evaluation Metrics for Object-Based Video Segmentation

Of note is the paper's use of the turning angle function to get differences of polygons and shapes. For a good survey of that field, check out Veltkamp and Hagedoorn.

Link Reference

@inproceedings{ErdemSankur2000 
   organization = {X European Signal Processing Conference (EUSIPCO)},
   author = {Ç. E. Erdem and B. Sankur},
   title = {Performance Evaluation Metrics for Object-Based Video Segmentation},
   year = {2000},
   month = {September},
   location = {Tampere, Finland},
   institution = 
}

- posted by David @ 11:16 AM

Color-based Skin Tracking Under Varying Illumination: Experiments

To evaluate their algorithm for skin tracking, Boston University's IVC used hand drawn binary masks.

Link

- posted by David @ 10:47 AM

Automatic Performance Evaluation for Video Text Detection

A researcher at MSR Asia, Hua does research on video text processing. His evaluation methods are very concrete, taking into account things like descenders and ascenders. This focus on detail makes human-generated ground truth a difficult proposition. Hua uses the use of MPEG-7 content set, but only 45 frames are used in the documented experiment.

This short paper gives a quick intro to the file format and the performance evaluation. The file is specified as a C Struct, so it is presumably a binary format. The other Hua paper goes into more detail about how the evaluation is performed, but this one gives an overview of the gui tool for performance evaluation.

It also operates as an add for the MS Vision sdk.

Link

Another paper with the same title goes into detail about the actual evaluation. It provides an interesting, if application specific, attempt to create 'difficulty independent' evaluations.

Link

- posted by David @ 1:01 PM