Framework for Multiple Selection

In the current release of ViPER-GT, 4b5, the 'view mediator' keeps track of the current selection with a SingleViperSubTree - a class that contains a node in the viper object model and labels the associated path as 'selected'. So, to enhance this, I switched to a simpler ViperSelectionSet, which is just a set of viper object nodes. In order to support some features, I keep track of the thing that was most recently added to the selection in a SingleViperSubTree - now called the 'Primary Selection'.

I modified a few of the classes as necessary, and have not yet notices any negative effects to the refactoring. In fact, some code is actually cleaner due to the simpler selection model, although we have, in a sense, two selection models now. There are still some more features to add to really get multiple selection working on the back end, and Charles and I haven't started integrating it on the front end at all.

If the SelectionSet class doesn't work out, I may have to do the more general viper-sub-tree selection model, which I still haven't implemented.

- posted by David @ 1:23 PM

Proposal for a Simple EPF Editor

So, we would like to have a visual editor for evaluation configuration files, the EPF files that ViPER-PE uses to calculate the different between two video metadata files. The simplest thing to implement is likely a variation on the schema editor, with the schema fixed and the ability to associate metrics and thresholds with the different attributes. There are filters, which can be input/output and candidate/target, and there are evaluations of the three types.

The simplest evaluation editor would support one epf file at a time, and one evaluation type at a time, with the tree view containing a five nodes: the four filters and the evaluations. Somewhat more useful would be the ability to drag between epf files, and possibly to keep a set of snippets or generic epf nodes available for drag-and-drop into new or existing files.

From here, it is not difficult to see that it would be very useful to support dynamic evaluation of these epf files. The simplest thing would be filter testing - including hiding input-filtered items and modifying the display (e.g. color) of output-filtered items. Finally, it would be nice to give overlay-type display of result data.

- posted by David @ 1:37 PM

PETS 2005

PETS is an IEEE annual conference discussing performance evaluation of tracking and surveillance, something ViPER was (almost) designed for. Perhaps I'll finally go this year. In 2000, there wasn't enough data for a good horse race, and most of the results for papers revolve around screenshots of one of the few included sample sets. 2001 is improved. I'll finish reading 2002's and both 2003 reports (VS and ICVS) over the weekend, and then 2004 when I can find it.

- posted by David @ 3:00 PM

Crossing Off Bugs means Finding New Bugs

So, I've managed to fix a few bugs on the big list below, but I keep coming up with more. So, I fixed the I-Frame Descriptor bug, testing it on a nice sized video (24,211 frames) and discovered that it slowed rendering of the timeline to a crawl.

- posted by David @ 1:02 PM

4.0 beta 4.1 Problems

Well, I just noticed that fragmentation count was working incorrectly for polygons - counting each tile as a separate fragment. I've already fixed it. If someone has a pressing need, I can release a patch before beta 5 comes out. Editing polygon truth for this has also highlighted a few weaknesses in the editor, mentioned yesterday. If you have any problems, drop me a line at viper-bugs, post to the viper google group, or add a bug on the sourceforge project page.

Update: I just found out that I broke the dice metric as well, for all shapes (not frame spans, though). Um, I'll have to post another beta soon, I suppose, as that is a key element of most evaluations. I really should put my set of example gt files and test evaluations into the ant build process. Perhaps it is time to switch to Maven...

- posted by David @ 12:42 PM

another beta

I released beta 4 this morning, hoping that I had polygon intersection and metrics working, but not having tried them out, I didn't know. When I got in to campus this morning, I found an annoying error in my tiling code (the goal is to tile polygons with convex polygons and do the math on those; right now, these are all triangles). This led to a bunch of fixes, and I now have a core of a unit test suite for the new functionality. Next comes integration testing, by way of new sample target/candidate data sets that use the 'polygon' data type. Editing polygon data in gt makes me want to see two RFE/bugs get implemented, morphing polygons and calculating screen distance in real pixels, not image pixels. I'll work on the first one in my spare time and bug Charles about the second one. Beta 4.1 should be out this evening/early in the morning, and I'll include a sample polygon-tracking evaluation, as mentioned two entries ago.

- posted by David @ 6:12 PM

Implementing Polygon Calculations in ViPER

There are still a few more bugs to fix with viper-pe before it is properly equipped to deal with polygon data. I'd been meaning to implement non-convex polygon comparison for a while. It is a fairly straightforward addition, as I already have the code to deal with bags of convex polygons (from the 'multiple' match type) and will only need to add the code to break up the polygon into peices.

As a quick hack, I've written code that breaks down and simple (non-self-intersecting) polygon into triangles. This can be extended to non-simple polygons by first converting them into simple polygons. Further, it can be sped up by using a better tile shape than triangles - first trapezoids, then more-ary convex polygons.

As usual, when I come back to viper-pe and viper-full after a long absence in the bliss that is the ant-built viper-light, I had to fix a variety of makefile-related bugs. I think the next version of os x, or maybe xcode, is supposed to come with ant built in, so maybe I can finally toss the makefiles for good soon. Anyway, somehow a lot of the files that were chmod+x are no longer so, and I'll have to fix that. I've tested the 'Framewise' evaluation in the previous entry by comparing a file to itself, but the real test comes with more structured examples, and the multiple-match or optimum-match filter methods. I plan to set those up tomorrow, and push out a new version in the evening.

- posted by David @ 4:06 PM

Evaluating Road Sign Data

Detecting of Road Signs

Frame-by-Frame Evaluation

The first type of evaluation I'll describe is the simplest - a framewise evaluation of 'how many pixels did I find correctly?', 'how many frames did I identify as containing road signs?', and 'how many road signs did I pick out accurately in each frame?' This uses a 'FRAMEWISE' evaluation type.

#BEGIN_FRAMEWISE_EVALUATION

OBJECT SIGN
   LOCATION : matchedpixels missedpixels falsepixels \              fragmentation arearecall areaprecision \              [arearecall .75] [areaprecision .75]

#END_FRAMEWISE_EVALUATION

This will give output that includes information about the number of pixels, frames, and signs-per-frame that were found. You will get results for each frame, followed by a result for all frames together, as follows:

Total for all frames
Detection Accuracy: 0.6363636363636364
Object Count Recall: 0.4666666666666667
Object Count Precision: 1
Pixels matched: 25952.979005067133
Pixels missed: 19660.520994872095
Pixels falsely detected: 39947.02099488817
Fragmentation metric: 0.7891750242695977
Object Area Recall: 0.22076486873855497
Box Area Precision: 0.4176735870141997
Localized Object Area Recall: 0.1037037037037037
Localized Box Area Precision: 0.1349206349206349

This doesn't take into account tracking - in each frame, the best polygon match found is used. To find the best tracks, we will have to switch to an object-based approach (or, for more specific analysis, a tracking one).

Tracking Road Signs

The second evaluation type uses the OBJECT evaluation, checking how many tracks of signs were detected correctly, and how well.

#BEGIN_OBJECT_EVALUATION

OBJECT SIGN [- -]
   LOCATION : [dice -]

#END_OBJECT_EVALUATION

Reading Road Signs

The final type of evaluation gives the coarsest, but also likely the most useful, results. This is the goal-based evaluation, where we test to see the type of the sign, not just the location.

#BEGIN_OBJECT_EVALUATION

OBJECT SIGN [- -]
   CLASS : [- -]

#END_OBJECT_EVALUATION

- posted by David @ 1:19 PM

Slow Updating of Site

Sourceforge has disabled cron access to the computers, so the site has not been updated every night for the past month or so. Instead, it only updates when I specifically request an update. I'll have to modify the site change instructions to reflect the new paradigm. In the meantime, I'll also work on making the bake_site script more efficient.

- posted by David @ 11:46 AM