DPRG mailing list archives: [DPRG] webcam robot vision limitations

Fri Jun 29 01:05:48 CDT 2007

On 6/28/07, Ed Okerson <ed at okerson.com> wrote:
> Yes, I was in San Jose when they started working on this project.
>
> I came across another interesting page on optic flow:
>
> http://www.centeye.com/pages/techres/opticflow.html
>
> After reading this description of optic flow, it seems to me that much of
> the processing necessary to generate the object vectors is already coded
> in an MPEG2 or 4 encoder.

The motion estimation in the typical MPEG encoder is just looking for
the best match, which is not necessarily the strongest correlation.
The encoder doesn't care if the motion vector represents the real
motion of the object, only that the motion vector results in the
smallest residual. Also, the motion estimation is typically done on a
reconstructed image, not the original pixels, which further reduce how
well the vectors maps to physical motion. In addition, the
hardware/software may not be doing a full search and the search may be
heavily short circuited (early termination).

If you are using a DSP that has motion estimation hardware assist
instructions, you can sometimes use them for cross-correlation, but in
many cases the accelerators are so tuned for MPEG-like motion
estimation they aren't overly useful for cross-correlation. It just
depends on how specialized the instructions/function unit is.

> The arrows look amazingly similar to those shown on Yaw right and move
> forward samples on the optic flow web page above.  So if you have a camera
> that can send an MPEG video stream, you could just analyze the motion
> vectors for optical flow.

Sadly, doing this type of stuff in the compressed domain is rarely of
any value, especially with the type of MPEG encoder you might find in
a low end USB camera. You might be able to do basic motion detection
using the motion vectors, but I wouldn't expect much more then that.

                             --Clem