DPRG mailing list archives: [DPRG] computer vision frame rate

Fri Apr 27 18:58:00 CDT 2007

Hi Chris,

It appears to me that a significant amount of your time is spent decoding
JPEG images.  Have you considered using a camera module that gives you raw
image data instead of JPEG?  Some of the guys in the Home Brew Robotics
club in San Jose were working on doing this with an FPGA.  I see they have
now formed a company around this:

http://www.roboticore.com/products.htm

I know they were doing some image pre-processing with hardware blocks in
the FPGA before passing it to the MicroBlaze processor, but surely you
would gain a lot even by having one of these style camera modules
connected directly to the bus of your ARM.

Ed Okerson

> Hello,
>
> The computer vision stuff I'm working on is starting to
> work with a webcam, ARM9 SBC and RC toy car robot. Here
> are some basic timings that give an idea of what to
> expect.
>
> The webcam is:    Labtec Notebook, 640x480 CMOS, 15 fps
> The computer is:  Technologic TS-7260 ARM9 200 MHz
>
> 1. Capture/sync frame 0 (usually corrupted)       80 ms
> 2. Capture/sync frame 1 (use this one)            10 ms
> 3. Decode JPEG, initial image processing         290 ms
> 4. Threshold feature image for clear corridor    160 ms
> 5. Goto step 1 and repeat
>
> I'm using EmbedCV for the computer vision stuff. With a
> little optimization, I think that I can get a little
> better than 2 fps in a look/drive loop. Beyond this, a
> lower camera resolution is required.
>
> This has been very interesting so far. Several thoughts
> come up.
>
> 1. Computer vision frame rate is determined by how long
>    it takes to process each frame.
>
> Ok, I know this is obvious... If a camera has a frame
> rate of 15 fps, then roughly 70 ms separate each image
> in the video stream. If the computer vision processing
> takes just 140 ms, then the frame rate of the system
> drops down to 5 fps. More likely, the algorithms will
> take longer than this. It is very difficult to maintain
> a good frame rate without a fast or parallel computer.
>
> The alternative is a pipeline of slow computers to
> maintain frame rate. But then latency increases. I
> suspect this is what many network cameras do as high
> latency is not a problem for that application.
>
> 2. Vision frame rate is determined by computation and
>    perception rather than hardware
>
> The more the computer must think about each image, the
> slower it will "see" regardless of camera frame rate.
> So I think visual perception depends on what kind of
> algorithms are required. If an environment requires
> something cheap and fast like color image segmentation,
> then the frame rate can be fairly high. But if the task
> requires object recognition, then the frame rate drops
> as it takes longer to process each image frame.
>
> The system must be able to accommodate this. It can't
> have a fixed main control loop that is disturbed by
> change in the vision processing. That leads to either
> a task or multi-thread/process based design.
>
> Chris
> _______________________________________________
> DPRGlist mailing list
> DPRGlist at dprg.org
> http://list.dprg.org/mailman/listinfo/dprglist
>