Hi everyone,

I've recently taken the plunge into using Mac OS X and I've got some  
questions about doing video capture and also how the ARToolkit  
wrapper for it works.

I notice that by default the ARToolKit interface captures from the  
camera in some kind of YUV format. I am using a firewire camera and  
wanted to extract images out using 640x480 RGB format (which the IIDC  
spec allows for) so I modified some #defines to use RGB format. It  
supplies the images as RGB but when I ran the Shark profiling tool  
(which BTW is the nicest profiling tool I've ever seen!) I noticed  
that QuickTime is doing a YUV to RGB conversion.

I looked into this as well as one of my students (both of us aren't  
Mac guru's though) and it seems like QuickTime is the only way to  
talk to a firewire camera, and it will only talk to the camera in YUV  
with a conversion to whatever format you want (with associated speed  
penalties).

Is there anyway to talk directly to the firewire camera and retrieve  
frames in the format I want (according to the IIDC standard)?

Is it possible to talk to a firewire camera without using QuickTime?

thanks,
Wayne

------------------------------------------------------------------------ 
---
Dr Wayne Piekarski - Researcher / Lecturer      pho:         
+61-8-8302-5070
                                                 fax:         
+61-8-8302-3381
Assistant Director - Wearable Computer Lab      mob:         
+61-407-395-889
School of Computer and Information Science      ema:   
wayne@c ..............
University of South Australia                   www: http:// 
www.tinmith.net

>> I am using a firewire camera and wanted to extract images out  
>> using 640x480 RGB format (which the IIDC spec allows for) so I  
>> modified some #defines to use RGB format. It supplies the images  
>> as RGB but when I ran the Shark profiling tool (which BTW is the  
>> nicest profiling tool I've ever seen!) I noticed that QuickTime is  
>> doing a YUV to RGB conversion.
>
> Yes, you would have noticed Quicktime's internal YUV-RGB  
> transformation. As far as I can tell, Quicktime acquires the IIDC  
> video data in a YUV-411 format, and converts to RGB. This is a  
> highly optimised operation, especially on a G4 or G5, but it is  
> still overhead, mainly because the amount of data being moved  
> around is more than doubled. But, as far as I was aware, all IIDC  
> cameras return only YUV-411 data on the wire. Correct me if I am  
> wrong there.

Well, if you want 30 frames per second, that appears to be the case  
with the cameras I have.  YUV411 at 640x480 and YUV422 at 320x240.    
I think many of the cameras will do RGB at 15 fps at 640x480, though,  
but most of us don't care about that (for speed reasons).

There may be newer cameras that do straight RGB;  most of the older  
ones will only create streams that are < 200 Mbps, and straight RGB  
is about 222Mbps.

On 29/06/2005, at 8:05 AM, Wayne Piekarski wrote:
> I've recently taken the plunge into using Mac OS X and I've got  
> some questions about doing video capture and also how the ARToolkit  
> wrapper for it works.
>

Congratulations, and as I wrote the Mac video lib, I can probably help.

> I notice that by default the ARToolKit interface captures from the  
> camera in some kind of YUV format.

The video format used by the toolkit is determined at runtime, and I  
added support for two common YUV formats, 2vuy and yuvs. These are  
yuv422 formats, identical except for the byte ordering.

> I am using a firewire camera and wanted to extract images out using  
> 640x480 RGB format (which the IIDC spec allows for) so I modified  
> some #defines to use RGB format. It supplies the images as RGB but  
> when I ran the Shark profiling tool (which BTW is the nicest  
> profiling tool I've ever seen!) I noticed that QuickTime is doing a  
> YUV to RGB conversion.

Yes, you would have noticed Quicktime's internal YUV-RGB  
transformation. As far as I can tell, Quicktime acquires the IIDC  
video data in a YUV-411 format, and converts to RGB. This is a highly  
optimised operation, especially on a G4 or G5, but it is still  
overhead, mainly because the amount of data being moved around is  
more than doubled. But, as far as I was aware, all IIDC cameras  
return only YUV-411 data on the wire. Correct me if I am wrong there.

> I looked into this as well as one of my students (both of us aren't  
> Mac guru's though) and it seems like QuickTime is the only way to  
> talk to a firewire camera, and it will only talk to the camera in  
> YUV with a conversion to whatever format you want (with associated  
> speed penalties).

YUV411 to YUV422 comes almost free in terms of computational load.

> Is there anyway to talk directly to the firewire camera and  
> retrieve frames in the format I want (according to the IIDC standard)?
>
> Is it possible to talk to a firewire camera without using QuickTime?
>

I will have to look up the docs, but yes you can talk directly with  
the firewire driver. You can create a driver which instantiates  
IOKit's firewire classes. There may already be some low level APIs  
which do some of the heavy lifting for you. Bypassing Quicktime comes  
at a cost though.. it handles a lot of different components and keeps  
things simple for the user.


Philip Lamb
Visiting Research Associate,
Human Interface Technology Laboratory
University of Washington, Seattle, WA.
+1 206 965 8234