[ARFORUM] Mixing Video an CG in realitme

Hi Stefan,

We have that working with the current version of the ARToolKit software we
are developing here at the HIT Lab. We have a laptop with firewire video
input that can capture video frames at 30fps. Then we render 3D computer
graphics on the ARToolKit tracking markers.

We have not released this into the public domain yet, but should be able
to do so by the end of October. It'll be put onto the ARToolKit website -
http://www.hitl.washington.edu/artoolkit/

With this software there is no programming needed appart from the 3D
graphics application code - ARToolKit takes care of the tracking and video
compositing for you.

The laptop we are using has a GeForce2 Go chip in it which is a pretty low
end GeForce card (17 million polygons/sec), so practically and GeForce
card would do the trick..

Please let me know if you have any questions about this..

	Mark

On Tue, 9 Oct 2001, Stefan Gruen wrote:

> Hello everybody,
>
> for building a video-see through AR-system I have the following problem.
> Suppose, that I want to  overlay the video stream of a fire wire camera in
> realtime with the computer graphics which my computer produces. One idea
> would be to mix video stream and the computer graphics in the computer and
> connect the HMD to the graphics card. What kind of graphic cards do you
> recommend for mixing (sorry: no SGI). And much more important: is there any
> software with which I can mix the video stream and the computer graphics in
> realtime? Or do I have to program it for my self?

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Mark Billinghurst               | Human Interface Technology Laboratory
grof@h ..................        | University of Washington, Box 352-142
fax: +1-206-543-5380            | Seattle, WA 98195

==^================================================================
EASY UNSUBSCRIBE click here: http://topica.com/u/?a84Ao5.a9zIQj
Or send an email To: arforum-unsubscribe@t .........
This email was sent to: webmaster@e ............

T O P I C A -- Register now to manage your mail!
http://www.topica.com/partner/tag02/register
==^================================================================

From:	Mark Billinghurst <grof@h ..................>	Received:	Oct 9, 2001
To	arforum@t .........
Subject:	Re: [ARFORUM] Mixing Video an CG in realitme
Hi Stefan, We have that working with the current version of the ARToolKit software we are developing here at the HIT Lab. We have a laptop with firewire video input that can capture video frames at 30fps. Then we render 3D computer graphics on the ARToolKit tracking markers. We have not released this into the public domain yet, but should be able to do so by the end of October. It'll be put onto the ARToolKit website - http://www.hitl.washington.edu/artoolkit/ With this software there is no programming needed appart from the 3D graphics application code - ARToolKit takes care of the tracking and video compositing for you. The laptop we are using has a GeForce2 Go chip in it which is a pretty low end GeForce card (17 million polygons/sec), so practically and GeForce card would do the trick.. Please let me know if you have any questions about this.. Mark On Tue, 9 Oct 2001, Stefan Gruen wrote: > Hello everybody, > > for building a video-see through AR-system I have the following problem. > Suppose, that I want to overlay the video stream of a fire wire camera in > realtime with the computer graphics which my computer produces. One idea > would be to mix video stream and the computer graphics in the computer and > connect the HMD to the graphics card. What kind of graphic cards do you > recommend for mixing (sorry: no SGI). And much more important: is there any > software with which I can mix the video stream and the computer graphics in > realtime? Or do I have to program it for my self? +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Mark Billinghurst \| Human Interface Technology Laboratory grof@h .................. \| University of Washington, Box 352-142 fax: +1-206-543-5380 \| Seattle, WA 98195 ==^================================================================ EASY UNSUBSCRIBE click here: http://topica.com/u/?a84Ao5.a9zIQj Or send an email To: arforum-unsubscribe@t ......... This email was sent to: webmaster@e ............ T O P I C A -- Register now to manage your mail! http://www.topica.com/partner/tag02/register ==^================================================================

From:	yohan baillot <baillot@a ...............>	Received:	Oct 9, 2001
To	arforum@t .........
Subject:	Re: [ARFORUM] Mixing Video an CG in realitme
You need to use a scan converter able to overlay video in the black zones of your vga signal. Look at the CORIOgen http://www.overlaytools.com/coriogen-e.htm it is expensive but do the job great Yohan ___________________________________________ Yohan Baillot VRLab / Naval Research Laboratory 4555 Overlook Avenue SE ______ Code 5580, Room 119A /\ \ Washington DC 20375 / \ \ Phone (202) 404 7801 / \______\ Mobile (703) 732 5679 \ / / Fax (202) 404 1122 \ / / baillot@a ............... \/_____ / http://ait.nrl.navy.mil/people/baillot/ http://ait.nrl.navy.mil/vrlab/projects/BARS/BARS.html ___________________________________________ ----- Original Message ----- From: "Stefan Gruen" <gruenvog@l .........> To: <arforum@t .........> Sent: Tuesday, October 09, 2001 11:38 AM Subject: [ARFORUM] Mixing Video an CG in realitme > Hello everybody, > > for building a video-see through AR-system I have the following problem. > Suppose, that I want to overlay the video stream of a fire wire camera in > realtime with the computer graphics which my computer produces. One idea > would be to mix video stream and the computer graphics in the computer and > connect the HMD to the graphics card. What kind of graphic cards do you > recommend for mixing (sorry: no SGI). And much more important: is there any > software with which I can mix the video stream and the computer graphics in > realtime? Or do I have to program it for my self? > > > > Thank you > > STefan > ----------------------------------------------------------- > Dr. Stefan M. Grünvogel > > phone: +49 (0)221 250-1053 > fax: +49 (0)221 250-2050 > @: mailto:gruenvogel@l ......... > http://www.lmr.khm.de > > LMR-Institute > LABORATORY FOR MIXED REALITIES > at the Academy of Media Arts > Coloneum, Gebäude C 1 > D-50829 Cologne, Germany > > ==^================================================================ EASY UNSUBSCRIBE click here: http://topica.com/u/?a84Ao5.a9zIQj Or send an email To: arforum-unsubscribe@t ......... This email was sent to: webmaster@e ............ T O P I C A -- Register now to manage your mail! http://www.topica.com/partner/tag02/register ==^================================================================

From:	Stefan Gruen <gruenvog@l .........>	Received:	Oct 9, 2001
To	arforum@t .........
Subject:	[ARFORUM] Mixing Video an CG in realitme
Hello everybody, for building a video-see through AR-system I have the following problem. Suppose, that I want to overlay the video stream of a fire wire camera in realtime with the computer graphics which my computer produces. One idea would be to mix video stream and the computer graphics in the computer and connect the HMD to the graphics card. What kind of graphic cards do you recommend for mixing (sorry: no SGI). And much more important: is there any software with which I can mix the video stream and the computer graphics in realtime? Or do I have to program it for my self? Thank you STefan ----------------------------------------------------------- Dr. Stefan M. Grünvogel phone: +49 (0)221 250-1053 fax: +49 (0)221 250-2050 @: mailto:gruenvogel@l ......... http://www.lmr.khm.de LMR-Institute LABORATORY FOR MIXED REALITIES at the Academy of Media Arts Coloneum, Gebäude C 1 D-50829 Cologne, Germany ==^================================================================ EASY UNSUBSCRIBE click here: http://topica.com/u/?a84Ao5.a9zIQj Or send an email To: arforum-unsubscribe@t ......... This email was sent to: webmaster@e ............ T O P I C A -- Register now to manage your mail! http://www.topica.com/partner/tag02/register ==^================================================================

From:	Mark Billinghurst <grof@h ..................>	Received:	Oct 10, 2001
To	arforum@t .........
Subject:	Re: [ARFORUM] Mixing Video an CG in realitme
On Wed, 10 Oct 2001, yohan baillot wrote: > > 2. Which camera type should one choose for the different solutions. > > Any. I think AR toolkit does take camera parameters but I dont > think they compensate in the displayed video image for the > camera distortion, juts for the tracking. In both case you have to take > a low distortion camera else the registration will be bad as you go > from the center of the screen Actually, ARToolKit does compensate for image distortion in the camera. The object registration should look good over all the screen. The video from the camera is warped to match the lens distortion. This can be turned off if desired. > > 4. In your perspective: which solution do you prefer? > > Personnaly I do not support video see-through AR because > the resolution of the display is yet to be improved before I > can feel confortable being operated by a surgeon wearing > an AR system using a video feed. However, vision tracking > has a better accuracy than open loop sensing but it is rather > hard to do and is not yet done commercially for arbitrary > environments. ARToolKit also works with optical see through displays. In that case you're using the video signal for tracking, but only displaying the stereo graphics in the headset. However it is getting difficult to find good, hi-resolution see through displays now unfortunately. I hope this helps, Mark +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Mark Billinghurst \| Human Interface Technology Laboratory grof@h .................. \| University of Washington, Box 352-142 fax: +1-206-543-5380 \| Seattle, WA 98195 ==^================================================================ EASY UNSUBSCRIBE click here: http://topica.com/u/?a84Ao5.a9zIQj Or send an email To: arforum-unsubscribe@t ......... This email was sent to: webmaster@e ............ T O P I C A -- Register now to manage your mail! http://www.topica.com/partner/tag02/register ==^================================================================

From:	yohan baillot <baillot@a ...............>	Received:	Oct 10, 2001
To	arforum@t .........
Subject:	Re: [ARFORUM] Mixing Video an CG in realitme
Hello Stefan, > Therefore I have some more questions: > 1. Which of these solutions is also cappable of managing stereo video input > (two cameras)? Both. In both case you just need to double the work. Obviously there may be a problem to do think in parallel for the software solution so you may loose performance here. > 2. Which camera type should one choose for the different solutions. Any. I think AR toolkit does take camera parameters but I dont think they compensate in the displayed video image for the camera distortion, juts for the tracking. In both case you have to take a low distortion camera else the registration will be bad as you go from the center of the screen > 3. Which solution is easier to calibrate? The calibration is not the same. I cannot speak for the ARtoolkit side and Mark will I am sure, but for the hardware solution since the video feed is not going into the computer, the video is not used for the tracking. Therefore you have to rely on your sensors (gyro, ...) and calibrate them. How? in our case we calibrate from a known position and looking in a known direction and we display what the user should see in virtual. His/her job is to align the graphic with the real world to determine the unknown transformation between the display origin and the sensor. You have also to calibrate once the fov, and this can be done with a target with pattern of know size. > 4. In your perspective: which solution do you prefer? Personnaly I do not support video see-through AR because the resolution of the display is yet to be improved before I can feel confortable being operated by a surgeon wearing an AR system using a video feed. However, vision tracking has a better accuracy than open loop sensing but it is rather hard to do and is not yet done commercially for arbitrary environments. Have a good day Yohan > > > Thank you a lot > STefan > > > -----Ursprungliche Nachricht----- > Von: Mark Billinghurst [mailto:grof@h ..................] > Gesendet: Mittwoch, 10. Oktober 2001 02:02 > An: arforum@t ......... > Betreff: Re: [ARFORUM] Mixing Video an CG in realitme > > > > Hi Stefan, > > We have that working with the current version of the ARToolKit software we > are developing here at the HIT Lab. We have a laptop with firewire video > input that can capture video frames at 30fps. Then we render 3D computer > graphics on the ARToolKit tracking markers. > > We have not released this into the public domain yet, but should be able > to do so by the end of October. It'll be put onto the ARToolKit website - > http://www.hitl.washington.edu/artoolkit/ > > With this software there is no programming needed appart from the 3D > graphics application code - ARToolKit takes care of the tracking and video > compositing for you. > > The laptop we are using has a GeForce2 Go chip in it which is a pretty low > end GeForce card (17 million polygons/sec), so practically and GeForce > card would do the trick.. > > Please let me know if you have any questions about this.. > > Mark > > On Tue, 9 Oct 2001, Stefan Gruen wrote: > > > Hello everybody, > > > > for building a video-see through AR-system I have the following problem. > > Suppose, that I want to overlay the video stream of a fire wire camera in > > realtime with the computer graphics which my computer produces. One idea > > would be to mix video stream and the computer graphics in the computer and > > connect the HMD to the graphics card. What kind of graphic cards do you > > recommend for mixing (sorry: no SGI). And much more important: is there > any > > software with which I can mix the video stream and the computer graphics > in > > realtime? Or do I have to program it for my self? > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > +++ > Mark Billinghurst \| Human Interface Technology Laboratory > grof@h .................. \| University of Washington, Box 352-142 > fax: +1-206-543-5380 \| Seattle, WA 98195 > > > ==^================================================================ EASY UNSUBSCRIBE click here: http://topica.com/u/?a84Ao5.a9zIQj Or send an email To: arforum-unsubscribe@t ......... This email was sent to: webmaster@e ............ T O P I C A -- Register now to manage your mail! http://www.topica.com/partner/tag02/register ==^================================================================

From:	Mark Lennerton <mjlenne1@n ...........>	Received:	Oct 10, 2001
To	arforum@t .........
Subject:	RE: [ARFORUM] Mixing Video an CG in realitme
Dr. Grünvogel If you're interested in real time merging (and who's not) one idea is to use keying hardware to mix your graphics and video. In this mixing application your graphics rendering would appear upon the keying matte which can be green or blue (most popular for chromakeying) or clear or black matte for intensity keying. Note that you would need to render the key matte when rendering the graphics. Here the graphics signal would be considered the foreground or 'talent' and the real world video would be the background. The background would replace the corresponding pixel if it was a key match. This hardware costs anywhere from $1.4K to $17K US and it appears you get what you pay for. The higher end brands have better noise filtering and even registration synchronization and culling zones. Panasonic and Sony produce low to mid quality and Ultimatte produces the best quality. Good luck. Hope this helps, Mark Lennerton Capt USMC -----Original Message----- From: yohan baillot [mailto:baillot@a ...............] Sent: Tuesday, October 09, 2001 5:04 PM To: arforum@t ......... Subject: Re: [ARFORUM] Mixing Video an CG in realitme You need to use a scan converter able to overlay video in the black zones of your vga signal. Look at the CORIOgen http://www.overlaytools.com/coriogen-e.htm it is expensive but do the job great Yohan ___________________________________________ Yohan Baillot VRLab / Naval Research Laboratory 4555 Overlook Avenue SE ______ Code 5580, Room 119A /\ \ Washington DC 20375 / \ \ Phone (202) 404 7801 / \______\ Mobile (703) 732 5679 \ / / Fax (202) 404 1122 \ / / baillot@a ............... \/_____ / http://ait.nrl.navy.mil/people/baillot/ http://ait.nrl.navy.mil/vrlab/projects/BARS/BARS.html ___________________________________________ ----- Original Message ----- From: "Stefan Gruen" <gruenvog@l .........> To: <arforum@t .........> Sent: Tuesday, October 09, 2001 11:38 AM Subject: [ARFORUM] Mixing Video an CG in realitme > Hello everybody, > > for building a video-see through AR-system I have the following problem. > Suppose, that I want to overlay the video stream of a fire wire camera in > realtime with the computer graphics which my computer produces. One idea > would be to mix video stream and the computer graphics in the computer and > connect the HMD to the graphics card. What kind of graphic cards do you > recommend for mixing (sorry: no SGI). And much more important: is there any > software with which I can mix the video stream and the computer graphics in > realtime? Or do I have to program it for my self? > > > > Thank you > > STefan > ----------------------------------------------------------- > Dr. Stefan M. Grünvogel > > phone: +49 (0)221 250-1053 > fax: +49 (0)221 250-2050 > @: mailto:gruenvogel@l ......... > http://www.lmr.khm.de > > LMR-Institute > LABORATORY FOR MIXED REALITIES > at the Academy of Media Arts > Coloneum, Gebäude C 1 > D-50829 Cologne, Germany > > ==^================================================================ EASY UNSUBSCRIBE click here: http://topica.com/u/?a84Ao5.a9zIQj Or send an email To: arforum-unsubscribe@t ......... This email was sent to: webmaster@e ............ T O P I C A -- Register now to manage your mail! http://www.topica.com/partner/tag02/register ==^================================================================