Hi everyone !

I was looking into using my own library that capture and display images 
using pixel copy or texture rendering. I basically replace the frame 
grabbing of AR toolkit by my own function and the rendering in ARToolkit by 
my own function (this is building on work that GeorgiaTech did that 
improves ARToolkit using OpenCV). However I noticed that when I do that 
ARToolkit does not work anymore, or my video appear upside down, and the 
object move the opposite direction of the video. I dug in the code and 
found out that the texture or pixel rendering was done to render the image 
up-side down with respect to the way it is captured, which explained the 
problem. The pixel copy rendering is done using glZoom (zoom,-zoom) which 
put the image up side down and the texturing is done from top to bottom but 
with Y texture coordinates increasing which basically revert the image as 
well. Now if I want to use my own library to capture and render the image, 
I need to pass to ARToolkit an image that is reverted vertically from what 
I capture, which involve an array copy which is a performance hit. I cannot 
change my library to revert the rendering as ARToolkit does because that 
would mean that I have to use glZoom (-zoom,-zoom)  as well if I want to 
render pixels that are not reverted with respect to texture

I was wondering if there was a reason why this approach was taken or if 
that is simply because the logic of image processing and transform 
derivation had been designed inverted from the start and the image display 
inverted was done to fix the problem quickly. If the reason was the later, 
could I request that next release do display what is captured, not the 
frame inverted?

thanks

Yohan

yes it does, thanks much for the explanation

Yohan

On Thu, 5 Aug 2004, Philip Lamb wrote:

> Yohan,
>
> The phenomenon you observe derives from the fact that ARToolKit was
> originally written on Linux with drivers that delivered buffers where
> the first byte in the buffer equated to the first component of the
> upper-left most pixel in the image, and subsequent bytes proceeded in
> a row-first fashion. The rest of ARToolKit and its subsequent ports
> to Irix, Windows and Mac OS X were all based on this specification.
>
> The reason then for drawing the image "upside down" in OpenGL is that
> OpenGL expects pixel data to begin with the first component of the
> lower-left most pixel. So drawing pixel data using OpenGL involves
> flipping the y-coordinate system. This has almost no CPU cost.
>
> Now for (presumably) worthwhile reasons, Windows DS capture delivers
> bottom-up buffers. If we were to change the type of buffers expected
> by ARToolKit to bottom-up, we would break usage on the other 3
> platforms (Linux, Mac OS X, Irix). We'd like to keep the original
> specification that ARToolKit use top-down buffers, because the code
> is being most actively developed by the original creator Dr. Kato on
> Linux, and its possible that he may release his toolkit with natural
> feature tracking, so we're retaining the Linux method as the spec.
>
> What this means for Windows users is three possibilities: (1) (What
> we do most often here in our lab) is to turn the camera upside down
> and use the WDM capture dialog's "mirror horizontally" setting. (2)
> (What is done in ARToolKit 2.68.2) pass "flipV" parameter to
> DSVideoLib on windows which makes it do an in-place vertical flip
> (actually not that costly in terms of CPU.. try profiling it on your
> machine to see this) or (3) (What Thomas did in his release) patch
> your own copy of ARToolKit to work with bottom-up buffers.
>
> Hope this clears up some of the confusion.
>
> Regards,
>
> Phil.
>
> At 5:38 PM -0400 4/8/04, Yohan Baillot (Contractor) wrote:
> >Hi everyone !
> >
> >I was looking into using my own library that capture and display
> >images using pixel copy or texture rendering. I basically replace
> >the frame grabbing of AR toolkit by my own function and the
> >rendering in ARToolkit by my own function (this is building on work
> >that GeorgiaTech did that improves ARToolkit using OpenCV). However
> >I noticed that when I do that ARToolkit does not work anymore, or my
> >video appear upside down, and the object move the opposite direction
> >of the video. I dug in the code and found out that the texture or
> >pixel rendering was done to render the image up-side down with
> >respect to the way it is captured, which explained the problem. The
> >pixel copy rendering is done using glZoom (zoom,-zoom) which put the
> >image up side down and the texturing is done from top to bottom but
> >with Y texture coordinates increasing which basically revert the
> >image as well. Now if I want to use my own library to capture and
> >render the image, I need to pass to ARToolkit an image that is
> >reverted vertically from what I capture, which involve an array copy
> >which is a performance hit. I cannot change my library to revert the
> >rendering as ARToolkit does because that would mean that I have to
> >use glZoom (-zoom,-zoom)  as well if I want to render pixels that
> >are not reverted with respect to texture
> >
> >I was wondering if there was a reason why this approach was taken or
> >if that is simply because the logic of image processing and
> >transform derivation had been designed inverted from the start and
> >the image display inverted was done to fix the problem quickly. If
> >the reason was the later, could I request that next release do
> >display what is captured, not the frame inverted?
> >
> >thanks
> >
> >Yohan
>
>
> --
> Philip Lamb
> Research Student,
> Human Interface Technology Laboratory New Zealand,
> University of Canterbury, Christchurch, NZ.
> +64 3 3642987 x3070
>


_______________________________________________________________________

 Yohan BAILLOT

 Virtual Reality Laboratory,
 Advanced Information Technology (Code 5580),
 Naval Research Laboratory,
 4555 Overlook Avenue SW,
 Washington, DC  20375-5337

 Email	: baillot@a ...............
 Work		: (202) 404 7801
 Home 	: (202) 518 3960
 Cell 	: (703) 732 5679
 Fax  	: (202) 767 1122
 Web		: http://ait.nrl.navy.mil/vrlab/projects/BARS/BARS.html

_______________________________________________________________________

Yohan,

The phenomenon you observe derives from the fact that ARToolKit was 
originally written on Linux with drivers that delivered buffers where 
the first byte in the buffer equated to the first component of the 
upper-left most pixel in the image, and subsequent bytes proceeded in 
a row-first fashion. The rest of ARToolKit and its subsequent ports 
to Irix, Windows and Mac OS X were all based on this specification.

The reason then for drawing the image "upside down" in OpenGL is that 
OpenGL expects pixel data to begin with the first component of the 
lower-left most pixel. So drawing pixel data using OpenGL involves 
flipping the y-coordinate system. This has almost no CPU cost.

Now for (presumably) worthwhile reasons, Windows DS capture delivers 
bottom-up buffers. If we were to change the type of buffers expected 
by ARToolKit to bottom-up, we would break usage on the other 3 
platforms (Linux, Mac OS X, Irix). We'd like to keep the original 
specification that ARToolKit use top-down buffers, because the code 
is being most actively developed by the original creator Dr. Kato on 
Linux, and its possible that he may release his toolkit with natural 
feature tracking, so we're retaining the Linux method as the spec.

What this means for Windows users is three possibilities: (1) (What 
we do most often here in our lab) is to turn the camera upside down 
and use the WDM capture dialog's "mirror horizontally" setting. (2) 
(What is done in ARToolKit 2.68.2) pass "flipV" parameter to 
DSVideoLib on windows which makes it do an in-place vertical flip 
(actually not that costly in terms of CPU.. try profiling it on your 
machine to see this) or (3) (What Thomas did in his release) patch 
your own copy of ARToolKit to work with bottom-up buffers.

Hope this clears up some of the confusion.

Regards,

Phil.

At 5:38 PM -0400 4/8/04, Yohan Baillot (Contractor) wrote:
>Hi everyone !
>
>I was looking into using my own library that capture and display 
>images using pixel copy or texture rendering. I basically replace 
>the frame grabbing of AR toolkit by my own function and the 
>rendering in ARToolkit by my own function (this is building on work 
>that GeorgiaTech did that improves ARToolkit using OpenCV). However 
>I noticed that when I do that ARToolkit does not work anymore, or my 
>video appear upside down, and the object move the opposite direction 
>of the video. I dug in the code and found out that the texture or 
>pixel rendering was done to render the image up-side down with 
>respect to the way it is captured, which explained the problem. The 
>pixel copy rendering is done using glZoom (zoom,-zoom) which put the 
>image up side down and the texturing is done from top to bottom but 
>with Y texture coordinates increasing which basically revert the 
>image as well. Now if I want to use my own library to capture and 
>render the image, I need to pass to ARToolkit an image that is 
>reverted vertically from what I capture, which involve an array copy 
>which is a performance hit. I cannot change my library to revert the 
>rendering as ARToolkit does because that would mean that I have to 
>use glZoom (-zoom,-zoom)  as well if I want to render pixels that 
>are not reverted with respect to texture
>
>I was wondering if there was a reason why this approach was taken or 
>if that is simply because the logic of image processing and 
>transform derivation had been designed inverted from the start and 
>the image display inverted was done to fix the problem quickly. If 
>the reason was the later, could I request that next release do 
>display what is captured, not the frame inverted?
>
>thanks
>
>Yohan


-- 
Philip Lamb
Research Student,
Human Interface Technology Laboratory New Zealand,
University of Canterbury, Christchurch, NZ.
+64 3 3642987 x3070