Win-captute with BitBlt and PrintWindow and Opengl examples

[ErosOlmi]

Attached a script I did some time ago about capturing DC of windows where the mouse is located.

Try to change %STRETCH to 0 and see difference.

Regards
Eros

[Petr Schreiber]

Hi Eros,

works good here,
although I felt a bit sick when I saw Start button zoomed to whole window, %STRETCH = 0 worked good to make it small again :)


Thanks,
Petr

[holomind]

Hi Nice Script,

i have modified it a bit, so it now uses antialize, and the aspect-ratio is corrected, so it has the ratio of the source-hdc.

the idea is similar to a screenloupe or livewindows-program i made earlier.
http://www.autohotkey.com/forum/topic11588.html

seems ahk and thinbasic is not soo different after all. i like the way dll-aliases are defined very much. (and do similar things in ahk to make code more readable.)

i will try to port my PrintWindow-Calls to this script and get more experience with thinBasic.

is it possible to take the current hdc and bitblt it to a texture in a TBGL Window ?
which means the texture mapped on a rectangle would "mirror" the window under the cursor ?

is there a function already to save the hdc into a Bitmap? this way one could store it temporary and then load it as texture . (direct bitblt would be much more effient...)

thinBasic looks good too me, and Parts are better than AHK, but AHK has a muuch bigger community and collection of scripts. would be nice to have the power of thinbasic with the community of AHK ;)



greetings
Holomind

[Petr Schreiber]

Hi holomind ,

The image filtering looks very nice


Thanks,
Petr

[holomind]

Hi holomind ,

The image filtering looks very nice


Thanks,
Petr


Hi Petr,
as you seem to be the OpenGL Guru on this Forum here, do you have an idee how i could copy the HDC (Desktop-Buffer) into a GL-Texture in near Realtime (means without saving it to disk and using LoadTexture) .
>> is it possible to take the current hdc and bitblt it to a texture in a TBGL Window ?
>> which means the texture mapped on a rectangle would "mirror" the window under the cursor ?

Do you think this is possible to do, or really hard ?

[Petr Schreiber]

Hi Holomind,

I am still very far from being guru :D

You can pass a texture to TBGL both from disk ( TBGL_LoadTexture ) and from memory ( TBGL_MakeTexture ).

I must admit it has been some time I tried to do a tricks you describe, so I have to refresh my memory on this.
Possible problem could be that the bitmap you would copy would not be with sides of power-of-two.

That is no problem on OpenGL 2.0+ cards, but older cards require the power-of-two sizes.

So yes, I think it is possible, will look on it ASAP. Using BitBlt & OpenGL is not realisable I think, as Microsoft says combining GDI and OpenGL graphics is anything but good and safe idea.

I will try to find a way and post it here, it is just about converting the given HDC bitmap to RGB or RGBA sequence.


Petr

[holomind]

Hi Holomind,

I am still very far from being guru :D

You can pass a texture to TBGL both from disk ( TBGL_LoadTexture ) and from memory ( TBGL_MakeTexture ).

I must admit it has been some time I tried to do a tricks you describe, so I have to refresh my memory on this.
Possible problem could be that the bitmap you would copy would not be with sides of power-of-two.

That is no problem on OpenGL 2.0+ cards, but older cards require the power-of-two sizes.

So yes, I think it is possible, will look on it ASAP. Using BitBlt & OpenGL is not realisable I think, as Microsoft says combining GDI and OpenGL graphics is anything but good and safe idea.

I will try to find a way and post it here, it is just about converting the given HDC bitmap to RGB or RGBA sequence.


Petr


Thanks, this helps a lot, mostly to decide if its possilble at all.

As i can controll the size of the HDC in BitBlt or similar, the Image can be optimized for Power of 2 sizes. (but i guess this is only a speed improvement, but not square images will work too).

as you point out, the conversion from hdc-format to RGBA will be the speed-killer, because they use a different format. and converting each pixel will be slow by definition. (bitblt is much faster than copy pixel one by one)

The Idea behind this all is to grab the output of a window and mirror it to a polygon in the opengl space. then this "thumbnail" can be rendered, moved and even rotated and zoomed very fast. doing animation with strechblt is possible but not really fast. (but actually fast enough) .

the basis will be for several programs which have the same core and different guis. there are several programs i like to understand , code myself and then tweak, because most of them are closed source. (and i like to tweak my programs and enhance with new little features, like the antialize in this case.) Like Expose, Scalable-Framework, Sphere-XP, Virtual-Desktop-Manager and perhaps also a Software-Emulated Compiz/XGL for Window-XP (which does not exists i think). This might be easier in Windows-Vista, but i dont like Vista and it should be possible in Windows-XP also. Because on the Same Hardware i can run Ubuntu with XGL. So it is "theoretically" possible.
The architecture is quite different (GDI-Plus vs. XGL with GL-X-Server) so it cannot be compared directly. XGL and Vista (or OSX) improve speed, because they Render the WM_Paint function directly into a Texture. Which will be quite hard with Windows-XP, but perhaps can be Tricked with a Graphic-Mirror-Driver like used in VNC, where only the Changes need to pe updated and not the full Image all the time. ) A Modified Mirror-Driver then could Paint directly into the opengl-Texture. And one step further to an architecture like Quarz on mac or XGL on Linux. With the benefit to have a "better" Desktop-manager and using "old" window-Applications. (Perhaps this is all Nonsense, because Vista solves the Problem, but Doing and Programming it oneself is just more fun..)

the gl-function glcopyteximage2d looks helpful for this task. (perhaps TBGL_MakeTexture uses this already)  http://www.opengl.org/documentation/specs/man_pages/hardcopy/GL/html/gl/copyteximage2d.html

This means one can copy pixels into a texture with GL_READ_BUFFER. Which moves the Problem, because how do i get my hdc into the READ_BUFFER (i guess Front or (better) Backbuffer of OpenGl). I read somewhere that you can grab a screenshot by copying the desktop into the frontbuffer of directx/opengl. which would enable to caputure moving video etc.

Another thing i need for the application is to make the opengl-window transparent, at the places where there is only black background (or other color , to allow black for painting).
But this can be solved easily by GDI/MFC Window Transparency. Or Similar to the Clip-Window-Function. I have seen some examples in this forum here. Running Fullscreen with Black background is good in enough for the start.

Even if Microsoft recommends that its not so fast, i guess it will be much faster doing this in memory than saving a screenshot as BMP to file/disk and then read it in again with LoadTexture. (loading from disk should always be slower than moving arround in memory).

Greetings
Holomind

[Petr Schreiber]

Thanks for lot of ideas,

TBGL_MakeTexture accepts RGB and RGBA buffer, or complete BMP or TGA from memory, so should be usable.

Your project ideas are very exciting, I will try to keep you supplied with functions you need for this job.

glCopyTexImage2D is already used in TBGL_RenderToTexture, but it cannot be used to catch other are than that in OpenGL window.

Your idea with masked / irregular window is definitely possible on Vista, real guru Patrice Terrier did something similar, as well as fast Aero emulation for Windows XP :) On XPs I did not tried that so cannot say yes/no until I try it.

I am doing some experiments to grab window2bmp, no success yet but I should be close :)


Thanks,
Petr

[Petr Schreiber]

I got some first results,

still not ideal, but quite usable, see the picture.

I will have quite busy days starting from tommorow and ending with Saturday, I will try to do as much as possible, but to be honest I think I will have more time to dedicate to this functions from April 27th.

So please keep me updated, but it is possible my reaction time won't be fast.


Thanks,
Petr

[holomind]

Thanks, this looks great,
do you need to modify the TBGL-Core or is this Pure thinBasic code ?

Perhaps you can post the script you have finished so far, i can experiment too.

I too have a busy week, so no hurry with your experiments.

Greetings
Holomind

[kryton9]

Interesting work guys. Thanks.

[holomind]

Your idea with masked / irregular window is definitely possible on Vista, real guru Patrice Terrier did something similar, as well as fast Aero emulation for Windows XP :) On XPs I did not tried that so cannot say yes/no until I try it.


i was referring to the sample scripts shipped with thinbasic.
SampleScripts\UI\Regions WinRegions_Sample.tBasic

a little gdi window-clipping is sufficient. i dont need translucent windows or glass/aero .

its only needed if the window-manager would run on top of the "normal" windows and one needs to remove the background. but it could be even easier.
like in Transcolor: http://www.autohotkey.com/docs/commands/WinSet.htm#TransColor
perhaps this is a direct DLL-Call or a WM_Message...

Just Make the Background of the Opengl-Window #FF00FF and set TransColor to #FF00FF, then the background gets transparent, and only the Vectors are seen and the black background disapears.

acutally its even possible to combine thinbasic with AHK, because thinbasic provides normal windows, which can then be modified with little AHK-Scripts.

eg. the one-line ahkscript would do the trick

WinSet, TransColor, FF00FF, "Title of my thinbasic Opengl-Window"


Putting the Window Always on Top is also Easy. See the Link to the documentation of WinSet above

But i guess, this little window-manipulations are also easy to do in thinbasic.

[Petr Schreiber]

Hi Holomind,

you can try following code, but it is:

Very slow ( "thanks" to moving big blocks of data )
Not working correctly for some windows
If the window is covered with other window, than that cover window is in the bitmap




Uses "TBGL"
Uses "UI"
Uses "OS"


dim WinHandle as dword
$APP_TITLE_TO_CAPTURE = "thinBasic programming language"

' -- We will shell out Internet Explorer as our capturing victim
dim t1 as long = gettickcount
WinHandle = win_findbytitle($APP_TITLE_TO_CAPTURE, %WIN_FindTitleStart)
if winHandle = 0 then
OS_shellexecute("open", "iexplore", "www.thinbasic.com", "", 1)
while WinHandle = 0
WinHandle = win_findbytitle($APP_TITLE_TO_CAPTURE, %WIN_FindTitleStart)
if gettickcount - t1 > 20000 then stop ' -- If we cannot find it in 20 seconds, better to stop
wend
end if

' -- Create and show window
Dim hWnd As Dword = TBGL_CreateWindowEx("Window capture early demo", 640, 480, 32, %TBGL_WS_WINDOWED or %TBGL_WS_CLOSEBOX, 200,200)
TBGL_ShowWindow


' -- This will size the target window to acceptable dimensions
Dialog set size WinHandle, 512, 512
Dialog set loc WinHandle, 0,0
win_setForeground(WinHandle)

InitAllNeededWin32APIs()

DIM FrameRate AS DOUBLE

' -- Resets status of all keys
TBGL_GetAsyncKeyState(-1)

' -- First snapshot
Window2BMP(WinHandle)
dim UpdateTime as long= GetTickCount


' -- Main loop
While TBGL_IsWindow(hWnd)
FrameRate = TBGL_GetFrameRate

' -- We will take a photo each 100ms
if gettickcount - UpdateTime > 100 then
Window2BMP(WinHandle)
UpdateTime = GetTickcount
end if

TBGL_ClearFrame
TBGL_Camera 2, 2, 2, 0, 0, 0

tbgl_UseTexturing %TRUE
tbgl_BindTexture 1

tbgl_Rotate sin(gettickcount/1000)*20,0,0,1
TBGL_BeginPoly %GL_QUADS
tbgl_TexCoord2d 0,0
tbgl_vertex -1, -1, 0

tbgl_TexCoord2d 1,0
tbgl_vertex 1, -1, 0

tbgl_TexCoord2d 1,1
tbgl_vertex 1, 1, 0

tbgl_TexCoord2d 0,1
tbgl_vertex -1, 1, 0
tbgl_endPoly

TBGL_DrawFrame

' -- ESCAPE key to exit application
If TBGL_GetWindowKeyState(hWnd, %VK_ESCAPE) Then Exit While

Wend

TBGL_DestroyWindow


sub Window2BMP( whandle as dword )
if whandle = 0 then exit sub

LOCAL hBMPDC, hMemDC, hMemBMP AS dword
LOCAL rc AS RECT
LOCAL bm AS BITMAP
LOCAL bmi AS BITMAPINFO
LOCAL bmpFileHdr AS BITMAPFILEHEADER
LOCAL bmpInfoHdr AS BITMAPINFOHEADER
LOCAL sBmpFile AS STRING

GetWindowRect (whandle, rc)

hBMPDC = CreateDC( "DISPLAY", "", "", "" )
hMemDC = CreateCompatibleDC( hBMPDC )
bmi.bmiHeader.biSize = SIZEOF( bmi.bmiHeader )
bmi.bmiHeader.biWidth = ( rc.nRight - rc.nLeft )
bmi.bmiHeader.biHeight = ( rc.nBottom - rc.nTop )
bmi.bmiHeader.biPlanes = 1
bmi.bmiHeader.biBitCount = 24
bmi.bmiHeader.biCompression = %BI_RGB

hMemBmp = CreateDIBSection( hMemDC, bmi, %DIB_RGB_COLORS, 0, 0, 0 )
GlobalLock (hMemBmp)
SelectObject (hMemDC, hMemBmp)

GetObject (hMemBmp, SIZEOF( bm ), varptr(bm))

BitBlt (hMemDC, 0, 0, bm.bmWidth, bm.bmHeight, hBMPDC, rc.nLeft, rc.nTop, %SRCCOPY)
bmpFileHdr.bfType = CVI( "BM" )
bmpFileHdr.bfSize = LEN( bmpFileHdr ) + LEN( bmpInfoHdr ) + bm.bmWidthBytes * bm.bmHeight
bmpFileHdr.bfOffBits = 54
bmpInfoHdr.biSize = 40
bmpInfoHdr.biWidth = bm.bmWidth
bmpInfoHdr.biHeight = bm.bmHeight
bmpInfoHdr.biPlanes = 1
bmpInfoHdr.biBitCount = 24
bmpInfoHdr.biSizeImage = 54 + bm.bmWidthBytes * bm.bmHeight

local sBuffer as string
local bmptr as dword = bm.bmBits
sBuffer = PEEK$(VARPTR(bmpFileHdr), SIZEOF(bmpFileHdr))+PEEK$(VARPTR(bmpInfoHdr), SIZEOF(bmpInfoHdr))+PEEK$( bmptr, bm.bmWidthBytes * bm.bmHeight )
'tbgl_setWindowTitle(hWnd, "Dim:"+STR$(bm.bmWidth)+STR$(bm.bmHeight)+STR$(rc.nRight - rc.nLeft)+STR$(rc.nBottom - rc.nTop))
'sbuffer = tbgl_getwindowbmp(whandle)
tbgl_MakeTexture sBuffer, %TBGL_FILE_BMP, bm.bmWidth, bm.bmHeight, 1, %TBGL_TEX_MIPMAP, 16

' -- Cleanup
DeleteDC (hBMPDC)
DeleteDC (hMemDC)
GlobalUnlock (hMemBmp)
DeleteObject (hMemBmp)


end sub

sub InitAllNeededWin32APIs()
' -- Win32 structures and functions
TYPE RECT
nLeft AS LONG
nTop AS LONG
nRight AS LONG
nBottom AS LONG
END TYPE

TYPE BITMAP
bmType AS LONG
bmWidth AS LONG
bmHeight AS LONG
bmWidthBytes AS LONG
bmPlanes AS WORD
bmBitsPixel AS WORD
bmBits AS BYTE PTR
END TYPE

TYPE BITMAPINFOHEADER
biSize AS DWORD
biWidth AS LONG
biHeight AS LONG
biPlanes AS WORD
biBitCount AS WORD
biCompression AS DWORD
biSizeImage AS DWORD
biXPelsPerMeter AS LONG
biYPelsPerMeter AS LONG
biClrUsed AS DWORD
biClrImportant AS DWORD
END TYPE

TYPE BITMAPINFO
bmiHeader AS BITMAPINFOHEADER
bmiColors AS LONG
END TYPE

TYPE BITMAPFILEHEADER WORD
bfType AS WORD
bfSize AS DWORD
bfReserved1 AS WORD
bfReserved2 AS WORD
bfOffBits AS DWORD
END TYPE

%SRCCOPY = &H00CC0020 ' (DWORD) dest = source
%BI_RGB = 0&
%DIB_RGB_COLORS = 0 ' color table in RGBs
' size of a device name string
%CCHDEVICENAME = 32

' size of a form name string
%CCHFORMNAME = 32

TYPE DEVMODE
dmDeviceName AS ASCIIZ * %CCHDEVICENAME
dmSpecVersion AS WORD
dmDriverVersion AS WORD
dmSize AS WORD
dmDriverExtra AS WORD
dmFields AS DWORD
dmOrientation AS INTEGER
dmPaperSize AS INTEGER
dmPaperLength AS INTEGER
dmPaperWidth AS INTEGER
dmScale AS INTEGER
dmCopies AS INTEGER
dmDefaultSource AS INTEGER
dmPrintQuality AS INTEGER
dmColor AS INTEGER
dmDuplex AS INTEGER
dmYResolution AS INTEGER
dmTTOption AS INTEGER
dmCollate AS INTEGER
dmFormName AS ASCIIZ * %CCHFORMNAME
dmLogPixels AS WORD
dmBitsPerPel AS DWORD
dmPelsWidth AS DWORD
dmPelsHeight AS DWORD
dmDisplayFlags AS DWORD
dmDisplayFrequency AS DWORD
dmICMMethod AS DWORD
dmICMIntent AS DWORD
dmMediaType AS DWORD
dmDitherType AS DWORD
dmICCManufacturer AS DWORD
dmICCModel AS DWORD
dmPanningWidth AS DWORD
dmPanningHeight AS DWORD
END TYPE


DECLARE FUNCTION GetWindowRect LIB "USER32.DLL" ALIAS "GetWindowRect" (BYVAL hWnd AS DWORD, lpRect AS RECT) AS LONG
DECLARE FUNCTION CreateDC LIB "GDI32.DLL" ALIAS "CreateDCA" (lpDriverName AS ASCIIZ, lpDeviceName AS ASCIIZ, lpOutput AS ASCIIZ, lpInitData AS DEVMODE) AS DWORD
DECLARE FUNCTION CreateCompatibleDC LIB "GDI32.DLL" ALIAS "CreateCompatibleDC" (BYVAL hdc AS DWORD) AS DWORD
DECLARE FUNCTION CreateDIBSection LIB "GDI32.DLL" ALIAS "CreateDIBSection" (BYVAL hdc AS DWORD, pbmi AS BITMAPINFO, BYVAL dwUsage AS DWORD, BYVAL ppvBits AS DWORD, BYVAL hSection AS DWORD, BYVAL dwOffset AS DWORD) AS DWORD
DECLARE FUNCTION GlobalLock LIB "KERNEL32.DLL" ALIAS "GlobalLock" (BYVAL hMem AS DWORD) AS DWORD
DECLARE FUNCTION SelectObject LIB "GDI32.DLL" ALIAS "SelectObject" (BYVAL hdc AS DWORD, BYVAL hObject AS DWORD) AS DWORD
DECLARE FUNCTION GetObject LIB "GDI32.DLL" ALIAS "GetObjectA" (BYVAL hObject AS DWORD, BYVAL nCount AS LONG, byval lpObject AS dword) AS LONG
DECLARE FUNCTION BitBlt LIB "GDI32.DLL" ALIAS "BitBlt" (BYVAL hDestDC AS DWORD, BYVAL x AS LONG, BYVAL y AS LONG, BYVAL nWidth AS LONG, BYVAL nHeight AS LONG, BYVAL hSrcDC AS DWORD, BYVAL xSrc AS LONG, BYVAL ySrc AS LONG, BYVAL dwRop AS DWORD) AS LONG
DECLARE FUNCTION DeleteDC LIB "GDI32.DLL" ALIAS "DeleteDC" (BYVAL hdc AS DWORD) AS LONG
DECLARE FUNCTION GlobalUnlock LIB "KERNEL32.DLL" ALIAS "GlobalUnlock" (BYVAL hMem AS DWORD) AS LONG

end sub


Thanks a lot for the AutoHotKey tips, when I will have more time I will try how it could interact with TBGL window ... then steal their technology :D


Bye,
Petr

[ErosOlmi]

Absolutely wonderful script :o

[Petr Schreiber]

Once it will be more reliable,

it could be used for that famous 3D windows flipping from Vista :)

But it will be long run to optimize this, with small windows it is quite ok to update them at nearly realtime speed, but fullscreen ThinAir takes some time, at least on my PC.


Petr

[holomind]

Thanks Psch,

will try it soon, the code is not too long, and i hope i will understand it.

speed improvement can be done later, one big speed improvement would be to use a smaller texture, as in expose you dont need full resolution. after strechblitting the hdc into a memory-buffer, you only need to load a smaller "image" into the texture.

its normal that overlaying windows show up, because bitblt and strechblt work with the "Desktop" or "Frontbuffer" of Windows, there is no distinction between windows. its only clipped.

but the trick here is to use printwindow in combination of strechblt, as printwindow can capture "offscreen" windows. (but printwindow is slightly slower, but comparable to bitblt) .

also a refreshrate of 1frame / sec is fast enough for "refreshing" the images. it does not have to be realtime.

[Petr Schreiber]

Hi,

if you will have problems decyphering my code, let me know.
It is almost pure API, so should not be difficult.

Your idea with PrintWindow and others is cool, I will check it out but now I am in little bit hectic time so cannot promise anything very soon.

1 sec refresh is ok, but there must be way to do it fast and in realtime, which is a challenge which cannot be left unsolved :)


Thanks,
Petr

[holomind]

Hi, i have Printwindow working,
but there is something wrong in the order the windows are "Selected" into. Perhaps you get it working.

you will get some "smearing" on the desktop , because the printwindow-dc is not released correctly.
(you can "refresh" your screen with "windows-key D" 2times which minimizes all windows and forces a repaint of them , thus restoring your desktop from artefacts.

at least it works to have the "Browser-Window" captured, even if the opengl-window is on-top.

i also introduces CreateCompatibleBitmap, because it think you need to render on the compatible Bitmap and not compatibleDC (which will be Desktop) . i am confused.

you can see my expose clone for comparison, how i did the printwindow-calls there (maybe they are wrong there too!)

http://www.autohotkey.com/forum/topic13001.html (big, full expose script)

or

http://www.autohotkey.com/forum/topic11588.html (shorter only one printwindow)

knowing the speed of bitblt, i dont think its too slow.
(its the "normal" speed of bitblt-ing a window as fast as you can.)

[Petr Schreiber]

Well ... wow :D,

your autoHotKey scripts are very cool, I think especially the expose.ahk will stay active for most time, as I usually have incredible mess on desktop, now I can choose freely. Awesome job! I have seen similar tricks on Linux, but never on Windows, really useful and working well!

The modified thinBASIC script now works nice too, only some flicker when moving any window over IE one.


Thanks,
Petr

[holomind]

Hi,

Flip3d is nearly ready. I have improved the printwindow-calls so they dont flicker any more.
sometimes there are little flicker, but that is the nature of printwindow.

i have integrated the new Win_Get() function from Eros (see other post) and now reading all toplevel-windows and display them all at once in the gl-window.

startup is a bit delayed because the windows need to get captured.

the reading of window-list is dynamic, which means you can minimize or change order of your windows and it will be reflected in winflip directly. also the windows are all captured in realtime (each 100 or 400ms) and rewritten to the textures. you can check this if you have an animation in page like on www.thinbasic.com , or simply by scrolling the browser window.

now we only need to optimize it and add some keyboard, and make the perspective more usable.

thinbasic is really cool, didnt think this would be possible at all, and also in such a short time.
performance is not too bad.

see the script as attachment.

also the screenshot. (actually you can see my menu-bar from macbook, as i have programmed this inside of parallels (virtual maschine) , which gives even usable performance with opengl.) 

Please experiment with it and post your modifications of the script.


you need to apply this UI-Patch from Eros, to get the Win_Get() function !!!
http://community.thinbasic.com/index.php?topic=1689.msg12278#msg12278

Greetings
Holomind

[kryton9]

That is REALLY neat Holomind! Thanks!!

[ErosOlmi]

Fantastic job Holomind! :o

I suppose we can transform part of Window2BMP function (or even all function) into a compiled function in UI or TBGL module and get a good speed improvement.

[ErosOlmi]

you need to apply this UI-Patch from Eros, to get the Win_Get() function !!!
http://community.thinbasic.com/index.php?topic=1689.msg12278#msg12278


Very soon a new thinBasic version will integrate all recent changes ...

[Petr Schreiber]

Hi,

that is very nice!

Only thing which makes me nervous is that when I force window bitmap to keep the original size, the image is "damaged".

I have OpenGL 2.1 card, so non power of two textures should be ok ... not sure what could be wrong, will check MakeTexture for possible problems during weekend.


Thanks,
Petr

[holomind]

Hi,

no new code ? but its inteesting, when the window2bmp would be optimized. but i guess PEEK should be quite fast already. i guess the slowest part is also the printwindow function. the problem is, that always the full screen is grabbed and not only the changes. it would be way more efficient to intercept the api message WM_PAINT() of other windows. (e.g. like the VNC-Server Mirror-Driver does, Hmm, maybe the source-code of the driver is available).

Next implementations go into different directions.
a) cloning of existing window-managers
b) creating a new window-experience and do some experimental UIs

The 3d features are nice, but i will use it "flat" for simulating expose, which has a better usability than flip3d (which hides information again). the only thing needed now is some animation to move the textures to get nice expose-animation. one trick also is to capture the desktop with DllCall: PaintDesktop, and then switch to Fullsceen GL, and paint there the current windows (exactsizes) and then move out the windows and arrange them (either in a grid, or spacial) my old real-expose-clone script has the needed formulas already, only need to port the syntax to thinbasic. (and simplify some parts).

Other Tools to clone would be
* Scalable-Framework. (each window gets minimized and can be arranged manually at the screenborders)
* Perhaps even a Dock (where you have your windows in iconized form, perhaps as persistent window with seldom refreshes, eg. every 5 seconds to save cpu) instead of opengl this could also be done with pure GDI+, but opengl would very easly allow for hover-zoom like in the real mac-doc (even if i never use this feature ;)
* Expose of course
* Virtual-Desktop manager (Perhaps similar to Spaces), but experimenting with it
* ZUI (Raskin, Zoomable-User-Interface) would be really cool, and might be possible with opengl.
* Desktop-Loupe
* DeskMan
* Europa-Desktop
* SUN Looking Glass. (with the nice feature to turn around a window and write comments on the back)
* Desk-Loops
* or like http://update.docs.kr/entry/Products-Shock-Aero3D-en

They are all very similar, and i intend to put them all in one application, with configuration to change the gui/apperance.

if this mirroring of windows to gl-texture is smooth and fast enough, it might also be possible to "work" directly in the direct-x window and then zoom windows in and out by will. also simply scale the windows a bit, i sometimes also want to zoom out windows, and watch them on the side of the screen. similar to widgets in the sidebar.

many exciting ideas. lets see what of them all will be realized in the future. As this is mainly a hobby-project, i will give it as much time as it needs, and will not hurry ;)

ps. did you recognise alt-tab already works in the flip3d simulation. as the current window moves to the first in the stack.

greetings
daniel

[Petr Schreiber]

Hi Holomind,

I had not time to enhance it as I am busy now.
The respect of Z-order I did not noticed in winflib.tbasic, now I focused on it and can confirm it behaves that way :)

I am leaving for 2 days, so there won't be progress on this from my side, but I keep in mind I have to speed it up :)



if this mirroring of windows to gl-texture is smooth and fast enough, it might also be possible to "work" directly in the direct-x window and then zoom windows...


I am not sure I understand this sentence :)


Bye,
Petr

[ErosOlmi]

Fantastic Petr, always better effects.
Now ... ;D imagine free window/rectagle moving using the mouse in TBGL window
... ops sorry :D

[holomind]

if this mirroring of windows to gl-texture is smooth and fast enough, it might also be possible to "work" directly in the direct-x window and then zoom windows...


I am not sure I understand this sentence :)


This means we could simulate XGL or Compiz this way. Where you start the tbasic script and it opens Fullscreen, and then takes over as a window-manager. Them mouseclicks are translated into the 3d-window (like in SphereXP) and then you can interact with your windows. and you can have a 3d-environment, or Zoom Windows in and Out like in OSX. Eg. Minimize to thumbnail of one window etc.

Analyzing Scalable-Fabric: http://research.microsoft.com/~horvitz/Scalable_Fabric.htm?0sr=a
i realized that it makes many title-less windows, which then can be moved around. they contain a screenshot of the window when minimized, and also allow for live-refresh inside the thumbnail with the help of printwindow.

I dont do c# or .net, and the code is not maintained since 2005. so i plan to recode it with thinbasic or ahk.

For ZUI (Zooming Interface Demo see here), this can maybe also be done with our script.
http://rchi.raskincenter.org/index.php?title=Demos

No need, to hurry, but i wondered why nobody posted a modified version, as many have downloaded and commented on it.

One thing is also that i need to decide if its better to run all things in opengl, or if its better to split it in several windows and then move this little windows around. (but the command WinMove in AHK flickers (or better said the Win-API implementation of the DLL call. so Opengl would be more smooth).

By accident i also found some nice flip3d and virtual-desktop implementations. (even freeware).
http://update.docs.kr/entry/Products-Summary-Desktop-Utility-en

Greetings
Holomind

[holomind]

Hi, some news from the AHK-Front,

somebody managed to get OpenGL working there with Pure DllCalls.
http://www.autohotkey.com/forum/viewtopic.php?p=192897#192897

this means i will develop my Ideas Parallel in thinBasic and AHK. (Perhaps i can find a "compatiple" syntax, to copy paste parts of code...)

Greetings
Holomind

[ErosOlmi]

Good.
Whatever info can be of inspiration for many.

Thanks
Eros

[holomind]

Whatever info can be of inspiration for many.


Yes, i am linking this two forums a little bit so each can share their knowledge. Also on the AHK-Forum there are lot more users, and examples. And AHK can benefit from the strong OpenGL and other skills from this forum. (Also the ASM-Thread is amazing.) thinBasic could benefit from more users ;)

P.S: A comment to the current Script. It seems there is a bug in the PrintWindow function or i dont use it correctly. because some WM_Paint messages dont reach the source-window. and lead into broken rendering, because the real window is not refreshed correctly. (Perhaps we need some "Invalidate DC" or similar, so this window gets repainted. Or some ReleaseDC. ...

[Petr Schreiber]

Hi Holomind,

I must leave now, but if you wish to ride both languages at once, it might be easier for you to use TBGL just for window handling/swapping buffers/handling keys, and for graphics use "pure DLL calls".

To use OpenGL in thinBASIC, just write following on the beginning of the script


#include "thinBASIC_gl.inc" ' -- For basic statements of OpenGL ~1.1
#include "thinBASIC_glu.inc" ' -- For glu functions like gluLookAt
#include "thinBASIC_glext.inc" ' -- For OpenGL up to v2.1


Then the equvialent of AHK:


DllCall("opengl32.dll\glVertex3f", Float, -1.0, Float, 1.0, Float, 0.0)


would be this in thinBASIC ( similar to the way in C/PowerBASIC/...):


glVertex3f( -1.0, 1.0, 0.0)


See SampleScripts/TBGL/TBGL_UsingAPI.tBasic for more samples of calling OpenGL functions directly.


Thanks,
Petr

P.S. I am leaving in hour for 2 days out, so not sure if I can react here during that period.

[ErosOlmi]

holomind,

if you want some more fun in TBGL (OpenGL module developed by Petr), have a look at Bonus Packs at http://www.thinbasic.com/index.php?option=com_docman&task=cat_view&gid=23&Itemid=66
You will find tons of TBGL example to study from.

Have nice time.
Ciao
Eros

[holomind]

...if you want some more fun in TBGL (OpenGL module developed by Petr), have a look at Bonus Packs at http://www.thinbasic.com/index.php?option=com_docman&task=cat_view&gid=23&Itemid=66
You will find tons of TBGL example to study from.

Thanks for the hint, but this examples are exactly the reason i now understand opengl and tbgl. it was one of the first things i found at the website and motivated me to choose thinbasic to implement my ideas. the examples are quite good and easy enough. I like this forum, as the quality of code and the responsiveness in the forum is very good and can compensate quite good for the quantitiy of users at the AHK Forum.

[ErosOlmi]

Perfect.

Anyhow consider that TBGL module is not just an OpenGL wrapper but a lot more (entity system is an example).
Petr did (and still doing) a great job in keeping things simple even if "inside" they are not so simple.
So it is not immediate to compare TBGL to other OpenGL implementations.

Ciao
Eros

PS: I like your "battle" in AHK forum about DllCall. I see AHO is a fantastic scripting language but when dealing with external DLLs, repeating DllCall and specify all parameter type again and again is something that could be avoided implementing external functions/subs declarations. Of course it is not something simple to implement because programmers have to handle some internal dictionary of all declared functions and their parameters sequence/type/options. Anyhow a "soft battle" is worth, maybe in a separated forum post.

[Michael Hartlef]

Anyhow consider that TBGL module is not just an OpenGL wrapper but a lot more (entity system is an example).


Yes, it is much more. It does a lot of little things under the hood. But the good thing is that you can allways mix it with direct OpenGl calls.



Btw. Holomind. aus welcher Ecke Deutschlands kommst Du? Ich lebe in Leverkusen. :)

[holomind]

Hi, there is also some OpenGL discussion starting over at the AHK forum.
I didnt find an online-link to opengl / TBGL-documentation on this forum, so i attach it on this post, would be nice to put it in the downloads-section. or even better have it available as html online. (eg. perhaps in the wiki?) The AHK-Forum has no attachments for posts, so i attach the tbgl-documention it here. And people must come here to get it, which seems fair ;)

P.S: Can some Forum-Admin update this Forum-Thread-Topic to something more fitting, like
"Win-captute with BitBlt and PrintWindow and Opengl examples" ?
so People would know we discuss about opengl and some BitBlt , Printwindow etc. tricks.

[holomind]

PS: I like your "battle" in AHK forum about DllCall. I see AHO is a fantastic scripting language but when dealing with external DLLs, repeating DllCall and specify all parameter type again and again is something that could be avoided implementing external functions/subs declarations. Of course it is not something simple to implement because programmers have to handle some internal dictionary of all declared functions and their parameters sequence/type/options. Anyhow a "soft battle" is worth, maybe in a separated forum post.

there are 2 ways to do it i AHK,
a) you write a simple AHK-script-function which wraps it, this is "normal" AHK script, i did this already with my real-expose-clone script. you can even use include-files in AHK, so you only say
use "gl_wrapper.inc" and it defines all GL-Functions for you.

b) you can extend AHK itself to support it and make "real" AHK-Functions (=language syntax) out of it, which is less flexible as you need to compile it each time you update it.

i already did automatic conversion of many MFC functions, but nobody noticed this script up to now. should also be easy to convert all gl-functions from a .h header file
http://www.autohotkey.com/forum/viewtopic.php?t=13171

[ErosOlmi]

I didnt find an online-link to opengl / TBGL-documentation on this forum, so i attach it on this post

thinBasic_TBGL.chm is distributed with every thinBasic instalaltion. iT can be found under \thinBasic\Help\ directory. SO in reality there is no need to replicate it into download area of the web site. Instead we made a download version of thinBasic help material in order for potential users to evaluate thinBasic capabilities and syntax before doing a full download.

Tip: to attach any .chm file to thinAir (thinBasic editor) go to thinAir config screen and add .chm file under Help tab. In this way when you press F1 over a token in the script, thinAir will try to search in all .chm files configured.



P.S: Can some Forum-Admin update this Forum-Thread-Topic to something more fitting, like
"Win-captute with BitBlt and PrintWindow and Opengl examples" ?
so People would know we discuss about opengl and some BitBlt , Printwindow etc. tricks.

Maybe better to open a new dedicated thread. Sometimes too big threads are very difficult to follow because after a while many arguments are touched inside it.

Ciao
Eros

[ErosOlmi]

there are 2 ways to do it i AHK,
a) you write a simple AHK-script-function which wraps it, this is "normal" AHK script, i did this already with my real-expose-clone script. you can even use include-files in AHK, so you only say use "gl_wrapper.inc" and it defines all GL-Functions for you.
b) you can extend AHK itself to support it and make "real" AHK-Functions (=language syntax) out of it, which is less flexible as you need to compile it each time you update it.


Exactly the same here in thinBasic.

You can create script function wrapper.
You can create compiled modules using thinBasic SDK (like TBGL) in many compiled languages like Power Basic, FreeBasic, C, Delphy, ASM
But when you have to deal with external standard DLLs, you can just declare DLL exported functions and use them as native functions.

[Petr Schreiber]

Hi guys,

I am back and with some very basic windows dragging over the screen, try the attachement.
It is not bullet proof, but could serve for starting something better.


Petr

P.S. I cannot test any of the newer win capture scripts from thinAir, as it hangs, no idea why :(

[kryton9]

Very nice Petr, thanks!

[holomind]

if you want some more fun in TBGL (OpenGL module developed by Petr), have a look at Bonus Packs at http://www.thinbasic.com/index.php?option=com_docman&task=cat_view&gid=23&Itemid=66
You will find tons of TBGL example to study from.

For those (like myself) who missed it, very nice NeHe implementations in thinBasic:
http://community.thinbasic.com/index.php?board=87.0

and the "essential" opengl-bible: (thanks for the tipp in the AHK-Forum)
http://www.glprogramming.com/red/chapter01.html
(opengl in AHK is now mature enough, as i ported the first example from redbook to AHK-Code with minor syntax corrections, because of some specialties in AHK, with some dll-wrapper functions you can now copy/paste c-code examples from NeHe or RedBook, if they are simple enough.. Its only a matter of time to port them, but no technical/experience/framework problem any more..)

i now wonder if i should go with the TBGL_* functions or use gl* functions directly in thinBasic to keep the scripts more "standards"-based. but as somebody already mentioned there are some things simpler when using TBGL_.

Its a bit paradoxic to use c-syntax and not compiling the programms, but i really like interpreted languages for rapid-protoyping and i cant stand it to compile scripts or include unneccessary .h header files (hmm. the dll-call wrappers are quite similar to .h files ;)

thinbasic and AHK mutate to an c-interpreter for functions in opengl, gdi+ and other mfc-dlls.

wow, i learned more opengl in the last 2weeks than reading opengl or direct3d "game-development" books from my local (not so small) bookstore. i really like these forums ;)

[holomind]

I am back and with some very basic windows dragging over the screen, try the attachement.


works good for me, and gives new ideas for implementing the ideas above. some comments...

http://www.youtube.com/watch?v=jWUwxe8JWVA&NR=1
(spherexp is nice, but i dont like it, that i cant tweak the code for my needs, also it has not been updated for a long time...)

it would be great to zoom the current window below the cursor with an hotkey or perhaps the wheelmouse ? should not be difficullt, only apply some glTransformation on the selected vector-matrix.

i would perfer to have realtime-update of the selected window. if the miniature-windows are bigger one could translate mouseclicks into the source-window. (similar to sphere-xp)

i did dome experiments with WinSet, Transcolor, but did not manage to get the opengl-window transparent (only the title gets transparent) . i guess the problem is that the Swapbuffer paints ontop of the "normal" window and the transcolor is ignored, because the normal window is behind the opengl-canvas. but it could be possible to set an alphachannel while calling glClearColor(...., Alpha)

with your demo one can "simulate" expose (gridview) and scalable-fabric a little bit, by arranging the windows. later the window-position should be stored in memory (static vars) so the windows get arranged automatically if you start the opengl-window-manager.

(but scalable fabric might be implemented easier with many little title-less windows, which contain a miniaturized mirror of the window, where the window is moved outside of the view-port, would be like having many live-windows.ahk scripts running at the same time. i dont know if using opengl here is the easiest way, because the background would need to be transparent and clicktrough. (clicktrough could easily be accieved by using WinSet Transcolor , #... 255 )

(sorry for the long post, but perhaps somebody likes these crazy ideas for new ways of thinging about window-managers).

one notice to the distorted windows when using PrintWindow. There seems to be a glitch that the real hDC of the source-window (= is the real desktop in the same time) gets not repainted, if the Printwindow calls WM_Paint, while the WM_Paint should draw onto the desktop. (seems to be a race condition). One trick (found in the ahk-forum) is to move a transparent window ontop of the distorted window (or the whole desktop) then a WM_Paint is forced for the windows which were temporarily behind the windows. (this should also work automatically if the openglwindow has screensize and is always-on-top).

zooming the whole scene and beeing able to resize the miniature-windows would also allow some experiments to simulate ZUI (see other post) from Raskin. ZUI is only a Prototype but explained in detail in the Book "The humane interface".

another idea is to use your current script to simulate a preview-window of a virtual-desktop-manager (like apple-spaces), then moving a thumbnail could also move the window on the real desktop. (in AHK easy with winmove ,x,y, "window-title". )

clicking on the miniaturewindow i would expect to animate it into the real size of the source-window and then move this window to foreground. (this would work much better if running the script in fullscreen instead of a window. and copy the desktop with dllcall(gdi/PaintDesktop)

i am curious how this script (window-manager) will evolve.

for some real speed improvements over printwindow, using a mirrordriver would be cool.
http://www.demoforge.com/dfmirage.htm
http://msdn2.microsoft.com/en-us/library/ms797878.aspx

to simulate the vista-window-rendering architecture one could create a virtual-display with the mirrordriver big enough (would need more memory) to hold the windows without overlapping them. and let windows think its a normal display. then these windows can be copied easily with bitblt and similar to Window2BMP or others. .... and only use the mirrored windows in the opengl fullscreen-window. (viola vista-ui clone) now it gets to crazy ;)
The next improvement would be to intercept the WM_Paint Messages and redirect their Painting directly into the texture (which is only a mapped allocated memory-region) to improve speed.

this way one could get vista-ui with the stability/compatibility of windows-xp.

adding some network-functions one can also extend this into an Remote-Desktop.

Also the idea of doing "Remote-Control" of the Local-Desktop "inside the opengl-window" gives nice possibilites to think about it. also VNC is quite usable, so it "could" be as usable in opengl.

Turning the idea arround, we could also use an VNC-Server with driver-hook to control the local computer instead of copying windows manualle with BitBlt and PrintWindow. (Having no network-Latency on localhost this could also be very fast and responsive). the big difference of the mirror-driver and Bitblt/Printwindow is, that the mirrordriver only sends the "portions" which where modified in WM_Paint-Calls. (Or what about using RDP or Terminal-Service to grab the screen input) eg. a RDP-Client-Clone displaying windows directly inside opengl. (making the Terminal-Service Virtual-Screen Resolution big enough one can prevent overlapping of windows, which allows directy bitblt of hdc.

Also very similar Virtual-Maschines use some kind of mirror-driver to grab the screen inside the simulated computer. sometimes you even see similar "compression"-artefacts like in vnc when using a tool like VMWare or Parallels.

(Perhaps i will move these Theoretical-Ideas/Poste into a "Windowmanager-Philosophy-Thread" ;)

P.S. in one amazing thread over there they discuss the desktop-composition architecture of vista and the usage of mirror-driver and their alternatives. gives good insight how the different memorys are related and used.
http://groups.google.com/group/microsoft.public.development.device.drivers/browse_thread/thread/3cafff960f475869/86e16bc8ffc91dfb?hide_quotes=no

[holomind]

Hi, sorry for making this thread into a blogpost ;)

this might help to improve the speed of Window2BMP2Texture2D in theory. using hardware accelearation for copying pixeldata into a texture or gl-framebuffer.

http://www.songho.ca/opengl/gl_pbo.html

[ErosOlmi]

I will do some native function in next non Win9x compatible version of thinBasic because, as you may know, PrintWindow API is available only WinXP or above OSs.

We are finishing a thinBasic development still compatible with Win9x. The next release I think will be the last compatible.
After that, we will change version from 1.x... to 2.x... Version 2.x will be compatible only with WinXP or above.

Ciao
Eros

[holomind]

After that, we will change version from 1.x... to 2.x... Version 2.x will be compatible only with WinXP or above.
Most GDI or GDIPlus also work with windows-2000 if you copy the gdiplus.dll in the project-folder or windows/systems32. PrintWindow is also Available on win2k with this trick. I think its the right decision to not support win95/98 actively as vista is getting more popular.

[Petr Schreiber]

Holomind,

what a storm of ideas, I feel like a ZX-Spectrum before polynomial regression calculation - possible but mammoth task :)
Your idea with PBO is good, but keep in mind not all 3D accelerators support this extension.

The window zoom is definitely possible, even some deformations and so...
The spherical desktop was nice, but I did not liked how the windows crossed each other.

For such a complex project I propose to write a feature list, discuss it and then start developing. Adding code part by part often results in inefficient solution and spaghetti code ( although in other situations I like spaghetti :) ).


Thanks,
Petr

[holomind]

For such a complex project I propose to write a feature list, discuss it and then start developing. Adding code part by part often results in inefficient solution and spaghetti code ( although in other situations I like spaghetti :) ).

We can start a new Thread for a joint-venture development, and define some features, perhaps with polling for features. I dont know if its worth to make a real project out of it. but it will be fun do try it anyways. Featurelist and code-exchange would be best in the wiki. (as the wiki also supports versioning, i tried this in the ahk-forum and it worked better than posting code as posts all the time.) wiki is a "shared" document.

there are also some features i dont like in spherexp, which could be improved.

[holomind]

Your idea with PBO is good, but keep in mind not all 3D accelerators support this extension.
Seems to be a very new extention, because the VBO or PBO examples (binary.exe) files dont work either on my rather new pc (1year with decent Nvidia gfx), and neither with the vmware 3d-emulation.

On my Mac i have a GPU with "shared memory". Maybe this is an advantage in this case, as the CPU und GPU can access the same address-space and dont need to move data around. Running Opengl inside an Virtualisation gives the whole thing another twist. But the speed inside VMware is accaptable.

I found some nice opengl tutorials
http://www.codesampler.com/oglsrc.htm

About the projects for developing. My first implementation will be a rewrite of the script "real-expose-clone" in ahk and thinbasic. to get more experience with opengl. then i can try to implement some of the more crazy ideas like mentioned before.

I am still looking for a way to get the opengl-window transparent so it can be blended with exitsting desktop more easily. i have no idea why alphachannel does not work in glclearcolor(..). Perhaps a bug in vmware-3d-driver.
(ahk: winset transcolor does not work). But for the real-expose-clone its no problem, as i will copy the desktop with dllcall:paintdesktop and start the animation from there in fullscreen.

[ErosOlmi]

I found some nice opengl tutorials
http://www.codesampler.com/oglsrc.htm

Very very interesting web site.
Thanks for sharing

[holomind]

Your idea with PBO is good, but keep in mind not all 3D accelerators support this extension.
I have to try this extensions again later.
Here is a very good tool to see which extentions or gl-version your gfx card supports:
http://www.realtech-vr.com/glview/download.html
http://wiki.daimi.au.dk/cpntools-help/use_cardinfo_for_windows.wiki (cardinfo.exe)

VMWare-Fusion emulates Opengl1.1 (and a bit 1.2) , (calls itself Microsoft GDI-Render 1.1.0)
Parallels emulates Opengl1.2 (calls itself "Parallels and Apple 1.2...)
My other Card Geforce7600 supports 100% gl2.0 which should be enough. (i updated my nvidia-drivers, now i also can run the unpack/pack example with PBO)
Native Macbook (Intel 950 GMA) supports 100% gl1.2 and about 66-90% features up to gl2.1

Parallels can run the Offscreen Framebuffer-Objects, (more compatible but slower)
VMWare does not ("missing EXT_..") (even if its way faster on other opengl examples)
http://www.codesampler.com/oglsrc/oglsrc_14.htm (Second Example)

But Parallels and VMWare both dont support "VBO". (or PBO "unpack" "pack" examples).

[Petr Schreiber]

Holomind,

thanks for the links.
The OpenGL Extensions Viewer is really very well done!
I like the percentage of version support listed.

I think your 7600 card should support even GL 2.1, I have GF 6150 and got 2.1 support ( and yes, even PBO/FBO).
The database of cards is also very useful, although it does not relflect the latest driver releases yet.

I do not like Intels so much as they do not support GLSL, but for basic stuff they are usable.


Thanks,
Petr

[holomind]

Hi, i found a way to software-emulate opengl1.5 (or maybe even higher).
its not really fast, but for testing a new feature it should be sufficient.

Mesa3d is a Software-Implementation of Opengl. MesaWinBinaries6.0 supports opengl1.5
(there is also a newer Mesa7.0 which even supports gl2.1 (software-emulation=slow) but
there is no binary to find. but you have the sources to compile them with VC6/7/8, if somebody
knows how to do it).
http://sourceforge.net/project/showfiles.php?group_id=3&package_id=27802 (binary, old but working)
http://www.mesa3d.org/README.WIN32 (instruction for compiling)
http://www.mesa3d.org/relnotes-7.0.html (opengl 2.1 api)
http://www.mesa3d.org/

there is no installer but in the folder lib of the mesawinbinaries.zip are 4 dlls which you can put in your project-folder and it will use the software-"opengl32.dll" instead of the one from your hardware-driver. (even opengl-extension-viewer works with it and can use functions from gl1.5)

using this software-emulation will not be fast, but its a good for compatibility. this way i can develop under vmware with software gl1.5 (or gl2.1 when somebody can compile it for win32) . and then use it on a real PC with full speed later. i didnt see a way to mix hardware-accellerated opengl32.dll with the software-version. 100%software or 100%hardware.

greetings
holomind.

[Petr Schreiber]

Hi,

Mesa is very interesting project, hope they will put out v7 DLLs soon.
Compilation from sources is stylish, but... :)

With v6 Mesa DLLs I get a GPF quite frequently ( especially for more complex programs ), but as some of the v7 releases are marked as bugfixes I presume new version could be more stable.



Petr