3D ind 2D, further study by Mark Brown

[mabrown]

Hi Everyone,

This is my first post to the forum. I hope you guys won't mind me reviving a 2 year old thread.

I came across Mike's amazing example via Petr Schreiber following an email I sent him asking for permission to use one of his excellent particle samples. My main area of interest is 3D imagery (I like to model and render ships, don't know why...) and have long wondered about the possibility of combining real time 3D via OpenGL or DX or some 3D engine, with pre-rendered imagery (for smoke, spray and the like). Unfortunately I didn't have the time then to pursue the idea (still don't really) and I suspect Petr didn't really have the time either.

I have continued to ponder the problem though. I am hopeless with Blender. I also don't find its default renderer of good enough quality for what I want to achieve (although it seems to be a better than average general purpose renderer). I use MoI3D to model and Kerkythea to render my images and so, today has been spent trying to get this to work with a KT render. I have been partially successful but there are many problems to solve. I'm also not a thinBasic programmer. Most of my hobby programming time has been spent with the Euphoria programming language.

I placed a cube in Blender at the absolute origin and exported it as obj to KT. I converted the cube to m15 for tbgl using Petr's utility. I then needed to export the camera from Blender to KT and used a Python script from the KT forum to do that. Attached are a couple of screen shots which show either a slight camera misalignment or a scaling issue. This is with the simplest of transformations. I don't fully understand the KT file format where the camera is concerned as the numbers don't seem to have a direct relationship to the radians which i believe Blender uses internally?

Any thoughts you guys may have on this would be greatly appreciated. I think I'm going to need some input to work these transformations out.

Mark

[Petr Schreiber]

Hi Mark,

welcome to the forums :)!

Do you think you could post the code and blend file here?
Do you use Blender 2.49 or did you jumped the 2.5 beta 0 train already?

Blender 2.49 default renderer is not bad, but not super.
Friend of mine uses YafRay and it is whole different class.

I will have a look at your code and file once you post it here (ideally as ZIP, using "Additional Options/Attach" when editing the post).

The problem could be swapped axes or incorrect field of view.


Petr

P.S. The credit for Blender export utility goes to Michael Hartlef. You can get the latest exporter here (http://community.thinbasic.com/index.php?topic=593.msg3116#msg3116).
This way you can export also the normals directly from Blender, without need to calculate them on each load.

For importing from OBJ, please use also updated OBJ2M15 (http://community.thinbasic.com/index.php?topic=1418.msg10052#msg10052). Again it provides normal import.

For comfortable using, I recommend latest TB Beta (http://community.thinbasic.com/index.php?topic=2899.msg21972#msg21972).

[Michael Hartlef]

Hi Mark, welcome to the forum.

Yes, like Petr said, it would be nice if you could attached your files so we can have a look at it.

[mabrown]

Great to speak with you again Petr and nice to meet you Michael.

I've attached what I have been messing around with which is simply Mike's code with some changed camera transformations and some remarks to help me remember what I've done. I've also included the KT stuff and blend file.

I solved the scale problem and created a new one! The image I rendered in KT was 512x512 but the camera exported from Blender was 800x600. Now however, it appears that I have it the other way around. I now have a mask which is larger than the rendered cube! It *should* all be 512x512. What this proves to me is that all this needs to be automated. I need to be able to set up my scene in KT and read the required camera transformations from the KT file. I really want to get Blender out of the picture as my workflow is MoI3D (obj) to KT where I apply materials and render.


Do you use Blender 2.49 or did you jumped the 2.5 beta 0 train already?

Still (mostly avoiding!) using 2.49. Have you tried 2.5?


Friend of mine uses YafRay and it is whole different class.

Yes, YafRay is definitely nice. What sets KT apart for me though is the scene & material editor it comes with which makes it independent of the modeler you use. The disadvantage of that is that there are no documented command line options making integration with other software a bit difficult.


P.S. The credit for Blender export utility goes to Michael Hartlef. You can get the latest exporter here.
This way you can export also the normals directly from Blender, without need to calculate them on each load.

Actually I used OBJ2M15 for this but have downloaded Michael's exporter to give it a try.

Mark

[Michael Hartlef]

Ok, that line in the XML file:

Parameter Name="Focal Length" Type="Real" Value="1.095840"

In TBGL a focal lenght of 45 has to be used. Does this value 1.095840 represent the 45 in KT? I see in your Blend file that the camera lens setting is 35 and that 'D' switch is not active. This is different from my initial example.

About the differentx between the XRot 50 and 40 in TBGL. I think the magic number here is 90. The rotation on the X-Axxis in Blender is different from TBGL. I think by 90 degrees.

[Michael Hartlef]

Ok, I think I mixed up the terms. I think TBGL has a FOV with 45 degrees. So you have to calculate this to the camera lens length in KT.

[Petr Schreiber]

After few experiments,

it seems it is definitely field of view problem.
Luckily, TBGL supports:


TBGL_EntitySetFOV(%MY_SCENE, %ENT_CAMERA1, 98)


That 98° I experimentally set to match the box. Now we just need to find out focal length and field of view relationship, and we can calculate it precisely.

The TBGL default FOV is 90° (which is 45° "FOV" in OpenGL as Michael noted), and it means the field of view from up to down (vertical).


Petr

[Petr Schreiber]

Hmm, and I even found out how to find it.

When you open that Blend file, you can go to "Editing (F9)", and you see the "Lens" property.
Value is 35 ... when you hit the "D" button, you see the FOV in degrees, which is 49.13.

And ... tada ... 49.13 * 2 = 98.26, which is almost perfectly the experimental value :)

So as it seems, you just need to pick the degrees property (which behaves like in OpenGL in Blender), and multiply it by two to get proper TBGL FOV of 98.26:


TBGL_EntitySetFOV(%MY_SCENE, %ENT_CAMERA1, 49.13 * 2)


The image render, if you use latest TB Beta, can be modified to:


sub RenderBack2D( centerX as long, centerY as long, fsize as long )
if fsize = 0 then exit sub

TBGL_PushStateProtect %TBGL_LIGHTING ' -- Light will not affect following
TBGL_Color 255,255,255 ' -- white base color
TBGL_PushMatrix
TBGL_UseDepth 0
TBGL_Translate centerX, centerY, 0
TBGL_Scale fsize, fsize, 1
TBGL_CallList %DLS_backimg
TBGL_UseDepth 1
TBGL_PopMatrix
TBGL_PopStateProtect ' -- If light was previously on, we will turn it on again

end sub


ThinBASIC now supports setup of ambient light and automatic constant enumeration as well, so I did few minor changes to the example.

Petr

[mabrown]

Thanks so much guys. I had read about the FOV situation in Mike's original thread and that some additional functionality was to be added to TBGL to address it. Thanks Petr for the fix and the explanation, you are a very talented programmer.

Might I also add that the lighting changes look sweet! The render and the ball look like they were rendered together. Fantastic! The current example has made me realise that I must also consider lighting angles, intensities, etc for this. I'm going to need to examine the KT xml code for that too. I think the possibilties for post-process render effects using this example are very exciting. My ideas for this are already racing far ahead of my abilities.

I need to study all this and I might be a while doing so. I also need to learn thinBasic (which is fortunately nice and easy to read). I will doubtless return with a pile of questions (and hopefully the beginnings of a simple post-process effects app).

One final question for tonight... what is my best option for adding gui elements to this? I seem to remember that Michael did some work on an example which, when I tried it at the time, was a little bit cranky on my old computer. Has there been any progress on TBGL gui elements?


Mark

[Petr Schreiber]

Hi Mark,

the GUI commands are currently not present in TBGL, but you can go other way - pick the elements of world of 3D graphics and classic UI (buttons, textboxes, listboxes, listviews, grid, ...).

In recent ThinBASIC versions, you can have one rendering surface in dialog. We call it "TBGL canvas".

I attach code for you, you will see, that besides the dialog integration, nothing has changed, and you can code as usual.

For more info on dialog canvas and callback mechanism, you can read article in ThinBASIC Journal #2 (http://community.thinbasic.com/index.php?topic=2310.msg17554#msg17554).

[Michael Hartlef]

Adding to that you could use TBGL Sprites and the ability to react on the MouseOver events.

[mabrown]

Hi Petr and Mike, many thanks for the input and particularly for the modified example. Combined with some code I found elsewhere on the forum for adding menus, this should be all I will need.

I've now come up hard against my lack of maths skills. The camera transformations in the KT xml are represented, I think, as some sort of matrix (I was really bad at this stuff in high school). I also *think* that they are represented similarly (the same?) by Blender but the Blender UI shows us these transformations in a human readable form, loc x,yx, rot x,y,x etc. This is convenient for this example because those numbers can be plugged into the TBGL code.

The line in the KT xml, as created by that Blender to KT python script I used looks like this:

<Parameter Name="Frame" Type="Transform" Value="1.000000 0.000000 0.000000 0.000000 0.000000 -0.642788 0.766044 -13.000000 0.000000 -0.766044 -0.642788 10.000000"/>

Numbers 4, 8 & 12 appear to be camera location. The remainder I'm assuming is a 3x3 rotation matrix?

The python code which creates this is I think:

m = camera_object.matrixWorld

file.write('<Parameter Name="Frame" Type="Transform" Value="%.6f %.6f %.6f %.6f %.6f %.6f %.6f %.6f %.6f %.6f %.6f %.6f"/>\n' %\ (m[0][0],-m[1][0],-m[2][0],m[3][0], m[0][1],-m[1][1],-m[2][1],m[3][1], m[0][2],-m[1][2],-m[2][2],m[3][2]))

It looks to me like there is a direct relationship between the Blender and KT matrix which is simply being written as is to the KT xml. Tomorrow I will search for some detail on the Blender camera matrix and see what I can find out about it. I'm then going to need to find a way to extract rotation from the matrix which I'm guessing isn't simple.

Why was I born with such a maths challenged brain?

Any thoughts much appreciated.

Mark

[Petr Schreiber]

Hi Mark,

there is no need to be afraid of math (unless it is evening before math exam).

The matrix you got here really is major part of 4x4 transformation matrix.

Here little visualisation:


1,000,000,00


0,000,64-0,77


0,000,76-0,64


0,00-13,0010,00



The vectors in RGB are most probably vectors of X, Y and Z axis. They default to (1,0,0), (0,1,0) and (0,0,1) when object is not rotated, but once you turn it, things change. You can see the X axis is still (1,0,0), so the turn was done around it.

TBGL provides TBGL_EntitySetXYAxis, TBGL_EntitySetXZAxis and TBGL_EntitySetYZAxis, so you can supply just 2 of 3 vectors, and the rotation is adjusted.

I can see the exporter does some sign manipulations, maybe that will have to be slightly arranged as well.

Blender also swaps YZ so, for TBGL, the matrix becomes:


1,000,000,00


0,000,77-0,64


0,000,64-0,76


0,0010,00-13,00



So you can easily setup the camera without need for TBGL_EntitySetRot as:


TBGL_EntitySetPos(%MY_SCENE, %ENT_CAMERA1, 0, 10, -13)

TBGL_EntitySetXYAxis(%MY_SCENE, %ENT_CAMERA1, 1.000000, 0.000000, 0.000000, _ ' X vector
0.000000, 0.766044, 0.642788) ' Y vector (Z vector calculated automagically)

[mabrown]

Hi Petr,

Thanks so much for this. I reckon this is everything I need to make a serious go at this.

Looking forward to getting home to try this out. I soon shouldn't need to worry about using Blender.

Thanks again to you and to Michael for the excellent examples.

Mark

[mabrown]

Finally got some time to play with this again today but after several hours I'm well and truly stuck.

I have tried to add a little to the example in order for it to load separately from a menu the pre-rendered image, the masking object and the camera from the kerkythea file which created the render. I have attached a zip of the program as it stands. The masking object is clearly not in the same spot as the pre-rendered object but is close. Could this be that problem Michael encountered where "...the magic number here is 90. The rotation on the X-Axis in Blender is different from TBGL. I think by 90 degrees" except this time expressed in a matrix in radians (I'm way out of my depth here!). If it is, how do I subtract 10 degrees from this?

Petr, I need use of your maths brain :?

Mark

[Michael Hartlef]

Just for the books... I think you get the 10 degrees from the 50 in blender and 40 in TBGL. But it is a not a difference of 10 but the sum of it. 90 Degrees.

[mabrown]

Hi Michael,

Thanks for that clarification. I should probably also have added that for the original camera transformation from the earlier example, the camera alignment is fine. The camera transformation in kerkythea for the working situation is:

"1.000000 0.000000 0.000000 0.000000 0.000000 -0.642788 0.766044 -13.000000 0.000000 -0.766044 -0.642788 10.000000 "

and for the non-working situation as shown in my previous post:

"-0.621298 0.10577 -0.776401 39.5321 0.783573 0.0838653 -0.61561 32.3305 2.8431e-008 -0.990846 -0.134985 6.54387 "

So, it seems to me that looking at the two transformations, in the working example the camera has not been rotated around its X-axis? Perhaps this relates to what you said earlier: "The rotation on the X-Axis in Blender is different from TBGL. I think by 90 degrees"?

I still also need to work out how the kerkythea focal length and "film height" relate to tbgl FOV. I'm guessing that could be wrong in this example too.

[Petr Schreiber]

Hi Mark,

I think the signs should be swapped as well, but you do it only for some components.
Also - the cube is 10x10x10 and you place sphere at 2.5, 2.5, 2.5.

I will check tomorrow, with more clear head.


Petr

[mabrown]

Hi Petr,

Merry Christmas (and to you too Michael).

There is no rush on this at all so please take as much time as you want/need before looking at this.

The sign manipulations were arrived at by studying the Blender->Kerkythea exporter. They could easily be wrong.

The 2.5, 2.5, 2.5 sphere location was arrived at experimentally. It is curious and I don't really understand why it is visible at all. Shouldn't it be inside the cube and therefore completely masked by it?

Mark

[Petr Schreiber]

Hi Mark,

this seems to be quite tricky.
Is there any way to display camera transformation matrix directly in Kerkythea editor?

What is very odd is that Kerkythea matrix of camera seems to point with Y vector down, and points with Z vector forward, while the coordinate system is Z - up oriented. To make it even funnier, in XML the Y vector is written in place of Y vector with some of the signs reversed.

I just tried a lot of combinations, but still not getting the right results. Some kind of more technical documentation to Kerkythea would be handy, but I didn't found anything suitable yet.

Petr

[mabrown]

Hi Petr,

I feel a little better now about the hours I've spent trying to get this to work :)

In the KT scene editor you can get to the camera transformation via menu->settings->advanced. This brings up a tree view. If you expand Scenes->My scene->Cameras you'll see a list of cameras. If no cameras have been added to the scene all you will see is the "current view". Selecting "current view" or a camera will show settings for a camera. "Frame" is the camera transformation. The information available is the same as the xml (as far as I can tell) and probably won't tell us much unless perhaps we experiment with those values to see what the camera does?

Documentation for KT seems hard to come by but I'll check the KT site & forum and see what I can come up with.

Mark

[mabrown]

I did a search of the KT forum and found nothing useful about how camera transformation is handled. The only posts of interest related to FOV:

http://www.kerkythea.net/phpBB2/viewtopic.php?p=1623
http://www.kerkythea.net/phpBB2/viewtopic.php?p=1670

Edit: Petr, there is some sample C++ code on the KT forum for a KT xml exporter. It isn't GPL'd or similar so I won't post it here but I will send it to you by PM.

Mark

[Petr Schreiber]

Hi Mark,

thanks for the PM!

This is what I researched before reading your posts:


Local px, py, pz As Double
Local Xx, Xy, Xz As Double
Local Yx, Yy, Yz As Double
Local Zx, Zy, Zz As Double

px = camera_transform_matrix(4)
py = camera_transform_matrix(8)
pz = camera_transform_matrix(12)

xx = camera_transform_matrix(1)
xy = camera_transform_matrix(5)
xz = camera_transform_matrix(9)

yx = camera_transform_matrix(2)
yy = camera_transform_matrix(6)
yz = camera_transform_matrix(10)

zx = camera_transform_matrix(3)
zy = camera_transform_matrix(7)
zz = camera_transform_matrix(11)

TBGL_EntitySetPos(%MY_SCENE, %ENT_CAMERA1, px, py, pz)
TBGL_EntitySetXYAxis(%MY_SCENE, %ENT_CAMERA1, _
Xx, Xy, Xz, _ ' X vector
yx, yy, yz) ' Y vector (Z vector calculated automagically)

' -- Now we have perfectly (?) replicated the setup, now we should turn it upside down, as TBGL has up vector ... up
TBGL_EntityTurn(%MY_SCENE, %ENT_CAMERA1,0,0,180)

... there still seems to be problem with fov... will investigate further. Now I must go offline.
Quite a challenge!


Petr

[mabrown]

Awesome work Petr! I just plugged your code in and it works great!!

I fiddled with the FOV until I got a value which is very close.

I will be very interested to see what the final numbers for FOV are as I'm finding it hard to see the relationship there too.

[Petr Schreiber]

Got it.

I can tell you this was blood, sweat and tears (as long as Googling can produce such a effects).

The equation is:
FieldOfView = 2 * RadToDeg(2 * Atn(FilmHeight/2/FocalLength))

So just plug the following in your code:


Uses "Math"

...

Local FocalLength As Double = Grab$(camera, "<Parameter Name=""Focal Length (mm)"" Type=""Real"" Value=""", """/>")
Local FilmHeight As Double = Grab$(camera, "<Parameter Name=""Film Height (mm)"" Type=""Real"" Value=""", """/>")
Local FieldOfView As Double = 2 * RadToDeg(2 * Atn(FilmHeight/2/FocalLength))
TBGL_EntitySetFOV(%MY_SCENE, %ENT_CAMERA1, FieldOfView)



I found the solution on following website: http://www.worldserver.com/turk/quicktimevr/calculators.html

To avoid confusion, the code from the last posts simply imports the data from Kerkythea 1:1, that means it has Z vector as up. This is why the move up/down goes so odd, as it slides along Y axis, which in Kerkythea lies on the ground.

You could swap the components to preserve Y as up vector, but I think it would only bring more troubles in future.

I think now you have all what you need for combining renders and TBGL, I am looking forward to your creations!


Petr

[Michael Hartlef]

Petr,

I love your determination on things like that. Of course, you allready have a great knowledge to resolve a problem like this. You are definately Mr. TBGL! :eusaclap:

[Petr Schreiber]

Thanks Mike,

when I reverse engineered the equation, it even makes sense!
Please see the image.

But ... I also discovered I made one mistake in entity camera design - that is, you need to pass the FOV*2.
The reason for this was bad illustration in one of the tutorials I learned from, resulting in the fact I added the redundant multiplication by two :oops: Quite a faux pas. I will update the help file with the information about this ... ahem ... feature.


Petr

[mabrown]

Hi Petr,

Fantastic. I would *never* have been able to work this out. That equation (to me) looks nothing like the equations tried on the KT forum. A couple of people on the KT forum appear to have had trouble with compositing and FOV with KT/Blender. I don't recall seeing mention of "film height" on the KT forum at all. They needed you on it over there Petr :D

Undoubtedly Mr TBGL!

Happy New Year Gentlemen :occasion:

[mabrown]

A little test before I retire for the night.

The attached image is of a model of HMAS Yarra II, rendered in KT @ 1024 * 768. Polygons about 60 or 70k.

The little sphere is sitting embedded in the back of the funnel and looks perfectly aligned. Thankyou again Petr.

Tomorrow, if I can find time, I'll add some code to enable snapping of the camera to standard views (top, side, etc) so that objects like particle emitters can be added to the scene. Will also need a button to restore camera to KT render position. Then it's time to play with particles.

I think this just might work :)

Mark

[Petr Schreiber]

Hi Mark,

thanks for the picture. The ship looks great!

Please note you can specify multiple cameras in TBGL, just use TBGL_EntitySetUse to choose which one you want to set currently active.


Happy new year :),
Petr

[mabrown]

Thanks Petr.

This has started the new year on a high note for me so I am most grateful for the effort you have put into this. Initially I want to try to recreate some of the functionality of a brilliant little trueSpace plugin called Primal Particles from a company called Primitive Itch. After moving from trueSpace to MoI3D for modeling and KT for rendering, that plugin is one thing I really miss.

Multiple cameras are definitely on my list. Lights also. I'd like a tree view if possible to manage that. I think you said that only one canvas (is that what I should call it?) is available for gui elements? I'm guessing that area could hold tree view and other buttons etc? I'm thinking expandable areas there like MoI has for scene management, tools etc.

I'm going to raid your particle code tomorrow...

Mark

[Petr Schreiber]

Hi Mark,

yes, only one canvas, but you create any number of viewports inside of it. I suspect MoI has it done this way (although they use Direct3D). So you can have multiple areas with totally independent views, cameras and content. This approach is very good from performance point of view - no need to switch between multiple OpenGL contexts.

TreeView is supported control element of the UI module, so you can place it on the dialog without problem.


Petr

[Michael Hartlef]

This has started the new year on a high note for me so I am most grateful for the effort you have put into this. Initially I want to try to recreate some of the functionality of a brilliant little trueSpace plugin called Primal Particles from a company called Primitive Itch. After moving from trueSpace to MoI3D for modeling and KT for rendering, that plugin is one thing I really miss.


Ahhh.... Truespace.... and Primal Partical. Have thme both too. Truespace was my first 3D app. Since then I feel most comfortable with 3D apps that have icons for the buttons. :)
I am really curious what you come up with.

[mabrown]

Since then I feel most comfortable with 3D apps that have icons for the buttons. :)


Hi Michael,

I'm still pretty fond of trueSpace. I know that many people found the UI a chore but I never had a problem with the icons for buttons. The trick was to set it up how you wanted it. It was (is?) highly configurable. It also had some excellent plugins written for it. Given that M$ is giving away for free what will probably be the last version of trueSpace, I reckon it is definitely worth adding to one's toolkit.

You will have a good idea of where I'm going with the particles side of this. I really hope I can do your code justice.

Mark