Carl Friedrich: Second Coming to Monte Carlo

[mike lobanovsky]

Hi Rob,

Here's an interpretation of your code to prove TB and TBGL can also be interfaced easily with other language implementations more appropriate for computationally intensive tasks than a purely interpretative environment of newLisp. The data interchange principles are quite common and the results are stored directly into thinBasic's native data array. I'm not at all sure newLisp is capable of doing just that.

Enjoy!

[RobbeK]

Wow , great Mike !! super ...

Did not know FBSL had a C language layer !
NewLisp -- I'll look what I can find , intercepting the console output stream (as I did) is not only iffy (what length ?) but also clumsy.

For TB there's Oxygen of course (attached).

I must have a further and deeper look -- I had to shift the curve a little , meaning TB array(1) = O2 array(0) -- when writing code, one has to be aware of such things , evil things may happen .....


best Rob (thanks for sharing this !! what C is used, btw ? )

[mike lobanovsky]

Hello Rob,


Wow , great Mike !! super ...

Thanks a lot, I knew you would like it. :)


Did not know FBSL had a C language layer ! ... For TB there's Oxygen of course (attached).

Alongside built-in Dynamic C (DynC), it also has a built-in Intel-style assembly layer called Dynamic Assembler (DynAsm). The both layers are JIT compilers similar to Oxygen which I'm naturally very well aware of. But the challenge was to interface TB/TBGL with an "alien" language. :)


For TB there's Oxygen of course (attached).

Thanks for sharing the code. It's only natural that O2's integration with thinBasic is much deeper. I was watching the process since its very early stages and I sincerely admired what I saw. :)


... meaning TB array(1) = O2 array(0) -- when writing code, one has to be aware of such things ...

Exactly. But there's nothing prevents you considering O2(1) = TB(1) just skipping the O2(0) extra. Once you get used to either of the two approaches, whichever is better for your personal perception, the mosaic falls into place perfectly. :)


... what C is used, btw ? ...

DynAsm was my own baby from ground up. Now I'm not that young any more as I used to be so I decided DynC should not be reinvented completely from scratch. FBSL's Dynamic C layer is a stripped-down x86-only Windows-only loose-type-check no-GAS derivative of GNU LGPL Tiny C Compiler v0.9.25 library by Fabrice Bellard. My choice was mainly due to the terrific compilation speed and general compactness of this C compiler.

Similar to TCC, DynC is GNU LGPL and this very portion of FBSL sources will be released for public access together with FBSL v3.5 Final. Now its code is still under beta test and subject to further modification. The rest of FBSL will stay closed-source as it's always been, similar to thinBasic.

Thanks for your interest in FBSL but I guess this is not the proper place for any further propaganda. I'll be mentioning FBSL here only in relation to thinBasic which is naturally an indisputable king, and a generous host too, on this forum. :)

Regards,

[RobbeK]

Ah !!
Your name rings a bell now - I remember it from the console window of FBSL , second line about ASM .... congrats !!
TinyC , yes , also think to remember it .. used by X11 Basic (IIRC - about one year ago - and after more than decade of doing no coding at all - I decided to pick it up again (but not professional this time - those days are over ), I picked up the things from when I left (under a DosBox ;-).
I worked mainly with Lisp, but I don't think there are good free ones (some are out-of-proportion expensive) , and I do not know Lisp is up-to-date with the latest technologies (I mean for PC at home).
The search for a good BASIC alternative brought me here -- I have to add that while searching (I think I looked at more than 10 basic languages) that at the same time as the web became world wide - also the flamers and trolls came out their caves ;-) --- on some fora (forums) it is impossible to get some fundamental information ... It's much more difficult to get a fair, correct description than to download an application ! (I was used to the opposite ). So many excellent tools available now (and so less respect for the people who developed it -- that's my honest and true opinion).
OK enough ... attached the same compare (added TB nothing) in a bundle -- going to read the NewLisp documentation now, c I can find something .

best, Rob
'

[mike lobanovsky]

Thanks for this little benchmark Rob!

Regretfully, FBSL cannot compete with thinBasic in speed regarding deeply nested interpreted loops. A typical timing for your Sub calc() run in FBSL on my PC is

4375 msec

Press any key to continue...
give or take some 32 msec, while thinBasic fits into 2000 msec most of the time. The reason for this is that all FBSL variables are basically Variants strong-typed only when absolutely necessary for compatibility with 3rd-party API's, DynAsm, and DynC. The data type of loop iterators i, j, k, l, m may be changed freely within the loops at the user's option, so the loop frame code can't be byte-coded straightforwardly. The frame performs continuous type-checking and is ready to react accordingly if a change is detected. This was one of the reasons why FBSL eventually acquired its DynAsm and DynC layers to compensate for this inevitable deficiency in speed-critical tasks.

As for benchmarking JIT compilers such as O2, DynAsm, DynC, LuaJIT, Idle and others, I don't see so much sense in it as all of them would yield approximately equal results. This is because they are all straightforward native code compilers unable to optimise their output either for speed or size. This is an inevitable penalty for their very high compilation speeds. All optimizing static compilers produce native code of much higher quality but are turtle-slow and unsuitable for JIT tasks. For example, it takes GCC v4.3.3 approx 2.5 minutes to compile the 2.4MB of densely written FBSL source code. Do you realize how many weeks of my life I wasted waiting for the compiler to generate the FBSL binaries for just a single trial run? :) The non-optimizing TCC would do it in .2 or .3 seconds but the quality of resultant binaries would be incongruously worse.

Regards,

[RobbeK]

Thanks Mike,

Well regarding slow and fast compile times , "we can't always get what we want " , I think ....
(infact JIT compilers are new to me, but I like the concept very much -- especially the fact they are dynamic addressable/ editable during run-time )

A final remark about that Newlisp, using a method as in you demonstration seems nearly impossible -- there's also a more fundamental reason (which escaped me first) -- the Lisp data structures are built by pointers. (what looks (at first sight) somewhat space and time consuming has some serious advantages -- you can mix whatever you want, whatever data dimension in mind ).
p.e : ( 12 (+ 56 77) ("Thinbasic" "FBSL") ( (a) (b c) (d e (f) ) ) )
I attached an illustration about how it is done ...

Reading the documentation , it all turns out not that bad for practical use - definitions and data stay (that's the same in Lisp ! -- a program is an evaluation of a list ) resident and can be called any time .. I wrote something that can be used as a skeleton (very primitive, just to get the idea )
as an example, a fractal of course ;-) -- but the slow way -- I should have used TBGL

kind regards,
Rob

[mike lobanovsky]

Hi Rob,

Yes, newLisp's flattening of the lists is a mighty thing to build your matrix with. I'll use a similar approach with just a little bit more of FBSL code.

FBSL offers Variant arrays which resemble to some extent VB's Collections in that they can store simultaneously and in any order everything you like from simple literals, variables and other arrays to class instances and COM objects.

As for the static process memory allocation, Fbsl.dll doesn't do it with its built-in C allocator but this can be overcome using the GlobalAlloc() API of kernel32.dll. It allocates memory in the process heap and keeps it there, if not deallocated explicitly, till the process ends. The memory may be used to store shared data as well as FBSL's own code which will thus stay intact until needed again.

And here's of course a matching interpretation of your code. :)

[RobbeK]

Wow (again ;-) ... Mike !

Very fast with buffered graphics !! (attached - and a more complex curve).

Gives me an idea for the Newlisp --- something as :
(1st getting the value of the pointer to Newlisp - expand$ will do the trick)

ReDim tbarray(lisp("(getlen a)")
lit=Str$(VarPtr (tbarray(1)))
lisp(Expand$("(setq pt $lit)")) --


then .. Common Lisp has (pack ... ) to convert flat lists into memory blocks of bytes, floats , double floats , integers etc ....
then (cpymen .address from. .address dest. .amount. )
Have to read NewLisp docu once again ...

always good tips ! thanks

best Rob

[mike lobanovsky]

Rob,

Bravissimo!!! :D

Yep, Common Lisp seems more capable along these lines than newLisp for interfacing with TB processes but FBSL can hardly be bit here. It maps over 2,300 WinAPI functions directly into its own namespace automatically at script load time. The WinAPI function calls are byte-coded hence extremely fast, and any other extra DLL can also be mapped in its entirety with just a single "#DllImports DllNameHere" preprocessor directive.

Thanks very much for this fruitful exchange of ideas!

[RobbeK]

(I mean ticks)

What to say, Mike -- Have to learn FBSL too then ;-) I'm convinced ...

As for the NewLisp :
The pack is only for strings -- more directed to read one result from a DLL and convert it. (It seems NewLisp has more string related functions than the classis Common Lisp ).
So, I first tried to parse a result from NewLisp in TB string -> single array ... however, the transit from NL to TB is slow (see .... nr 2)
The next method seems to work fine (... 3) ; both string formats NL/TB are compatible , so I handed the address and length to TB , it is then parsed into a string array() (countable by item) -- I had to parse the string in NL too, the result is not usuable in TB -- a complete different architecture, however the count is the same and this is used to Redim the array into the exact needed length.
I again use strings now -- to get out of the dual meaning of "0" -- "move" and "angle 0" - the string avoids this. (though it is somewhat slower for drawing, but not too much).
BTW in Lisp this problem does not exist - false is "nil" and not 0 (or -1 or whatever number) -- nil is the nothing - so if conversion fails you get "nil" and not "0".
Things are ready for further work on the GeometricFractals 3D now, I'll use the NewLisp for setting up the model -- Lisp is strong for doing such things.

best Rob

[mike lobanovsky]

Hello Rob,

I like getting my chick counts too whenever I can! :D


... it is somewhat slower for drawing, but not too much ...

I wouldn't however call it "somewhat". Your stringified chicks are twice slower than singles; please see ChicksCount below. String manipulation, whether transparent or explicit, adds noticeable speed penalties in any language.

Any nonzero floating-point value in thinBasic will evaluate to TRUE so you can still use exact zeroes as plot markers while your zero angles may be denoted by relatively small floating-point non-zeroes. This won't affect plotting accuracy but will help preserve the evaluation algo simple.

I admire your newLisp exploration effort but if it can only manipulate strings, then perhaps newLisp isn't the best choice for math calc. OTOH I'm perfectly sure there are lots of other activities where newLisp would be the language of choice.

Here's my matching TB/FBSL interface solution that uses TB array allocation at a given memory address. Hope you'll like it too. :)

[RobbeK]

Hi Mike,

Of course, I like it !! --

And I stand corrected , a result of erroneous thinking ... yes , 0 will do (as long as no one uses this number in the "mother" string ).
On my (vintage laptop) the difference in speed was about -40% (which I found still acceptable because the model will be translated into a TBGL List , and is run-once).

Following your advice parsed into an array of singles ... , speeding up things -- it's good now (In the few tests I did , similar calculations run faster in NewLisp than TB).

(I'm just using the NewLisp because I was used to Lisp -- that's possibly the main reason - have to find out about NewLisp however, it has some Scheme related flavours, i noticed).

Thanks for ur "to the point" help -- u c it's productive ;-)


best Rob

[mike lobanovsky]

Hi Rob,

Thank you for benchmarking the scripts!

-40% is still a huge figure. Imagine that it will be well in excess of 100% in just three iterations, and 1,000%, in just thirty...


... similar calculations run faster in NewLisp than TB ...

Judging by the nature of your benchmarks, I presume what you probably meant was "... in NewLisp than FBSL"? ;) No way Rob, no way... :p Please see the pictures and the attached zip. DynC Parse() benchmarked alone within the FBSL buffer takes exactly 64 msec to complete on my PC. You can verify it yourself adding FBSL MsgBox()'es in there. And it could take yet up to 40% less time if hand-coded in DynAsm.

Don't you feel like rewriting the exact same script using Oxygen to create and parse the matrix into tbarray() to maintain the similarity of the tasks? We could then compare how fast it all runs using thinBasic's native features.

And thanks again for your co-operation!

P.S. Can I ask the moderators to please move this thread into a more appropriate subforum? It doesn't appear to have any direct relation to TBGL though it seems rather enternaining for my colleague and me. Thank you.

[RobbeK]

Hi Mike,

I did some benchmarking compare between TB and NewLisp, not FBSL.

but :

ExecBuffer buffer, Str$(VarPtr(out)) & " " & Str$(VarPtr(outsize)) & " " & Str$(lim)

is about 2x slower than

lisp(LispCode)
lisp(Expand$"(build-matrix $lit)")
out=Peek$(lisp("(address a)"),lisp("(length a)") )
lim=lisp("(length (parse a))")
ReDim tbarray(lim)
Parse ( out , tbarray , " ")

Is it a correct interpretation from the attachments that you get about the same times 187 vs 188 when using DynC parse ?

Benchmarking strings vs numbers is important because all this code is written with the concept of a 3D turtle in mind, hence my worries about val("m")=0 , because for a "space turtle" I have two work with 2 angles and a distance. Here it is very possible one of the angles is zero ... then
converting string to numbers ....
basic : eg " m 200 10 m 0 80 " -> 0, 200 ,10 ,0, 0 ,80 with the dual meaning of 0
lisp " m 200 10 m 0 80 " -> nil 200 10 nil 0 80 unique definitions.

There's a kind of artificial solution of course : angle = mod(angle,360) and giving m a value of something as 9999, then everything is defined in a unique way for the basic language. (all this already starts to smell very suspicious ;-)

All these difficulties only show that there's something wrong with my idea of using the "m" - it should be replaced with something as (0,0) in 3D where (x,y) = (HorizontlAngle,VerticalAngle) and 0 in 2D from the start.

Yes, great idea to use this code for benchmarking tests ? Did you try direct FBSL ? (not calling the dll in TB ?)
I will try something using GAS to see where it leads ...

"-40% is still a huge figure. Imagine that it will be well in excess of 100% in just three iterations,... "
It's the loss in speed from single=Val(item) it's within one loop, it should be constant , not ??
10 mSec -> 14 mSec an p.e. 10 Sec -> 14 Sec ?

best Rob ! Should I try TB / 02 noLisp ?? (Money for nothing and ticks for free !)

[mike lobanovsky]

Hello Rob,


Thanks for keeping up the conversation!


I did some benchmarking compare between TB and NewLisp, not FBSL.

I'm afraid this is not so. Your code from "TB Lisp chicks as singles.tbasic":

...
n=GetTickCount
lisp(Expand$"(build-matrix $lit)")
out=Peek$(lisp("(address a)"),lisp("(length a)") ) ' was the tick eater !!!
lim=lisp("(length (parse a))")
ReDim tbarray(lim)
Parse ( out , tbarray , " ")
calcanddraw()
MsgBox(0,Str$(GetTickCount-n))
...
and "FBSL's Chicks2.tbasic":

...
n = GetTickCount
ExecBuffer buffer, Str$(VarPtr(out)) & " " & Str$(VarPtr(outsize)) & " " & Str$(lim)

CalcAndDraw
MsgBox 0, Str$(GetTickCount - n) & " msec for 'Dim tbarray(size) As Single At ptr'"
...
as they appear in your "FBSL's Chicks2.rar" above in fact benchmark your Lisp calls plus the unavoidable relevant TB manipulations versus an FBSL call in its pure interpretative (BASIC) form which doesn't require additional TB processing. "Alien" language calls in both cases are the true bottlenecks of the technique. The problem is, the task they do is too small compared to the overhead (i.e. time penalty) of setting up each call. This is not like calling a native-code DLL which is very fast. All the calls here, in Lisp and FBSL alike, presuppose parsing your input every time, executing it, and finally presenting the required results in a TB-readable form.

For the results of these benchmarks on my PC please see the snapshots 1 and 2 below. My current test setup is in my signature. Today newLisp performs even worse than yesterday. Probably that's due to a different number or nature of current processes on my computer.


...but :
ExecBuffer buffer, Str$(VarPtr(out)) & " " & Str$(VarPtr(outsize)) & " " & Str$(lim)
is about 2x slower...
It's not so for me Rob, sorry. See pictures 1 and 2 again. :)


Is it a correct interpretation from the attachments that you get about the same times 187 vs 188 when using DynC parse ?
I do not forge my results, ever. That's why I always supply my benchmark screenies and the code for everyone to try and confirm or disprove my results on their computer. My today's results for the entire ExecBuffer call including DynC Parse() is presented in picture 3 below.


... Here it is very possible one of the angles is zero ... then
converting string to numbers ....
To the best of my knowledge, you can get exact floating-point zeroes in any decent language only if you hardcode them in the form "var = 0.0". Otherwise, any math operation with floats on an Intel FPU will introduce a small finite error which will help you differentiate true 0.0 from any other computed quantity.


...basic : eg " m 200 10 m 0 80 " -> 0, 200 ,10 ,0, 0 ,80 with the dual meaning of 0
lisp " m 200 10 m 0 80 " -> nil 200 10 nil 0 80 unique definitions.
Regarding what I said above, why don't you try and change your meaningful 0's (not "m" markers as they are treated by "If t Then"!) in this matrix to, say, 0.000001 for Singles which won't affect your plotting in any way but will help you preserve the simplicity of If/Then evaluation?


It's the loss in speed from single=Val(item) it's within one loop, it should be constant , not ??
10 mSec -> 14 mSec an p.e. 10 Sec -> 14 Sec ?
Exactly. If there's any way to avoid calling interpreted functions in a loop, you should eliminate such calls by all means. That's why I used DynC which helps implement it directly in native code. I guess using JIT Oxygen will have an exact same effect.


Should I try TB / 02 noLisp ?? (Money for nothing and ticks for free !)
:D Dire Straights were also my favorites some time in the past. Yes, please do! :)


Regards,

[RobbeK]

Hi Mike,

1) NewLisp gets very slow when calculations are at order :-(
tried :
==================================
(setq a '(m 85 m -85 m -85 m 85 m))
(setq b a)
(setq pi 3.141592654)


(define (build-matrix x)
(dotimes (i x)
(setq a (flat (replace 'm a b)))))


(define (cartesian)
(setq cart '())
(setq hk 0 x1 0 x2 0 y1 0 y2 0 unit 1)
(dotimes (i (length a))
(if (number? (a i) )
(inc hk (a i))
(begin
(setq x2 (add x1 (mul unit (cos (div (mul pi hk) 180)))))
(setq y2 (add y1 (mul unit (sin (div (mul pi hk) 180)))))
(push x2 cart) (push y2 cart)
(setq x1 x2 y1 y2))))
(setq cart (reverse cart)))
========================================================

sigh .. really disappointing ... the list processing related things run at good speed (but that's all)
In my days, if Lisp compiled stuff reached 1/2 speed against a good C compiler exec the result was acceptable, but this .. 1/5 or less .. (maybe Lisp as a scripting language is difficult .. don't know .... )

2) attached the TB with a drop of O2 - TB parsing string -> array seems good .
If you have time (and mood) can you benchmark this ? (don't trust mine anymore -- i mean for a general public - we have different results (and I'm not a forgerer either ;-) )

(really disappointed about NewLisp's speed - I expected a lot ... )
Ah, that pinch of O2 will not do more than 10% (estimating)

best Rob

[jack]

hello RobbeK, I tried unsuccessfully to run your newLisp script, anyway, interesting experiments that you and mike lobanovsky are doing, was wondering if you had considered ecl?
ecl is not as straightforward to embed as newLisp though.

[mike lobanovsky]

Hello jack,

Nice to meet a Vulcan at this time on a Dagoban weekend at this forgotten planet! :D

Can it be you didn't download newlisp.dll to run Rob's samples? You can get one from the "FBSL's Chicks2.rar" archive immediately above. It should be placed in the same directory as Rob's scripts. Likewise, Fbsl.dll from the same archive should be placed there to run any of mine. We've been the only parties to discuss this topic for quite some time so we've got our DLL's installed already.

Sorry for any inconvenience,

[mike lobanovsky]

Rob,

Sorry but your script doesn't produce any meaningful plot here so I need some time to analyze it.

[mike lobanovsky]

Rob,

1. There's nothing to plot or benchmark in your script as there's no matrix built or processed in there. Do you want me to write the relevant O2 code?

2.
don't trust mine anymore -- i mean for a general public - we have different results (and I'm not a forgerer either ;-))
No problem whatsoever Rob, just post your sreenshots in exchange whenever you can. I don't insist that what I'm getting on my PC is an ultimate truth because the results may vary on different platforms, and they may vary considerably. My only intention is the exchange of information, nothing else. The more we see, the more we know. :)

Regards,

[RobbeK]

Hi Jack, Mike

@ Jack :
Oops, this was just a piece of code written using the NewLisp IDE - but the execution speed was that low I abandoned it.
(I think the other examples (calling NewLisp) have the dll included -ECL , no -- thanks , I'll have a look).

@ Mike :
It probably needs 1.9.11 (or the most recent O2 module) -- I attached the bundle.

best

Rob

[mike lobanovsky]

Oops Rob,

The bundle runs just fine. My bad. :oops:

I guess this is the right time for me to install v1.9.11. I'll do it right away but it's already past two in the morning at my parts so I guess I'll have to leave it at that for tonight.

See you tomorrow Rob!

[mike lobanovsky]

G'morning,

So I've installed my v1.9.11.0 successfully and I wish to thank Eros, Charles and Peter for this update.

Your solution seems to be reasonable and rewarding, speed-wise. Please see picture 1 for your benchmark on my x64 Windows 7 I'm writing this message from. Yet my old FBSL benchmark (see picture 2) of DynC Parse() where the cartesian calc is still done in pure thinBasic isn't all that bad either, is it? And all this without direct access to a single thinBasic variable in the script! I'm satisfied with the performance results of this tiny Fbsl.dll which carries in itself the entire FBSL v3.5 with its three constituent languages, two JIT compilers, two preprocessors, extensive vector and matrix math library, and of course a whole bunch of other goodies developed and added to it in recent years.

Closer integration to speed up things to be on a par with Oxygen in the thinBasic environment might have been achieved using thinBasic's SDK. Who knows, perhaps one day... :comp5:

Anyway, thank you Rob also for an instructive journey into the boondocks of Lisp. Obviously thinBasic is sufficient to program most intricate projects without resourting to any "alien" solutions. On the other hand, newLisp et al. developers might've had a better look at interfacing their creations with speedy things like Oxygen, Dynamic Assembler, and Dynamic C.

Happy coding in thinBasic and O2! :drink:

[RobbeK]

Hi Mike,

Yes, indeed an interesting journey -- the NewLisp (well Lisp - old love never dies , probably ).
I'll take a glass of "Белгородский Герб" now to conclude things -- I more or less know now what I wanted to know ..

BTW the string transfer NewLisp -> TB takes about as much time as all the calculations and producing the image on the canvas together.
(so just in case someone wants to use it in the future
don't a$= Lisp("object") where object is a string , but a$=PEEK$(lisp("(address object)",lisp("(length object)") )
)

best, and thanks again !

Rob

[RobbeK]

Hi all,

Trying some other ways now ... DLL is compiled with Emergence Basic , primitive graphics this time.
The fractal pattern is still set up by TB. (it takes almost no code)
(source of the DLL attached at the bottom of the TB script)
best

Rob

[mike lobanovsky]

Hello Rob,

Nice to see you here again!. Yes, it works but it's not scripting. That's calling third-party API's directly, much like what peter is doing with his A.dll. You could probably write and compile your DLL in OxygenBasic too while I'd be compiling mine with some good optimizing static C compiler to see who's faster in the end.

But that would be another story... :)

Cheers,

[RobbeK]

Hi Mike,

Yep, the O2 dll also works fine, except I get an error on exit. (no idea, maybe the closing of the window -- no idea , I'll ask Charles (unless you know the answer ?! ).
Seems very speedy ....

best Rob

[mike lobanovsky]

Rob,

I noticed that in some other thread where peter tests his A.dll's written in OxygenBasic. Seems like the problem persits for some time. I think this might be a bug somewhere in the way an Oxygen DLL is compiled. DLL's are more intricate in their structure than a straight-forward PE executable. Static compilation is a relatively new feature in OxygenBasic so some latent glitches are likely to appear from time to time yet. Nonetheless I'm sure Charles will look into the matter if you address him with this issue here or on their forum.

I've also noticed that some of your scripts in this thread hang as soon as the StarT button is clicked for the second. Yet this glitch is not so important here as these are only short test samples rather than applications.

Regards,

[RobbeK]

Hi Mike,

Yes, that is my mistake .. I did not reset the strings when hitting StarT again (intended as run once, I should have eleminated that possibility )
I the meantime, working a little on the 2D stuff.
(attached)
Added "base" : can be line or any polygon (open or closed) now.
# it is the number of iterations
# An the number of changes during the Anim A action.
Step the increase of angle/step.
Added Notepad to save, copy, paste , comment some things in case.

Sometimes , the program hangs for a while till the end is reached -- (?).
Will be recoded for TBGL soon.

(the interesting part is |Anim A| - hope it will reach 3D --- )

best Rob