Cdecl (-> dll)

[RobbeK]

Hi,

Charles (any one else) ,

Is it possible to set up CDECL call convensions from DLLs written O2 (ThinBasic + O2.dll or Oxygen Basic ) ?

(Working on some DLLs for Lisp (all based on alien Win32 products (Linux) ,
Using the Aurora compiler as an Ersatz (but I think O2 produces faster code) .. I have to do something as

export foo
declare cdecl foo( x as "type" , .. , z as "type" .. ) , "type"
function foo ( ..... ) , "type"
(body)
return ...
end function

but even then single-floats givs a NaN , doubles work ... (Lisp is very secure on number typesn(like conversion is not done automatically )

t.i.a. Rob
................

[Charles Pegge]

Hi Rob,

For individual functions:

function foo cdecl (...) as double, export
...
end function



For groups of functions

an extern block can be used:


extern cdecl export

function foo(...) as double
...
end function

function bah(...) as double
...
end function

end extern

[RobbeK]

Thanks Charles !

[Charles Pegge]

Rob,

I remember we had a disussion about index bases. In recent releases of Oxygen, I added support for negative index bases. They could be useful for carrying array-header information in the negative range, or you may have a mathematical use for negatively biased indices :)

indexbase -100
sys a[100]={0,1,2,3,4,5,6,7}
print a[-99] '1
indexbase 0
print a[1] '1

[RobbeK]

Ah, thanks ;-)

O2 module for tB or Oxygen Basic (or both) ?

O2 works perfectly .. code runs ~4x faster than if compiled with the previous compiler.
(attached -- this Lisp had no Win32 graphical output -- so far so good )


best Rob

[mike lobanovsky]

@Charles:

Has anybody ever thought about the possibility of wrapping the indices of a BASIC array so that the iterator never goes out of bounds but rather wraps around automatically to its minimum or maximum again and again? Can there be any practical sense in it? :)

@Rob:

I'm dying to know what exactly is there in that Luminance HDR folder on your desktop? :)





Just wondering...

[Charles Pegge]

Rob,

indexbase is confined to Oxygen. All thinBasic arrays always start with 1, as far as I know.


Mike,

Cyclical indexing is a bit too specialised to become core, but it has uses in feeding wave audio buffer loops, for instance.

If the array size is a binary multiple - like 64k, then a simple bit mask can be applied to the index:

indexbase 0
...
i and=0xffff

[RobbeK]

Mike ,
http://qtpfsgui.sourceforge.net/ : High Density Range mapping / composing of images.
In digital photography the captured range is (or can be) rather low (film is better because it has a thickness , modern films have several layers to catch light with different wavelengths / intensities resulting in a wider density range ) .. however, the modern equivalent is to take several pictures of a scene at different exposure parameters -- it is then processed in a (super) additive way.
Aligning of the seperate images (in this case) is done by another formidable program : Hugin
http://hugin.sourceforge.net/ (Hugin was one of the raven of Odin (the poor man gave up one eye for wisdom ) -- the bird kept an eye on things (especially at the back - the old Germanic Gods were always in a mood for a fight ;-) .. IIRC the other raven was called Munin or something like ...

Charles & Mike ...
Indices : the Lisp / Scheme way
(in Scheme - that's the easiest -- you can set up the indices on forehand --
1) as a list (range 2 6) is identical with '(2 3 4 5) this can be fed to a macro -- here you see it is not restricted to a pure linear sturcture it is easy to set up '(2 4 8 16 etc.. )
thus : for ( [i '(1 2 3) ] ..... identical with [ i (range 1 4) ] ...
(as said) of course you can replace (range ... ) with any definition that outputs a sequence of numbers ..
2) Now it comes -> you also can turn this into a stream -- while (range) outputs a list , (in-range) outputs a stream (there are many constructions in lisp doing smae things sequential or paralel.
Now you can define something as (in-naturals ..) which takes the whole range of natural numbers (0 to infinity ) !!
Because this is a stream and not a list it can be interupted at the needed place/time/position. (i mean without the computer hanging on constructing an infinite list)

I'm not very familiar with it, but i do think Haskell goes even a little further , in a way trying to combine both by lazy lazy evaluation.
It will only do something if needed p.e. (in human language)
.
.
a=a lot of calculations , taking a lot of time ....
.
(nothing happens, because the output is not needed)
.
.
print (a) ... it starts calculating a
.
.
reversed -> if condition1 or condition2 or cond3 ... it will not evaluate/compile cond2 , cond3 ... when condition1 = t
with the opposite in the "and" construction ..

if false and .... / ends here


(I think Mike told me C has similar constructions .. )
(in (some) Lisp(s) you can turn this off by (do-all) forcing it into strict evaluation )

.. but I only used Haskell writing a few lines / it seems to works this way at first sight .. (Lisp - more than a decade )

best Rob

[Charles Pegge]

I deploy lazy evaluation on conditional expressions but it can be improved. in a nested expression: some of the components are calculated in advance and stored to temporary variables. These component calculations need to be in-lined, so that lazy-eval can jump over them. My next task :)

In Oxygen, local macros and functions can be used to customise the behaviour of array indices:

'3d static array example


indexbase 0
def xe 100
def ye 100
def ze 8


float b[xe*ye*ze]


macro bm(x,y,z) b[z*xe*ye+y*xe+x]


bm(1,2,3)=bm(1,2,4)

[RobbeK]

Hi Charles,

I think deep inside Lisp must use something similar, which makes a difference with languages like Haskell.
I mean Lazy Eval can only happen when controled by a macro ( if , when , cond , etc .... ), but it seems Haskell is 100% functional , (if is a function and not a macro ), for this reason lazy evalution will cover the whole code , for this reason calculations will not be done before the result is needed. But as it is very easy to write macro's in Lisp (using the same language syntax) this is not a big thing.
(btw even in assigning, Lisp can use paralel and sequential structures ,
p.e. (let ... ) binds locals paralel , (let* ... ) serial/sequencial ;-)
(maybe I will download Haskell (and the Glasgow Haskell Compiler) and play a little with it .

btw donwloaded the latest Oxygen now (i was working with 2012 one ) , that IUP stuff seems most interesting ;-)

best Rob

[mike lobanovsky]

Hello Rob and Charles,

@Rob:

1. Yes, C conditionals are always evaluated the "exclusive" (lazy) way, evaluation being stopped at the first occurrence of a condition that happens to be met. Traditional BASIC evaluation is "inclusive", i.e. it always checks the entire range of conditions specified, which is inefficient speed-wise and may cause bugs that are not evident and difficult to intercept. FBSL's BASIC has both strategies implemented natively while FBSL's ANSI C naturally supports lazy evaluation only.

2. Thanks for the info on the Luminance HDR package. I once wrote a texture loader that was supposed to handle images stored in a Radiance HDR format (.RGB, .RGBE) and now I saw that "Luminance HDR" label. So I thought it might be something that I missed altogether but now I realize I was simply caught by this pun on words - Luminance and Radiance. :)

@Charles:

Yep, I guess cyclic wave audio would be too special to have it implemented at the core level. Nonetheless thanks for answering. (anyway I said "just curious..." :) )

[RobbeK]

Hi Mike,

"inclusive" - maybe an "almost mistake" ? (and confusing imo)

----------------------------
Uses "Console"

Sub test(x As Integer) As Single
If (x<>0) And (1/x < 1) Then test=Sqr(x)
End Sub


PrintL test(7)
PrintL test(6)
PrintL test(0)


WaitKey
--------------------------------------
crashes of course -- here the reason is obvious, but it may be (very , very ) well hidden ....
.. and on top of this , logically , we may think .. "but I excluded x=0 , what happened ?? "

in Lisp the forcing into strict evalution may make a (very, very) little sense :
after all, one can include something to catch the exception ...

(when NaN? (print "Sorry, don't know how to handle this") (wait-for-human-help) )

it will not crash .. it's even possible to define things as :

inf + number = inf
inf + inf = inf
inf * inf = inf
inf / inf = NaN (most of the time, there may be a solution -- needs human interaction )
0 * inf = NaN etc ....

after all it's a programmable programming language -- getting O/T oops ,

best Rob

[mike lobanovsky]

Rob,

I'm using the words "inclusive"/"exclusive" here in the sense of the English set phrase "All inclusive": whatever conditions appear on one and the same code line (i.e. in one and the same expression) will be evaluated in their entirety.

Such conditionals as If (x<>0) And (1/x < 1) Then test = Sqr(x) are perfectly valid and safe for a C compiler but are forbidden in a vanilla BASIC. You must use a nested If block to avoid division by 0 here:

If x<>0 Then
If 1/x<1 Then
...........

The same concerns logical Or one-liners that may also generate similar pitfalls.

FBSL's BASIC features And Also (aka AndAlso) and Or Else (aka OrElse) operators that act exactly like their "lazy" C counterparts and can be safely used in such one-liner conditionals as yours. Besides, while FPU division by zero is a legal operation in FBSL's BASIC that yields a + or -INF result depending on the sign of x, integer division by zero can be intercepted in the latest FBSL version with On Error GoTo/Resume Next error trapping.


Regards,

[RobbeK]

Hé Mike,

OOps , my excuses, it's not about any nomenclature nor any form of word choice -- the "almost mistake" is about using such a construction inside a programming language

: (another try)
.. using (all) inclusive evaluation of a sequence of conditions can (imo) be regarded as an almost mistake ...

for the reason of equivalence between them , I put both within double quotes

"inclusive" = "almost mistake" and not "inclusive" = almost mistake ...

.. mea culpa (i'll skip the maxima ;-)

best Rob

[mike lobanovsky]

Morning Rob, :)


we may think .. "but I excluded x=0 , what happened ?? "

Usually such pitfalls occur only when porting some "alien" code to BASIC the quick and dirty way. A predominantly BASIC oriented coder would tend to use nested Ifs even when coding in other languages. The "inclusive" behavior of BASIC one-liner conditionals was set forth by the ancient ECMA/ANSI/ISO/IEC standards and has been respected by all the three generations of BASIC ever since.

Regards,

[RobbeK]

Hi Mike ,

I don't think strict evalution has to be disadvantageous (from there the "almost" ) , be it that lazy eval can take advantage of infinite data structures (like in Haskell p.e. let numbers = [ 1,2 ... ] , one can write such things because it is not evaluated , you can map something on this sequence and then pick a certain element (or a sequence of elements ) only then calculations will start and they will scope a finite subset of the "numbers" ...

This is Haskell or Scheme streams .. in Lisp calculations are strict , you have to build in a limit and work with countable data ..

p.e. (apply '+ (map (lambda (x) (/ x)) (sequence 1 1000)))

evaluated form right to left

'(1 2 ... 1000) (Lisp builds the complete list)
'(1 1/2 ... 1/1000)
(+ 1 1/2 .... 1/1000)
= Zeta(1) till 1000th term

Makes me think building such a thing as the Glasgow Haskell compiler must be a kind of "tour de force".
Another point is the lack of iteration mechanisms on those languages ... I don't like recursion at all (and in Scheme code I often see pseudo iterations a tail recursive [ (... n) ( (... (- n 1))) ] , I certainly prefer the Lisp do, dotimes, loop, for macro's (or the simple for ... next in Basic ).
btw , the return stack in something NewLisp is that limited high amount recursion is impossible anyway ...

I mean while lazy evaluation of conditions is one thing, lazy calculus may be another discussion point ?

best, and thanks for ur reaction ! Rob

(and in the end , things (lazy) may become what they say ;-) ... slow ... ( the overhead may increase significantly in deep nested calculations ... )

[mike lobanovsky]

Good evening Rob,


...you can map something on this sequence and then pick a certain element (or a sequence of elements ) only then calculations will start and they will scope a finite subset of the "numbers" ...
In FBSL, we also have dynamic Variant arrays, initialized or uninitialized, that can be redimensioned at run time very fast and can hold any data in any order and in any number since Variants are typeless. We can then apply an arbitrary routine to them using the Map(callback, array) function to process the array content in a specific task-dependent manner. Here callback is any user-defined function while array is the array (strong type static or Variant type dynamic) to process. I guess this functionality pretty much covers what you describe as Haskell-specific.


... evaluated form right to left
Any manner of evaluation or transformation can be set forth in the custom callback function for use with a given array in a call to FBSL's Map().


...the Glasgow Haskell compiler must be a kind of "tour de force".
Not in the least. It may have a more highly abstracted syntax with more things done behind the curtains but "how much do we want our data refined for us", as one very clever and talented FBSL user once asked me? Higher abstraction means lesser flexibility. We prefer to build our solutions from smaller Lego bricks to be in full control over what we're doing. :)


I certainly prefer the Lisp do, dotimes, loop, for macro's (or the simple for ... next in Basic ).
For/To/Step+/- is by no means simple. Modern BASIC's also feature Do While|Until/Loop While|Until, While/WEnd and Repeat/End Repeat blocks that pretty much cover all the above functionality. FBSL also has a For/DownTo/Step+/- loop, for that matter. :)


...the return stack in something NewLisp is that limited high amount recursion is impossible anyway ...
I can change the Size of Stack Commit field in the PE header of my Fbsl.exe when compiling its sources with GCC. I'm pretty sure you can also hack the NewLisp (or thinBasic :D ) binary in a hex editor to increase the value written by the compiler into this field of PE header to make its (or their :D ) recursion capabilities as deep as you like. However, I would also recalculate the binary's Image Checksum in PE Explorer after such a hack and fix its value in the PE header too. This would preserve perfect integrity of the binary from the standpoint of Windows PE loader.


I mean while lazy evaluation of conditions is one thing, lazy calculus may be another discussion point ?
Exactly.


Thanks again and have a nice night,

[John Spikowski]

RobbeK,

You can do something like this in Script BASIC.



FUNCTION add(arg)
add = arg + 1
END FUNCTION

value[0] = 10
value[1] = 11
value[2] = 12

side1{"colors"}{"R"} = 128
side1{"colors"}{"G"} = 64
side1{"colors"}{"B"} = 32
side1{"Title"}{"Top"} = "Top Title"
side1{"Title"}{"Bottom"} = "Bottom Title"
side1("Function"}{"add"} = ADDRESS(add())
side1{"vertex"}{"values"} = value


I think the perfect combo would be to embed Script BASIC in OyxgenBasic and use it for all your dynamic / typeless structures.

Just a thought.

[RobbeK]

Hi Mike,

The more I hear about FBSL, the more interesting it sounds !!
Abstraction, yes every abstraction hides information - on top, it may result in writing unnecessary code by sole reason of communication (protocol). On the other hand, it may come closer to human thinking patterns, the Haskell [ 1,2 ... ] is a good example -- we can "think" about an infinite sequence without calculating it -- however , were is the barrier (how far should it go) .. IIRC if you feed the GHC something as [ 2,4,6,8 ... ] it will expand it into something of the form 2N -- (does it more complex (non-linear extrapolation) too ??? - one should now if using this mechanism .. ).
& then there are the endless benchmark competitions ... In the 1980s when working with Lisp, i've seen them many times ... "look my code runs as fast as C " ... yes, and when you looked at the code, it even looked like C ;-) (imo there is no intention (should be) for such things : Lisp == Lisp and C==C btw in that bot competition, the main part of the Haskell team were among the first to exit the arena ..; but then statistics prove everything and nothing (in France there are mor children born in regio with a lot of storks .. the conclusion is yours .. ;-)
-- often statistics are used to link unrelated things , that's the trouble .. (politicians and such are rather strong into this ... )

Hi John,
Thanks !
O2 is an excellent tool and language -- used it in the Lisp to interact with the Win GDI lib (and one very time-critical process (in the example attaced , about 10.000.000 iterations of the complex mapping z -> z²+c )) The input is done with Japi (uses Java) , but it was too slow for graphical high performance output (I couldn't find double buffered graphics in the documentation .. )

thanks Rob


oops ... edit : for Mike -- the most general iteration macro in Lisp ;-)

-------------------------------------- (from the HyperSpecs -- as usual the "*" makes the difference between paralel and sequential assigment )

Macro DO, DO*


Syntax:


do ({var | (var [init-form [step-form]])}*) (end-test-form result-form*) declaration* {tag | statement}*

=> result*


do* ({var | (var [init-form [step-form]])}*) (end-test-form result-form*) declaration* {tag | statement}*

=> result*



Arguments and Values:


var---a symbol.

init-form---a form.

step-form---a form.

end-test-form---a form.

result-forms---an implicit progn.

declaration---a declare expression; not evaluated.

tag---a go tag; not evaluated.

statement---a compound form; evaluated as described below.

results---if a return or return-from form is executed, the values passed from that form; otherwise, the values returned by the result-forms.


Description:


do iterates over a group of statements while a test condition holds. do accepts an arbitrary number of iteration vars which are bound within the iteration and stepped in parallel. An initial value may be supplied for each iteration variable by use of an init-form. Step-forms may be used to specify how the vars should be updated on succeeding iterations through the loop. Step-forms may be used both to generate successive values or to accumulate results. If the end-test-form condition is met prior to an execution of the body, the iteration terminates. Tags label statements.

do* is exactly like do except that the bindings and steppings of the vars are performed sequentially rather than in parallel.

Before the first iteration, all the init-forms are evaluated, and each var is bound to the value of its respective init-form, if supplied. This is a binding, not an assignment; when the loop terminates, the old values of those variables will be restored. For do, all of the init-forms are evaluated before any var is bound. The init-forms can refer to the bindings of the vars visible before beginning execution of do. For do*, the first init-form is evaluated, then the first var is bound to that value, then the second init-form is evaluated, then the second var is bound, and so on; in general, the kth init-form can refer to the new binding of the jth var if j < k, and otherwise to the old binding of the jth var.

At the beginning of each iteration, after processing the variables, the end-test-form is evaluated. If the result is false, execution proceeds with the body of the do (or do*) form. If the result is true, the result-forms are evaluated in order as an implicit progn, and then do or do* returns.

At the beginning of each iteration other than the first, vars are updated as follows. All the step-forms, if supplied, are evaluated, from left to right, and the resulting values are assigned to the respective vars. Any var that has no associated step-form is not assigned to. For do, all the step-forms are evaluated before any var is updated; the assignment of values to vars is done in parallel, as if by psetq. Because all of the step-forms are evaluated before any of the vars are altered, a step-form when evaluated always has access to the old values of all the vars, even if other step-forms precede it. For do*, the first step-form is evaluated, then the value is assigned to the first var, then the second step-form is evaluated, then the value is assigned to the second var, and so on; the assignment of values to variables is done sequentially, as if by setq. For either do or do*, after the vars have been updated, the end-test-form is evaluated as described above, and the iteration continues.

The remainder of the do (or do*) form constitutes an implicit tagbody. Tags may appear within the body of a do loop for use by go statements appearing in the body (but such go statements may not appear in the variable specifiers, the end-test-form, or the result-forms). When the end of a do body is reached, the next iteration cycle (beginning with the evaluation of step-forms) occurs.

An implicit block named nil surrounds the entire do (or do*) form. A return statement may be used at any point to exit the loop immediately.

Init-form is an initial value for the var with which it is associated. If init-form is omitted, the initial value of var is nil. If a declaration is supplied for a var, init-form must be consistent with the declaration.

Declarations can appear at the beginning of a do (or do*) body. They apply to code in the do (or do*) body, to the bindings of the do (or do*) vars, to the step-forms, to the end-test-form, and to the result-forms.


Examples:


(do ((temp-one 1 (1+ temp-one))
(temp-two 0 (1- temp-two)))
((> (- temp-one temp-two) 5) temp-one)) => 4

(do ((temp-one 1 (1+ temp-one))
(temp-two 0 (1+ temp-one)))
((= 3 temp-two) temp-one)) => 3

(do* ((temp-one 1 (1+ temp-one))
(temp-two 0 (1+ temp-one)))
((= 3 temp-two) temp-one)) => 2

(do ((j 0 (+ j 1)))
(nil) ;Do forever.
(format t "~%Input ~D:" j)
(let ((item (read)))
(if (null item) (return) ;Process items until NIL seen.
(format t "~&Output ~D: ~S" j item))))
>> Input 0: banana
>> Output 0: BANANA
>> Input 1: (57 boxes)
>> Output 1: (57 BOXES)
>> Input 2: NIL
=> NIL

(setq a-vector (vector 1 nil 3 nil))
(do ((i 0 (+ i 1)) ;Sets every null element of a-vector to zero.
(n (array-dimension a-vector 0)))
((= i n))
(when (null (aref a-vector i))
(setf (aref a-vector i) 0))) => NIL
a-vector => #(1 0 3 0)


(do ((x e (cdr x))
(oldx x x))
((null x))
body)

is an example of parallel assignment to index variables. On the first iteration, the value of oldx is whatever value x had before the do was entered. On succeeding iterations, oldx contains the value that x had on the previous iteration.

(do ((x foo (cdr x))
(y bar (cdr y))
(z '() (cons (f (car x) (car y)) z)))
((or (null x) (null y))
(nreverse z)))

does the same thing as (mapcar #'f foo bar). The step computation for z is an example of the fact that variables are stepped in parallel. Also, the body of the loop is empty.

(defun list-reverse (list)
(do ((x list (cdr x))
(y '() (cons (car x) y)))
((endp x) y)))

As an example of nested iterations, consider a data structure that is a list of conses. The car of each cons is a list of symbols, and the cdr of each cons is a list of equal length containing corresponding values. Such a data structure is similar to an association list, but is divided into ``frames''; the overall structure resembles a rib-cage. A lookup function on such a data structure might be:


(defun ribcage-lookup (sym ribcage)
(do ((r ribcage (cdr r)))
((null r) nil)
(do ((s (caar r) (cdr s))
(v (cdar r) (cdr v)))
((null s))
(when (eq (car s) sym)
(return-from ribcage-lookup (car v)))))) => RIBCAGE-LOOKUP


Affected By: None.


Exceptional Situations: None.

----------------------------------------------

[Charles Pegge]

Well, I've got the lazy-evaluation working better in OxygenBasic. The lazy leapfrog is confined to non-nested boolean expressions which contain comparators. For example:

if (a>b) and (b<c) and (b<>b) then ...

This is a very common pattern and in this case, Lazy takes the opportunity to jump at each and whenever the comparator expression yields false.


I've also improved infinity and zero handling for floats:
Signed infinity and zero are supported. Could be useful for atn()

def infinity 1/0


double d=infinity
double e=-d
print d '#INF
print e '#-INF
print 1/d ' 0
print 1/-d '-0
print atn d 'pi/2
print atn e '-pi/2




If the number cannot be valued, the FPU will oblige by returning #qNAN

def nan 0/0


double n=nan
double m=n
print m
if m=n then print "invaluable!"



thinBasic_Oxygen latest:
http://www.thinbasic.com/community/showthread.php?12175



PS: Stripping unused code from a program is rather similar to lazy computation. The compiler has to backtrace the code, starting at the top level, tag all the invoked functions, then delete all the others. A kind of pruning operation. So I wonder if this algorithm, in some form, could be applied to computation as well.
In Oxygen, this mode is activated with #compact

[mike lobanovsky]

Hello Rob,

I agree, this do functionality is cool. But tastes (and tasks) differ and, while someone may be more interested in asking the computer a simple "Why?" to hear a competent "Because!" in response, others may be more fascinated by "How?". I belong to the latter bunch, hence my interest in the building blocks rather than the building proper. There wasn't a single toy left unbroken in my kid's room when I was a boy; all of them were taken apart to see how they did what they were supposed to do.

Please read this (http://blog.wolfram.com/2014/03/25/injecting-computation-everywhere-a-sxsw-update/) - that's the story of Hal 2014. I'm sure you're going to like it. :)

And yes, your "magma" is very impressive.

[RobbeK]

Sorry (very) late reply ..

Thank you both for the links and the fresh O2 ...

I switched to NewLisp now , the new FFI is much clearer , easier than on the previous Lisp I tried. (but not giving up the "classics" )
99% was very easy , 1% very difficult -- Lisp has no pointer data type , (mainly because it is useless anyway - mainly only strings use a "flat memory model" , the rest uses pointers (a very lot of them) to connect things -- i've used O2 to allocate memory in case I use a Java server (that needs addresses for fast graphics color-arrays (rgb) -> screen :: the openGL/glu/glut works directly without any problems ... )

Charles, don't know if the 2012 O2 edition is the standard/stable version at this moment ... if you compile ->DLL the caller program will generate a windows error on exit (i tried a few Lisps (via FFI and CFFI ) and Racket Scheme ffi/unsafe ) .. the newer version does it perfect and clean ....
(ah, yes, is there a function in O2 de-allocating the claimed memory , so I can give it back to the system - the arrays needed for Java swing (while r g b values are not represented by bytes or char bur 32bit integers !! -- )
best Rob
(in fact what I did , makes not much sense -- NewLisp is already perfectly embeddable in tB ...

hi Mike, demolishing things -- well , we have to get in , to get out , not ;-)

[Charles Pegge]

Hi Rob,

Earlier versions of Oxygen are deep alpha :)

My curiosity about Lisp is aroused, and I wonder whether Lisp S expressions could be useful in Basic. I like the way in which declarations, operators, functions and any other construct, existent and yet to be invented, all have uniform syntax.

I see no problem in representing classes, types pointers, and all the other things we use in Basic.

Concerning memory allocation: If O2 allocates the memory for Lisp then it can also free it at the end, but where the allocation comes from any outside system, one would need to know how it was created.

[RobbeK]

Hi Charles,

I was dreaming about dynamic allocation/de-allocation .. but it's no problem , for the graphics I set up 3 int arrays (int needed for Java) r g & b and oxygen returns @r , @g , @b to Lisp using it for calling Java that needs these pointers. NewLisp does have (address) but the data structures are wrong (except for strings, but then I have to set up huge null character strings to start and processing these is slow - and I stll have to simulate 32 bit integers )
For the moment I can do graphical windows where the w*h <= 1024² - it are flat memory models so the width and height do not matter , it always fits. (but not in an economic way ;-)

"all have uniform syntax..." yes ! exactly -- even the macro declarations !

(define-macro (my-if test true-action false-action)
(if (eval test) (eval true-action) (eval false-action)))

this gives enormous possibilities -- writing your own be it strict, be it lazy , be it whateve.



that eval is of course one of the powertools of Lisp , because data=code and code=date we can do something as

(setq L '(1 2 3 4)) a list L containing (1 2 3 4)

(eval (push '+ L)) -> 10 destructive mehtod : L changed
(eval (cons '+ L)) -> 10 constructive method : L did not change - the new formula only existed during calculations

to make it even more spectacular

(eval (copy (push '+ L))) --- constructive from a destructive mechanism !!!

all this means '+ is just another "data" , it can be replaced with anythihng during runtime -- formulae themselves can morph.
(making it an AI and proving tool).

I think ACL2 is build on Gnu Common Lisp :
http://www.cs.utexas.edu/users/moore/acl2/v6-4/manual/index.html?topic=ACL2____INTERESTING-APPLICATIONS


classes
Lisp comes with context's (and also FOOP)
something as p.e
(context 'circles) (context 'squares) and later from their specific definitions ...

(area:circle r) (area:square z)

a final teaser from Lisp :

;--------------------------------------------------------------------------------------------

9.17 Bayesian analysis

Statistical methods developed initially by Reverend Thomas Bayes in the 18th century
have proved versatile and popular enough to enter the programming languages of today. In
newLISP, two functions, bayes-train and bayes-query, work together to provide an easy
way to calculate Bayesian probabilities for datasets.
Here's how to use the two functions to predict the likelihood that a short piece of text is
written by one of two authors.
First, choose texts from the two authors, and generate datasets for each. I've chosen Oscar
Wilde and Conan Doyle.


(set 'doyle-data
(parse (lower-case
(read-file "/Users/me/Documents/sign-of-four.txt")) {\W} 0))

(set 'wilde-data
(parse (lower-case
(read-file "/Users/me/Documents/dorian-grey.txt")) {\W} 0))

The bayes-train function can now scan these two data sets and store the word frequencies
in a new context, which I'm calling Lexicon:


(bayes-train doyle-data wilde-data 'Lexicon)

This context now contains a list of words that occur in the lists, and the frequencies of each.
For example:
Lexicon:_always
;-> (21 110)


ie the word always appeared 21 times in Conan Doyle's text, and 110 times in Wilde's.
Next, the Lexicon context can be saved in a file:
(save "/Users/me/Documents/lex.lsp" 'Lexicon)
and reloaded whenever necessary with:
(load "/Users/me/Documents/lex.lsp")


With training completed, you can use the bayes-query function to look up a list of words
in a context, and return two numbers, the probabilities of the words belonging to the first
or second set of words. Here are three queries. Remember that the first set was Doyle, the
second was Wilde:

(set 'quote1
(bayes-query
(parse (lower-case
"the latest vegetable alkaloid" ) {\W} 0) 'Lexicon))
;-> (0.973352412 0.02664758802)

(set 'quote2
(bayes-query
(parse
(lower-case
"observations of threadbare morality to listen to" ) {\W} 0) 'Lexicon))
;-> (0.5 0.5)

(set 'quote3
(bayes-query
(parse
(lower-case
"after breakfast he flung himself down on a divan
and lit a cigarette" ){\W} 0) 'Lexicon))
;-> (0.01961482169 0.9803851783)

These numbers suggest that quote1 is probably (97% certain) from Conan Doyle, that quote2
is neither Doylean nor Wildean, and that quote3 is likely to be from Oscar Wilde.
Perhaps that was lucky, but it's a good result. The first quote is from Doyle's A Study in
Scarlet, and the third is from Wilde's Lord Arthur Savile's Crime, both texts that were
not included in the training process but - apparently - typical of the author's vocabulary.
The second quote is from Jane Austen, and the methods developed by the Reverend are
unable to assign it to either of the authors.

;-----------------------------------------------------------

I think I mainly use Lisp because in the 1980's I wrote a lot of interfaces between AutoDesk's AutoLisp and Siemens p-codes (CAD/CAM) - there's nothing that can be done in another language - but sometimes the solutions are very elegant.
It also comes with complex numbers on board and things like De Moivre's CIS function built - in ..


best Rob

[Charles Pegge]

Thanks for your Lisp material, Rob.

For the implementation of eval, Oxygen can do run-time compiling, albeit with some limitations on static var space.


To allocate dynamic memory in Oxygen, you can create a string of nulls and pass the strptr

s=nuls 1024 * 1024 * 3
p=strptr s

You can overlay any structure array onto p:

type pixel byte r,g,b
pixel px at p

px can now be used as an array

px[n].y ...

Strings have the advantage of being garbage-collected, but you can also allocate null memory bytes directly:

p=getmemory 1024 * 1024 * 3

and explicitly free it when done:

freememory p

[RobbeK]

Thanks Charles,

I can finish my FMI now (foreign memory interface ;-)
Is there an example of "eval"/run-time compiling somewhere so I can understand it ?

Picking in on those variadic function, in Lisp it is exctremely easy -- in the lambda list just put "&rest"

(defun foo (& rest) .... can be mixed with everything else &optional, &key etc ... mechanisms enough to crawl trough the parameter list.

As said NewLISP is perfectly embeddable in tB (it's a less than 300Kb , and it are only strings going trough the pipe ...
I made an example :
http://www.thinbasic.com/community/attachment.php?attachmentid=8930&d=1394703339

(from http://www.thinbasic.com/community/showthread.php?12376-Newlisp-TB-one-more-time&highlight=newlisp )

best Rob

[Charles Pegge]

Hi Rob,

Here is an example of dynamic compiling taken from OxygenBasic\examples\DynamicCompile\

This runs in JIT mode but I have encountered a problem when trying to run it as an EXE/DLL. In any case it requires Oxygen.dll to provide the run-time compiling.

You can of course, manipulate source code strings in thinBasic and do primary compiling in the usual way.





'-------------------------------------
'RUNTIME COMPILING FROM STRINGS
'WITH ACCESS TO ALL PARENTAL VARIABLES
'-------------------------------------


'-----------------------------------------------
function evaluate (string s,float b,e) as string
'===============================================
'
double x,y,z
sys a,p
string v,u,tab,cr,er
'
p=1
cr=chr(13,10)
tab=chr(9)
v=nuls 4000
mid v,p,s+cr+cr
p+=4+len s
'
a=compile s
er=error
if er then
print "runtime error: " er : exit function
end if
for x=b to e
call a
u=str(x) tab str(y) cr
mid v,p,u : p+=len u
next
'
freememory a
'
function=left v,p-1
'
end function





print evaluate "y=x*x*x",1,10
print evaluate "y=sqrt x",1,9

[RobbeK]

Thanks so much, Charles !

(forgot to search in the examples this time -- the idea adapting the Japi lib for Lisp came from your files however)

You can't directly give strings (or the input-stream) to exo2/gxo2 ?

best Rob

[Charles Pegge]

Hi Rob,

I have fixed that problem for binaries that include dynamic compiling.

A few notes for the above example:



'uncomment to create an oxygen-dependent binary
'#file "t.exe"


' if oxygen.dll is in a different directory then
' create a file named 'oxygen.cfg' containing its pathname,
' for example:
' ..\..\oxygen.dll
'
' this will be read by the binary to find and load oxygen.






Latest:
http://www.thinbasic.com/community/showthread.php?12175

[RobbeK]

Thanks Charles, got it !!

Is it possible to keep the dynamic compiling in memory only ? (or doesn't it matter by reason of the file/mem cache ...

I had an about similar problem in Scheme. Using bitmaps and even colors as objects made it very slow, so I built an memory image buffer simply composed with color lists '(color r g b) , but then I had to make a roundabout over "disk"

,-----------------------

(set! im (color-list->bitmap buffer n n ))
(save-image im "im.png" )
(set! bmp (read-bitmap "im.png"))
(new message% [parent mframe] [label bmp])
(send mframe show #t)

;----------------------------------

surprisingly this works very fast (no memory mapped files needed )


best Rob, and thanks for your expert help !

[Charles Pegge]

Hi Rob, dynamic compiling always takes place in memory. Technically speaking I use OLEstring memory (from the heap).

[RobbeK]

ah Ok, thanks Charles

I asked because the Lisp (exec "command" &optional stdin) is not exactly a classic shell - I have to read the documentation , it uses a classic shell command and returns all standard output (from the callee) as a list of strings (one for each line in standard out (stdout)).

However, if it works from inside a DLL then it will be no problem

.
.
function evaluate (string s, &rest) as something , export

build
compile s
&
call (compiled s)

return something
end function ????????????????????

.. it's more curiosity , (non-dyn) from a DLL via the ffi is exactly what I need .. (on the other hand dynamic compiling looks very Lispish ... )


best Rob

oops, reading this again .. what I had in mind is that the dynamic part is the program source as a string itself , then I could do something as :
(exec "exo2.exe -s s") , but if existing there are some extra problems concerning the output - (I have to write to the output stream - oxygen's "print" will return only an empty list as it is directed to a messagebox ... ) - - I first have to read both documentations ..

[Charles Pegge]

Hi Rob,

There is a small console.inc which you can use. It overrides "print".

You may want to derive your own, for Lisp interfacing standard i/o. You pobably wont need AllocConsole()

[RobbeK]

Many thanks, Charles ...

I should have all I need now ..
I never did it before in Lisp (using a List generated by an "alien" stdout ) ... let's see (after all , List structures are by nature variadic ;-)
(and even can be nested in other cases than this ).

Thanks again ( an idiot can ask more questions, than a wise man can answer ... :- )

Rob

[RobbeK]

addendum :

... it works, Charles, it works ... ! (intercepting the O2 output stream via console.inc ).

Now, i have the best of both worlds ;-)
(connecting data cells with pointers as in Lisp has advantages (no memory block shifts etc ..) , but produces a lot of orphans during manipulations - resulting in a need for heavy garbage collection -- and some other things like the amount of pointers to travel ) (Mr Lutz Mueller from NewLisp uses ORO memory management i.o. classic garbage collection -- one reference only .. no idea how it works ).

I'll shut up about Lisp now .. just one extra (related with variadics - how easy it is in Lisp (no patchwork needed)).

(define (foo)
(doargs (i) (print i "-")))

(foo 2 5 4 77) -> 2-5-4-77- (to the output stream) -- much more mechanisms available , they even work when defining macro's


best, Rob

[Charles Pegge]

That is good, Rob,

On data cells and pointers: maybe one could run a dual system which uses block shifts for small cells, and pointers for large cells. Keeping everything together in close proximity is better for CPU performance, due to memory-cacheing, which uses the high-speed core memory.

Another strategy is to use memory pools for small strings. With fixed allocation units, say 100 bytes, they would be easy to recycle, and the whole pool can be freed in one step, when all the strings are finished with.

[RobbeK]

Excellent ideas, Charles ..

For the moment, I got the graphic images in "foreign memory" , O2 creates, assigns, addresses and destroys them.
In the old days the (gc) was a serious problem - running processes were interupted - and people working with the code were reasonably frustrated about this.
Till someone had a brilliant idea - we turn off the garbage collector , code runs till the available memory is saturated , the program waits till garbage collection is turned on again, and we go home for the rest of the day ... ;-)

But with Scheme I sometimes notice that it spends >50% of its run time on gc. One can turn it off , and turn it on in idle time (like waiting for an input or so ) , but that's gambling ..

On the other hand other languages could benefit from this mechanism (on a micro scale) -- infact it's something as follow the pointers and use their (pointed) data till you encounter the null pointer. (but then recursively, making nested list possible ). -- oops, recursion may be skipped , a simple flat model will be more than enough for imperative languages ...

Speed of O2 will be close to assembler, I guess ?? (the most recent assemblers I used were for the Z80 and the 6502 ;-)
-- (well, I had a look how floating points are done these days - found it difficult, but later realized the document was outdated and there are much easier and shorter ways , i don't recall exactly but it had to do with SSE and SSE2 - and this was only about comparing two floats ... )
best Rob

http://upload.wikimedia.org/wikipedia/commons/1/1b/Cons-cells.svg memory model of '(42 69 613) -- that "quote" ' is there to mark it is a non-excecutable (=data) .. infact making (quote ... ) the opposite of (eval ... )

[Charles Pegge]

You can inspect the assembly code generated by OxygenBasic, on specific lines.

It will show you how to do Pentium floats, using the FPU. This device understands Reverse-Polish :)

I often use it to check the behaviour of the compiler.

float a,b,c


#show if b>c
#show a=sin(a+c)
endif

[RobbeK]

Thanks Charles,


attached : the "Electronic Numerical Integrator And Computer". (17,468 vacuum tubes !!)

best Rob

[RobbeK]

Hi Charles, (or any one else)

I did some things in Linux -- (there are much more packages available for Lisp's running under *NIX ... but , but the graphic packages are so old it's not so nice any more ... found some with a TI (texas instruments) signature from the mid 1980's still used as the standard .. , and one package (unfinished I think - Vigil (Russian)) not documented ) considers also flat memory arrays (skipping the Lisp pointer chains ) .. (something I could do with O2 running under Windows)
As I have everything I need under Win .. the question .. hm ... you by any chance , is there .. maybe .. an O2 compiled for Linux ?? ( I need those DLL's converted to .so 's ) ( I have fbc for Linux and can do it, but I prefer Oxygen for job ).

Speaking about fb, .. why does it use those "lib.... .a" files ? iirc you use Freebasic too ? .. what are these , static libs ?? The Lisp accepts *.so files, but fb seems to give an error ( related with the linker ?)

best Rob

[Charles Pegge]

Continuing this topic on the o2 forum:

http://www.oxygenbasic.org/forum/index.php?topic=1110.msg9453;topicseen#msg9453