Oxygen and Basic syntax

[ErosOlmi]

Charles,

I have a question about Oxygen that maybe is to much to ask but worth it.

I see Oxygen as a king, something really fantastic, unexpected. A super language per se. But at the same type difficult understand and to be used for people in search of a BASIC language like thinBasic is or try to be. I put myself in the list of users that find Oxygen difficult to understand due to my ignorance on CPU and ASM.

Now the question is: why not having Oxygen to parse a language more close to a Basic one? Maybe a little more limited in possibilities than what Oxygen is now but at the same time more close to what people are used or try to find here. I'm sure more people would use it so giving more tributes and satisfaction to the big big work you have done so far. I have to admit that latest version made a big improve on this side.

I know this is an unfair request due to the amout of time you have spent on current Oxygen but I think one of the biggest satisfaction for people writing programming languages is to see other people using and be productive with that language. And you merit thousands of users using Oxygen.

Ciao
Eros

[Charles Pegge]

Yes Eros, we can add a High level language layer (H2O) to Oxygen providing TYPE awareness and a kernel of functions with the BASIC syntax. This would enable programmers to convert critical sections of their thinBasic scripts into machine code without using Assembler. (Then invite Donald to evaluate it ;))

[ErosOlmi]

Well,

if you need any new interface with thinCore in order to simpify this job, let me know.
I didn't port all thinCore interfaces to FreeBasic SDK but if you need I will convert them all (or anything else from my side).

Donald seems very concentrated on HB. It seems a love/hate feeling. I will not bother him again (for the moment ;) )

Ciao
Eros

PS: I've tested in thinBasic: "9 * 10 + 9". It always results to 99. I was a little worried about that to have the same problem ;)

[Charles Pegge]

Thanks Eros.

I've got a version of thincore.bi matching your PB version as at 23 Mar 2008, though I have not tested all the functions..

- it's included in the Oxygen src folder.

One facility, which would enhance integration would be to provide a script pointer to the module so it can capture a chunk of script, and return its end position. ThinBasic can then skip over this block and continue as normal:

p=GetScriptPtr()

SetScriptPtr(p)

[ErosOlmi]

Charles,

an interface function like that is already present and used (for example) by thinDebug to get full script source code.
The problem is that it works only if the script is in clear text. If script is obfuscated, it does not return anything (by design) because it would go against the obfuscation concept and break the rule.

Eros

[Charles Pegge]

Another approach would be to provide a superstring so that normal quote marks (ascii 34) and single quote (ascii 39) can be used inside it. - going back to the rawtext idea.

I favour a comppund symbol - something that is not normally used in Basic C Javascript or XML

!!! ... !!!

[Petr Schreiber]

Hi,

good idea with high level. But please keep also the current state, I like it, because I have no other possibility to play with SSE comfortably :).

When I think of it ... using Tokenizer + Oxygen modules it is theoretically possible to write compiler of high level language producing EXEs, right?


Petr

[Charles Pegge]

Hi Petr,

Introducing Basic is definitely an addition on top of the instructions already available. - another layer on the onion.

But first we need H2O, which adds TYPE awareness with rules for converting between strings, integers and floating point. This will conceal the real CPU and provide a virtual machine with its own instruction set. BASIC can then encapsulate H2O.

I have not envisage producing standalone EXEs. Could either adopt FreeBasic's method of using the GNU library, assembler and linker or directly produce files in PE format. But I think deploying the run time module is the better option, since it will be reasonably small and potentially support many binaries. (that Oxygen Opengl1 demo is only 6.5 k of machine code :)).

[Petr Schreiber]

Thanks Charles,

I also checked the OpenGL "binary footprint" and I was pleasantly surprised.


Petr

[kryton9]

Eros, thanks for the request. Charles, thanks for putting in systems to allow for such syntax adaptions in. Great planning!!

I still want what is there to be there, because I want to learn assembler, at least enough to do simple things for now to learn.

[Charles Pegge]

Here is my proposal:

O2 will incorporate basic syntax, highly compatible with thinBasic. This will allow speed-critical sections of thinbasic source code to be dropped into Oxygen where they can be compiled and executed in machine code instead of going through the interpreter.

I have sketched out a system for doing this and it is entirely feasible to incorporate a small Basic compiler into the Oxygen module taking its present size of 135k up to around 200k.

With thinBasic 1.7, the functions can be declared and and used in the normal way.




uses "OXYGEN"

dim src as string

src="
basic
(
function arith(byval a as long, byval b as long, byval c as long) as long
function=a+b*c
end function
)
"

o2_asmo src

if len(o2_error) then msgbox 0,"Problem: "+o2_error()+o2_view (src) : stop

Declare Function arith(byval a as long, byval b as long, byval c as long) at o2_buf 0

msgbox 0,arith(1,2,3)

[Michael Hartlef]

Wow Charles. I'm watching this for a long time...


O2 will incorporate basic syntax, highly compatible with thinBasic.

and that would become very interesting. right now I'm working on my own AStar Pathfinding routines in thinBasic. Of course, it is slower than compiled code. I'm sure I want to try this in Oxygen. That should give it the speed burst it needs.

Keep up your awesome work!
Michael

[ErosOlmi]

Charles,

I think this is a great great addition and will burst Oxygen usage a lot.

Thanks a lot
Eros

[Charles Pegge]

Right here goes!

I think this will take about 2 months.

Oxygen Basic Outline Specification



DECLARATIVES type dim declare byval byref local static global

_________________________________________________

VARIABLE TYPES byte word dword qword long quad ext char wchar string (,,) pointer

_________________________________________________

OPERATORS + - * / \ ( ) = > < and or xor not ++ -- += -= *= /= and= or= xor= ptr * &
_________________________________________________

BLOCK STRUCTURES macro function sub gosub goto if then else elseif endif while wend do loop exit repeat select case

_________________________________________________

NUMERIC FUNCTIONS int round sin cos tan asin acos atan exp log val sqrt pi asc cvlong...

_________________________________________________

POINTER FUNCTIONS peek poke varptr strptr codeptr

_________________________________________________

STRING FUNCTIONS string$ trim$ mid$ str$ hex$ chr$() mklong$...

_________________________________________________

DIRECTIVES #ifdef #ifndef #endif #include basic(..) asm(..) o2(..)

[Petr Schreiber]

Good move Charles,

I will keep my fingers crossed for this, as it means big jump in usability for people normally afraid of assembler.


Petr

[kryton9]

I am in deep water here, but I had an idea. I hope I can express the idea with the correct terms.

Since thinBasic already exists, with syntax and parsing/tokenizing is done, perhaps you and Eros could work together and make a new directive that uses the current thinBasic parser/tokenizer and with the directive knows to then send the tokens to oxygen for doing its magic.

[Charles Pegge]

Hi Kent,

Once the compiler logic is done, there are many possibillities, for instance passing function scripts to oxygen then, if they compile, calling them instead of interpreting. This could be done behind the scenes, without requiring an explicit directive.

Another possibility is to assimilate the technology into thinCore so it can do its own JIT compiling on suitable script sections.

Oxygen could also be used as a standalone compiler. - but what am I letting myself in for :) ?

[kryton9]

Reading your post it seems anything is possible Charles.

It would be neat to see a test , maybe something like a loop benchmark that perhaps counts to some high number.
Something easy to write, but to see what sort of improvements in speed one could see.

First - thinBasic only.
Second - Just In Time compiled
Third - Stand alone compiled .exe

About Just In Time compilation, why make it try to select sections to compile and not just compile everything just in time?

[ErosOlmi]

Kent,

executing interpret code and just in time compiling are 2 complete different things. We cannot mix each other.
I would wait for Charles to come out with a solution and see. It will take some time (Charles said 2 months) because things are not so simple.

As Charles showed, if we will have the possibility to define a BASIC like source code to pass to Oxygen and have back a pure machine code buffer to be linked to a DEFINE thinBasic function, WOW, it will be great! It will greatly improve execution speed by some factors because it would be like calling an external function from inside a DLL!

To me Oxygen was already a dream with ASM like syntax but with BASIC like syntax it is a dream coming true!

[Charles Pegge]

About Just In Time compilation, why make it try to select sections to compile and not just compile everything just in time?



I can envisage some dynamic things normally done in an interpreter might require compiling every time they are encountered - then interpreting would be more efficient and safer - for instance Eval(..). But JIT compiling certainly gives you more flexibility than static compiling.

The other issue is the HUGE and ever growing instruction set of thinBasic and all its modules!

[Petr Schreiber]

This is looking great,

combining the power of compiler and interpreter in one language, preserving advantages of both in appropriate situations.


Petr