Further Developments in Asmosphere II

[Charles Pegge]

Hi Kent,

I've given this code a thorough workover and shaken a out a few bugs. The main issue remaining is that you can't really make floating point operations transparent at this level. - the FPU is a totally different beast. - we will have to speak to it in Reverse Polish or hide it in a floating point library.

This is Test_oop1.tbasic - working with the latest version: I hope it makes sense enough to play with. - There's more work to do on the manual.



uses "OXYGEN"
uses "FILE"

dim vv as single

dim src as string ="
; test_oop1
; 13:30 21/06/2008


initialise:
;-----------
; ebx holds address of runtime functions table
; assume esi edi and ebx are preserved


esi=getmemory 1000 ; pass address of sz holding length
; this replaces the previous varbase in esi


call testclasses
freememory esi

ret


;==========================
; classes
;==========================


;------------------
;GENERIC OOP MACROS
;------------------



def class type

def context
(
indexers `ecx` offset 0 ascending var %1 this
)

def params
(
def parambytes %1
indexers `ebp` offset 8 ascending
)

def locals
(
indexers `ebp` offset 0 descending
push ebp
mov ebp,esp
sub esp,%1
)

def release
(
mov esp,ebp
pop ebp
ret parambytes
)

def create
(
var %1 %2
%1_methods %2
)

def method
(
lea ecx, %1
proc )


;------------------
;define classes
;------------------

class class0
(
4 u 4 v 4 w
)

class class1
(
class0, 4 a 4 b 4 c
)

class class2
(
class1, 4 d 4 e
)


;define methods
;---------------

.init
(
context class0
params 0
locals 0
release
)

.put
(
context class1
params 4
var 4 aa
locals 16
var 4 i j
mov eax,aa
mov this.a,eax
mov eax,0
release
)

.get
(
context class1
params 0
locals 16
mov eax,this.a
release
)
.clear
(
context class2
params 0
locals 16
mov this.u,0
mov this.v,0
mov this.w,0
mov this.a,0
mov this.b,0
mov this.c,0
mov this.d,0
mov this.e,0
release
)


;define method calls
;-------------------


def class0_methods
(
def %1_init method %1 init
)

def class1_methods
(
class0_methods %1
def %1_get method %1 get
def %1_put method %1 put
)

def class2_methods
(
class1_methods %1
def %1_clear method %1 clear
)

'------------
testclasses:
'------------
(

indexers `esi` offset 100 ascending
def Result [#vv]

create class2 obj
obj_init
;using 32 bit floating point
;inline table of single constants
edx=data (ns 42.125 ns 0.61803 ns 3.14159)
obj_put [edx+4]
Result=obj_get
obj_clear
;Result=obj_get
ret

)


;=========================
; end classes
;=========================
"


' record o2 listing
file_save ("t.txt",o2_view(src))

O2_ASMO src

if len(O2_ERROR)>0 then
msgbox 0, O2_ERROR+$crlf+"Program will end now"
stop
end if

O2_EXEC

msgbox 0, vv





http://community.thinbasic.com/index.php?topic=1637.0

[kryton9]

Thanks Charles, I got it working, now I will study and play with it. Will write more in the future after some playing.

[Charles Pegge]

A lurid description of Asmosphere macros

Technical descriptions can be quite hard to follow so I thought of a way to describe these in more graphic terms which are easier to conceptualise.

Imagine a macro word as an amoeba. It can eat words both ahead of it and behind it. If the macro contains any %1s in its body, it will consume the first word to the right of it. If a %2 is present then it will consume the second word to the right and so forth. If there is a %0 then the word to the left is taken. If there is a %b in the macro body then it consumes all the words from the beginning of the line. If there is a %e then the macro will devour the rest of the line or even an entire block if a left bracket is encountered.

The macro substitutes all the %params in its body with the words it has swallowed, then bursts disgorging its contents into the script.

Macros may contain other macros - These are resolved recursively. If the macro contains itself or any of its progeny contains itself then the script will recursively inflate - in theory until the computer runs out of memory but the preprocessor imposes a limit to this source code inflation. If the source script inflates due to macro recursion by a factor of 10 then assembly is halted. In addition to this protection, recursive cycles are limited to 256 loops per word - so macros like def def def will not lock the assembler into a perpetual (non inflationary) loop.

Macros may also contain definitions of other macros - so it is possible to generate an entire vocabulary with a single word - to create a domain-specific mini-language.

If macros are defined between brackets then they have local scope within those brackets and do not affect code elsewhere. So it is possible to have several mini languages in the same script - all using similar words but for different purposes, without causing name conflicts.
The amoeba is confined by brackets.

[Petr Schreiber]

Hi Charles,

I was for 3 days out and once back you advanced again :)
May I ask you to make "The Asmosphere Assembler" topic with help + DLL + samples as a sticky one, with date of last update in thread name? ( like "The Asmosphere Assembler [UPDATED 18-Jun-2k8]")

Thanks a lot,
for nice new shiny toys to play with :)


Petr

[Charles Pegge]

Hi Petr,

Welcome back! That is a good idea but I don't have the option available to create sticky topics - only locked ones. Is there a magic switch Eros can throw somewhere?

PS

The preprocessor layer feels nearly complete but am tempted to add a few more runtime string functions - like str$ hex$ val - these are a bore to code up from scratch when they are so readily accessible.

[ErosOlmi]

I've created a dedicated forum for all your code and projects, Charles.
You should be able to add sticky and moderate posts in there.
I've also moved all posts and subforum regarding ASM.
Hope this is ok or ... let me know if you need other setup or subforums.

Ciao
Eros

[Charles Pegge]

Thank you Eros. I will now set up the sticky topic for the downloads.

PS: This is our first day of warmish weather. No need for gloves but I am still keeping my hat and furs on :)

[kryton9]

I liked the lurid description for asmosphere macros. It is going to take me some time to get my head around this stuff. But it is nice to get a glimpse of all our computers do for us behind the scenes. It is easy to forget with high level languages the work that goes on at the lowest levels.

[Petr Schreiber]

This is our first day of warmish weather. No need for gloves but I am still keeping my hat and furs on


First day !? I am BURNING here for a month. I need an igloo desperately... :)


Thanks for all the updates,
Petr

[Charles Pegge]

Stretching the syntax

With some very small tweaks it is now possible to define macros and variables with multisymbol and multiword names. This takes us into uncharted territory, but to give a flavour of what is possible this is all valid Asmosphere preprocessor:


def `@-/\-@` nop
def `++` inc
def `--` dec
def `<>` cmp eax,edx
def `:)` mov eax,1
def `;)` mov eax,2
def `:D` mov eax,3
def `:(` mov eax,-1
def `bye bye` ret
var 4 `the meaning of life`

@-/\-@
++ eax
-- eax
<>
:)
;)
:D
:(
mov the meaning of life,42
bye bye



http://community.thinbasic.com/index.php?topic=1845

[kryton9]

Charles, that scares me as much as it amazes me :)

[kryton9]

Ok, I decided to see if I can make any progress on this stuff and am lost :)

I thought well let me show what I am trying to do and maybe I can figure out how to study up and get to using this power.

I am going to go through the html version of The Art of Assembly
This page is a good start.
http://webster.cs.ucr.edu/AoA/Windows/HTML/HelloWorlda3.html

If you scroll down to the sample code for:
program DemoMOVaddSUB;

How can I take what I see in the online book as this and use with ASMosphere and Oxygen?
An example like this will go a long way to help, thanks Charles!

program DemoMOVaddSUB;



#include( "stdlib.hhf" );



static

i8: int8 := -8;

i16: int16 := -16;

i32: int32 := -32;



begin DemoMOVaddSUB;



// First, print the initial values

// of our variables.



stdout.put

(

nl,

"Initialized values: i8=", i8,

", i16=", i16,

", i32=", i32,

nl

);



// Compute the absolute value of the

// three different variables and

// print the result.

// Note, since all the numbers are

// negative, we have to negate them.

// Using only the MOV, ADD, and SUB

// instruction, we can negate a value

// by subtracting it from zero.



mov( 0, al ); // Compute i8 := -i8;

sub( i8, al );

mov( al, i8 );



mov( 0, ax ); // Compute i16 := -i16;

sub( i16, ax );

mov( ax, i16 );



mov( 0, eax ); // Compute i32 := -i32;

sub( i32, eax );

mov( eax, i32 );



// Display the absolute values:



stdout.put

(

nl,

"After negation: i8=", i8,

", i16=", i16,

", i32=", i32,

nl

);



// Demonstrate ADD and constant-to-memory

// operations:



add( 32323200, i32 );

stdout.put( nl, "After ADD: i32=", i32, nl );







end DemoMOVaddSUB;

[kryton9]

Or, since this is based on his High Level Assembly language perhaps it is not the best one to study?

If you know of or can point to another site to learn and then how that site's code can be converted into your modules for learning it will be a great step! Thanks again.

[Charles Pegge]

Hi Kent,

Yes the main problem is that hardly any of it is in standard intel x86 assembler - it is mostly a macro language and uses a library whose contents we have to guess :).

So we are immediately faced with a translation exercise.

With Asmosphere we can leave nearly all of the i/o to thinBasic and just focus on the assembler parts - using thinBasic variable addresses to pass values back and forth:

mov eax, [#v1]
add eax,[#v2]
mov [#v3],eax
ret

This is the simplest way to start - without using stack locals - and all variables provided by thinBasic - so they are all visible in thinBasic.

OOP structures can also be directly mapped onto TB UDTs and arrays.

PS:
Curiously enough, I found it easiest to learn assembler directly from reference manuals. - the amount of supporting code required to get going makes pure assembler examples quite hard to follow.

[Charles Pegge]

Charles, that scares me as much as it amazes me :)


Well it eliminates many of the normal constraints imposed on names. It could be used for the most fiendish obfuscation - or to bring clarity to abstract ideas with greater descriptive power.

Modern OS'es support a wide range of file name formats to our advantage so why not try it for variable names in programs?

Changing the fundamentals ()`"";'

One further freedom is the ability to reassign some of the symbols on which the Asmosphere preprocessor is based. You can change the brackets, quote marks or comment marks, (And also turn case sensitivity on and off.)

This is done with the symbols directive, which is shielded from normal parsing. The symbols directive can be invoked at any point in the program - wherever specialised syntax needs it.

a single bracket pair ()
3 quote marks `""
2 comment marks ;'
case sensitivity (c or n)

Default configuration:

symbols ()`"`;'n

same but now case sensitive:

symbols ()`"`;'c

now using curly braces and some exotic quote marks :)

symbols {}|^~;'n

He said | she said ^ they said ~ xyz ~ ^ |

Caveat with inline string leterals in thinBasic " and ' cannot be expressed so one must use duplicates or other symbols eg:

symbols ()```;;n

One further trick for those who prefer words instead of scoping brackets:

to do this we must first dis the brackets then restore them following the macro definitions:

symbols \\```;;n
def begin (
def end )
symbols ()```;;n


We now have the words: begin for left bracket and end for right bracket.

mov ecx,42
begin
dec ecx
jnz repeat
end

[kryton9]

Thanks for the posts. I will keep plugging away.

[kryton9]

Found this Charles, this might be the right combination. Take a look and see what you think.
http://www.doorknobsoft.com/tutorial/asm-tutorial-learning-assembly/

[kryton9]

make sure to go through section 2 and 3 as he does some graphics it seems.
They could be easy to miss, as I did the first few times at the site :)
http://doorknobsoft.com/tutorial/asm-tutorial-learning-assembly-part-2/
http://doorknobsoft.com/tutorial/asm-tutorial-learning-assembly-part-3/

[Charles Pegge]

Ah yes this is 16 bit dos oriented assembler with system calls using interrupts and data segments. This CPU mode is about to disappear forever as we shift to 64 bit operating systems. Fortunately the 32 bit coding is significantly cleaner than the old stuff, and 64 bit will be a further improvement with extra registers to play with.

To help find the right material you can use 32 bit register names as keywords when Google searching:
eax ecx edx ebx

[kryton9]

Thanks Charles, that will help by keying in with the proper search items.

[Charles Pegge]

New features in Asmosphere

The earlier Asmosphere (~May 2008) had a simple macro language without parameters. It also had a method of calling functions like a high level language (suitable for stdcall or Cdecl). It was also possible to pass string
literals, which would always leave their pointers in the eax register.

But linking to DLLs was quite hard work and relied on some of the procaddresses being passed with thinBasic variables.

To make coding more efficient and escape from the minutiae of assembly coding it was necessary to introduce a few more elements into the preprocessor layer.

Here is a recap of the recent Asmospsphere developments: (the ones that made it)

Structures and variables and arrays were introduced to represent data in similar way to higher level languages.

type

types can be specified in multiline or single line format.


type vec3
(
8 x
8 y
8 z
)

type rect 4 left 4 top 4 width 4 bottom


types can contain other types


type bounding_box
(
4 id
vec3 A
vec3 B
)


types can inherit other types

type vec5
(
vec3,
8 u
8 v
)



var
Variables can be defined with or without TYPE

the structure or number of bytes is given first. Dimensions are specified in brackets)


var 4 vv ww
var 8 xx yy zz

var rect rec
var rect recs(100)
var vec3 vecs(10,10)

variables use the familiar dot notation when referring to their elements

mov eax, rec.top

variables are associared with specified index(es) using statements like this:

indexers `ebp` offset 0 descending
indexers `esi` offset 4 ascending
indexers `ebp+eax*4` offset 0 descending


To help with variable management the following constants are available for use as literals

sizeof size of a type, element or variable
offsetof offset of a variable or element
spanof size of a dimension in an array variable




def

Macros can now support parameters %1..%9

def greeting messagebox 0,`%1 %2`,`Greeting`,0

greeting Hello World!


As you can see, prototypes are not used, Instead, the macro 'consumes'
adjacent words and incorporates them into its body. There a few restrictions where the %params may be placed.

Some %params are a little more specialised:

%0 takes the word to the left of the macro

%b takes all the words on the line to the left

%e takes all the words on the line to the right or an entire block if this is delineated by brackets.

%a Assignment: this takes the words on the line to the left and spits them out in the line following the next proc.


defs types and vars have much in common. They are all local and can therefore be confined with in scope by using brackets.

These macro words have priority over almost all other words. It is therefore possible to create instant mini-languages for a particular task in your program. They will be out of scope beyond the closing bracket - and any prior meanings of the words will be restored.


defs may contain other defs:
note the double %% to disguise the macro parameters on the primary expansion.

def floats
(
def + fadd %%1
def - fsub %%1
)



Ruining the syntax:

Breaking the rules macro names may be composed of several symbols, words or whole phrases:

def `floating point operations`
(
def + fadd %%1
def - fsub %%1
def A qword [edx+08]
def B qword [edx+16]
)


(
floating point operations
+ A
- B
)

[Charles Pegge]

RunTime support functions

These rely upon the ebx register, which holds the address of the
run time function table when your code is called with o2_exec

LoadLibrary

user32=loadlibrary `user32.dll`

FreeLibrary

freelibrary user32

GetprocAddress

msgbox getprocaddress user32,`MessageBoxA`
proc msgbox 0,`Hello`,`greeting`,0


data

create inline data
esi=data
(
nd 1.234
nd 3.14159
nl 2345
)



dataspace

create inline data / workspace
esi=dataspace 512

GetMemory

allocate system memory
esi=getmemory 512

FreeMemory

release memory
freememory esi

Bind

bind DLL calls
bind user32
(
MessageBox MessageBoxA
CreateWindowEx CreateWindowExA
)
messagebox 0,`hello`,`greeting`,0



copy0

for zstrings
copy0 dest,source

copy00

for wstrings
copy0o dest,source

copyn

for any string
copyn dest,source,length

[Charles Pegge]

New O2_ functions

These are intended to improve performance and flexibility. They work with the O2_Buf buffer selector which supports code buffers 0..511

O2_Exec

Call assembled code in the current buffer

O2_Asmo

Assemble from specified source string but leave the result in the current buffer

O2_Asmo_File

Assemble from specified source file but leave the result in the current buffer


O2_Len

return length of code in current buffer

O2_Get

Get code string from current buffer

O2_Put

Insert code string into current buffer

O2_Proc1 .. O2_Proc8

Exec code at predefined entry points. These are offset from the beginning of the code string at 8 byte intervals: 8 16 24 32 etc.

They support multitasking software such as sound and 3d frames which require setup, frame processing and termination calls while maintaining persistant data.

[Charles Pegge]

Conclusion

This is quite a lot to take in but there are a fair number of scripts included in the distribution to show how these new functions are used in practice. A chm manual is also available including the beginnings of an experimental code snippet generator called Omerlin codespells (possibly a wizard of Irish descent) , which will help to produce reliable standardised blocks of source code.

It should seldom be necessary to write code from scratch. The easiest way to learn and develop assembly code is to take a working script and gradually modify it. Assembly code is far more picky than higher level languages, so it is vital to know where an error is triggered even if you can't immediately tell why.