Dim myArray() as Long At StrPtr(myString(Index))?

[ReneMiner]

Can I create an array of Strings and copy the content of the String to an array of Numbers as this



Dim myString() as String : Redim myString(2)
Dim myNumbers() as Long

'even though it seems stupid, but just to fill it right
myString(1) = "0123012301230123" '...
myString(2) = "123412341234123412341234" '...

ReDim myNumbers(Len(myString(1))/SizeOf(LONG)) as Long At strPtr(myString(1))


now myNumbers(1 to 3) will hold the CVL of "0123" ???

can I do this thereafter:


ReDim myNumbers(Len(myString(2))/SizeOf(LONG)) as Long At strPtr(myString(2))


myNumbers() can be set to manipulate the string-content at myNumbers(Position) also?

[ErosOlmi]

There are two things to get

1.
In thinBasic Dynamic Strings OLE32 strings or (like they are called in C/C++) BSTR strings
An OLE32 string is a pointer to a pointer that points to a dynamic allocated area
VARPTR of a dynamic string will return the pointer to the string
STRPTR of a dynamic string will return the pointer to the dynamic allocated area
http://oreilly.com/catalog/win32api/chapter/api_0602.gif

An array of strings is in reality a sequence of DWORD pointers all of them pointing to a dynamic memory area.
VARPTR of a dynamic string will never change (the pointer of a string will never change)
STRPTR of a dynamic string will change every time the dynamic string is changed (every time the string is de-allocated and allocated again)

More info here: http://oreilly.com/catalog/win32api/chapter/ch06.html

2.
Numbers (LONG or whatever type) are stored as binary representation and not human readable form
So you cannot just map a "123" string into a LONG and have it as LONG number

[ErosOlmi]

Maybe this will clarify:


Uses "console"

Dim Counter As Long
Dim myString() As String
ReDim myString(2)

'even though it seems stupid, but just to fill it right
myString(1) = "0123456701234567" '...
myString(2) = "123412341234123412341234" '...

Dim myNumbers(Len(myString(1))/SizeOf(Long)) As String * 4 At StrPtr(myString(1))
For counter = 1 To UBound(myNumbers)
PrintL Counter, myNumbers(Counter)
Next
PrintL "---Press a key---"
WaitKey




'---Instead what you can do is to use a dynamic string to store whatever you would like
'---Imagine you want to store 1000 longs into a dynamic area
%NLONGS = 10
'---allocate a String able To store %NLONGS longs
myString(1) = String$(SizeOf(Long), %NLONGS)
'---Overimpose an array of %NLONGS longs over the same memory area of the dynamic string
Dim MyLongs(%NLONGS) As Long At StrPtr(myString(1))
'---Fill you array of longs
For Counter = 1 To %NLONGS
MyLongs(Counter) = Counter
PrintL Counter, MyLongs(Counter)
Next
'---Each 4 bytes of the string is the binary representation of a long
For Counter = 1 To %NLONGS
PrintL Counter, CVL(myString(1), SizeOf(Long) * (Counter-1))
Next
PrintL "---Press a key---"
WaitKey

[ErosOlmi]

Here another example using DIM/REDIM ... AT ...
Use dynamic strings inside UDT (not many languages allows this) to dynamically allocate memory buffers for dummy arrays.

Use thinBasic debugger to inspect dummy data.


Type MyType
nLongs As Long
sLongArray As String
nStrings As Long
sStringArray As String
End Type


Dim MyData As MyType


'---Allocate room for ... longs using a string buffer
MyData.nLongs = 10
MyData.sLongArray = String$(MyData.nLongs * SizeOf(Long), $NUL)


'---Allocate room for ... dynamic string using a string buffer
'---Remember: a dynamic string is apointer to a pointer that points to dynamic memory
MyData.nStrings = 10
MyData.sStringArray = String$(MyData.nStrings * SizeOf(Long), $NUL)


'---Now define some dummy arrays
Dim DummyLongs (MyData.nLongs ) As Long At StrPtr(MyData.sLongArray )
Dim DummyStrings(MyData.nStrings) As String At StrPtr(MyData.sStringArray )




Dim Counter As Long
'---Fill data in dummy long array
For Counter = 1 To MyData.nLongs
DummyLongs(Counter) = Counter
Next


'---Fill data in dummy string array
For Counter = 1 To MyData.nStrings
DummyStrings(Counter) = "String " & Counter
Next




'---How to re-dimension arrays????
'---Simple, just redimension the UDT string buffer and redim the dummy arrays
'---Here we will add other 10 elements
MyData.sStringArray = MyData.sStringArray & String$(10 * SizeOf(Long), $NUL)
MyData.nStrings = 20
'---and remap the dummy string
ReDim DummyStrings(MyData.nStrings) As String At StrPtr(MyData.sStringArray )
'---Fill data in dummy string array
For Counter = 11 To MyData.nStrings
DummyStrings(Counter) = "String " & Counter
Next

[ReneMiner]

Thanks for your replies, the last example ist the "most fitting" for my purpose but I'm still in doubt about which way would be the best.

Imagine I have a document (as a Mesh or a Bitmap or whatever) and I have stored all documents in memory as some string-array, like myBitmaps$(123) = I have 123 diffferent Bitmaps in memory. (do not worry about Bitmapheader or details it's just an example - in truth i want to use it to edit my own mesh-format, but that would make it too complicated here)

Now the user wants to perform common actions on myBitmap(1) so he has to choose myBitmap(1) as the current one to work with.

At that moment I want myNumbers()-Array to "contain" the data of myBitmap$(1), so for example myNumbers(24) will access the data of pixel 24 on currently active myBitmap$(1).
If user is done editing "myBitmap$(1)" and chooses to edit "myBitmap$(27)" the myNumbers() are supposed to access the myBitmap$(27)-data.

I could surely just count the data in into myNumbers() by parsing through the data-string and vice versa when the user changes the document to work with - but I thought using like a "mask" for the data-string, dimensioned at the same position in memory will perform this data-exchange much faster than I could do in a basic-count-in-loop, especially when those strings get very long.

Is it valid to ReDim myNumbers(123) At some different position?
Will Redim ... At... always Preserve the memory?
Is there some similar method as "Union" for this purpose which I have overseen?
Would it be better way for this, to just copy the memory as myCurrent$= myBitmap$(27) and let just myCurrent$ always share the same memory with myNumbers()?

[ErosOlmi]

Sorry I'm on travel driving so I cannot completely reply ...

Dim ... At
Redim ... At
SetAt ...
GetAt ...
Are all methods that define/redefine/map virtual variables over some memory allocated and handled somewhere else.
They just gives programmer the possibility to interpret some continuous memory area the way he/she prefer.

Virtual variables NEVER allocate or de-allocate memory even when virtual variables finish their life scope.
Virtual variables just use memory already allocated.

You can always change and redefine a virtual variable using redim ... At or SetAt ... to a new memory location.

The only important thing to remember in to map a virtual variable to a memory location that is fully allocated (using the method you prefer) otherwise GPF is just around the corner



Sent from my iPhone using Tapatalk

[ErosOlmi]

Here another interesting way to store big amount of data and quickly retrieve it: thinBasic Dictionary and dynamic strings



Uses "Dictionary"


Dim myDict As DWord
Dim pData As DWord
Dim MyString As String At 0 '---< Dummy virtual string. Its memory location will be set later when needed


'---Create a new thinBasic dictionary (collection of key/data pairs
myDict = Dictionary_Create(1000)


'---Add few key/data but you can add thoudands
'---Data can be any kind, even binary data of whatever len
Dictionary_Add(myDict, "KEY1", "ABCDEFGHIJKLMNO")
Dictionary_Add(myDict, "KEY2", "12345678901234567890")


'---Find a key and if found return a pointer to the dynamic string that holds data
pData = Dictionary_Exists(myDict, "KEY1")
If pData Then
'---Let the dummy string be placed over the real string ptr
SetAt(MyString, pData)
MsgBox 0, MyString
End If


'---Find another key and if found return a pointer to the dynamic string that holds data
pData = Dictionary_Exists(myDict, "KEY2")
If pData Then
'---Let the dummy string be placed over the real string ptr
SetAt(MyString, pData)
MsgBox 0, MyString
End If

'---IMPORTANT:
Dictionary_Free(MyDict)

[ReneMiner]

Dictionary is cool.
Another quest how to track/convert data then correct/fastest:

Assume I have a numerical UDT-Array as this:


Uses "Dictionary"

Type t_Vec3d
X as Double
Y as Double
Z as Double
End Type

Dim Positions() as t_Vec3d: Redim Positions(123)
Dim nPositions as Long = 123 ' this not urgently needed since there's Ubound()...
Dim myDict as Dword = Dictionary_Create(234)



Which way would be the fastest/correct one to convert Positions()-Array into a String at once so can I store it to Dictionary and how restore it fast from there without looping through the whole array of positions() using CVD/MKD$ on .X,.Y and .Z?

Do I need to convert it at all? Can't I just poke/peek that whole udt-array in one simple call into/from dictionary somehow? I guess there's a trick to it...

[ErosOlmi]

I always prefer not to use PEEK/POKE but try to use Virtual Variables.
Below an example that I hope will clarify something more.
It is long because I added many comment to explain various steps. Please read comments as a continuous speech.

In general the message I would like to give is: memory is just a sequence of bytes regardless the way you access or interpret it !
Using Virtual Variables you can interpret memory the way you want in different ways all at the same time.



Uses "Dictionary"

'---Here you structure
Type t_Vec3d
X As Double
Y As Double
Z As Double
End Type



'Dim Positions() As t_Vec3d
Dim MaxPositions As Long = 123
'ReDim Positions(nPositions)


'---Here we define a dictionary able to create 1000 buckets
'---1000 is not the max number of keys the dictionary will be able to store
'---but is the number of pre-allocated buckets for the internal hash table value calculation.
'---If different keys will produce the same hash value, dictionary will create
'---a linked list of keys for the same bucket
Dim myData As DWord = Dictionary_Create(1000)


'---We want to use this dictionary to store all main application info
'---So we store some of them
Dictionary_Add(MyData, "APP_NAME" , "Test dictionary power")
Dictionary_Add(MyData, "APP_AUTHOR" , "Rene Miner")
Dictionary_Add(MyData, "APP_COAUTHOR" , "Eros Olmi")


'---Ok, after few APP info, some real data
'---To store an array of t_Vec3d we just need to allocate the correct memory for it
'---If we know the definitive number of elements, it is quite simple to
'---calculate the needed memory: SizeOf(t_Vec3d) * MaxPositions
'---So we create a dictionary element (key + memory) for this
'---We will call it DATA_POS and will allocate enough memory full of nulls so
'---all memory will be initualized to nul
'---For later fast usage, we will also save a pointer to the first data byte
Dim pDataPos As DWord
pDataPos = Dictionary_Add(MyData, "DATA_POS" , String$(SizeOf(t_Vec3d) * MaxPositions, $NUL))


'---Dictionary_Exists can be also used to get back pDataPos
'pDataPos = Dictionary_Exists(MyData, "DATA_POS")


If pDataPos Then
'---All is OK
Else
'---Something went wrong, better to give error
End If

'---Here we assume all went OK


'----------------------------------------------------------
'---Now how to access single t_Vec3d element in the array?
'----------------------------------------------------------


'---Imagine we want to access t_Vec3d(100) element
Dim ThePosWeWant As Long = 100
'---1. easy way: create a DUMMY VIRTUAL t_Vec3d array variable
'--- pointing its memory to the correct memory position
Dim lVec3d(MaxPositions) As t_Vec3d At pDataPos
lVec3d(ThePosWeWant).x = 1
lVec3d(ThePosWeWant).y = 2
lVec3d(ThePosWeWant).z = 3
'---DIM ... AT will never allocate or de-allocate any memory but will only
'---create a dummy variable that uses the memory allocated by others


'---2. little less easy way: create a DUMMY VIRTUAL t_Vec3d single variable
'--- pointing its memory to the correct memory position making some element calculation
Dim lVec3dElem As t_Vec3d At pDataPos + (SizeOf(t_Vec3d) * (ThePosWeWant - 1))
lVec3dElem.x = 4
lVec3dElem.y = 5
lVec3dElem.z = 6

'---You can always set the memory to which a dummy variable point using SetAt
'---so if you want to access position 10 ...
ThePosWeWant = 10
SetAt(lVec3dElem, pDataPos + (SizeOf(t_Vec3d) * (ThePosWeWant - 1)) )
'---Now we have pointed lVec3dElem at position 10
lVec3dElem.x = 10
lVec3dElem.y = 20
lVec3dElem.z = 30

[ReneMiner]

I tried, but won't work to copy the data in one step from Dicitonary to myUDT.
This is the script I'm testing - not working - how can I copy the Array now?



' Empty thinBasic CONSOLE file template

Uses "Console", "Dictionary"


Type t_myType
X As Double
Y As Double
Z As Double
End Type

Dim myGlobalUDT() As t_myType
Dim myDict As DWord = Dictionary_Create(1000)
Dim lCount As Long

' fill udt with random elements
FillUDTRandom()
' display what we got
For lCount = 1 To UBound(myGlobalUDT)
PrintL "Element" + Str$(lCount)
PrintL " X=" + Str$(myGlobalUDT(lCount).X) + " Y=" + Str$(myGlobalUDT(lCount).Y) + " Z=" + Str$(myGlobalUDT(lCount).Z)
Next lCount
' should "write" data from my UDT to Dictionary-Slot "myKey01"
StoreMyData("myKey01")

FillUDTRandom()
For lCount = 1 To UBound(myGlobalUDT)
PrintL "Element" + Str$(lCount)
PrintL " X=" + Str$(myGlobalUDT(lCount).X) + " Y=" + Str$(myGlobalUDT(lCount).Y) + " Z=" + Str$(myGlobalUDT(lCount).Z)
Next lCount

' and put data away now
StoreMyData("myKey02")

If RestoreMyData("myKey01") Then
For lCount = 1 To UBound(myGlobalUDT)
PrintL "Element" + Str$(lCount)
PrintL " X=" + Str$(myGlobalUDT(lCount).X) + " Y=" + Str$(myGlobalUDT(lCount).Y) + " Z=" + Str$(myGlobalUDT(lCount).Z)
Next lCount
EndIf

WaitKey
' ----------------------------------------------------
Sub FillUdtRandom()
' just assign some random elements filled with rnd-numbers
Local i As Long
Randomize
Local nFields As Long = Rnd(2, 9)

ReDim myGlobalUDT(nFields)

For i = 1 To nFields
myGlobalUDT(i).X = Rnd
myGlobalUDT(i).Y = Rnd
myGlobalUDT(i).Z = Rnd
Next i

End Sub
' -------------------------------------------------
Sub StoreMyData(ByVal sKey As String)

Local pData As DWord = Dictionary_Add(myDict, sKey, String$(UBound(myGlobalUDT) * SizeOf(t_myType),$NUL))

Local lDummy(UBound(myGlobalUDT)) As t_myType At pData

' next line causes Error, also if I omit the parens
lDummy() = myGlobalUDT()

End Sub
' --------------------------------------------------
Function RestoreMyData(ByVal sKey As String) As Boolean

Local pData As DWord = Dictionary_Exists(myDict, sKey)

If Not pData Then Return FALSE

Local lDummy(StrPtrLen(StrPtr(pData))/SizeOf(t_myType)) As t_myType At pData
ReDim myGlobalUDT(UBound(lDummy))


myGlobalUDT() = lDummy()

Function = TRUE

End Function




Is there an undocumented Heap_Copy or similar that I've overseen?

[ErosOlmi]

oops, I think there is a bug in Dictionary_Exists :oops:
Function is returning a pointer to the pointer and not pointer to data.
I'm checking ...

[ReneMiner]

so copy of the whole array at once as


lDummy = myGlobalUDT
'or
lDummy() = myGlobalUDT()

is not the problem and is valid in thinBasic?
With or without parens?

Edit: I forgot something important - should free dicitonary at the end!

[ErosOlmi]

myGlobalUDT without any point followed by UDT element is considered by thinBasic just a sequence of bytes: a string.

The idea is not to copy data over and over but to allocate once and point to its memory using virtual variable to change data.
Copying memory from one location to the other is one of the slowest operation you can do into an application.
It require a lot of CPU resources and data exchange from CPU to memory.
This is very very slow.
Instead allocating the memory (the way you prefer) and than changing its content in place is very very fast.

Anyway, I discovered some bugs in dictionary and I'm checking/fixing them, sorry.

[ReneMiner]

So i might try making something up with Heap then to store my memory until user decides to change it. But I can not make all my global udt-arrays virtual variables- the whole program redims them over and over again with any single shape that the user draws - that would urge me to rewrite all geometrics... a few thousand lines.... uarrrrgghhhh - I don't like the idea...

I want to allow the user to draw his 3d-Image in different "Groups" since the user can edit only in one "Group" at the time - I use flexibilty of global UDT-arrays for all single elements of the current group.
Each Group is one 3d-object, consisting of the same basic elements (faces, corners, colors etc.) but different counts of and different values in them. So "Group Roof" would use different material/color/texture and has different amount of faces as "Group Wall" -

Now I intend to store the current groups data in some space in memory - where it does not change as long as it's just stored.
The user might add more groups now and change, which group to edit - so I want to store the current group data to some place and fill the global UDT-Arrays with the data of the group to edit, so there is always a backup of the current group before any changes were made and I don't have to rewrite it all since all the functions to create stuff are based on data-to-work-with is stored in the global UDT-Arrays.

If Not Uses "Dictionary" Then 'what would you suggest?

Edit: And however I try, thinBasic will not allow me to copy from one UDT-Array to another in one step - even if same-size :(

If I use

myUDT = Dummy
thinBasic complains missing parens. If i try

myUDT() = Dummy()
it results in Error- Index out of bounds

@Eros: do you ever check your PMs?

[ErosOlmi]

In order:


when you have complex applications like the one you are doing, one of the first thing to think about is data structure and data handling.
The way to do is to try to generalize it and wrap into general functions. This will allow you to later change strategy (if needed) without the need to change back you application.
If the programmer does not think to data structure first or if the application has many points where a global structure is used using the global variable name, sooner or later there will be problems.
I fallen into this situation many times in the past and now I've learned it
Regarding what to suggest instead of Dictionary, any method that allocate some memory and return a pointer to it (so you can just keep track of the pointer) ... is valid.
Copying one array data over another array of data having the same exact size is just a matter of memory copying.
In this case you can use the following thinBasic function:


Memory_Copy(SourceAddress, DestinationAddress, Size)

Where you just need to indicate the source memory area where to copy from, the destination memory area where to copy to, the number of bytes to copy.
If you are copying from one UDT array to another UDT array and the UDT has no dynamic strings inside it, you just need to do:


Memory_Copy(VARPTR(SourceUDTArray(1)), VARPTR(DestinationUDTArray(1)), Size(SourceUDTArray(1)) * NumberOfDestinationUDTArrayElements )

If there are dynamic strings involved, the above is not possible because a dynamic string is always a pointer and the string content is allocated in another place (StrPtr(...)). In this case every situation is different from the other.
Personal Message? Sorry: I saw you PM few days ago while I was in a meeting. I thought "I will reply later" but "later" I forgot to reply :mad:
Will do now.


Ciao
Eros

[ReneMiner]

...you just need to do:



Memory_Copy(VARPTR(SourceUDTArray(1)), VARPTR(DestinationUDTArray(1)), Size(SourceUDTArray(1)) * NumberOfDestinationUDTArrayElements )
...

Thats what I was looking for- but I could not find this anywhere in the help..., I put it on the list ;)

[ErosOlmi]

Memory_Copy, Memory_Get, Memory_Set
are functions just recently introduced, if I remember well in beta version 1.9.2.0.

They were suggested by Petr Schreiber: http://www.thinbasic.com/community/project.php?issueid=375

The same effect can be achieve using classic Basic PEEK/POKE set of functions but they are more ... cryptic :)

[ReneMiner]

Revive this thread - because it fits the topic ;)

I already made a suggest about this- but I fear it'll get lost since we "trashed" the support-section yesterday with a lot of posts.


The idea is the following: Since I use this virtual arrays a lot and mostly as String or Heap I would like it to have some simplified, smarter syntax as follows:

currently:


[Dim|Local] lLong(StrPtrLen(StrPtr(sString))/SizeOf(Long)) As Long At StrPtr(sString)
'or
[Dim|Local] lLong(Heap_Size(myPtr)/SizeOf(Long)) As Long At myPtr


where Long and Strptr/Pointer to Heap are always twice in these codelines. Thats especially annoying and cumbersome if you have an array of Pointers/Strings because you have to calculate Pointer-Index/Positions twice or store it to some local variable. And those codelines become very large and vulnerable for loosing overview, parenthesis and all that stuff...

Since its always a complete overlay onto some known size of memory (StrPtrLen|Heap_Size) - it would be great to have for this special purpose some cool call as this:


Local lLong() As Long At Strptr(sString)
'or
Local lLong() As Long At Heap(myPtr)

where the empty parenthesis signalize to size the array ((StrPtrLen|Heap_Size) / SizeOf(type)) - or maybe like this


Local lLong(ALL) As Long At Strptr(sString)
'or
Local lLong(*) As Long At Heap(myPtr)
' or
Local Array lLong As Long At StrPtr(sString)
Array lLong SetAt Heap(myPtr)

[Petr Schreiber]

I finally understand it - thinBasic would automatically dimension the array based on type + destination size. If string is 24 bytes, and the data type is long, it would automagically make the dimension 1..24/sizeof(long), so 1..6.

I think this should be doable. Maybe instead of empty parenthesis, which have already different meaning of "will be dimensioned later" it could have some additional sign to do what Rene needs.


Petr

EDIT: Rene edited the post, originally there were just ()

[Charles Pegge]

Mapping arrays onto string space is very useful. The concept can be extended a little further by giving the variable array a dual personality. So for re-dimensioning or block data transfer, it can be treated as a simple string. Then there is no need to have both the array and its mapped string as two separate entities.

To do this, you need some kind of casting:

dim ss() as long

cast string ss=string(100*sizeof(long),chr(0)) 'allocate 100 elements

ss(10)=42 'behaves like a normal integer array

cast string ss+=string(100*sizeof(long),chr(0)) 'allocate 100 more elements

cast string ss="" 'release

From what I understand of thinBasic memory management, I think this would fit in quite well.


Charles

[ReneMiner]

And another question about virtual place-over. This time Linked List- module

can I place some virtual array over LL_Data somehow?

I tried different ways as



llItem = LL_FindByName(llRoot, "myItem")

Local lData As Long At LL_Data(llItem)

Local lData As Long At Strptr(LL_Data(llItem))
Local lData As Long At Varptr(LL_Data(llItem))



but all this crashes...

Now before I try the whole day and get frustrated in the end: would it be possible to do something similar to dictionary?

...play thinBlocks now until I get the answer :)

[Robert Hodge]

If the point here is that you have a memory address, and a data type, and you want to overlay an array of items of the given data type, AND let's assume that you can't have multi-dimensional array bounds but just one, then you are saying you want the size of the area to be used as a parameter for the dimension size. So, the AT clause itself is 'contributing' to the DIM size.

Rather than using ALL or *, how about this:


Local lLong(At Strptr(sString)) As Long

By embedded the AT right inside the dimension expression, you're 'killing two birds with one stone'.

Not sure if Eros would like to parse this, but it is unique, and could not possibly be confused with anything else.

... just a thought ...

[ReneMiner]

In a multi-dimensional array you would of course need to use the "old way" to size the dimensions. But on straight one dimensional ones this would be great - and also very straightforward - and unique anyways :D

[Robert Hodge]

The trick in doing this is extracting the size information from the AT expression.

If you code


Local lLong(At Strptr(sString)) As Long


the Strptr(sString) part directly provides an address, but where does the size come from? Here's where trouble begins (grammatically, that is).

If we say Strptr(sString) has two "components" to its value - an address and a size - where does does size part come from? If it's from Strptr, then this function has to be redefined somehow, because right now it only has a single value - an address. If it's from the underlying variable sString, then we would be asking TB to 'understand' the semantics of the expression Strptr(sString) and to "know" you wanted to 'look inside' STRPTR to (a) look at the expression inside, (b) confirm that it's a simple string variable name and no other type of expression, (c) get the pointer to the string, (d) determine its length, and (e) divide that length by the size of the type of the declaration (LONG). That's asking a lot.

Then, suppose you wanted to do this string overlay thing, not with a string, but with a pointer to a string:


DIM sPtr as STRING PTR

... code to set sPtr

Local lLong(At sPtr) As Long

Then what would this mean? That lLong is an array of LONG's that overlays a single pointer to a string? There is no way to know what the length is.

I think there's two things we can take from this.

First, if we allowed an AT expression by itself, it already means an address. There should be no need for a call to STRPTR. TB should be able to analyze the data type of the AT expression, and if it's a STRING, it would automatically assume STRPTR, otherwise it would use a VARPTR value.

Second, if the datatype of the AT expression already is a pointer, TB would never attempt to do an 'overlay define' of the pointer itself, but would use the pointer as the starting point of the overlay. However, it's clear that existing syntax already covers this, so the code above is really the same as


Local lLong(some expression) As Long At sPtr

Given that, we can simplify the proposed syntax, and make it easier compile at the same time, by doing this:


Local lLong(At sString) As Long

Now - and here is where I think it gets interesting - suppose you didn't want the size of sString as the basis of all this, but some other (byte) length. It's important that the size in question is in bytes, because the whole point of the AT expression is that the length gets divided by the type-size. Suppose you had a variable called nLen that had the number of bytes you wanted to overlay-define, but you want to do this as an array of LONG. We can add a FOR keyword to the AT expression:


Local lLong(At sString FOR nLen) As Long

Here, suppose sString had a length of 80, and LONG's are 4 bytes, so that is 20 LONG's in the string. Now, if nLen = 40, the overlay would be only over the first 40 bytes, and 40/4 = 10.

Taken together, that means,


' assume LEN(sString) = 80 at this moment - this is just to simpify the explanation.
' In practice, you would have a LEN(sString) expression involved.

Local lLong(At sString) As Long

' same as ...

Local lLong(20) AT STRPTR(sString) ' LEN(sString)/SIZEOF(LONG) = 80/4 = 20

' and ...

DIM nLen AS LONG = 40
Local lLong(At sString FOR nLen) As Long

' same as ...

Local lLong(10) AT STRPTR(sString) ' nLen/SIZEOF(LONG) = 40/4 = 20

The nice thing about the AT/FOR notation is, if you wanted to, you could use a string PTR expression for the AT part, since the FOR part would be supplying the size. And, there is an implicit division by the sizeof the type, so the resulting code is shorter:


' instead of this:

Local lLong(nLen / SIZEOF(LONG)) As Long At sPtr

' you could do this:

Local lLong(At sPtr FOR nLen) As Long

[ReneMiner]

no, for nLen is no good- results in same long line because one has to calculate this again and have two times the pointer & size in that line - not an improvement

The idea was an overlay at Heap or at StrPtr with full Heap_Size or StrptrLen(Strptr) by default if just giving a few things:

-position to start from (At)
-varname & layover-vartype/udt (x As ...)
- is it at heap or at some string - or maybe at something else in future


from these parameters place the vartype as often as it fits complete into either Heap_Size(given pointer) or into StrPtrLen(given pointer)
probably best to name it what it does:


Virtual vartype Array varname over String | Heap (ptr)
'or
Virtual Array varname As vartype over String | Heap (ptr)

[Robert Hodge]

OK, let me try this again ...

First, I am not totally sure I understand what you mean by "one has to calculate this again and have two times the pointer & size in that line". I think there is language issue here ... did you mean that the pointer and length are each specified in two places on the declaration? (You didn't mean "multiply by 2", right?)

Anyhow, I think we are getting closer here. What I didn't take into account the last time is the issue of heap storage. When you set a pointer based on Heap_Alloc or Heap_Realloc, you can use Heap_Size to get the size in bytes of the memory that was allocated.

When a pointer is declared, you know what it is a type TO, because you declare it on the DIM as STRING PTR, LONG PTR, etc. But, you don't know by looking at it where the pointer came from.

I don't think it is correct to be referencing functions like STRPTR and VARPTR as if they were keywords, to try to make this distinction. What's needed, I believe, is a data type name. And, for heap-related items, it would appear that you have to explicitly say it came from the heap, because you want TB to go do a Heap_Size behind the scenes to figure out its size.

Further, let's agree that if you say that a pointer is pointing to a string, and it's in the heap, the you DON'T want to use Heap_Size to get it's length. Why? Because if it has a data type of STRING, but you are trying to USE it as a string, and it has its internal length factor set as a normal string does, you DON'T want to override that. That is an extremely easy way to overwrite memory and crash, IMHO.

So, let's take the AT requirements one step at a time.

First, NO multi-dimensional arrays would be supported. It's just too complicated.

Second, inside the parens of the dimension, you begin with AT variable_name.


If variable_name is of type STRING PTR, then the data portion of the pointed-to string is the base address
If variable_name is of type non-STRING PTR, then variable contains the base address
If variable_name is of type STRING, then STRPTR(variable_name) is implied as the base address
If variable_name is not of type STRING, then VARPTR(variable_name) is implied as the base address


Next, there is an optional FOR clause, which indicates the 'span' of memory pointed to by the AT clause.


If the FOR clause is omitted, and the variable_name is of type STRING or STRING PTR, the current string length is the memory size.
If the FOR clause is omitted, and the variable_name is NOT of type STRING or STRING PTR, then there is nothing else to go on, and so whatever type is being defined (like LONG in the example we have been considering) the implied dimension would be 1.
If the FOR clause is present the numeric expression represents (by default) a count of the number of bytes of memory.
If it is desired to have the FOR count represent a count of some other data type units (like WORD, for instance) the type name would be specified after the count expression, as in, FOR nLen WORD.



Now, how does all that help for heap-based memory?

Suppose you had some heap memory pointed to by pMem. You want to overlay define it as an array of LONG. How would you do it?

Given the synatx above, you could do it as


LOCAL lLong (AT pMem FOR HEAP_Size(pMem)) AS LONG

Here, if we went no further, the drawback is that pMem gets mentioned twice; not nice.

If we could convey somehow that the pointer is a "heap pointer", the call to HEAP_SIZE could be 'deduced'. We need a little more syntax:


LOCAL lLong (AT pMem HEAP) AS LONG

Somehow, the idea of "heap" has to be explicitly mentioned. The exact way that got done depends on what Eros can handle. Some other possibilities:


LOCAL lLong (AT HEAP pMem) AS LONG
LOCAL lLong (ATHEAP pMem) AS LONG

Because this variable lLong is now pointing to the heap, it (or the address of it) could conceivably be used in a Heap_Free call. That suggests that you might not need to create a separate pointer to it, but could do something like this (in some cases):


LOCAL lLong (AT HEAP HEAP_ALLOC(n)) as LONG

So, the syntax of the AT-clause would become:


( AT [ HEAP ] variable_name [ FOR expression [ type_name ] ] )

Would this meet your requirements?

[ReneMiner]

this


LOCAL lLong (AT pMem HEAP) AS LONG



would be great - but anyway is probably better to forget about "AT" and use something else instead, alike this


Local lLong( Over Heap [At] myPtr ) as Long
'or
Local lLong( Over String [At] myPtr ) as Long


maybe one could make it "Over_Heap", "Over_String" instead of two seperate keywords, but this "Over..." would signalize: "hey computer, use the full available size which is either Heap_Size or StrPtrLen and cover it with AS many of type LONG as fit over this".

"OVER" could replace "AT" with meaning "just use the full size for the layover"


Perhaps in future there comes some addiditonal module, somewhat similar to Dictionary or LinkedList wich can be target for virtual layovers too. That's why I like the seperate "Over" more:
AT means "starts At",
OVER means starts At until Start + Len(whatever it is) and to know what it is needs some specification as "Heap", "String" or maybe something else in future.

For all other cases (multidimesional arrays or not complete overlay) there's still the current syntax...

Edit: Forum is very slow today...

Anyway, depends on whatever is doable...

[Charles Pegge]

With regard to accessing byte length, this is what you can do with thinBasic string overlays



'ARRAY OVERLAY WITH LENGTH IN BYTES
'(THIS IS A PROPERTY OF BSTRINGS AS USED BY THINBASIC)

dim ss as string

ss="ABCDE"

dim ps(1) as long at strptr(ss)-4 'bstring length field

msgbox 0,hex$(ps(1)) 'gives byte length=5


Charles

[ReneMiner]

Eros, as long as you don't read this- it makes no sense to discuss

"multidimensional" (at least 2)




Dim a(19) as String
a(1) = mkl$(1)+mkl$(2)+mkl$(3)'...
' a(2) = I don't care, some...bytes whatever
'...

Dim lLong( Ubound(a), Over String Array a() ) as Long


just kidding :D

No- what we reallly need for an Interpreter to release its full powers to a maximum is:


ByString("<treat this string content as if it was code>")

that would be the - I don't know how to call it - ultimative step to computer understands humans from their speach.
At least a bit of this we already have from Call/Call_ifExists/Variable_Exists and thinbasic_GetVariableInfoEx-function. I would appreciate if tB evolutes into this direction :yes:

[Robert Hodge]

Eros, as long as you don't read this- it makes no sense to discuss

"multidimensional" (at least 2)




Dim a(19) as String
a(1) = mkl$(1)+mkl$(2)+mkl$(3)'...
' a(2) = I don't care, some...bytes whatever
'...

Dim lLong( Ubound(a), Over String Array a() ) as Long


just kidding :D

No- what we reallly need for an Interpreter to release its full powers to a maximum is:


ByString("<treat this string content as if it was code>")

that would be the - I don't know how to call it - ultimative step to computer understands humans from their speach.
At least a bit of this we already have from Call/Call_ifExists/Variable_Exists and thinbasic_GetVariableInfoEx-function. I would appreciate if tB evolutes into this direction :yes:

What you talking about is what Rexx does with INTERPRET. Here are examples from the ooRexx manual:


/* INTERPRET example */
data="FRED"
interpret data "= 4"
/* Builds the string "FRED = 4" and */
/* Processes: FRED = 4; */
/* Thus the variable FRED is set to "4" */
/* Another INTERPRET example */
data="do 3; say "Hello there!"; end"
interpret data /* Displays: */
/* Hello there! */
/* Hello there! */
/* Hello there! */

The main issue in doing this is a matter of timing and when and where various interpreter resources are available.

[ReneMiner]

time for a suggest...let's make one...as Interpret() - see who's first

Edit: guess I was (http://www.thinbasic.com/community/project.php?issueid=415)

[ErosOlmi]

Please, even if I do not reply do not think I do not read all those interesting posts.
I'm always interested into such development ideas and I always follow those threads.

Regarding "virtual" variables I think thinBasic has already a strong variable overlay system.
Maybe some new features are needed but, as you have discussed here, there are many situations to handle. I cannot just handle one special situation.


Regarding dynamic interpreting from string, thinBasic has already a dedicated module called "Eval" that is able to do some interesting things.
There are examples on how to use it in \thinBasic\SampleScripts\Eval\
But Eval is a module and is not aware of current script execution. And more, Eval is in reality another programming language inserted into thinBasic main one.

What Rene is asking since some time (and I appreciate this) is the possibility to load piece of main source code on the fly and interpret them like they were there written by the programmer.
A sort of dynamic "include & execute" of thinBasic source code of thousands of lines.

What I can say is that I'm working of this dynamic source code handling but is very very complex due to internal thinBasic structure.

That's all for now.

Ciao
Eros

[ReneMiner]

Instead of writing this:



Local lLong(StrptrLen(Strptr(foo))/SizeOf(Long)) As Long At StrPtr(foo)
'or
Local dDword(Heap_Size(kap)/SizeOf(Dword)) As Dword At kap


i would like to have it this way so it does not need to specify pointer twice which is especially cumbersome if one has to calculate a pointers index.



Local lLong() @String(StrPtr(foo), Long)
' or
Local dDword() @Heap(kap, Dword)

' other possible(?) syntax
Local Array lLong @String(StrPtr(foo)) As Long
' or
Local Array dDword @Heap(kap) As Dword


which -of course- is always about some complete overlay on either heap or string...

[Petr Schreiber]

Another wild idea, for string overlay:


String foo = "ABCDEFGH"
Dim lLong(AUTO) As Long At foo


ThinBasic would detect foo is string variable, not a number and it would create array of 2 LONG elements (because foo is 8 bytes long and LONG is 4 byte).



String foo = "ABCDEFGHI"
Dim lLong(AUTO) As Long At foo


Here the lenght of foo is not precisely matching multiple of sizeof(long). Here it would again allocate 2 LONG elements.




String foo = "ABCDEFGHIJKL"
Dim lLong(2) As Long At foo


Here the dimension is explicitly specified, so it would create two element array, even if there is place for three elements.


Petr

[ReneMiner]

but what about heap? would "Auto" detect, it can not be a string- so has to be some heap then?

And I'm not sure about overlay-possibilities at LL or Dictionary-pointers.

[Petr Schreiber]

One more try, maybe better:


String foo = "ABCDEFGH"
Dim arrA(FILL) As Long At String foo ' -- Overlays over string, UBOUND will be 2

String foo = "ABCDEFGHI"
Dim arrA(FILL) As Long At String foo ' -- Overlays over string, UBOUND will be 2

String foo = "ABCDEFGHIJKL"
Dim arrA(FILL) As Long At String foo ' -- Overlays over string, UBOUND will be 3

Dword hHeap = Heap_Alloc(8)
Dim arrB(FILL) As Long At Heap hHeap ' -- Overlays at heap, UBOUND will be 2

Dword hHeap = Heap_Alloc(9)
Dim arrB(FILL) As Long At Heap hHeap ' -- Overlays at heap, UBOUND will be 2

Dword hHeap = Heap_Alloc(12)
Dim arrB(FILL) As Long At Heap hHeap ' -- Overlays at heap, UBOUND will be 3


The "@" require you to press Shift + @ key on keyboard (2 key presses), and "At" also requires two key presses, so there is no typing speed saving in using it.

Linked lists are not continuous in memory by design, so it makes little sense to overlay anything over LL as a such.
Dictionary is also not continous.


Petr

[ReneMiner]

on my keyboard i only have to press Alt Gr + Q to get @ - (in purebasic the @ means somewhat like "at VarPtr/StrPtr")

[ReneMiner]

...
Linked lists are not continuous in memory by design, so it makes little sense to overlay anything over LL as a such.
Dictionary is also not continous

Petr

But I don't think about overlay to LL or dictionary as such. I think about overlay onto the stored "data-pointer" - which stores string mostly, alike this:



Uses "Dictionary"

Dword myDict = Dictionary_Create(100)
String myData = mkl$(1) + mkl$(2) + mkl$(3) '...
String myKey = "nobody_cares"
Dword myPtr = Dictionary_Add(myDict, myKey, myData)
' until here is still "normal"
' ----
' but the line below is the future:

Dim foo(FILL) As Long At Peek(Dword, Dictionary_Exists(myDict, myKey))
' or perhaps other syntax
Dim foo(FILL) As Long At Dictionary myPtr
Dim foo(FILL) As Long At String Dictionary_Find(myDict, myKey)
' whatever...

similar thoughts about LL... ;) - both would increase the amount of possibilities when it comes to dynamic types, subtypes, inheritage etc...
Both have functions that return a data-string. So that string is at least stored local to the function-result - even if not continous in dictionary or LL...and it's a string...

Edit: I think this would weave LL and Dictionary very much into Core. LL sadly does not return useable pointers...

[Petr Schreiber]

Thanks for the explanation Rene,

I think overlay over single item is no problem then :)


Petr

[ReneMiner]

...


String foo = "ABCDEFGH"
Dim arrA(FILL) As Long At String foo ' -- Overlays over string, UBOUND will be 2
...

...
Petr

perhaps "...At 'String'" is wrong here and we need a new keyword as "StrVar" so one could overlay a (local) function-result onto any string content directly using String-Keyword just like this:


Dim arrC(FILL) As Long At String = "ABCDEFGH"

String foo = "IJKLMNOP"
Dim arrD(FILL) As Long At StrVar foo

Dword kap = Heap_AllocByStr("12345678")
Dim arrE(FILL) As Long At Heap kap

Dim arrF(FILL) As Long At Dict pBucket '?

Dim arrG(FILL) As Long At String = LL_Data(myLL_Item)
'or
Dim arrH(FILL) As Long At LLItem LL_GetByNumber(myLL_Root, myItemNumber)

but I think the "At String ="-way will be read-only...
PS. I extended Heap_Get$ (http://www.thinbasic.com/community/project.php?issueid=412#top)-Example just "a tiny little bit"