some arrays goodies

[primo]

some array goodies
some features available only in thinbasic
Dim SomethingValue(500) As Long
SomethingValue(31)=31,32,33,34,35,36,37,38,39,40
the array from item 31 to 40 filled with these numbers
we can also use :
Array Assign SomethingValue(31)=31,32,33,34,35,36,37,38,39,40

Dim TextMatrix() As String 'dynamic string array
myText = "12_345_6789"
nLines = Parse(myText, TextMatrix, "_")
For i=1 To UBound(TextMatrix)
PrintL TextMatrix(i)
Next
the parse results fill the array items neatly without any extra code: note this :nLines = Parse(myText, TextMatrix, "_")

there are too many other goodies i will post from time to time
If you have other goodies about arrays you can add here
the complete example:

Uses "console"

Dim TextMatrix(), myText, nLines As String
Dim SomethingValue(500) As Long
Long i
SomethingValue(1)=1,2,3,4,5,6,7,8,9,10
SomethingValue(31)=31,32,33,34,35,36,37,38,39,40
Array Assign SomethingValue(401)=401,402,403,404,405
For i = 1 To 20
PrintL "index = "+Str$(i)+" "+SomethingValue(i)
Next
PrintL "******************************************"
For i = 31 To 40
PrintL "index = "+Str$(i)+" "+SomethingValue(i)
Next
PrintL "******************************************"
For i = 401 To 405
PrintL "index = "+Str$(i)+" "+SomethingValue(i)
Next
PrintL "******************************************"
myText = "12_345_6789"
nLines = Parse(myText, TextMatrix, "_")
For i=1 To UBound(TextMatrix)
PrintL TextMatrix(i)
Next

WaitKey

[kcvinu]

Hi,
Thanks for introducing these features in TB.

[Petr Schreiber]

Primo,

did you know you can easily list array values without for cycle? Check this:


long SomethingValue(500)

SomethingValue(1)=1,2,3,4,5,6,7,8,9,10
SomethingValue(31)=31,32,33,34,35,36,37,38,39,40
SomethingValue(401)=401,402,403,404,405

msgbox 0, join$(SomethingValue, ",", "0", 1, 10)
msgbox 0, join$(SomethingValue, ",", "0", 31, 40)
msgbox 0, join$(SomethingValue, ",", "0", 401, 405)


Petr

[ErosOlmi]

:D

I like those ... challenges

Here we have "virtual variable" usage. Fill an array, read data as a matrix using a "virtual" variable
Virtual variable are variable that do not allocate/deallocate any memory but use memory allocated for other variables to read/write data (also in different ways)


'---An array of 9 elements
Long SomeValue_Array(9)
'---A matrix of 9 elements that share the same memory of the memory allocated for the array
Long SomeValue_Matrix(3, 3) At VarPtr(SomeValue_Array(1))

'---Fill the array
SomeValue_Array(1)=1,2,3,4,5,6,7,8,9

'---Read as matrix
MsgBox 0, Join$(SomeValue_Matrix, ",", $CRLF)


Have fun.
Eros

[primo]

Thanks Petr and Eros, i was sure there are many golden eggs in thinbasic.
the classic programmer (like me) who are grown with QB or VB find these shortcuts and tips very strange, but it is in fact very useful and handy and time saving.
thanks

[axelmoe]

Yes absolutly true. Many many thanks

Kind regards,

Axel:D

[kcvinu]

Last day I had a little discussion on array ReDim in facebook with Petr Schreiber. My question is;
Assume that i am going to fill an array from an infinite source. (By infinite source, i mean, i don't know how many elements are there to fill my array.) So i have declared an array with one element and use ReDim in each time i put something to that array. i.e, Put something to that array and use ReDim to increase the size. Well, I know this will slow down my program execution. Then the other way which i know is, to declare a bigger array. say, 500 elements. In that case, what if my array filling operation completes in 200 elements ? Do i need to check the last element's position and delete the rest ?. I am seeking suggestion on this matter. Thanks in advance.
-kcvinu

[ErosOlmi]

It depends on data types and if array is one dimension or multiple dimensions (matrix).

With a single dimension array I would allocate, let's say, one thousand or one million elements.
Than fill the elements I need keeping track of the last.
If I need more, again Redim Preserve ... to other one thousand or one million elements.
At the end Redim Preserve ... to the last filled element.
You need to keep REDIM to the minimum because every REDIM is slower depending on how much memory is allocated and need to be reallocated, copied and free.

Another option is to use an Hash Table: http://www.thinbasic.com/public/products/thinBasic/help/html/index.html?hash_table.htm
If you use the array index number for the hash table key, is like having an array but with the flexibility of an hash table that dynamically allocates data when needed.
http://www.thinbasic.com/public/products/thinBasic/help/html/index.html?hash_set.htm
But again an Hash Table needs to have an idea on how many elements you need to create in order be be able to calculate hash values and reduce duplicated hash.

Another option is to use and AVL Tree.
No need to know number of elements in advance, just add data with a key easy to be retrieve.
http://www.thinbasic.com/public/products/thinBasic/help/html/index.html?tree_avl.htm

There are examples in \thinBasic\SampleScripts\DataStructures\ using Hash Tables and AVL Trees that fill/search/delete data in a blink of an eye

[primo]

i find this array strange (but very useful and handy) , it is used by Parse:
Dim NumbersMatrix() As Long
this syntax is not available in classic Basics
and then prepare the string myText
and then
nLines = Parse(myText, NumbersMatrix, "_")
it dimensions the NumbersMatrix as necessary according to the number of "_" in the myText, and since the MyText contains numbers and the NumbersMatrix() is of type Long then every array cell will have the number 12 or whatecer separated by "_"
i think it is possible to accept data (from mars as an example) as long string separated by space or "_" and then parse it to an array.
here is big string of one million numbers added to one million array cells (created internally by Parse)
i hope my talk about parse is correct .

Uses "console"
Dim NumbersMatrix() As Long
Dim myText As String
myText = Repeat$(1000000, "12_")
Long i, nLines
nLines = Parse(myText, NumbersMatrix, "_")
For i=990000 To UBound(NumbersMatrix)-1
PrintL Str$(i)+" "+Str$(NumbersMatrix(i)*2)
Next
WaitKey

[kcvinu]

@Eros,
Wow! AVL is superb.. very fast. Really, I love to code in thinBasic but now i can only do some experiments. Because all my projects are demanding unicode support. So eagerly waiting for the latest thinBasic version. Till that time, i would like to play with these kind of features. Thanks. :)

[ErosOlmi]

:oops: I'm working on it, slowly but I'm working.

[ReneMiner]

I have another suggest for those who have a homogeneous array of numbers that's elements count is unknown in advance but the size altogether will not exceed 2GB.
No redim needed if elements shall be added, all done by the powers of thinBasic.
Use a string.
How?


Uses "console"

' this will store the numbers:
Dim myNumericArray as String

' access the numbers virtually:
Dim myNumber() as Number At 0

Dim i As Long ' just to list our arrays elements

' add a few elements, type Ext equals Number, that's why MKE$:
myNumericArray = MKE$(0.12345) ' the first element be 0.12345
' just append a bunch of new elements:
myNumericArray &= MKE$(1,2,3,4,5)
' and some more
myNumericArray &= MKE$(6.666, 7.777)

' now let's read what we have:
' put the virtual array onto the string:
Redim myNumber(Strptrlen(Strptr(myNumericArray))/SizeOf(Number)) At Strptr(myNumericArray)

' list what we have
For i = 1 to UBound(myNumber)
PrintL i, myNumber(i)
Next

WaitKey

Just an idea. Typed this on the mobile, not error-checked. Perhaps the $-sign is wrong on the phone? If so someone may correct it.
But you could tell me if string reallocates faster than Redim :)

[Petr Schreiber]

Rene,

thank you for your example! This approach could be further mixed with stringBuilder to reach maximum efficiency.
Each &= means reallocation in your case, while stringBuilder reallocates only when you reach its internal capacity, which keeps doubling :)


Petr

[kcvinu]

@Petr Schreiber,
Could you please post an example with StringBuilder ?

[primo]

@kcvinu
until Petr reply, i say i have tried string builder just recently, and it is very efficient, here its thread http://www.thinbasic.com/community/showthread.php?12447-StringBuilder-for-ThinBASIC-GitHub
look the Petr example there
while its page is here: https://github.com/petrSchreiber/thinBasic_StringBuilder/wiki
look more of its methods available
it is genuine and in the github page is its source code
to create it:

'Dim stringBuffer As String
Dim sb As StringBuilder
sb = New StringBuilder()

and to build big string from some words

For i = 1 To 20000
sb.Add("Appending is fine!")
Next

the result is instantaneous
here is we save the big string to a file because the Console can't display more than a few thousands of charaters)
note what in the string builder we copy to a usual string
bigString = sb.ToString() to display it.

Uses "StringBuilder", "Console", "File"

Function TBMain()

'Dim stringBuffer As String
Dim bigString As String
Dim sb As StringBuilder
sb = New StringBuilder()

Long i

For i = 1 To 20000
sb.Add("Appending is fine!")
Next

bigString = sb.ToString()

FILE_Save("test.txt", bigString)

PrintL "big string saved as file test.txt .... Press any key to quit..."

WaitKey

End Function

[kcvinu]

@primo,
Thanks for the example. Its really fast for creating and saving that 352 KB text file. I think the variable named Stringbuffer is useless in your code. Let me check the links you provide. Thanks again. :)

[primo]

Thank you kcvinu for the notes.
i have tried also to loop up to a million loops for the sentence of 20 characters. and using AddLine method (listed in page https://github.com/petrSchreiber/thinBasic_StringBuilder/wiki ) which adds a new line after reading one string. in one second it saves about 21 MB txt file.
but never try to open this big text file in notepad it will hang your computer. a good editor i see to open these big files is EmEditor
https://www.emeditor.com/text-editor-features/large-file-support/files-up-to-248gb/
but the notepad++ can open it ( https://notepad-plus-plus.org/download/ ) note that opening the files which have many new lines is easier than continuous file.

Uses "StringBuilder", "Console", "File"

Function TBMain()

'Dim bigString As String
Dim sb As StringBuilder
sb = New StringBuilder()

Long i

For i = 1 To 1000000
sb.AddLine("Appending is fine!!!")
Next

'bigString = sb.ToString()

FILE_Save("test.txt", sb.ToString())'bigString)

PrintL "big string saved as file test.txt .... Press any key to quit..."

WaitKey

End Function


there are other things to discover yet. thanks for Petr and Eros for this jewel .

[kcvinu]

@primo,
Wow ! one million lines of text. Obviously it's a huge file. I am assuming that you might see your computer hung when you opened it in notepa. Am i right ?

[primo]

kcvinu wrote: I am assuming that you might see your computer hung when you opened it in notepa. Am i right ?
in windows xp the notepad does not hung if we created the txt file using sb.AddLine even with slow performance , but it hung and freeze if we created the file with sb.Add ie as one line. to be sure the end of the file displayed add "finish":
9663
For i = 1 To 1000000
sb.Add("Appending is fine!!!")
Next
sb.Add("finish")
the universal viewer is able to load any file
the freeware version of universal viewer is here:
http://www.uvviewsoft.com/uviewer/download.htm

but only EmEditor is able to show the 20 MB one line file as one line, when we move the cursor to the end of the file we see this
ln 1 col 20000006
ie line 1 column 20,000,006

[kcvinu]

@primo,
Thanks a lot for introducing Universal Viewer. Its a must have program. :)

[Petr Schreiber]

Hi,

primo basically nailed the stringBuilder usage, here a little example:


Uses "console", "stringBuilder"

' this will store the numbers:
Dim builder as new StringBuilder()

' access the numbers virtually:
Dim myNumber() as Number At 0

Dim i As Long ' just to list our arrays elements

' add a few elements, type Ext equals Number, that's why MKE$:
builder.Add(MKE$(0.12345)) ' the first element be 0.12345
' just append a bunch of new elements:
builder.Add(MKE$(1,2,3,4,5))
' and some more
builder.Add(MKE$(6.666, 7.777))

' now let's read what we have:
' put the virtual array onto the string:
string builderData = builder.ToString()
Redim myNumber(Strptrlen(Strptr(builderData))/SizeOf(Number)) At Strptr(builderData)

' list what we have
For i = 1 to UBound(myNumber)
PrintL i, myNumber(i)
Next

WaitKey


I may add direct pointer to string in stringBuilder as functionality, if there is interest. That would skip the step with assignment to temporary string variable, making it even faster :)


Petr

[ErosOlmi]

I may add direct pointer to string in stringBuilder as functionality, if there is interest
It would be great.

[ReneMiner]

If I understood right the stringbuilder allocates more size than the actual length of the string. But sb.Length() then returns the number of chars that are occupied.
Then assuming sb gets a new method named DataPtr() it were like


Redim myNumber(sb.Length/SizeOf(Number)) At sb.DataPtr


?
However we would access the single elements of the array myNumber using parenthesis and index. No matter what type myNumber really is, even fixed-size Udt can be used.
SizeOf(...) tells how many bytes we need, the index of myNumber(...) tells which bytes we need.

Let's say we have the array as in the example stored to builder and we want to list the arrays elements without the use of a virtual variable.

Cumbersome we have to calculate:


For i = 1 To builder.Length/SizeOf(Number)
PrintL i, CVE(Memory_Get(builder.DataPtr + (i - 1) * SizeOf(Number), SizeOf(Number)))

Next


How about methods to stringbuilder similar to current _.Char() but
_.Element(index, SizeOf(datatype)) : return an elements data, means a sequence of bytes
_.Elements(SizeOf (datatype)) : return UBound of elements stored to sb



For i = 1 to builder.Elements(SizeOf(Number))
PrintL i, CVE (builder.Element( i, SizeOf(Number)))

Next


or even better tell stringbuilder the SizeOf elements or the required datatype and it will remember it:



Dim builder As New stringbuilder

builder.SetType(Number) ' default Byte/Char
' or
builder.ElementSizeSet(SizeOf(Number)) ' default 1, don't want division by 0...
builder.Add(MKE$(0.123))
builder.Add(MKE$(1,2,3,4))
builder.Add(MKE$(5.5,6.6,7.7))
For i = 1 To builder.Elements()
PrintL i, CVE(builder.Element(i)))
' ...


I don't know if possible for Udt if using _.SetType, but having to give the size once only instead of repeating it every time were great

@ Eros,
how about
Alias MKE$ As MKNumber$
Alias CVE As CVNumber
For better readability and understanding?

[ErosOlmi]

@ Eros,
how about
Alias MKE$ As MKNumber$
Alias CVE As CVNumber
For better readability and understanding?

Yes,good point.
I will add.

[Petr Schreiber]

Hi friends,

I added the DataPtr property (http://www.thinbasic.com/community/showthread.php?12447-StringBuilder-for-ThinBASIC-GitHub&p=93448&viewfull=1#post93448), which allows you to directly access the internal buffer of stringBuilder object!

@Eros: Can we please have this in the next release of thinBasic?


Petr