Maximum width of canvas = 1348921, but why?

[EmbeddedMan]

OK, here I am with my silly virtual canvas questions again.

Here is a sample program:


Uses "UI"
Uses "OS"
Uses "GDIP"

'---Constant declarations
Begin ControlID
%ID_Canvas,
End ControlID

%CANVAS_LENGTH = 1348921 ' Width of graph in pixels
%CANVAS_HEIGHT = 380 ' Height of graph in pixels
%VP_WIDTH = 1000 ' Width, in pixels, of viewport into canvas

'------------------------------------------------------------------------------
' Program start point
'------------------------------------------------------------------------------
Function TBMain() As Long
Local hDlg As DWord '---Used to store window handle of main dialog

'---Create a new dialog
hDlg = Dialog_New Pixels, 0, "Canvas Test", -1, -1, 1020, 420,
%WS_DLGFRAME |
%DS_CENTER |
%WS_CAPTION |
%WS_SYSMENU |
%WS_OVERLAPPEDWINDOW


'---Show dialog in modal mode
'---cbDialog function is the callback function handling dialog events
'---Application control will pass to dialog callback till dialog will exists
Dialog Show Modal hDlg, Call cbDialog

End Function

'------------------------------------------------------------------------------
' Callback procedure for main window
'------------------------------------------------------------------------------
CallBack Function cbDialog() As Long
Select Case CBMSG

Case %WM_INITDIALOG
Local i As Long

'---Add controls
Control Add Canvas, CBHNDL, %ID_Canvas, "", 10, 10, %VP_WIDTH, %CANVAS_HEIGHT + 20, %WS_BORDER | %WS_CHILD | %WS_VISIBLE | %SS_NOTIFY, Call cbDialog

Canvas_Attach(CBHNDL, %ID_Canvas, TRUE)
' Set the canvas to be very wide (virtualy)
Canvas_SetVirtual(%CANVAS_LENGTH, %CANVAS_HEIGHT)
Canvas_SetView(0, 0)

' Write something on the canvas so we can know when it moves
For i = 1 To %CANVAS_LENGTH Step 100
Canvas_SetPos(i, 100)
Canvas_Print(i)
Next i

End Select

End Function

Now, if you run this program, it will work just fine. You can use the thumb to drag the canvas underneath the viewport, and you can see all of the numbers (showing that you have access to the whole canvas).

But if you use %CANVAS_LENGTH = 1348922 (just one more than is in the program above) then it will not work. The canvas will not scroll around, and you can't see any more of the numbers.

But why? My computer has tons of RAM (32GB). Could it be possible to use canvases that are much, much bigger than this? (say 21600000 pixels wide) I have 250 data points coming in from my embedded system every second, and I'd like to be able to graph 24 hours of data on one canvas.

Everything works wonderfully up to 1348921.

I think it has to do with the total size (area) of the canvas, because if I reduce the canvas height I can increase the width.

Is it because 1348922 * 380 * 4 is getting close to 2^31?

*Brian

[ErosOlmi]

A lot of data.

Yes, problem is allocated memory.
thinBasic is a 32bit application and has the limit of 2GB per process. Even if you have plenty of memory, every 32bit process has this limit.

You need to find another strategy.
You need to have data stored somewhere, show some rolling screen that shows only the last x minutes.
When you need to show the whole day or more, reload data and show them.
To store data you can use new SQLite module and manage a local DB where to put all the data you get.

A quick and dirty way to go further the limit of 1348921 pixels wide is to reduce the height of the Canvas.
Set %CANVAS_HEIGHT = 100 (for example) and you will be able to set a much bigger %CANVAS_LENGTH.

[EmbeddedMan]

Ahh! Very good. That makes complete sense. Thank you.

Expanding the canvas was just my lazy attempt at getting more recording time. I will have to bite the bullet and do things right! :D

What are the limits to arrays? Can I, for example, have a long array that has 1073741823 elements (2GB in size)? Or do all variables in a TB program together need to fit into 2GB?

*Brian

[primo]

what about 2 canvases !!! when the graphics reaches the end of the first wide canvas it begins to plot to the second canvas.

i don't want to cover your last question, here it is again since it is important:

What are the limits to arrays? Can I, for example, have a long array that has 1073741823 elements (2GB in size)? Or do all variables in a TB program together need to fit into 2GB?

*Brian


Uses "UI"

Dim x,y As Double

Begin ControlID
%ID_Canvas
%ID_Canv2
End ControlID

Function TBMain() As Long
Local hDlg As DWord '---Used to store window handle of main dialog

hDlg = Dialog_New Pixels, 0, "canvas wide graphics ", -1, -1, 1200, 775,
%WS_DLGFRAME |
%DS_CENTER |
%WS_CAPTION |
%WS_SYSMENU |
%WS_OVERLAPPEDWINDOW

Dialog Show Modal hDlg, Call cbDialog
End Function

CallBack Function cbDialog() As Long
Select Case CBMSG
Case %WM_INITDIALOG
Control Add Canvas, CBHNDL, %ID_Canvas, "", 50, 10,1000, 200, %WS_BORDER | %WS_CHILD | %WS_VISIBLE
Control Add Canvas, CBHNDL, %ID_Canv2, "", 50, 210,1000, 200, %WS_BORDER | %WS_CHILD | %WS_VISIBLE
Canvas_Attach(CBHNDL, %ID_Canvas, %TRUE)
Canvas_SetVirtual(1000, 380)
Canvas_SetView(0,0)
Canvas_Width(2)
Canvas_Clear %BLACK
Canvas_SetView(1000,0)

'Canvas_Scale(-20, -5, 20, 20)

For x = 0 To 2000 Step 0.1

y = Sin(x)*20* Rndf(1, 2)*Sin(x/3)^2

Canvas_Width(3)

Canvas_Line( (x*10, y +100), (x*10, y +100) , Rgb(255, 255, 0) )

Canvas_Redraw

Next

''''wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
Canvas_Attach(CBHNDL, %ID_Canv2, %TRUE)
Canvas_SetVirtual(1000, 380)
Canvas_SetView(0,0)
Canvas_Width(2)
Canvas_Clear %BLACK
Canvas_SetView(1000,0)

'Canvas_Scale(-20, -5, 20, 20)
'mark the beginning of canvas 2 graphics with a circle
Circle(10,70,10,Rgb(0,255,0),Rgb(255,0,0))
For x = 0 To 2000 Step 0.1

y = Sin(x)*20

Canvas_Width(3)

Canvas_Line( (x*10, y +100), (x*10, y +100) , Rgb(255, 255, 0) )
Canvas_Redraw
Next
End Select
End Function

Function Circle(x As Long, y As Long, r As Long, colr, colrFill)
Canvas_Ellipse(x-r, y-r, x+r, y+r, colr, colrFill)
End Function

[ErosOlmi]

What are the limits to arrays? Can I, for example, have a long array that has 1073741823 elements (2GB in size)? Or do all variables in a TB program together need to fit into 2GB?


Limit is per process.
You have to consider that thinBasic Core engine has its memory needs plus every module has its memory needs.
Sum all together and subtract from 2GB: the rest is what remains to your script.

I think you have to consider also security: imagine you have all data in memory and you application crash (whatever application not only thinBasic).
What about your data? You have lost all.

Better strategy would be to store data into some persistent support (files or local DB or remote DB) as first step, than read back data to show it.
One application can get data and store, another application can read back data and show.

[ErosOlmi]

Regarding drawing data into a Canvass, it is better to draw it inside a Timer event and not during %WM_INITDIALOG event.
Otherwise you will not get any window until your drawing is finished and your app will quite sure hangs.

So during %WM_INITDIALOG event just setup your window and controls and create a timer.
Then draw during timer events.

[EmbeddedMan]

As an aside, for anybody who's interested:

One of the functions of my application is to graph 5 different variables for a long period of time (currently about 80 minutes), and then save off all of the data to a .csv file with my app the way it is now as well as a .bmp image of the graph (maxed out canvas width with 380 pixel height).

So TB has no problem with this - and I'm generating 1348921 x 380 pixel .bmp files just fine. It's so fantastic to be able to have a complete record of everything the user saw during the run.

Thanks TB!

*Brian

[ErosOlmi]

eheh ...
maybe one day you will let us know about how look your applications :)
We are very curios.

[EmbeddedMan]

Limit is per process.
You have to consider that thinBasic Core engine has its memory needs plus every module has its memory needs.
Sum all together and subtract from 2GB: the rest is what remains to your script.

I think you have to consider also security: imagine you have all data in memory and you application crash (whatever application not only thinBasic).
What about your data? You have lost all.

Better strategy would be to store data into some persistent support (files or local DB or remote DB) as first step, than read back data to show it.
One application can get data and store, another application can read back data and show.


Yes, ideally, that is the case. That requires more complex internal operation. One of the requirements of this program is that the entire graph (all 24 hrs of it, if possible) be 'scrollable' at all times. So the user should be able to scroll back to any point in the graph, zoom in and out, all while new data is coming in. I.e. "live".

As it is right now, I'm also saving every new bit of data off into the .csv log file, which (after the fact) can be loaded into the program (canvas) and scrolled around to see what happened during the run. This actually works extremely well. So I'm not worried about data that's in RAM getting lost during a crash.

It's very good to know about the 2GB total limit - TB has no way to allocate or use memory outside of this limit, correct? That's just fine - I will need to be clever with how I store and display things if I want to run for longer periods.

[EmbeddedMan]

eheh ...
maybe one day you will let us know about how look your applications :)
We are very curios.

Sure thing! Here is a very simple run I did for just a few seconds at my desk (not inside a human heart - those tests are very expensive). I called it "PIGrapher" because the first use for the tool was to allow me to tune the PI parameters for our motor speed control. (We only use PI, not PID for this device.)

Of course, now people at work see how useful the tool is, and a lot more people want to use it for many other uses! I consider that a success. But with success comes lots of new feature requests, like "can it graph for 24 hours?" :D

9811

*Brian

[ErosOlmi]

TB has no way to allocate or use memory outside of this limit, correct?
Correct.

Only way is to develop a thinBasic 64bit but actually is impossible because the compiler I use for development, PowerBasic, is only 32bit and there are no plan to release a 64bit version.
I'm considering to abandon PowerBasic for something else but will be a looooong way

[EmbeddedMan]

Correct.

Only way is to develop a thinBasic 64bit but actually is impossible because the compiler I use for development, PowerBasic, is only 32bit and there are no plan to release a 64bit version.
I'm considering to abandon PowerBasic for something else but will be a looooong way

OK, that is very good to know. No worries at all - I'm certain I can rework my application to get much longer recording times while still staying within the 2GB limit.

*Brian

[ErosOlmi]

An example like the one attached can be useful?
Is a small program showing a continuously moving display of curves without storing any data.

I've made few changes in UI module to allow such scrolling so before to be able to release I need to release a new thinBasic version.

9814

[EmbeddedMan]

An example like the one attached can be useful?
Is a small program showing a continuously moving display of curves without storing any data.

I've made few changes in UI module to allow such scrolling so before to be able to release I need to release a new thinBasic version.

9814

Very cool! That would be a very useful mode for my application - infinite recording ability (streaming data to disk .csv file) but you only get 1 'screenful' of graph to see.

Is your app using a canvas? (yes, obviously, since you named it canvas.)

What changes did you to to enable this functionality?

*Brian

[ErosOlmi]

Hi Brian,

I'm still not ready to release a new thinBasic version because I'm in the middle of a big change. So here how to create the above example just replacing UI module.

I've added "Canvas_GetSize(Width, Height)" in UI module. The rest is something that was already there in UI module.
Attached thinBasic_UI.dll. Please copy it into \thinBasic\Lib\ directory replacing your current one. Please have at least thinBasic 1.10.4 installed: http://www.thinbasic.com/community/showthread.php?12778-thinBasic-1-10-x

Here Canvas_Continuous example


' Here is a small program showing a continuously moving display of curves.
' This may be useful for monitoring any real time data.

Uses "UI"

begin ControlID
%IDC_GRAPHIC1
%IDC_BUTTON1_EXIT
%IDC_BUTTON1_STRT
%IDC_BUTTON1_STOP
%IDC_TIMER1
end ControlID

CallBack Function ShowDIALOG1Proc()

Static TimerInterval As Long
Static WidthVar, HeightVar, Xstep As Long
Static StationaryXpos, yPosRed, yPosBlue, yPosGreen As Single
Static prevYposRed, prevYposBlue, prevYposGreen As Single
Static xRightSpace As Long
Static hWndSaveFocus As Dword

Select Case CBMsg
Case %WM_INITDIALOG
' Initialization handler
TimerInterval = 30 ' in microseconds - can be changed
Canvas_GetSize WidthVar, HeightVar

Xstep = 2 ' number of pixels moving to the left per time inteval
Canvas_Width Xstep

xRightSpace = 10 ' blank space on right side of curves in pixels - can be changed
StationaryXpos = WidthVar - xRightSpace

Randomize Timer
yPosRed = Rnd(2, HeightVar - 2)
yPosBlue = Rnd(2, HeightVar - 2)
yPosGreen = Rnd(2, HeightVar - 2)
prevYposRed = yPosRed
prevYposBlue = yPosBlue
prevYposGreen = yPosGreen

CONTROL ENABLE CBHndl, %IDC_BUTTON1_STRT
CONTROL DISABLE CBHndl, %IDC_BUTTON1_STOP

Case %WM_NCACTIVATE
If IsFalse CBwParam Then
' Save control focus
hWndSaveFocus = win_GetFocus
ElseIf hWndSaveFocus Then
' Restore control focus
win_SetFocus(hWndSaveFocus)
hWndSaveFocus = 0
End If

Case %WM_COMMAND
' Process control notifications
Select Case CBCtl
Case %IDC_BUTTON1_EXIT
If CBCtlMsg = %BN_CLICKED Then
Dialog End CBHndl
end If

Case %IDC_BUTTON1_STRT
If CBCtlMsg = %BN_CLICKED Then
CONTROL DISABLE CBHndl, %IDC_BUTTON1_STRT
CONTROL ENABLE CBHndl, %IDC_BUTTON1_STOP

DIALOG SET TIMER CBHndl, %IDC_TIMER1, TimerInterval
Randomize Timer
End If

Case %IDC_BUTTON1_STOP
If CBCtlMsg = %BN_CLICKED Then
CONTROL DISABLE CBHndl, %IDC_BUTTON1_STOP
CONTROL ENABLE CBHndl, %IDC_BUTTON1_STRT
DIALOG KILL TIMER CBHndl, %IDC_TIMER1
end If

End Select

Case %WM_TIMER
Select Case CBwParam
Case %IDC_TIMER1
'---Kill the timer until we have finished drawing
'---Keeps drawing time as low as possible
DIALOG KILL TIMER CBHndl, %IDC_TIMER1

Canvas_BitmapCopy3 CBHndl, %IDC_GRAPHIC1, Xstep + 1, 1, WidthVar - xRightSpace, HeightVar,1, 1
Canvas_Line( (WidthVar - xRightSpace, 1), (WidthVar - xRightSpace, HeightVar), %White) ' erase previous points

yPosRed = yPosRed + Rnd(2,8) * Sgn(Rnd(-yPosRed+2, HeightVar-2-yPosRed)) + 3 ' You can exchange these
yPosBlue = yPosBlue + Rnd(2,8) * Sgn(Rnd(-yPosBlue+2, HeightVar-2-yPosBlue)) ' random functions with
yPosGreen = yPosGreen + Rnd(2,8) * Sgn(Rnd(-yPosGreen+2, HeightVar-2-yPosGreen)) - 3 ' any real time measurements

Canvas_Line ( (StationaryXpos - Xstep, prevYposRed), (StationaryXpos, yPosRed), %Red)
Canvas_Line ( (StationaryXpos - Xstep, prevYposBlue), (StationaryXpos, yPosBlue), %Blue)
Canvas_Line ( (StationaryXpos - Xstep, prevYposGreen), (StationaryXpos, yPosGreen), %RGB_GREEN)

prevYposRed = yPosRed
prevYposBlue = yPosBlue
prevYposGreen = yPosGreen

'---Restore timer after drawing
DIALOG SET TIMER CBHndl, %IDC_TIMER1, TimerInterval

End Select
End Select
End Function

Function TBMain()
Local hDlg As Dword

Dialog New Pixels, %HWND_DESKTOP, "Continuously Moving Curve Display",
-1, -1, 543, 382,
%WS_POPUP | %WS_BORDER | %WS_DLGFRAME |
%WS_CAPTION | %WS_SYSMENU | %WS_MINIMIZEBOX | %WS_CLIPSIBLINGS |
%WS_VISIBLE | %DS_MODALFRAME To hDlg

Control Add Canvas, hDlg, %IDC_GRAPHIC1, "", 8, 8, 528, 336', %WS_CHILD Or %WS_VISIBLE
canvas_Attach hDlg, %IDC_GRAPHIC1
Canvas_Color -1, %White
Canvas_Clear

Control Add Button, hDlg, %IDC_BUTTON1_EXIT, "E&xit", 456, 352, 80, 24
Control Add Button, hDlg, %IDC_BUTTON1_STRT, "&Start", 8, 352, 112, 24
Control Add Button, hDlg, %IDC_BUTTON1_STOP, "St&op", 128, 352, 112, 24

Dialog Show Modal hDlg, Call ShowDIALOG1Proc

End Function

[primo]

Thanks Eros for this canvas exciting feature and the educative example, it is suitable for a huge number of situations such as monitoring earthquakes data , vibrations before volcanoes eruptions, and heart pulses.

[ErosOlmi]

eheh :D

Really happy how from a simple request can come out many ideas.
That's what I think passionate all of us.

I have some more interesting examples on Canvas but I will publish next week because I will have a working week-end at the company.
We have an BIIIG important go-live in march and we are a bit late on schedule.

[primo]

Hi Eros, regarding the experimental thinBasic_UI.dll
when we try the example C:\thinBasic\SampleScripts\UI\RichEdit\RichEdit_001.tbasic it outputs an error
line number 130
line code: "CONTROL ADD RTF_GETCLASS, HDLG, %ID_RICHEDIT,....%WS_EX_CLIENTEDGE
token found: %WS_EX_CLIENTEDGE
when we comment %WS_EX_CLIENTEDGE and open a sample.rtf file it is not displayed
i post this report for your attention but i know the above thinBasic_UI.dll is not official and for testing the Big Canvas only in TB 1.10.4.0

[ErosOlmi]

Thanks a lot.
I will have a look.

I would like to release a new complete thinBasic version 1.10.5 but quite busy at work.
I hope to be able to release by this week-end

Ciao
Eros

[ErosOlmi]

Ciao Primo,

here it is working fine under Windows 10 :confused:
What OS are your using? RichEdit is a complex control loaded from different DLL depending on the OS version.
9825

[primo]

my OS is windows xp/32 . when i replace the new thinBasic_UI.dll which have size 170kb with the original thinBasic_UI.dll size 156kb i can run again the RichText control.
if the thinbasic update needs dropping support for richText in winxp then thats okay , i use usually the TextBox which is usually easier to use than RichText. i prefer the Continuous canvas feature than the rich Text.
in fact i have windows 7/64bit installed in the same machine but i don't use it, my brain is hard wired with windows xp like millions of other people

[ErosOlmi]

No, no: I didn't want to force to move, just needed to understand to have a clue on where to check.
I too still use XP at work due to some old software still needed.

I will investigate because in reality I didn't change anything on how the RichEdit Dll is loaded from the OS.

[catventure]

As you know I use the richedit control in the TAB program...

Anyhow, just letting you know I tried the richedit001 sample program and it worked OK on Vista

(with or without the %WS_EX_CLIENTEDGE)

Using thinbasic 1.10.0.0

[primo]

i have rebooted and tried also in windows 7/64bit on the same machine and also issue the same error with RichEdit examples, while the original DLL for TB 1.10.4 works okay
here is a screen shot from windows xp
https://s18.postimg.org/87jlk8oux/error.png

just a guess in the dark: what if the compression utility used to compress the dlls somehow ...(Fuzzy Logic sometimes works !!!). don't know since the catventure report it works in vista

[ErosOlmi]

Thanks to all.
I think it is related to new or changed SDK functions between Core engine and Modules I'm developing.
Please wait till I will release complete thinBasic 1.10.5