Word list maker

[sandyrepope]

This is a program I wrote just to learn thinbasic. I don't know if anyone will want it but, at least, it is free.

Thanks
Sandy

I Think I've got the bug fixed and am uploading a new version.

I found another bug and am uploading a newer version.

[Petr Schreiber]

Hi Sandy,

very useful proggie !

Just one thing - when I open file for analyse then it loads just its first line. And when I want open another it does not respond. I have no idea why it does not work, I will check it more.

To workaround this I used "new" CountWord function:


function CountWord()

local sBuffer as string = FILE_LOAD(sFile)
local sLines() as string
local sWordsNew() as string, sWordsID as long
local i, n, c as long

n = parse(sBuffer, sLines, $CRLF)

for i = 1 to n
A_line = sLines(i)
' -- First we will replace "weird" characters with spaces
A_line = Replace$( A_line, any chr$(0 to 31)+"0123456789:-|(){}&#91;]';,.?!/\^!@#$%^&*_<>=+"+$DQ+CHR$(123 to 255), with $SPC)
' -- Then we will reduce multiple spaces to single space
A_line = trimfull$(A_line)

if len(A_line) > 0 then
' -- Number of words ? Just parsecount ...
A_count = parsecount(A_line, any $SPC + $crlf)
Word_Count = Word_Count + A_count

if A_count = 1 then
control append text hDlg, %ID_TXTBUFFER, lcase$(A_line)
control append text hDlg, %ID_TXTBUFFER, $crlf
else
WC = SPLIT(A_line," ",Word_Array)
for c = 1 to WC
control append text hDlg, %ID_TXTBUFFER, lcase$(Word_Array(c))
control append text hDlg, %ID_TXTBUFFER, $crlf
next
endif
endif
next

end function



Thanks,
Petr

[Michael Clease]

I think i know, in the word count function sandy tries to close the file handle by passing the filename (sFile) instead of the handle (fHandle)


File_close(sFile)

should be

File_Close(fHandle)

[Petr Schreiber]

Good eye Abraxas !,

but it still behaves a bit odd - it seems to me that it never leaves:


while FILE_EOF(fHandle) = %FALSE
...
wend


Maybe there is problem in some of the file functions - it reads just first line and then it continues to return null length string.


Bye,
Petr

[Michael Clease]

function CountWordV2()
LOCAL FileBuffer as STRING
FileBuffer = file_LOAD(sFile)
' -- First we will replace "weird" characters with spaces
' any chr$(0 to 43)+";.?!/\^"+CHR$(58 to 255), with $SPC
FileBuffer = Replace$( FileBuffer, any chr$(0 to 31)+"0123456789:-|(){}&#91;]';,.?!/\^!@#$%^&*_<>=+"+$DQ+CHR$(123 to 255), with $SPC)
' -- Then we will reduce multiple spaces to single space
FileBuffer = trimfull$(FileBuffer)
WC = SPLIT(FileBuffer," ",Word_Array)
control Set text hDlg, %ID_TXTBUFFER, ""
for Loop_Count = 1 to ubound(Word_array)
control APPEND text hDlg, %ID_TXTBUFFER, lcase$(Word_Array(Loop_Count))
control append text hDlg, %ID_TXTBUFFER, $crlf
next
end function

try that its a bit quicker.

[Michael Clease]

try changing fHandle from a STRING to a LONG.

should defining variables AS LOCAL outside functions (in the main loop) make any difference to program execution.

[sandyrepope]

Sorry about the program not working. I must have made some changes that aren't working. I'll work on it today and get it working right today. Then I'll post a version with the bugs fixed.

Thanks
Sandy

[Petr Schreiber]

Hi Sandy,

no problems with some little bugs.
I am looking forward to new version, Abraxas pointed out the most criticall parts very profesionally and fast!


Bye,
Petr

P.S. ( Sounds of squeaking tooth ) yes your version is much faster Abraxas :)
I wanted to not change the source too much to keep little changes watchable, but your solution is the best for this job.

[Petr Schreiber]

Ok,

this could be even faster, because we will add text in one go and not each word separately ;):


function CountWord()

LOCAL FileBuffer as STRING = LCASE$(file_LOAD(sFile))

' -- First we will replace "weird" characters with spaces
' any chr$(0 to 43)+";.?!/\^"+CHR$(58 to 255), with $SPC

FileBuffer = Replace$( FileBuffer, any chr$(0 to 64)+chr$(91 to 96)+CHR$(123 to 255), with $SPC)
' -- Then we will reduce multiple spaces to single space
FileBuffer = trimfull$(FileBuffer)
WC = PARSE(FileBuffer, Word_Array, $SPC)

control Set text hDlg, %ID_TXTBUFFER, join$(Word_Array, $CRLF)

end function


Bye,
Petr

[Petr Schreiber]

To not list one word more times use this version,
it also does A-Z sort of the words found so it is easier to browse :)


function CountWord()

local i as long
LOCAL FileBuffer as STRING = LCASE$(file_LOAD(sFile))
' -- First we will replace "weird" characters with spaces
' any chr$(0 to 43)+";.?!/\^"+CHR$(58 to 255), with $SPC

FileBuffer = Replace$( FileBuffer, any chr$(0 to 64)+chr$(91 to 96)+CHR$(123 to 255), with $SPC)
' -- Then we will reduce multiple spaces to single space
FileBuffer = trimfull$(FileBuffer)
WC = PARSE(FileBuffer, Word_Array, $SPC)

' -- We will sort the array in A-Z order
array Sort Word_Array(), ascend
' -- ... to group same words together for easier elimination of duplicates
for i = WC to 2 step -1
if Word_Array(i) = Word_Array(i-1) then Word_Array(i) = ""
next
' -- Duplicates were replaced with empty strings so we need to sort again
array Sort Word_Array(), ascend
control Set text hDlg, %ID_TXTBUFFER, trim$(join$(Word_Array, $CRLF), $CRLF)

end function


Bye,
Petr

[Michael Clease]

now thats just showing off. :P

[Petr Schreiber]

Ok,
I stop here.
But first corrected versions lacked the check for multiple occuriences, I had to fix it, otherwise I could not sleep in peace anymore ;D


Bye,
Petr

[sandyrepope]

I wasn't sure how to just replace the zip file so I just uploaded the newer version. It is the second one listed.

Abraxas,

try changing fHandle from a STRING to a LONG.

I tried this and it seems to have fixed the program although I don't know why this would work.


I think i know, in the word count function sandy tries to close the file handle by passing the filename (sFile) instead of the handle (fHandle)

I made this change also.
Thanks you.

Please let me know if this works on your computer now. It works on mine.

Thanks
Sandy

[Petr Schreiber]

Hi sandy,

thanks a lot for corrected version, it works perfectly!

Just for bigger files ( or better - files with lot of words ) the continuous adding of words to textbox during loading could slow down the time a lot. My test file ( 2 393 bytes ) takes with latest script 2109 ms to load, optimized version above about 15ms - just because putting the text at once.

To upload only the last version you just have to uncheck old verision in "Additional Options" and "Attach:" the new one.


Thanks a lot,
Petr

[kryton9]

Sandy, very neat program. Abraxas and Petr, you guys really added to it too. Getting caught up in graphics programming forget all the useful other types of applications one can write. Thanks Sandy for showing and sharing of a creative program!

[Michael Clease]

Sandy in the help file for FILE_OPEN it declares that you pass a STRING (filename), STRING (mode) and returns a NUMBER (a unique number (filelock) reference), you were using a string by mistake. I've done it loads of times.

perhaps Eros could edit the help files to a specific minimum data type.

I have found a few that I wasnt sure what was going to be returned do i use a LONG, WORD, DWORD, QUAD.

P.S. Sandy i really like the idea behind the program it looks quite interesting. I didnt know menus were that easy. i was going to write my level editor with TBGL menus and buttons but i am now rethinking.

p.p.s thanks Kryton.

[sandyrepope]

P.S. Sandy i really like the idea behind the program it looks quite interesting. I didnt know menus were that easy. i was going to write my level editor with TBGL menus and buttons but i am now rethinking.

p.p.s thanks Kryton.

Thank you for the kind words. I really can't take credit for the menu as all I did was use the one in samplescripts/ui/menu. I just changed it to fit my needs.

Thanks
Sandy

[sandyrepope]

I tried to use my program earlier today and found that it won't run. I haven't yet been able to find out what is wrong with it but when I do I'll put up a new version.

Sorry for any inconvenience this might cause. At first glance, it appears to be a problem with the menu system.

Thanks
Sandy

[Michael Clease]

Its working fine for me, both the count and main program.

version 1.6.0.9 of TB.

[sandyrepope]

After some more checking I think it might be a problem with my windows xp-home. I get the information I've attached to this reply. Unfortunately, I don't understand any of it.

Sandy

[ErosOlmi]

That error is about a GPF (General Protection Fault) occurred during execution. In particular, GPF occurred in OLE32.DLL that is one of the most important DLL of the Operating System. This doesn't mean 100% that the problem is inside OLE32.DLL, maybe (for example) some other application (in this case thinBasic.exe) has called a function inside OLE32.DLL with the wrong params. Understanding what's going on is not so simple.

Your scripts are working fine here too so do not know what to say.
Maybe if you give the exact point where you get GPF can help me to investigate a bit.

Ciao
Eros

[Petr Schreiber]

Hi Sandy,

I have Windows XP SP2, with all updates before SP3, and no problem.
Did you upgraded to SP3 already?


Thanks,
Petr

[sandyrepope]

Maybe if you give the exact point where you get GPF can help me to investigate a bit.

It seems to be line 63 which is: MENU ADD POPUP, hMenu, "File", hPopup1, %MF_ENABLED


Did you upgraded to SP3 already?

No.

What I don't understand is that the menu under UI in the samplescripts works just fine.

Thanks
Sandy

[Petr Schreiber]

Sandy,

can you try to run the script not using F5 but F8?
It takes you to the debugger. Just keep pressing F8 to see, if this way you get different line.


Petr

P.S. Maybe you already did this :-[

[sandyrepope]

can you try to run the script not using F5 but F8?
It takes you to the debugger. Just keep pressing F8 to see, if this way you get different line.

Using F8 was how I got line 63. I had the debugger going and when this line was highlighted I pressed F8 and got the error. I've tried it several times and this is always how it happens.

Sandy

[ErosOlmi]

Sandy,

I've tested your scripts using thinBasic 1.6.0.9 under Win98, WinME, Win2K and WinXP but not GPF at all.
I've checked the osurce code of MENU but I was not able to find anything strange.
If I cannot replicate the problem, for me is very hard to do something.

What I can suggest for the moment is to try to execute again but before:

switch off temporarly any anti virus software
switch off temporarly any instant messaging software (Messenger, Skype, ...)
switch off temporarly any personal firewall (many personal firewalls check application behave so some usage of OLE32 can be considered "dangerous")


Than let me know if this make any difference.

Ciao
Eros

[sandyrepope]

Eros, I'll try all of that later this weekend when I get a chance but for now I have a question.

Shouldn't the UI/menu example give the same error if it was any of that?

The example works just fine which is why I was thinking that it was my script.

Thanks
Sandy

[ErosOlmi]

Well, it is not always logic or known how some programs works, so I have no answer. What's undert the curtains of a so complex chains of event make the things very difficult to say.

Anyhow, please make also this test: change hMenu, hPopup1, hPopup2, hPopup3 variables from DWORD to LONG and see if it makes any difference.

Ciao
Eros

[sandyrepope]

Anyhow, please make also this test: change hMenu, hPopup1, hPopup2, hPopup3 variables from DWORD to LONG and see if it makes any difference.

I tried that and it didn't make any difference.

One thing I've tried is to comment out the four 'menu add popup' lines and there was no error messages. Of course, there was no menu system either.

I'll keep looking.

Thanks
Sandy