Greets,

After taking way longer than I first envisioned, I've got my most important program functioning and spitting out the results that I had hoped for. But and there's always a but, I'm running into random crashes of thinBasic and the thinCore.dll. So I come looking for any guidance on the issue, before I fully consider rewriting a bunch of code.

This thread gives some insight into what I work with, as far as data and files; http://www.thinbasic.com/community/s...ad.php?t=11464

Some of the files are rather large, such as for the U.S. of A., where the data in the files is a grid of 10801x10801. I subsequently changed these files into quarters and their make up is a grid of 5401x5401. Using some sample code given by Eros in the linked thread, I load the file into a string using FILE_Load and then parse it into two separate strings. One string is the first six lines which give geographic location and then the second string reads the rest of the file. The second string then gets parsed into MyMatrix(), giving me my 5401 rows and 5401 columns. After that, I get into the data manipulation and then write out the data to a file.

Before the manipulation loop ends, I go and Redim MyMatrix() and then the program starts all over reading the next data file, parsing into two string and parsing the data string into MyMatrix(). This is when I get random crashes from thinBasic, listing thinCore.dll as the faulting module.

Memory usage is not so great to blow through the Win32 limit. At a certain point parsing the data string thinBasic may get up to around 1.2gb of memory use before settling back down to around 1gb of memory use until MyMatrix() gets Redim'd. I have inserted a Sleep(10000) to allow a little time for the memory drain to occur before the loop starts all over again. I tried taking out the Redim MyMatrix() command and instead leaving MyMatrix at (5401,5401) and using nested loops to reset the elements back to a null string of "". That crashed also.

The crashes can come at any time, but they appear after the first process and always back at the top of the code, where I am trying to Parse another file. The error messages are strange, as in one from yesterday where it said the variable SourceFileName should be of range 1 to 1, but was 2372. What is strange is SourceFileName is a string variable.

So is there any thoughts as to what I can do to make the processing of the large matrix more secure? Could I programmatically generate a new name for MyMatrix(), so that subsequent loops might create MyMatrix1(), MyMatrix2() and so on, in the hopes that any repetitive use of the matrix might be a contributing factor to the crashes? Otherwise I fear that I am going to have to consider stepping through the source file line by line and incurring, what I believe would be, a much longer processing time.

Thanks for any assistance,

Lance

P.S. for Eros: I'm more than happy to share thinBasic source and my data files. I can upload to my web site if you wanted to take a look. But you MUST promise not to laugh at my spaghetti code!