Storing massive amounts of data in INI files

[marcuslee]

Where did I say the size of the file would speed it up ?

I guess your comment about reducing bashing made me think speed.

you say "The data and format will vary across the board: numbers and strings, long and short, precise (floating point) and integer."

but will this data be same for each record?

The way I picture it now (which could change) is that there might be a certain number of variables that need to be saved ... say anywhere from 25 to 1000 or more depending on the complexity of the game.

That will be for just one iteration of the game. I want to be able to keep track of the decisions you make in the game so each set of variables will be needed in an archival mode.

Opening a file for "random" could work, I suppose. The player wouldn't need to save the entire file to memory. If you play the game long enough, you may not have enough ram to do that (for a really complex version of the game). In archival mode, the player could simply reopen the file and get the needed data.

Does this make sense?



Mark

[Michael Clease]

Not really.

I just dont have enough information about what your are trying to do.

Its like saying "I want to build a house" then not telling me what materials to build the house from, I can keep guessing but no surprises I wont.

Mike

[marcuslee]

I just dont have enough information about what your are trying to do.

Well, I could give you more info on the game, but I think that might just confuse you.

So, how about a small code example of what I just described to get started.

Type tRecord
  a     As Byte
  b     As Byte
  c     As Byte
  d     As Integer
  e     As Integer
  f     As Integer
  g     As Integer
  h     As Integer
  i     As Integer
  j     As Integer
  k     As Integer
  l     As DWord
  m     As DWord
  n     As DWord
  o     As DWord
  p     As DWord
  q     As DWord
  r     As DWord
  s     As DWord
  t     As DWord
  u     As DWord
  v     As DWord
  w     As DWord
  x     As DWord
  y     As DWord
  z     As DWord
  aa    As Extended
  bb    As Extended
  cc    As Extended
  dd    As String
  ee    As String
  ff    As String
  gg    As String
  hh    As String
  ii    As Currency
  jj    As Currency
  kk    As Currency
  ll    As Currency
  mm    As Double
  nn    As Double
  oo    As Double
  pp    As Double
  qq    As Double
  rr    As Double
  ss    As Double
  tt    As Double
End Type

Okay, now ... a little bit more about my game idea.

Now, there would be a limitless amount of these records. It would just depend on how long you play the game. Each turn of the game would be one record, and if you liked the game, and it was challenging and engaging, you could easily advance the game a thousand times or more.

It is a simulation city builder, sort of like sim city without the graphics. The closest that comes to it now is a game on Facebook called Metropolis. Of course, as I always dream big, I would want it to be bigger than that. Who knows if I will get there! Probably not, but trying is half the fun.




Mark

[Michael Clease]

I do think your over estimating the size of the data, try this.

Type tRecord
  a     As Byte
  b     As Byte
  c     As Byte
  d     As Integer
  e     As Integer
  f     As Integer
  g     As Integer
  h     As Integer
  i     As Integer
  j     As Integer
  k     As Integer
  l     As DWord
  m     As DWord
  n     As DWord
  o     As DWord
  p     As DWord
  q     As DWord
  r     As DWord
  s     As DWord
  t     As DWord
  u     As DWord
  v     As DWord
  w     As DWord
  x     As DWord
  y     As DWord
  z     As DWord
  aa    As Extended
  bb    As Extended
  cc    As Extended
  dd    As String
  ee    As String
  ff    As String
  gg    As String
  hh    As String
  ii    As Currency
  jj    As Currency
  kk    As Currency
  ll    As Currency
  mm    As Double
  nn    As Double
  oo    As Double
  pp    As Double
  qq    As Double
  rr    As Double
  ss    As Double
  tt    As Double
End Type


Dim FileBuffer As String
Dim FileName   As String = APP_ScriptPath + "file.ini"
Dim sMsg       As String
Dim FirstValue As Byte

  FileBuffer = String$(SizeOf(tRecord)*1, 64)  

'   This must be called after filling the buffer or you can use REDIM
Dim MyRecord(1) As tRecord At StrPtr(FileBuffer)

  sMsg  = "Record size *1         = " + Str$( SizeOf(tRecord)*1         )+" bytes"+$CRLF
  sMsg += "Record size *100     = " + Str$( SizeOf(tRecord)*100/1024  )+" kbytes"+$CRLF
  sMsg += "Record size *1000   = " + Str$( SizeOf(tRecord)*1000/1024 )+" kbytes"+$CRLF
  sMsg += "Record size *10000 = " + Str$( SizeOf(tRecord)*10000/1024 )+" kbytes"+$CRLF
 
 MsgBox 0, sMsg

Dont forget you are in control of the data so if you keep a thousand records in memory and then dump them all to a file and start a new set you can.

This is one of the challenges in writing that sort of game, I would search for examples how other people have organised the internal usage of memory.

[marcuslee]

I do think your over estimating the size of the data, try this.

Perhaps. The problem I foresee is that the data could be large if you play the game long enough. Of course, I could program a limit so that nothing gets larger than what I dim it for.

Dont forget you are in control of the data so if you keep a thousand records in memory and then dump them all to a file and start a new set you can.

I don't see why I would need to keep everything in memory at one time, so worrying about the size and speed of the game may be moot in the end anyway. More experimentation is needed.

This is one of the challenges in writing that sort of game, I would search for examples how other people have organised the internal usage of memory.

Could you point me in the right direction here? I'm not familiar with this type of thing and/or resource.



Mark