Keywords, constants, UDTs and classes for each module

[Petr Schreiber]

Hello Eros,

to make the idea proposed here possible, I created for you a clumsy, but working generator of list of all the module keywords, constants, UDTs and classes :P

We have app_listKeywords, app_listEquates, app_listUdts and app_listClasses at disposal, however they list items for all currently imported modules.
How to workaround it? I generate script for each of detected modules and execute it alone, with just that module used. And then I subtract the Core items, which I cannot filter out otherwise.

Hellish? Yes. Works? Yes! :P

uses "file", "os", "console", "ini"


function TBMain()
  string moduleName()
  long moduleCount = module_getNames(moduleName)
  
  printl strformat$("Found {1} modules", moduleCount)
  
  ' -- Prepare and clean output directory
  string tokenExporterPath = app_sourcepath + "tokenExporters\"
  if not dir_exists(tokenExporterPath) then dir_make(tokenExporterPath)
  file_kill(tokenExporterPath+"*.*")
  
  ' -- Create generator scripts
  module_createCoreTokenExporter(tokenExporterPath)
  
  printl               
  print "Creating generator for "
  for i as long = 1 to moduleCount
    if moduleName(i) = "trace" then iterate for
    
    print moduleName(i) + iif$(i < moduleCount, ",", "") in 14
    module_createTokenExporter(tokenExporterPath, moduleName(i))
  next
  
  ' -- Launch generator scripts
  string generator()
  long generatorCount = dir_listarray(generator, tokenExporterPath, "*.tBasic", %FILE_NORMAL | %FILE_ADDPATH)
  
  printl
  printl
  print "Launching generator for "
  for i = 1 to generatorCount
    print moduleName(i) + iif$(i < moduleCount, ",", "") in 14
    os_shell(app_path + app_name + " " + generator(i), %OS_WNDSTYLE_HIDE, %OS_SHELL_SYNC)
  next
  
  ' -- Generate INI file
  string outputFile = app_sourcepath + "moduleTokens.ini"
  if file_exists(outputFile) then file_kill(outputFile)
  
  string csvFile()
  long csvFileCount = dir_listarray(csvFile, tokenExporterPath, "*.csv", %FILE_NORMAL | %FILE_ADDPATH)
  
  string coreKeywords()
  parse(file tokenExporterPath + "core.csv", coreKeywords, ",")
  
  string csvKeywords()


  printl
  printl
  print "Generating output file " + outputFile  
  for i = 1 to csvFileCount
    if endswith(csvFile(i), "\core.csv") then iterate for
    print "."
    
    parse(file csvFile(i), csvKeywords, ",")
    
    array_removeItems(csvKeywords, coreKeywords)
    ini_setKey(outputFile, "Modules", file_pathsplit(csvFile(i), %PATH_FILE), join$(csvKeywords, ",")) 
  next
  
  printl "DONE" in 10
  waitkey


end function


function module_getNames(byRef moduleName() as string) as long


  string modules()  
  long   moduleCount = dir_listarray(modules, app_path + "Lib", "thinBasic_*", %FILE_NORMAL)
  redim  moduleName(moduleCount)
  
  for i as long = 1 to moduleCount
    moduleName(i) = lcase$(modules(i))
    moduleName(i) = grab$(moduleName(i), "_", ".dll")
  next
  
  return moduleCount


end function


function module_createCoreTokenExporter(path as string)


  string fileName = path + "core_tokenExporter.tBasic"                                                                                          
                                                                     
  file_save(fileName, "save_file("""+path+"core.csv"", app_listKeywords("","") + "","" + app_listEquates("","") + "","" + app_listUdts("","") + app_listClasses("",""))")


end function


function module_createTokenExporter(path as string, moduleName as string)


  string fileName = path + moduleName + "_tokenExporter.tBasic"
                            
  file_save(fileName, "uses """+moduleName+"""
                       save_file("""+path+""+moduleName+".csv"", app_listKeywords("","") + "","" + app_listEquates("","") + "","" + app_listUdts("","") + app_listClasses("",""))")


end function


function array_removeItems(byRef baseArray() as string, byRef items() as string)
  string baseArrayContent = "," + join$(baseArray, ",") + ","
  
  string token
  for i as long = 1 to countof(items)
    token = ","+items(i)+"," 
    if instr(baseArrayContent, token) then
      baseArrayContent = replace$(baseArrayContent, token, ",")
    end if
  next
  
  baseArrayContent = trim$(baseArrayContent, ",")
  parse(baseArrayContent, baseArray, ",")
end function

Petr

[ErosOlmi]

Wow, Petr
Always great ideas and code from Brno

I'm setting up a virtual machine in order to test your script because in my development environment every module is under specific sub-directory and not all into \thinBasic\Lib directory like they are when installing thinBasic.

I will let you know.
Eros


Capture.PNG

[ErosOlmi]

Got it, impressive job Petr

Attached "moduleTokens.ini" file with all keywords/equates/classes for each module generated by your script.

Now question is: to use it?
I mean, when runtime error 18 "Unknown Keyword" occurs I have the unknown token.
Using such an INI file I would need to go though all keys getting key value (the list of tokens present in a module) and search for the unknown token.
If found, I can suggest to add USES "ModuleFound"

Is this your idea?

[Petr Schreiber]

Hi Eros,

I am glad you got it working.
Yes, that is exactly my idea


Petr