Hi,

I've been all over the web, and tried some of the code from several Basic dialects. But I can't seem to find any simple code that will just take a single line/string (not a file), which might contain quoted strings, and extract out either symbolic tokens or just the elements. Bint32 was mentioned several times, but I haven't been able to locate the source for that.


The type of code I'm looking for, doesn't require while...wend or if...endif support. It would just need to be able to extract e.g.


from
sVar := "whats happening" (or sVar = "whats happening?" or sVar="whats happening?" )

into
{
"sVar",
":=",
"whats happening?"
}


Is there anything like that around? One of the problems with trying out each mention of 'tokenizer' or 'lexer' on the web, is that you never know ahead of time, whether it will handle quoted strings.


Sorry if this is off-topic or already addressed, but I really have searched a lot, for the past month.