Possible approaches for testing

[Petr Schreiber]

Hi Eros,

there are multiple approaches of writing tests, and multiple types of tests actually. Without going into too much depth - think of them as a system for "good sleep". It takes time to write the tests, but it pays off in the future - you have army of little guardians checking the correctness of your functionality.

Testing is such a broad topic... and I am no test engineer really, so a brief overview of 2 kinds of tests which can help us now:
- "command test" -> you basically simulate all test cases for functionality via multiple tests. This is what I did with ARRAY EXTRACT. The results are verified via the ut_assert* functions. In case the condition fails, you are notified
- "regression test" -> you remember my example where different order of CASEs caused false positive error? Fixed for now, but what to do to prevent it from returning in future? The best is to write test function, which does execute previously problematic code. This way, in case it breaks again (because of optimization, human mistake, ...), you are notified by failing test again.

The regression set should be run before each release, to make sure, or better, to minimize, the risk of breaking anything.

[Petr Schreiber]

Case study: Array Extract added to ThinBASIC

User requests a functionality to be able to extract data matching specified criteria. One of them is "Starts with". So we need to:
- prepare representative enough data set for tests
- perform the extraction, and verify that:
-- expected number of items is returned
-- expected content of new array items is returned

Let's start with the data set. I think it should not be too huge, and it should be easy to understand. So let's set target for filtering everything starting with "H":


String items(5) = "Hi there", "Hello", "Howdy", "Ciao", "See ya"


Then we need to perform the command:


String processed()
Array Extract items, StartsWith "H" InTo processed


And as a last step, verify, those 2 criteria mentioned, can be done via:


ut_AssertEqual(3, CountOf(processed)) ' -- Because 3 items start with H

ut_AssertEqualText("Hi there", processed(1))
ut_AssertEqualText("Hello" , processed(2))
ut_AssertEqualText("Howdy" , processed(3))


It is good practice to cast single type of verification per test, so this would actually end with two tests. The test_ prefix is mandatory in my testing engine, rest is up to you:


Function test_ArrayExtract_StartsWithModifier_CorrectItemCount()

String items(5) = "Hi there", "Hello", "Howdy", "Ciao", "See ya"
String processed()

Array Extract items, StartsWith "H" InTo processed

ut_AssertEqual(3, CountOf(processed))

End Function

Function test_ArrayExtract_StartsWithModifier_CorrectItemContent()

String items(5) = "Hi there", "Hello", "Howdy", "Ciao", "See ya"
String processed()

Array Extract items, StartsWith "H" InTo processed

ut_AssertEqualText("Hi there", processed(1))
ut_AssertEqualText("Hello" , processed(2))
ut_AssertEqualText("Howdy" , processed(3))

End Function


...the name of the function should be descriptive enough, to give you idea about what is being tested.

Following the same logic, the other functions, covering EndsWith, Contains and Collate Ucase modifier are made:
https://github.com/ErosOlmi/ThinBASIC_On_GitHub/blob/master/Regression/1.9.16.x/tests/tests_Core_ArrayExtract.tBasicU

Curious question - is this coverage enough? ;) (no - but can you guess why?)


Petr