Integrating swWxGuiTesting into wxWidgets?

[Reinhold Füreder]

Dear wxWidgets developers,

It seems as if I am able to spend some time on my wxWidgets GUI testing
project (http://www.wefi.net/swWxGuiTesting/) in the next few weeks.
Some of the last equally moderate, but significant changes to
swWxGuiTesting are: (1) make it work with the latest testing release
2.7.0 of wxWidgets under MSW and Linux (GTK2; Ubuntu Dapper), (2)
abandon/remove the dependency on Xerces-C++ in favour of using expat
which is already included in wxWidgets (this was e.g. suggested by Vadim
Zeitlin), (3) demonstrate facilitated handling of user-defined modal
dialogs based on timer (suggested by Julian Smart).

My idea/goal is to spend some time on integrating swWxGuiTesting
with/into wxWidgets until middle/end of November, if this (a) makes
sense, (b) is desirable, and (c) is feasible...

In my mind there are quite a couple of issues involved:

(a) Inclusion of swWxGuiTesting:
* Where: contrib vs. core
* Renamings: files, classes, namespaces
* Coding convention adaptations
* Folder restructurings: additional splitting into (1) core, (2) C&R
* Incorporating parts of remaining swFramework dependency into
wxWidgets: Which ones? How? Where? Maybe first put in the parts of
swFramework as-is, so that these can be moved into wxWidgets, i.e. taken
over by wx, step by step?

(b) Almost part of (a) is the fact that the currently utilised CMake
build system should/must be replaced with the standard wxWidgets one(s):
according to Julian this is the bakefile system -> bakefile usage?
CppUnit dependency?

(c) So far swWxGuiTesting was built and tested under MSW (Windows 2000,
XP; VC6) and Linux (Ubuntu Dapper, Red Hat 8; GTK2) -- what about other
platforms? Also links with (b).

(d) Important further work according to the conclusion section of the
MPhil thesis chapter to achieve robust regression testing or continuous
integration means decoupling the testing framework and the
system/application under test:
* Detection and handling of errors and test case failures as soon as
possible
* Isolate execution of each single test case -- currently all test cases
are run "on/at" the same instance of wxApp!

Reason for the tight coupling -- is this still the case??? (In fact this
tight coupling has the big advantage of normal "debuggability" during
development and ideally should be switchable on/off)
-> wxEntry + Exit to startup wxWidgets system from console "mode" can be
done only once -> all tests are executed on only one global wxApp
instance
* Even using CppUnit2 does not solve this isolation-related problem,
does it?
* Alternative, in particular vital for continuous integration, but also
in general (cf. isolation, independency of single test cases):
# CTest like: each test case must be turned into a single executable
# Also max timeout for crash, dead lock, ... detection possible
? Open Issue: How to notify test driver/runner of problem details? (Only
success vs. failure?) Or complex interprocess communication for more
details?

(e) Tutorial/cookbook that is more developer-friendly than the thesis
chapter, and reflects the new "host/environment" wxWidgets in particular
with respect to the aforementioned part (a) as well as the subsequent
part (f).

(f) Improvements:
* Most importantly there are improvements that are linked directly with
(a), that is incorporating relevant parts of swFramework related to
handling of (1) modal dialogs, (2) toolbars, (3) popup menus, ...
E.g. possible solutions for modal standard dialogs: (a) ignore them --
i.e. test cases must not use/access them, (b) look in all
implementations and potentially modify the behaviour if being in test
mode?

* C&R:
    - looked up containers => cache them somehow to prevent duplicate
lookups and variable declarations.
    - do not emit code directly + fill map of include directories during
emitting => all include directories (without duplicates) can be actually
emitted first, before the rest of the code follows.


What do you think? What is your opinion/suggestion?

Thanks,
	Reinhold