Wine
A 'Book testing' approach to Wine testing
Wine needs regression tests. I think it really needs something for both
the graphical aspects (widgets look, message sequence, ...) and system API
aspects (open/close, semaphores, ...). Getting a test suite that provides
a good coverage of the API will take time but it has to be done. So what is
this 'book testing' ?
I bought, a few years ago, programming books about Windows 95. And one
day (recently) I realized that these books contain lots of example programs
that one can run in Wine. I also realized that many of these programs have
glitches when run in Wine. So the idea is to use these programs to test Wine.
The advantages:
- These programs are not open-source but their source is available to
anyone who has the book. So if something does not work I can look at the
source, and even read the book to understand what the program expects from
Windows. Basically this means I have a program with source ode and
excellent documentation. Isn't that better than trying why a closed-source
commercial application does not work?
- The other nice thing is that these programs are small. So you are
less likely to get lost than if you try to debug some big open-source
Windows application.
- By essence these books both concentrate on a specific aspect of
Windows (general GUI programming or Systems programming for instance) but
try to have a good coverage of this aspect. So you get a better idea of
which part of Wine you are testing than if you take a single random big
application.
So as I said I have some Windows programming books (4 to be precise)
and I have started testing Wine with them. The first one alone, the well
known Petzold - Programming Windows 95, contains 84 programs. That's a lot
of makefiles and other things to write so I have developped a small perl
script and packaged it as the 'Windows Programming Book Test Kit' (please,
please, help me find some other name). This kit is under the X11 license
(so that it is compatible with Wine's license) and you can download it
right there:
wpbtk-0.2.0.tar.gz
Note that you can also get the kit together with all the data for
'Programming Windows 95' from the test results
for that book.
What does the script do?
- I do not write my observations in am HTML page. I write it in a text
file and the script then converts it to a web page. This way I can change
the look of the page easily, and, even more importantly, I can add sections
generated by the script.
- One such example is the API cross-reference section: the script
makes a list of the Windows functions used by each example and also makes
a list of the examples using each API.
- To do the above the script generates a batch file which uses dumpbin
to collect information about the import and export sections of each
executable and dll.
- Since 0.2.0 the script uses winemaker to generate the makefiles and is
now 100% book-independent. For most examples winemaker a good 90% of the job.
All you have to do after is a little bit of fine-tuning to cut down on the
warnings (e.g. imported but unused libraries) and to correct the things that
winemaker gets wrong.
Note that I do not consider this to replace a proper test suite for
Wine, even less a correct regression test suite. I consider it instead to
be a complementary approach.
