[uf-discuss] requirements testing

[Mark Rickerby]

Hi all,

For the last few months, I've been considering possibilities for using
HTML to define test cases, with references to various testing formats
from around the web:

http://microformats.org/wiki/requirements-testing
http://microformats.org/wiki/requirements-testing-examples

Through my work, I've had to write a lot of tests, and in some cases I
would rather that they weren't bound to a specific framework or
language. It would also be good if they were readable by people who
aren't necessarily familiar with specific programming language syntax.
Hence HTML makes a lot of sense to use here.

One of the key things I've noted is that the existing tools already
based on an HTML format (like Selenium and FitNesse) use <table>
elements to define assertions. These tables are effective, but clunky,
and not particularly natural to write, when compared to the native
syntax of various unit testing frameworks (eg: JUnit, SimpleTest,
Test::Unit)...

Heres an example from the Selenium documentation:

<table border="1" class="table">
<colgroup>
<col width="23%" />
<col width="55%" />
<col width="21%" />
</colgroup>
<tbody valign="top">
<tr><td>type</td>
<td>nameField</td>
<td>John Smith</td>
</tr>
<tr><td>typeAndWait</td>
<td>textBoxThatSubmitsOnChange</td>
<td>newValue</td>(
</tr>
</tbody>
</table>

Great idea, but it seems a kinda strange mash of HTML - especially if
this is intended to be hand coded. It strikes me that the natural
expression of a web test case is a bullet point list or an ordered
list. Here's an example using the Selenium semantics, which seems a
litttle bit more clear:

<ul class="test">
 <li>
  <span class="command">type</span>
  <span class="target">nameField</span>
  <span class="value">John Smith</span>
 </li>
 <li>
  <span class="command">typeAndWait</span>
  <span class="target">textBoxThatSubmitsOnChange</span>
  <span class="value">newValue</span>
 </li>
</ul>

If we removed the selenium commands, and reduced the test case to
natural language semantics we would have something like:

<ul class="test">
  <li>type "John Smith" in nameField</li>
  <li>type "newValue" in textBoxThatSubmitsOnChange and wait</li>
</ul>

This example gets to the root of the problem of documenting acceptance
tests in HTML: there are different levels of abstraction that will all
produce the same imperative command.

I was just wondering if anyone had any thoughts, criticism or
suggestions relating to this. The concept has been kicked around for a
while, so I thought I would push it out on the list, to see if anyone
else could see any significant overlaps with the existing microformats
work. Pointers to other testing tools / languages that I haven't yet
covered would be much appreciated, as would better HTML examples.

Also - is there such a thing as rel-test? Has anyone used this - or
has intentions to use this, and if so - where and what for?

Thanks,
Mark

--
----------------------------------------------------------------------
mark rickerby
http://maetl.coretxt.net.nz
http://www.mcs.vuw.ac.nz/~maetl/
----------------------------------------------------------------------