[uf-discuss] FW: New microformats test-suite idea

[Glenn Jones]

As the dev list does not seem to be working at the moment, reposted
here.

Just added Operator to testrunner, but you have to use Firefox.
> If you go to http://ufxtract.com/testsuite/hcard/1.0/hcard1.htm and
press Alt X you can see a working demonstration of the testrunner.

Glenn

-----Original Message-----
From: Glenn Jones 
Sent: 03 May 2008 12:11
To: 'microformats-dev at microformats.org'
Subject: New microformats test-suite idea

Hi all

I have been working on a concept for a new microformats test-suite. I
need a comprehensive test-suite before I can move the development of
UfXtract forward. Rather than just build something in isolation I
thought it would be nice to find a way to share this work with the
community. 

I have written two POSH patterns "testsuite" and  "testfixture".  They
follow the principles of microformats design:
*	They are self-describing and created with HTML ,with no hidden
metadata
*	Building a test should be easy even for those who are HTML
authors 
*	They are not linked to any one programming language and should
be easy to share
*	They allow for the creation of an in-browser Testrunner

For example go to http://ufxtract.com/testsuite/hcard/1.0/

The earliest tests for hcard used vcards to describe the expected
output. As the community has moved forward it has designed microformats
which are independent of external specification. So this test-suite is
designed around the concept of a standardised data structure.  In this
case, expressed in JSON, but they could be converted into XPaths to test
XML or other languages.

I have started to build a small console app, which will spider the HTML
and create NUnit/C# class files for my build tests. Although this is
specific to my own parsers development, it should be easy to do the same
for other programming languages and projects.

Parsing "testsuite" and  "testfixture".
I have already setup UfXtract to parse these patterns into JSON/XML. It
would not take much for other microformats parser developers to
construct profiles for these POSH patterns.  
Here is an example of the output.
http://lab.backnetwork.com/ufXtract/?url=http%3A%2F%2Fufxtract.com%2Ftes
tsuite%2Fhcard%2F1.0%2Fhcard1.htm&format=test-fixture&output=json


The Testrunner

If you go to http://ufxtract.com/testsuite/hcard/1.0/hcard1.htm and
press Alt X you can see a working demonstration of the testrunner.
   
I have observed that most parser developers are using comparative
testing as their main tool to quickly understand how the complex rules
and optimizations are applied. So I have built a JavaScript Testrunner
which allows for simple comparative testing between parsers.

It uses a number of techniques to standardise both access to the parsers
API's and the JSON output. ***Please note that at this stage the JSON
standardisation process can cause a test to be marked as failed when it
could be judged to have passed***. Most of the current differences in
parser output are down to whether a value is stored as single property
or an array of properties.

At the moment the Testrunner is only working with the testfixure , it
would not take much to extend the Testrunner to run a whole test-suite. 

I would love to add Operator and other parsers to the Testrunner. 

Proof of concept
This is very early proof of concept stuff. What I would like to ask is a
number of questions before moving it forward.

*	Are people interested in the idea of shared test-suites?
*	What do you think to the approach?
*	Can you see any big issues with the concepts?
*	If you already have tests/test-suites, would you be willing to
add them to the project?
*	Would you be interested in contributing to a project like this?
 
Hope you like the idea.

PS: the JavaScript needs serious re-factoring; I will simplify it soon.


Glenn Jones