[uf-dev] hatom test suite useage?

[Ryan King]

On May 16, 2007, at 5:03 PM, Ryan King wrote:

> Is anyone else using the test suite for hAtom? (besides me and X2V)

Ok, I was mainly asking this because I'd like to do some major  
refactoring on the test suite.

For a bit of context, several of us working on creating hCard and  
hCalendar test last year. Another group created an hAtom test suite.  
Those have since been merged into one [1].

Since that time I've learned quite a bit about how we can do things  
better. Specifically, I've read these documents: [2] [3] [4]. You  
should read those, too.

So, to cut to the chase, here's my plans for our test suite, based on  
our experience:

1. organize tests into categories based on complexity

    CSS used abcdef (atomic, basic, composite, detailed, evil and  
failure)[5]. I'm not sure if we need a system that complex or  
detailed, but we definitely need to start with atomic tests and work  
up towards more complexity.

    My experience backs this up as well. I'm currently trying to get  
my hAtom parser compliant with the hAtom test suite, but all of the  
tests are what would probably be considered 'composite' or 'detailed'  
tests. That means I have to implemented a large part (if not all) of  
hAtom before I can get a single test to pass. This is not very good  
for morale. :D

2. Organize the tests in a canonical ordering, from simple to complex.

    I sort of did this with hCard and hCalendar, though I couldn't  
explain fully at the time why it worked. At the time, I just put them  
in the order that I was implementing them, which naturally went from  
simple to complex.

3. Make the tests more navigable.

    The major use case considered when building the suite was parsing  
code that could run automated tests. Since then, we've had things  
like Operator, where the testing is done by hand. We need to make  
that easier to do.	

    For this I want to start with creating cover pages and navigation  
links, as described in [6]

4. Provide more documentation on the purpose of each test. [7]

5. Explicitly state the status of each test.[8] Some tests are still  
works in progress, but it's tough to know that if you aren't actively  
work on them.

----

Unless there's objections or better suggestions for these problems,  
I'm going to start implementing them for hAtom. If we like them for  
hAtom, we can backport them to hCard and hCalendar as well.

-ryan

1. http://hg.microformats.org/tests
2. http://www.w3.org/TR/2005/NOTE-test-metadata-20050914/
3. http://www.w3.org/Style/CSS/Test/testsuitedocumentation.html
4. http://www.w3.org/TR/2005/NOTE-test-metadata-20050914/
5. http://www.w3.org/Style/CSS/Test/guidelines.html#kinds
6. http://www.w3.org/Style/CSS/Test/ 
testsuitedocumentation.html#usingtestsuites
7. http://www.w3.org/TR/2005/NOTE-test-metadata-20050914/#purpose-def
8. http://www.w3.org/TR/2005/NOTE-test-metadata-20050914/#status-def