test-suite-2009

<entry-title>Test Suites</entry-title>

So, you want to parse microformats in your app? Awesome. There are parsers in many languages.

A critical effort to assist in interoperability and completeness of microformat parsing is the Test Suite. This page documents the effort to produce test cases for each microformat and parsing pattern, so that you can verify the operations of your parser.

'NB: This page is new. There is a limited test suite available already for hCard, hCalendar, hReview and hAtom. It is somewhat outdated and unmaintained, but should give you a starting point. This effort hopes to produce something far more robust and extensive, as well as setting a precedent and framework for future microformats and test suites.

This page will be expanded.

Get the Tests

hCard

???

hCalendar

???

hAtom

???

hReview

???

Geo

???

Goals of Test Suite Project

We need to support specifications with better tests. There's no other way for parser writers to verify the correctness and completeness of their code without them. Microformats are getting adopted by big players, and implementations are having bigger and bigger impact. It's critical that implementers don't reinvent the testing wheel for each new project.

• Verify the correctness of microformats parsing in parsers
• Indicate the completeness of parser implementations
• Ensure that global parsing patterns like abbr-design-pattern and value-class-pattern are implemented accurately
• Provide a base set of tests for implementations to build their automated test scripts
• Encourage implementations to put their tests into this combined effort

Test Structure

Each test takes the form of:

• Valid HTML 4 or XHTML 1.0 compatible input file
• JSON output file, representating parsed data structure

JSON is the output format because:

• Parsers are available for most languages, and is less ambiguous for this purpose than inventing some bespoke XML format.
• It's useful that all tests return the same output format. Early tests returning vcf (vcard), ics (iCal) and Atom were useful for early implementations, but parsers and consuming cases are now more sophisticated so providing raw data to verify against is more useful.

There is to be one test suite per format. In some cases, such as geo and adr with hCard, the address and geo portions will be included in the geo and adr test suites and hCard parsers will be expected to include cases from all three suites.

Each suite is organized as follows

• /stable contains tests for the stable, specified aspects of a specification.
  • /stable/new is the initial location for each test. Tests are checked into new first, pending peer-review. After review, the test is moved into verified.
  • /stable/verified contains tests for stable aspects of a specification which have been sanity checked by at least one other. These tests will be presented (and should be) 100% correct and reliable.
• /experimental is a repository for brainstorm tests. If a brainstorm developers to point of building an experimental implementation, the tests will be filed under /experimental/brainstorm-name. This keeps them separate from the stable branch, but makes it easy to merge them in if a brainstorm results in an update to the spec.
• /old contains tests that don't match the proposed test suite format/criteria. They're useful and should be ported to the new form, but are filed separately.

Establishment Tasks

• Get the existing test cases from hg.microformats.org and into a Github repository.
  • The collaboration functionality of Github is well suited to the development of new tests.
• Make people aware of it!
• Get feedback from other parser authors that the proposed HTML+JSON test cases are suitable.
  • Document the JSON object format conventions.
• Fill in all the blanks on this page.
• Rewrite the existing hCard, hCalendar and hAtom tests to transform into JSON rather than native formats
• Produce a tool in JavaScript to generate the test result JSON from an HTML input sample
  • Making it easier to produce new test case results
  • Generate a test case from any web page using a bookmarklet
  • Produce a reference implementation