test-suite-2009: Difference between revisions
Jump to navigation
Jump to search
(Updated wrt adopting the µfxtract test-fixture format) |
m (Replace <entry-title> with {{DISPLAYTITLE:}}) |
||
| (11 intermediate revisions by 4 users not shown) | |||
| Line 1: | Line 1: | ||
{{DISPLAYTITLE:Test Suites}} | |||
'''Note: This page reflects 2009 era test suite efforts for a set of stable microformats as of that time. See the current [[test-suite]] page for modern [[microformats2]] test suite efforts. [[User:Tantek|Tantek]] 17:18, 2 December 2012 (UTC)''' | |||
---- | |||
So, you want to parse microformats in your app? Awesome. There are [[parsers]] in many languages. | So, you want to parse microformats in your app? Awesome. There are [[parsers]] in many languages. | ||
| Line 5: | Line 9: | ||
A critical effort to assist in interoperability and completeness of microformat parsing is the Test Suite. This page documents the effort to produce test cases for each microformat and parsing pattern, so that you can verify the operations of your parser. | A critical effort to assist in interoperability and completeness of microformat parsing is the Test Suite. This page documents the effort to produce test cases for each microformat and parsing pattern, so that you can verify the operations of your parser. | ||
==Get the Tests== | |||
specific microformats: | |||
; [[hCard]] | |||
: http://ufxtract.com/testsuite/hcard/ | |||
: ... | |||
; [[hCalendar]] | |||
: [[value-class-date-time-tests#hCalendar|value class pattern date and time hCalendar tests]] | |||
: http://ufxtract.com/testsuite/hcalendar/ | |||
: ... | |||
; [[hReview]] | |||
: ... | |||
; [[hAtom]] | |||
: ... | |||
; [[adr]] | |||
: ... | |||
; [[geo]] | |||
: http://ufxtract.com/testsuite/geo/default.htm | |||
: ... | |||
; [[hResume]] | |||
: http://ufxtract.com/testsuite/hresume/default.htm | |||
: ... | |||
patterns across microformats: | |||
; [[value-class-pattern]] | |||
: [[value-class-pattern-tests]] | |||
; [[include-pattern]] | |||
: ... | |||
==Goals of Test Suite Project== | ==Goals of Test Suite Project== | ||
We need to support specifications with better tests. There's no better way for parser writers to verify the correctness and completeness of their code without them. Microformats are getting adopted by big players, with large, stable implementations and are having bigger and bigger impact. It's critical that implementers don't reinvent the testing wheel for each new project. | We need to support specifications with better tests. There's no better way for parser writers to verify the correctness and completeness of their code without them. Microformats are getting adopted by big players, with large, stable implementations and are having bigger and bigger impact. It's critical that implementers don't reinvent the testing wheel for each new project. | ||
We need to provide a suite to: | We need to provide a suite to: | ||
* Verify the correctness of microformats parsing | * Verify the correctness of microformats parsing | ||
* Indicate the completeness of a parser implementation | * Indicate the completeness of a parser implementation | ||
| Line 35: | Line 50: | ||
==Test Structure== | ==Test Structure== | ||
Each test takes the form of: | Each test takes the form of: | ||
* Valid HTML 4 or XHTML 1.0 compatible file, containing the HTML mark-up for the test itself, along with the assertions used to validate the test. | * Valid HTML 4 or XHTML 1.0 compatible file, containing the HTML mark-up for the test itself, along with the assertions used to validate the test. | ||
* Assertions are described using a <code>test-fixture</code> mark-up pattern, invented by Glen Jones for his [http://www.ufxtract.com/testsuite/µfextract tests]. ''This pattern needs to be documented here.'' | * Assertions are described using a <code>test-fixture</code> mark-up pattern, invented by Glen Jones for his [http://www.ufxtract.com/testsuite/ µfextract tests]. ''This pattern needs to be documented here.'' | ||
This generic format has the following advantages: | This generic format has the following advantages: | ||
| Line 63: | Line 77: | ||
==Establishment Tasks== | ==Establishment Tasks== | ||
* Get the existing test cases from <code>hg.microformats.org</code> and into a Github repository. | * Get the existing test cases from <code>hg.microformats.org</code> and into a Github repository. | ||
** The collaboration functionality of Github is well suited to the development of new tests. | ** The collaboration functionality of Github is well suited to the development of new tests. | ||
* Get the existing <code>test-fixture</code> tests integrated | * Get the existing <code>test-fixture</code> tests integrated | ||
** Needs Public Domain release from | ** Needs Public Domain release from Glenn Jones. Done [[test-fixture]] | ||
* Make people aware of it! | * Make people aware of it! | ||
* Compile documentation for <code>test-fixture</code> | * Compile documentation for <code>test-fixture</code> | ||
| Line 77: | Line 90: | ||
Later, thought should be given to how assertions can be used to test implied meanings in microformats. For example, we cannot test ‘hCard is an organisation’ using a simple assert, since that is not a field of hCard (but would be a data field in an implementation). Initially, these implied meanings should just be documented in the tests, and applicable implementations can take note. | Later, thought should be given to how assertions can be used to test implied meanings in microformats. For example, we cannot test ‘hCard is an organisation’ using a simple assert, since that is not a field of hCard (but would be a data field in an implementation). Initially, these implied meanings should just be documented in the tests, and applicable implementations can take note. | ||
== past tests == | |||
There are also some older test suites available for [[hCard]], [[hCalendar]], [[hReview]] and [[hAtom]]. They are somewhat outdated and unmaintained, but should give you a starting point. The effort described on this page hopes to produce something far more robust and extensive, as well as setting a precedent and framework for future microformats and test suites. | |||
==Related== | ==Related== | ||
* [[test-fixture]] | |||
* [[parsers|Parsers]] | * [[parsers|Parsers]] | ||
* [[debugging-tools|Debugging Tools]] | * [[debugging-tools|Debugging Tools]] | ||
* [http://www.ufxtract.com/testsuite/ ufXtract test suite] | * [http://www.ufxtract.com/testsuite/ ufXtract test suite] | ||
Latest revision as of 16:33, 18 July 2020
Note: This page reflects 2009 era test suite efforts for a set of stable microformats as of that time. See the current test-suite page for modern microformats2 test suite efforts. Tantek 17:18, 2 December 2012 (UTC)
So, you want to parse microformats in your app? Awesome. There are parsers in many languages.
A critical effort to assist in interoperability and completeness of microformat parsing is the Test Suite. This page documents the effort to produce test cases for each microformat and parsing pattern, so that you can verify the operations of your parser.
Get the Tests
specific microformats:
- hCard
- http://ufxtract.com/testsuite/hcard/
- ...
- hCalendar
- value class pattern date and time hCalendar tests
- http://ufxtract.com/testsuite/hcalendar/
- ...
- hReview
- ...
- hAtom
- ...
- adr
- ...
- geo
- http://ufxtract.com/testsuite/geo/default.htm
- ...
- hResume
- http://ufxtract.com/testsuite/hresume/default.htm
- ...
patterns across microformats:
Goals of Test Suite Project
We need to support specifications with better tests. There's no better way for parser writers to verify the correctness and completeness of their code without them. Microformats are getting adopted by big players, with large, stable implementations and are having bigger and bigger impact. It's critical that implementers don't reinvent the testing wheel for each new project.
We need to provide a suite to:
- Verify the correctness of microformats parsing
- Indicate the completeness of a parser implementation
- Ensure that global patterns like abbr-design-pattern and value-class-pattern are implemented accurately
- Provide a base set of tests for implementations to build their automated test scripts
- Encourage implementations to put their tests into this combined effort, rather than working alone
Test Structure
Each test takes the form of:
- Valid HTML 4 or XHTML 1.0 compatible file, containing the HTML mark-up for the test itself, along with the assertions used to validate the test.
- Assertions are described using a
test-fixturemark-up pattern, invented by Glen Jones for his µfextract tests. This pattern needs to be documented here.
This generic format has the following advantages:
- It is language implementation agnostic, so never has to be ported into other languages (e.g. PHPUnit tests ported to JUnit ported to YUITest ported to…)
- The test case and assertions are contained in the same physical file. One file per test is easier to manage than multiple files associated in name.
- The test is self-documenting. Because the test is written in HTML, it can contain explanation alongside the assertions, can be verified easily by hand, as well as parsed by machines.
- The
test-fixtureformat can be tested using itself. - Dedicated automated test suites for languages (PHP, Python, JavaScript, Ruby, etc.) can be generated from the HTML using a single script. One new test case written using
test-fixturecan create tests for all languages. - It's useful that all tests be in the same output format. Early tests for microformats returned native formats, such as
vcf(vcard),ics(iCal) and Atom. These were useful in the context of early implementations, but parsers and consuming cases are now more sophisticated so providing assertions to verify against is more useful.
There is to be one test suite per format. In some cases, such as geo and adr with hCard, the address and geo portions will be included in the geo and adr test suites and hCard parsers will be expected to run cases from all three suites.
Tests using global patterns, such as value-class and abbr-design-pattern should be included in tests for all vocabularies, where appropriate.
Each vocabulary suite is organized as follows
- /stable contains tests for the stable, specified aspects of each vocabulary.
- /stable/new is the initial location for a new test. Tests are checked into new first, pending peer-review. After review, the test is moved into verified.
- /stable/verified contains tests for stable aspects of a specification which have been sanity checked by at least one other. These tests will be presented as (and should be) 100% correct and reliable.
- /experimental is a repository for brainstorm tests, for experimental implementations. The tests will be filed under /experimental/brainstorm-name. This keeps them separate from the stable branch, but makes it easy to merge them in if a brainstorm is successful and results in an update to the spec.
- /old contains tests that don't match the proposed test suite format/criteria. They're useful and should be ported to the new form, but are filed separately.
Establishment Tasks
- Get the existing test cases from
hg.microformats.organd into a Github repository.- The collaboration functionality of Github is well suited to the development of new tests.
- Get the existing
test-fixturetests integrated- Needs Public Domain release from Glenn Jones. Done test-fixture
- Make people aware of it!
- Compile documentation for
test-fixture - Fill in all the blanks on this page.
- Rewrite the existing hCard, hCalendar and hAtom tests to transform into JSON rather than native formats
- Encourage production open source scripts to convert a
test-fixturedocument into a runnable test case in major languages:- PHP, Perl, Ruby, Python, .NET, Java
- Produce
test-fixturetest cases fortest-fixtureitself.
Later, thought should be given to how assertions can be used to test implied meanings in microformats. For example, we cannot test ‘hCard is an organisation’ using a simple assert, since that is not a field of hCard (but would be a data field in an implementation). Initially, these implied meanings should just be documented in the tests, and applicable implementations can take note.
past tests
There are also some older test suites available for hCard, hCalendar, hReview and hAtom. They are somewhat outdated and unmaintained, but should give you a starting point. The effort described on this page hopes to produce something far more robust and extensive, as well as setting a precedent and framework for future microformats and test suites.