<h1>Hash brainstorming</h1>

__TOC__

== The Problem ==
Checksums (MD5 & SHA-1 hashes) are offered for files to prove they haven't been tampered with and to uniquely identify them. They are very useful, but they are not used as much as they could be. The current method involves a manual process of hashing the downloaded file (with programs that are not installed by default on all operating systems) and then comparing the value to the one listed. An easy and automatic way to use them would be preferrable to present methods.

== Participants ==
* Ant Bryan
* Charles Iliya Krempeaux
* [http://stoneship.org/ Denis Defreyne]

== Proposal ==
A microformat for MD5 and SHA-1 hashes could make them more usable. [http://mdhashtool.mozdev.org/index.html MD Hash Tool], another extension, or download managers could be modified to use them automatically.

<pre>
<span class="download">
<a rel="bookmark" href="...">Download OpenOffice.org
<span class="checksum md5">e0d123e5f316bef78bfdf5a008837577</span>
</a>
</span>
</pre>

=== Use with hAtom ===
For example, with RSS and Atom feeds, we have something called an "enclosure". With an "enclosure" you are being told that this file (that the enclosure points to) is "attached" to this item. (And that you might want to go and download it.)

Now, having checksum information (like a MD5 checksum) could be very useful for this. Especially in the context of hAtom -- the Microformat variation of Atom. And the rel-enclosure Microformat.

So, if we combined the two -- combined this semantic HTML for "downloading" and rel-enclosure -- then we might get something like this:

<pre>
<span class="download">
<a rel="bookmark enclosure" href="...">Download OpenOffice.org
<span class="checksum md5">e0d123e5f316bef78bfdf5a008837577</span>
</a>
</span>
</pre>

Note that I've added "enclosure" to the "rel" attribute of the <a> element. This could be used in other Microformats and semantic HTML too.

=== Issues ===

The proposal above has a few issues.

* Visibility of metadata: The type of checksum (MD5, SHA1, …) should not be inside an attribute, since it is metadata that should be visible.
* Checksum type attribute: There would need to be a massive number of class names for all possible checksum types: md2, md4, md5, sha1, sha256, sha384, sha512, tiger, ripemd128, ripemd160, etc.

The second sub-section also mentions use with hAtom, even though rel-enclosure is not tied to hAtom.

== Proposal #2 ==

Based on the two issues mentioned above, I propose a following hash format similar to the one in this example:

<code><pre><span class="checksum">The <span class="type">MD5</span> checksum of this download is <span class="value">e0d123e5f316bef78bfdf5a008837577</span>.</span></pre></code>

This introduces a <code>type</code> attribute for the kind of checksum (MD5, SHA1, RIPEMD-160, etc), and <code>value</code> for the actual checksum.

== Hashes and UIDs ==

Hashes an UIDs (unique identifiers) have quite a bit in common. Hashes are, usually, unique as well, except in a few edge cases where two different items have the same MD5 sum. Also see [[uid]] and related pages.

I think the difference between hashes and UIDs is too big for the two concepts to be merged. Hashes are, in my experience, not used to reference items. For example, you wouldn't say "give me the MP3 file with MD5 sum d2c550efaecc2bed137f7e40255177fe" but you could say "give me the book with ISBN ISBN0596002815". [[User:DenisDefreyne|DenisDefreyne]] 16:02, 5 Aug 2007 (PDT)

downloads-brainstorming

2007-08-05T22:53:19Z

DenisDefreyne:

hash-examples

2007-08-05T21:57:02Z

DenisDefreyne: Adding reference to media:hash from Yahoo MRSS

<h1>Hash examples</h1>

A microformat for cryptographic hashes, such as MD5 and SHA-1.

__TOC__

== Participants ==
* Ant Bryan
* Charles Iliya Krempeaux
* [http://stoneship.org/ Denis Defreyne]

== Real-World Examples ==
Currently, MD5 and SHA-1 checksums are either listed on a webpage or email (see Example #1) or stored in a separate file such as (filename.ext.md5 or filename.ext.sha1) (see Example #2). There is no standard or automatic way to use them. Verifying a file after you have the hash is not complex, but it is more than the average user is used to doing (see [http://www.openoffice.org/dev_docs/using_md5sums.html OpenOffice.org: Using MD5 sums]). MD5 checksums are 32 digit hexadecimal numbers, while SHA-1 checksums are 40, and SHA-256 checksums are 64.

== Who offers MD5/SHA-1 checksums with software ==
This is only a small sampling.
*[http://httpd.apache.org/ Apache HTTP Server] in .md5 file from web.
*[http://www.cisco.com/ Cisco] MD5 for versions of IOS from Software Center on Cisco website.
*[http://developer.apple.com/darwin/ Darwin] MD5 on web.
*[http://fedora.redhat.com/ Fedora Project] SHA-1 on web and SHA1SUM file on ftp.
*[http://www.freebsd.org/ FreeBSD] on web and in CHECKSUM.MD5 and CHECKSUM.SHA256 files.
*[http://gcc.gnu.org/ GCC] on ftp as md5.sum file.
*[http://www.gentoo.org/ Gentoo] as .md5 file on ftp.
*[http://www.gnome.org/ GNOME] as MD5SUMS-for-gz and MD5SUMS-for-bz2 files on ftp.
*[http://www.gnupg.org/ GnuPG] SHA-1 on web.
*[http://www.kde.org/ KDE] on web and on ftp as MD5SUMS file.
*[http://www.knoppix.org/ Knoppix] in .md5 and .sha1 file.
*[http://www.mysql.com/ MySQL] MD5 on web.
*[http://www.openoffice.org/ OpenOffice.org] MD5 on web.
*[http://www.openssh.com/ OpenSSH] SHA-1 in release announcement.
*[http://www.openssl.org/ OpenSSL] .md5 and .sha1 files linked to from web.
*[http://www.perl.org/ Perl] link to .md5 on web.
*[http://www.postgresql.org/ PostgreSQL] in a .md5 file.
*[http://www.python.org/ Python] MD5 on web
*[http://www.ubuntu.com/ Ubuntu] as MD5SUMS on ftp.
*[http://www.x.org/ X.org] md5sums file on ftp.

=== Example #1: [http://download.openoffice.org/2.0.1/md5sums.html OpenOffice.org MD5 sums] ===
<pre>
English Application Binaries

e0d123e5f316bef78bfdf5a008837577 OOo_2.0.1_LinuxIntel_install.tar.gz
35d91262b3c3ec8841b54169588c97f7 OOo_2.0.1_LinuxIntel_install_wJRE.tar.gz
cc273fe9d442850fa18c31c88c823e07 OOo_2.0.1_SolarisSparc_install.tar.gz
ff6626c69507a6f511cc398998905670 OOo_2.0.1_SolarisSparc_install_wJRE.tar.gz
ce099d7e208dc921e259b48aadef36c1 OOo_2.0.1_Solarisx86_install.tar.gz
4fb319211b2e85cace04e8936100f024 OOo_2.0.1_Solarisx86_install_wJRE.tar.gz
66bd00e43ff8b932c14140472c4b8cc6 OOo_2.0.1_Win32Intel_install.exe
2d86c4246f3c0eb516628bf324d6b9a3 OOo_2.0.1_Win32Intel_install_wJRE.exe
</pre>

=== Example #2: [http://mirrors.csol.org/knoppix/ Knoppix MD5 and SHA-1 sums in separate files ] ===
<pre>
KNOPPIX_V4.0.2CD-2005-09-23-EN.iso.md5:

1188f67d48c9f11afb8572977ef74c5e *KNOPPIX_V4.0.2CD-2005-09-23-EN.iso

KNOPPIX_V4.0.2CD-2005-09-23-EN.iso.sha1:

56857cfc709d3996f057252c16ec4656f5292802 *KNOPPIX_V4.0.2CD-2005-09-23-EN.iso

</pre>
Note: This directory also contains filename.ext.md5.asc and filename.ext.sha1.asc files containing the same checksums and PGP signatures in one file.

== Existing Practices ==
As described above, I believe almost all solutions are manual (see [http://www.openoffice.org/dev_docs/using_md5sums.html OpenOffice.org: Using MD5 sums]), an 8 step process on Windows and 3 steps on Linux. [http://mdhashtool.mozdev.org/lfinfo.html Link Fingerprints] which are used by [http://mdhashtool.mozdev.org/index.html MD Hash Tool], a Firefox extension, is one exception. Here is a Link Fingerprint example:

<pre>
http://example.org/OOo_2.0.1_LinuxIntel_install.tar.gz#!md5!e0d123e5f316bef78bfdf5a008837577
</pre>
A Link Fingerprint begins with a traditional URL, then #!md5!, then the MD5 hash.

[http://www.metalinker.org Metalinks] are an XML file format (.metalink) that contain mirrors and checksum information for downloading files. They are used by download programs/managers and mainly open source projects. After a download finishes, the checksum is automatically verified.

Brad Fitzpatrick also [http://brad.livejournal.com/2162507.html suggested referring to "files/patches/changesets" by their unique digest.]

Some HTTP server applications compute a hash over the response body to serve as an effective ETag. The server must still compute the body but can benefit from reduced network utilization and reduced downstream cache thrashing. Such applications must be willing to risk a hash collision, albeit scoped to a single URL.

Including a hash in a URL can lead to great cacheability, since the TTL can likely be set to an infinite value. Such URLs are often referred to as ''versioned URLs''.

== media:hash in Yahoo Media RSS Module ==

Yahoo's [http://search.yahoo.com/mrss Media RSS Module] adds <code><media:hash></code> to RSS feed enclosures, like this:

<code><pre><media:hash algo="md5">dfdec888b72151965a34b4b59031290a</media:hash></pre></code>

== Related Pages ==

* [[hash-brainstorming]]

appcast-brainstorming

2007-08-05T21:48:06Z

DenisDefreyne: Redirecting appcast-brainstorming to downloads-brainstorming

#REDIRECT [[downloads-brainstorming]]

downloads-formats

2007-08-05T21:46:43Z

DenisDefreyne: Renaming appcast to downloads where appropriate

appcast-formats

2007-08-05T21:46:12Z

DenisDefreyne: appcast-formats moved to downloads-formats

#REDIRECT [[downloads-formats]]

downloads

2007-08-05T21:45:49Z

DenisDefreyne: Renaming appcast to downloads where appropriate

==Related pages==
* [[downloads-brainstorming]]
* [[downloads-examples]]
* [[downloads-formats]]

appcast

2007-08-05T21:45:16Z

DenisDefreyne: appcast moved to downloads

#REDIRECT [[downloads]]

downloads-examples

2007-08-05T21:44:46Z

DenisDefreyne: Renaming appcast to downloads where appropriate

appcast-examples

2007-08-05T21:44:19Z

DenisDefreyne: appcast-examples moved to downloads-examples

#REDIRECT [[downloads-examples]]

appcast-brainstorming

2007-08-05T21:41:41Z

DenisDefreyne: Merging appcast-brainstorming into downloads-brainstorming

This appcast-brainstorming is obsolete; please visit [[downloads-brainstorming]].

2007-08-03T09:39:07Z

DenisDefreyne: fixing markup

<h1>Appcast Brainstorming </h1>
The purpose of this page is to capture software updates practices in the wild, as one effort to codify practices currently used in the automatic update system called Sparkle.

__TOC__

== Authors ==

* [[User:DenisDefreyne|Denis Defreyne]]
* [http://factoryjoe.com/ Chris Messina]

== Context ==

Sites like [http://iusethis.com iusethis.com], [http://versiontracker.com versiontracker.com], [http://macupdate.com macupdate.com] and [http://download.com download.com] among others should be investigated and documented for object names and attributes.

Among common data in a software update changelog are changes or fixes, completed bugs, new features or modified behavior, and known issues. A version number is also typically supplied, but takes on many forms and is not always numeric.

Lastly, this work should be seen as compatible with hAtom, possibly as a prelude to a format that could be embedded as the payload of an hfeed object.

Work should be carried out on the [[appcast-examples | Appcast Examples]] page.

== Related Work ==

* [[downloads-brainstorming]] seems to have a similar goal. Could these two pages be merged? [[User:DenisDefreyne|DenisDefreyne]] 02:56, 28 Jul 2007 (PDT)
** I would agree that they're related; the Appcast work is specially aimed at making it possibly to transmit Sparkle-like updates in HTML and hAtom. I would be nice to be able to subsume the downloads work with the appcast microformat. [[User:Chris_Messina|Chris Messina]] 010:02, 28 Jul 2007 (CDT)
* [[hash-examples]] has a section about hAtom integration. The example uses a "download" class, which contains a rel-enclosure link as well as a span with "md5" and "checksum" classes.

== Sparkle-specific Appcast enhancements ==

Sparkle adds a few extra features to appcasts:

* External release notes URL: used when the release notes are not included in the appcast itself
* MD5 sums and DSA signatures: used for some extra security
* Advanced version string: Sparkle determines the application version from the enclosure name, which is assumed to be APPNAME_VERSION.zip; the advanced version string allows this convention to be overridden

The appcast microformat should probably support these features as well.

The MD5 sum and DSA signature is specific for an enclosure. It is, however, not possible to add these extra attributes to a [[rel-enclosure]] link; a larger "enclosure" microformat may be necessary (also see [[hash-examples]]).

== Proposal ==

This proposal is mostly inspired by appcasts with Sparkle extensions. It has a few issues, mostly due to the fact that it is not "general" enough.

* Each hentry can have one or more <code>download</code>s. Having more than one download in an appcast microformat used by Sparkle is not useful, as Sparkle would only use the first enclosure it sees. It may be useful in multi-* appcasts (see below).
* Each hentry has one <code>entry-version</code> which holds the application version.

Example:

<code><pre><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<title>Appcast</title>
</head>
<body>
<h1>Appcast</h1>
<div class="hentry">
<h2 class="entry-title">Adium <span class="entry-version">1.0.4</span></h2>
<p>Updated on <abbr class="updated" title="2007-06-20T18:00+01:00">24 June</abbr>.</p>
<div class="entry-content">
<ul>
<li>Fixed a crash introduced in 1.0.3 which could occur when accounts disconnected or status changed</li>
<li>Fixed group chat when message history is enabled</li>
</ul>
</div>
<p class="download">
<a href="http://adiumx.cachefly.net/Adium_1.0.4.dmg" rel="enclosure">Download</a>.
MD5 sum: <span class="checksum md5">0f182a2c88373e6d77c69fa15c910f08</span>.
</p>
</div>
</body>
</html></pre></code>

[[User:DenisDefreyne|DenisDefreyne]] 03:40, 1 Aug 2007 (PDT)

== Multi-app/arch/OS/... appcasts ==

A single downloads page can have downloads for different applications, different versions, different architectures, different operating systems, etc. The proposal above does not take this in account. (The [http://dev.mysql.com/downloads/mysql/5.0.html MySQL downloads page], for example, has many apps, many architectures and many OSes.) A way to distinguish between applications, architectures, OSes, ... would therefore be necessary.

Here's a probably incomplete list of different download properties:

* Architecture: PPC, x86, 68k, architecture-independent, …
* OS: Mac OS X, Windows XP, OS-independent, …
* Download type: Ruby Gem, gzipped tarball, Mac OS X DMG, … (Cannot use MIME types for this, as not all different download types have their own MIME type)

On second thought, all this is download-specific, so this is probably better suited for the [[downloads-brainstorming]] page.

[[User:DenisDefreyne|DenisDefreyne]] 03:40, 1 Aug 2007 (PDT)

== Software using appcasts ==

This is a list of programs and frameworks that use appcasts or appcast-like data.

* [http://sparkle.andymatuschak.org/ Sparkle] uses Sparkle-enhanced RSS appcasts.
* [http://metaquark.de/appfresh/ AppFresh] uses Sparkle-enhanced appcasts and [http://iusethis.com/ iusethis].
** I'm not sure how the iusethis integration works, but it looks like AppFresh scrapes HTML pages.

== Related Pages ==

* [[appcast]]
* [[appcast-examples]]

appcast-brainstorming

2007-08-03T09:38:40Z

DenisDefreyne: Adding "Software using appcasts" section

&;h1>Appcast Brainstorming </h1>
The purpose of this page is to capture software updates practices in the wild, as one effort to codify practices currently used in the automatic update system called Sparkle.

__TOC__

== Authors ==

* [[User:DenisDefreyne|Denis Defreyne]]
* [http://factoryjoe.com/ Chris Messina]

== Context ==

Sites like [http://iusethis.com iusethis.com], [http://versiontracker.com versiontracker.com], [http://macupdate.com macupdate.com] and [http://download.com download.com] among others should be investigated and documented for object names and attributes.

Among common data in a software update changelog are changes or fixes, completed bugs, new features or modified behavior, and known issues. A version number is also typically supplied, but takes on many forms and is not always numeric.

Lastly, this work should be seen as compatible with hAtom, possibly as a prelude to a format that could be embedded as the payload of an hfeed object.

Work should be carried out on the [[appcast-examples | Appcast Examples]] page.

== Related Work ==

* [[downloads-brainstorming]] seems to have a similar goal. Could these two pages be merged? [[User:DenisDefreyne|DenisDefreyne]] 02:56, 28 Jul 2007 (PDT)
** I would agree that they're related; the Appcast work is specially aimed at making it possibly to transmit Sparkle-like updates in HTML and hAtom. I would be nice to be able to subsume the downloads work with the appcast microformat. [[User:Chris_Messina|Chris Messina]] 010:02, 28 Jul 2007 (CDT)
* [[hash-examples]] has a section about hAtom integration. The example uses a "download" class, which contains a rel-enclosure link as well as a span with "md5" and "checksum" classes.

== Sparkle-specific Appcast enhancements ==

Sparkle adds a few extra features to appcasts:

* External release notes URL: used when the release notes are not included in the appcast itself
* MD5 sums and DSA signatures: used for some extra security
* Advanced version string: Sparkle determines the application version from the enclosure name, which is assumed to be APPNAME_VERSION.zip; the advanced version string allows this convention to be overridden

The appcast microformat should probably support these features as well.

The MD5 sum and DSA signature is specific for an enclosure. It is, however, not possible to add these extra attributes to a [[rel-enclosure]] link; a larger "enclosure" microformat may be necessary (also see [[hash-examples]]).

== Proposal ==

This proposal is mostly inspired by appcasts with Sparkle extensions. It has a few issues, mostly due to the fact that it is not "general" enough.

* Each hentry can have one or more <code>download</code>s. Having more than one download in an appcast microformat used by Sparkle is not useful, as Sparkle would only use the first enclosure it sees. It may be useful in multi-* appcasts (see below).
* Each hentry has one <code>entry-version</code> which holds the application version.

Example:

<code><pre><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<title>Appcast</title>
</head>
<body>
<h1>Appcast</h1>
<div class="hentry">
<h2 class="entry-title">Adium <span class="entry-version">1.0.4</span></h2>
<p>Updated on <abbr class="updated" title="2007-06-20T18:00+01:00">24 June</abbr>.</p>
<div class="entry-content">
<ul>
<li>Fixed a crash introduced in 1.0.3 which could occur when accounts disconnected or status changed</li>
<li>Fixed group chat when message history is enabled</li>
</ul>
</div>
<p class="download">
<a href="http://adiumx.cachefly.net/Adium_1.0.4.dmg" rel="enclosure">Download</a>.
MD5 sum: <span class="checksum md5">0f182a2c88373e6d77c69fa15c910f08</span>.
</p>
</div>
</body>
</html></pre></code>

[[User:DenisDefreyne|DenisDefreyne]] 03:40, 1 Aug 2007 (PDT)

== Multi-app/arch/OS/... appcasts ==

A single downloads page can have downloads for different applications, different versions, different architectures, different operating systems, etc. The proposal above does not take this in account. (The [http://dev.mysql.com/downloads/mysql/5.0.html MySQL downloads page], for example, has many apps, many architectures and many OSes.) A way to distinguish between applications, architectures, OSes, ... would therefore be necessary.

Here's a probably incomplete list of different download properties:

* Architecture: PPC, x86, 68k, architecture-independent, …
* OS: Mac OS X, Windows XP, OS-independent, …
* Download type: Ruby Gem, gzipped tarball, Mac OS X DMG, … (Cannot use MIME types for this, as not all different download types have their own MIME type)

On second thought, all this is download-specific, so this is probably better suited for the [[downloads-brainstorming]] page.

[[User:DenisDefreyne|DenisDefreyne]] 03:40, 1 Aug 2007 (PDT)

== Software using appcasts ==

This is a list of programs and frameworks that use appcasts or appcast-like data.

* [http://sparkle.andymatuschak.org/ Sparkle] uses Sparkle-enhanced RSS appcasts.
* [http://metaquark.de/appfresh/ AppFresh] uses Sparkle-enhanced appcasts and [http://iusethis.com/ iusethis].
** I'm not sure how the iusethis integration works, but it looks like AppFresh scrapes HTML pages.

== Related Pages ==

* [[appcast]]
* [[appcast-examples]]