The Disappointing Formats

The ODF-to-OOXML harmonization effort being hosted by the German standards group, DIN, is Europe's best effort to resolve our Mexican Standoff between Microsoft, Sun and IBM. Even though harmonization is laughably complex and will not work unless the applications are harmonized too, the best and brightest of Germany are left to hope for success.

Hope for a harmonization of formats does reflect one truth: that the whole point of this ugly exercise is to have one single FORMAT -- which is really the same thing as having interoperability. Anything else ridiculously undermines common sense.

We have e-mail. E-mail works because there is one standard format. (Despite there being a host of different e-mail protocols, they are open enough to enable everyone to field and deliver e-mail across different systems.) This is called interoperability.

So, too, HTML is the single agreed format which enables us all to view web pages from different brands and versions of browsers and across a good (though not great) variety of hardware and software operating system environments. This too is called interoperability (though bad people have tried and are still trying to corrupt HTML).

E-mail interoperability and web page interoperability define what we mean by interoperability. It means when software applications of any kind can read the content of an electronic document and present the document's original and intended layout faithfully.

Why don't we have interoperability of this nature for editable documents?

The answer is obvious. Because there is a monopoly of the desktop operating system and of the office suite, and because the monopolist guards its position of control over the main document format. I would too, if I were in their place (and I hope that you would do everything in your power to stop me if I were in their place).

Here's what they are guarding:

REDMOND, Wash. — January 24, 2008 — Microsoft Corp. today announced second quarter records for revenue(s) ... of $16.37 billion. Compared to the year ago period, [this figure] represent growth of 30%.

This record performance is attributable largely to strong sales of Office 2007, which it goes without saying is the Trojan Horse insertion device for the new OOXML file format which drives Microsoft's strategy to kill Adobe Systems, kill SAMBA, stop GNU/Linux adoption, and in Steve Ballmer's words, "... win the Web!"

Despite great efforts to intercept and weaken Microsoft's control of the main editable document format, there has been no effect on the company's transition from the old binary formats to the new XML-based style of document format.

ODF was originally conceived -- at least in the minds of some -- as a death-bomb to the Office & format hegemony, but it has been executed badly and now is being shaped to carve out a small share of the office suite market for Sun and IBM. OpenOffice.org's and ODF's execution is so poor, in fact, that Sun & IBM are merely carving out a minority share of an obsolete market for old fat-client applications which may only have another 2 years to live -- at best. Most shocking is the low level of Sun's and IBM's market ambitions, not so much the familiar vendor tactics of obfuscation they're using to promise interoperability while delivering an application-tied file format with the same flaws as Microsoft's.

They would have done better to kill Microsoft Office -- dead -- by engineering full interoperability into ODF and closing off Office 2007's chances in the market-place with fit aggression. Make ODF a single Universal Document Format by enabling the format to handle all of Microsoft's document application features (without dropping "foreign elements") and enable OpenOffice.org to function successfully in a mixed office suite environment, making application change in the enterprise at least a possibility. Thus, by end-of-life of Office 2003 (sometime in 2009), CIO's would have been given time to orchestrate full migrations to other, perhaps more open and less wasteful, platforms.

This is where ODF went wrong. The chance to disrupt the monopoly in the market-place has come and gone (Gary describes here precisely how the buy-side has been left out). If you don't believe me, go to the mailing lists; the ODF community's distaste for format-level interoperability sits in plain sight in the record of the OASIS ODF Technical Committees over the span of the last several years.

Even KDE developers can't get documents to work interoperably between KOffice and OpenOffice.org. Here is KDE's lead developer:

"One thing I have always dreamed to be possible is that when I write a doc in KOffice I can then open it in OOo to use that one feature that's useful to me and then save it and continue in KOffice without loosing lots of data.

"Its still a dream, of course. Most features are lost on opening and saving it in OOo, but its a nice goal[.]"

Thomas Zander | KDE | 27 Sep 2007

The history of a single KDE interoperability proposal to the OASIS ODF Technical Committee reveals Sun's and IBM's success at keeping the OpenOffice.org application's unique document settings from being fully specified in the ODF standard:

From the original KDE interoperability proposal to the OASIS ODF TC ...

"In order to improve interoperability with
the OpenDocument format, we need to
standardize the settings a bit more. 

Right now the contents of settings.xml is
not part of the standard, it's up to each
implementation to save whichever settings
they want...."

David Faure | KDE | 25 Nov 2005

Two years later, D Faure submits an action item enclosing his two-year old suggestion, verbatim ...

"Here are my suggestions for which
settings we could standardize upon..."

David Faure | KDE | 19 Mar 2007

The ODF TC minutes note that a response, ostensibly from the TC Chairman, Michael Brauer (Sun Microsystems), is required ...

"* Action items
(Document settings), reposted by David,
follow up from Michael needed ..."

Lars Oppermann | Sun Microsystems | 4 Apr 2007

The KDE interoperability action item stays alive in the OASIS ODF TC minutes ...

"* action items
michael - follow up on docuemnt settings,
in porgress" [SIC]

Lars Oppermann | Sun Microsystems | 25 Apr 2007

The KDE interoperability action item stays alive in the OASIS ODF TC minutes ... 

"* action items
michael - follow up on document settings,
in progress"

Lars Oppermann | Sun Microsystems | 9 May 2007

The KDE interoperability action item stays alive in the OASIS ODF TC minutes ... 

"* action items
michael - follow up on document settings,
in progress"

Lars Oppermann | Sun Microsystems | 18 May 2007

The KDE interoperability action item stays alive in the OASIS ODF TC minutes ...

"* action items
michael - follow up on document settings,
in progress"

Lars Oppermann | Sun Microsystems | 24 May 2007

Next minutes reveal the action item gone ...

"* action items
no new status"

Lars Oppermann | Sun Microsystems | 7 Jun 2007

The action item has recently reappeared in the TC notes, but without action. Meanwhile, the relevant application-specific markup remains in the OpenOffice.org application's source code, where only tag names and not their functionality are identified. (See lines 169-211.)

I have been on record criticizing Microsoft's bad specification of its new XML format. I still stand by that criticism.

Reasonable people recognize the importance and necessity of holding ODF to the same standard of criticism. In light of the performance of leadership of the OASIS ODF Technical Committee shown here, it is impossible not to point out similar fundamental omissions in the specification of ODF.

This is why we've missed our chance.

The harmonization process will call again for the interoperability features in ODF and OOXML which KDE leadership where calling for THREE YEARS ago. Over that time span, Microsoft has been allowed to ready another office suite platform. This is a disgraceful loss of a chance to intercede in the monopoly grip on the market while the opportunity was ripe.

Office 2007 is in and now we must shift our attention to the next forum of the conflict: Documents & the Internet. The drama of ODF & OOXML at ISO is merely theater at this point.

Comments

[orcmid]

I don't think we are ready to figure out a common format for office productivity documents, and there are problems we have not stumbled on yet, let alone resolved. At the same time, I share your concern and interest in the harmonization activity. It is the only way we will learn the dimensions of the problem and the pitfalls of underspecified formats. (I just had an experience that suggests even the Zip format may be eroding and fragmenting. Drat.)

Concerning e-mail formats, I think there is oversimplification. There is indeed more than one e-mail format, even though there is often a single carrier. That is, plaintext is co-opted to carry in-band wrappers for a different format. "HTML-formatted" e-mail is quite different than RTF-formatted e-mail and these differ from plaintext e-mail and plaintext e-mails differ from each other, etc.

I'm thinking that the degree of success in having there appear to be a single format in everyone's favorite mail reader/browser is an example of harmonization, not convergence on a single format.

(There are still awkward cases of abstraction sheer, as when list servers mess up the embedded format, especially for digests, or when digital signatures force unexpected format constraints.)

In many ways, e-mail provides a nice case study on interoperability, harmonization, and ways incoherence remains, especially when clueless intermediaries put their fingers all over the mail. (E.g., I just learned my hosting service is protecting me from viruses in ways I didn't know about and that kept me from receiving some important mail.)

[orcmid]

The first time I read it, I dismissed your remark about needing to harmonize application (programs) too quickly.

I think that is actually a big deal. It is what people deal with. The formats are below everyone's direct attention and, while standardization and assurance of interchange is essential, it is insufficient, I think, with regard what works for people who use products that use the formats.

I think we will run into that at some point. I'm not at all sure what the next step would be. Harmonizing browsers may be child's play compared to office document software, and look at how we struggle with that.

[Sam]

Orcmid-

You point out in your first comment that e-mail is really multiple formats.

I believe there's one and the others are *contained*.

This is precisely what CDF does.

With the pain of having to harmonize both formats and both applications in this present mess, we think it is easier to leap ahead to the Web around CDF. It will save 3 years in which Microsoft Office would be reconsolidating its dominance ... AGAIN.

So simple, no one thought of it.

[TripleII]

Where did you get the idea ODF has " it has been executed badly and now is being shaped to carve out a small share of the office suite market"

http://en.wikipedia.org/wiki/OpenOffice.org#Market_share
"Although Microsoft Office retains 95% of the general market as measured by revenue,[50] OpenOffice.org and StarOffice have secured 14% of the large enterprise market as of 2004[51] and 19% of the small to midsize business market in 2005."

How about 1 in 6 home users use OO?

http://www.imaginaryplanet.net/weblogs/idiotprogrammer/?p=83398725
"With OpenOffice.org closing in on 15% of Open Internet users..."

The days of MS's 100% margins are coming to an end. When 1 in 6 use a free alternative, it gets harder and harder to justify the $449.

The above is MS's worst nightmare.

TripleII

[tohiio]

It's very easy to say ODF should have been open to office in the first place, that would have killed office... How do you make an open format interoperable with a bulky proprietary binary (crappy ie not fully interoperable with office tools)... Micro$oft would have killed all interoperability at the first occasion by releasing a new binary format...

[Sam Hiser]

tohiio-

To an extent that is always a risk. But Microsoft will necessarily blow up its installed base and document application interfaces -- i.e., it's developer relationships -- if it tries to dodge an interop scheme that is cleverly implemented.