PlexNex

File Format Fatigue Sets In

On the eve of ISO's notorious BRM week in Geneva (when national standards bodies try to find reasons to pass Microsoft's format, OOXML, as an international standard like ODF v1.0), we are visited by a strange call for truce from the ODF side's Patrick Durusau -- Sun's employed editor of the OASIS ODF v1.2 (and friend of the erstwhile interop player & my very own, the once and ever, OpenDocument Foundation, the Foundation having joined the august company of Disgruntled OASIS ODF TC Alumnae including Arbortext, Boeing, Corel, Speedlegal, Stellent|BEA|Oracle & others who tired of beating their heads against Jonathan Schwartz's erected brick wall).

Perhaps the most telling part is that Patrick chose to publish his letter in Adobe's PDF fixed file format, so thoroughly has the ODF v OOXML debate deteriorated into distraction from the main point -- which was and still is interoperability ... or rather the lack of it: the inability of word-processors of different vendors, vintages and varieties to open, read and edit files with business-process-level fidelity ... like e-mail and web pages. (The need is for a true interchange format -- what we've called a Universal Document Format for editable documents. Peter Seebach gets right to the point in "OOXML: What's the big deal?" | IBM Developerworks | 19 Feb 2008)

PDF is the real winner here.

click the image for Adobe's James King's insights: "Bits & Pieces"...

At least we have that: a great & useful, open-enough & archive-able fixed document format with aesthetic document layout-presentation nous. This alongside the other news that Blue-ray has won out definitively against HD DVD in the hi-def video medium war. There's reassurance in at least some finality -- somewhere.

Not so lucky are we here in the document format war -- where all conclusions are certain to be inconclusive and the uncertainty has a 50-percent chance of carrying the years 2008 AND 2009.

Patrick's letter, among other things, is a loser's cry for help (with CV attached) because even the EU-sponsored harmonization directive on ISO to work with the German standards body, DIN, parented by the Fraunhoffer braintrust to report on the harmonization potential of ODF & OOXML is proving a waste of time, the directive likely about to produce the conclusion that merging ODF & OOXML is a technical impossibility even if the sponsoring applications -- OpenOffice.org & Microsoft Office -- are harmonized too (a commercial impossibility given how tightly Microsoft, Sun & IBM covet the office application market shares).

The calls for civility, patience, calm & restraint appeal nicely to all parties at this point, tired of personal attacks, misdirection, mud-slinging, character assassination, vitriol & pointless emotionalism. The calls appeal to Microsoft most of all, which moves forward in its new phase of dominance of the web with a brilliant ramp of the truly brilliant & actually innovative Office 2007 product suite -- which implants a version anyway of the OOXML format which is the questionable case before ISO.

Eyes and ears with any interest in new balance in software markets must now ignore everything on the table in the Document Format War as it is being dished today, must look ahead -- skate to the puck (as Gary Edwards says) -- to documents and the Internet.

Here, the only place to look is the W3C. But given the high stakes in owning formats, there's a real risk that the argy-bargy visited by the vendors upon us in the ODF-OOXML Cycle (2005 - 2008) will encumber, subsume & overwhelm the W3C in future days.

Let's hope not, for documents ought to be open & free if we are to have data-sovereignty and if basic applications are ever going to work for us(ers).

'Harmonization' Synthesis

Dennis Hamilton, alias Orcmid, has an interesting comment with links on the many views of ODF & OOXML harmonization.

IBM's | Sutor's Dissonance

Bob Sutor's Warsaw deck (6 slides, PDF) is to-the-point & true with regard to this magic word, 'interoperability'. Too bad ODF isn't an interoperable format and a harmonization effort, if successful, would take another 3 years.

It reminds me of the good feelings Simon Phipps' words inspired when we could hear him talk about Sun's commitment to open source back in 2003 & 2004 when OpenOffice.org was all promise -- and Sun hadn't yet discovered its allergy to GNU/Linux.

Both parties speak well, but the evident intent on both Sun's & IBM's parts to create an alternative office suite monopoly -- a DUOPOLY (Sun & IBM on the one side, Microsoft on the other) -- through the force of rip&replace political mandates is disgraceful, and the execution lame.

Doc Searls said the great thing about open source and Free Software is that the users have taken over their own supply ("DIY-IT" | Linux Journal | 1 Feb 2004). It is high time this became true in the document space, for CIOs and policy bureaucrats who have signed up for ODF are going to get a Massachusetts comeuppence when the difficulty of deploying OpenOffice.org in a decentralized enterprise is acknowledged. (By now it would be easier to go straight to Ubuntu.)

CIOs & policy bureaucrats! Look past the ODF & OOXML miasma to the Internet -- where the problem of document interoperability can reasonably be solved.

The Disappointing Formats

The ODF-to-OOXML harmonization effort being hosted by the German standards group, DIN, is Europe's best effort to resolve our Mexican Standoff between Microsoft, Sun and IBM. Even though harmonization is laughably complex and will not work unless the applications are harmonized too, the best and brightest of Germany are left to hope for success.

Hope for a harmonization of formats does reflect one truth: that the whole point of this ugly exercise is to have one single FORMAT -- which is really the same thing as having interoperability. Anything else ridiculously undermines common sense.

We have e-mail. E-mail works because there is one standard format. (Despite there being a host of different e-mail protocols, they are open enough to enable everyone to field and deliver e-mail across different systems.) This is called interoperability.

So, too, HTML is the single agreed format which enables us all to view web pages from different brands and versions of browsers and across a good (though not great) variety of hardware and software operating system environments. This too is called interoperability (though bad people have tried and are still trying to corrupt HTML).

E-mail interoperability and web page interoperability define what we mean by interoperability. It means when software applications of any kind can read the content of an electronic document and present the document's original and intended layout faithfully.

Why don't we have interoperability of this nature for editable documents?

The answer is obvious. Because there is a monopoly of the desktop operating system and of the office suite, and because the monopolist guards its position of control over the main document format. I would too, if I were in their place (and I hope that you would do everything in your power to stop me if I were in their place).

Here's what they are guarding:

REDMOND, Wash. — January 24, 2008 — Microsoft Corp. today announced second quarter records for revenue(s) ... of $16.37 billion. Compared to the year ago period, [this figure] represent growth of 30%.

This record performance is attributable largely to strong sales of Office 2007, which it goes without saying is the Trojan Horse insertion device for the new OOXML file format which drives Microsoft's strategy to kill Adobe Systems, kill SAMBA, stop GNU/Linux adoption, and in Steve Ballmer's words, "... win the Web!"

Despite great efforts to intercept and weaken Microsoft's control of the main editable document format, there has been no effect on the company's transition from the old binary formats to the new XML-based style of document format.

ODF was originally conceived -- at least in the minds of some -- as a death-bomb to the Office & format hegemony, but it has been executed badly and now is being shaped to carve out a small share of the office suite market for Sun and IBM. OpenOffice.org's and ODF's execution is so poor, in fact, that Sun & IBM are merely carving out a minority share of an obsolete market for old fat-client applications which may only have another 2 years to live -- at best. Most shocking is the low level of Sun's and IBM's market ambitions, not so much the familiar vendor tactics of obfuscation they're using to promise interoperability while delivering an application-tied file format with the same flaws as Microsoft's.

They would have done better to kill Microsoft Office -- dead -- by engineering full interoperability into ODF and closing off Office 2007's chances in the market-place with fit aggression. Make ODF a single Universal Document Format by enabling the format to handle all of Microsoft's document application features (without dropping "foreign elements") and enable OpenOffice.org to function successfully in a mixed office suite environment, making application change in the enterprise at least a possibility. Thus, by end-of-life of Office 2003 (sometime in 2009), CIO's would have been given time to orchestrate full migrations to other, perhaps more open and less wasteful, platforms.

This is where ODF went wrong. The chance to disrupt the monopoly in the market-place has come and gone (Gary describes here precisely how the buy-side has been left out). If you don't believe me, go to the mailing lists; the ODF community's distaste for format-level interoperability sits in plain sight in the record of the OASIS ODF Technical Committees over the span of the last several years.

Even KDE developers can't get documents to work interoperably between KOffice and OpenOffice.org. Here is KDE's lead developer:

"One thing I have always dreamed to be possible is that when I write a doc in KOffice I can then open it in OOo to use that one feature that's useful to me and then save it and continue in KOffice without loosing lots of data.

"Its still a dream, of course. Most features are lost on opening and saving it in OOo, but its a nice goal[.]"

Thomas Zander | KDE | 27 Sep 2007

The history of a single KDE interoperability proposal to the OASIS ODF Technical Committee reveals Sun's and IBM's success at keeping the OpenOffice.org application's unique document settings from being fully specified in the ODF standard:

From the original KDE interoperability proposal to the OASIS ODF TC ...

"In order to improve interoperability with
the OpenDocument format, we need to
standardize the settings a bit more. 

Right now the contents of settings.xml is
not part of the standard, it's up to each
implementation to save whichever settings
they want...."

David Faure | KDE | 25 Nov 2005

Two years later, D Faure submits an action item enclosing his two-year old suggestion, verbatim ...

"Here are my suggestions for which
settings we could standardize upon..."

David Faure | KDE | 19 Mar 2007

The ODF TC minutes note that a response, ostensibly from the TC Chairman, Michael Brauer (Sun Microsystems), is required ...

"* Action items
(Document settings), reposted by David,
follow up from Michael needed ..."

Lars Oppermann | Sun Microsystems | 4 Apr 2007

The KDE interoperability action item stays alive in the OASIS ODF TC minutes ...

"* action items
michael - follow up on docuemnt settings,
in porgress" [SIC]

Lars Oppermann | Sun Microsystems | 25 Apr 2007

The KDE interoperability action item stays alive in the OASIS ODF TC minutes ... 

"* action items
michael - follow up on document settings,
in progress"

Lars Oppermann | Sun Microsystems | 9 May 2007

The KDE interoperability action item stays alive in the OASIS ODF TC minutes ... 

"* action items
michael - follow up on document settings,
in progress"

Lars Oppermann | Sun Microsystems | 18 May 2007

The KDE interoperability action item stays alive in the OASIS ODF TC minutes ...

"* action items
michael - follow up on document settings,
in progress"

Lars Oppermann | Sun Microsystems | 24 May 2007

Next minutes reveal the action item gone ...

"* action items
no new status"

Lars Oppermann | Sun Microsystems | 7 Jun 2007

The action item has recently reappeared in the TC notes, but without action. Meanwhile, the relevant application-specific markup remains in the OpenOffice.org application's source code, where only tag names and not their functionality are identified. (See lines 169-211.)

I have been on record criticizing Microsoft's bad specification of its new XML format. I still stand by that criticism.

Reasonable people recognize the importance and necessity of holding ODF to the same standard of criticism. In light of the performance of leadership of the OASIS ODF Technical Committee shown here, it is impossible not to point out similar fundamental omissions in the specification of ODF.

This is why we've missed our chance.

The harmonization process will call again for the interoperability features in ODF and OOXML which KDE leadership where calling for THREE YEARS ago. Over that time span, Microsoft has been allowed to ready another office suite platform. This is a disgraceful loss of a chance to intercede in the monopoly grip on the market while the opportunity was ripe.

Office 2007 is in and now we must shift our attention to the next forum of the conflict: Documents & the Internet. The drama of ODF & OOXML at ISO is merely theater at this point.

Denying CDF: "There Be Dragons"

Carta Marina, 1539

The industry of denial of the importance of the W3C's Compound Document Format ("CDF") is going to be a sysiphean ordeal for the blindered ODF & OOXML choruses ...

"CDF: the common format you've never heard of" | O'Reilly XML.com | Kurt Cagle | 29 Nov 2007

Already, the number of HTML documents that exist dwarf (by a few orders of magnitude) the total number of Microsoft Word documents. As editing increasingly moves onto the web, its safe to say that the document of choice will be neither ODF nor OOXML, both of which gain their power on the basis of supporting legacy word processing systems. Instead, what seems to be emerging from the W3C is something that is not an office suite because it didn’t evolve from one, but that nonetheless is capable of most if not all of the same functions that office suite documents pose.

Note also this -- dated Sept 2007 -- W3C editor's draft of CDF's CDI WICD profile for "conforming document-authoring tools".

In plain English, this means CDF is in fact designed for the desktop as well as devices and the web. Let me repeat that: CDF is a format for office suites.

Andy Updegrove was either lying or revealing a willingness to believe anything IBM tells him; and IBM got poor W3C CDF working group members to deny CDF's applicability to the office suite tools to keep the sinking ODF ship afloat (see statements attributed to W3C's Chris Lilley).

How can you blame the W3C for wanting to stay clear of the politicized & unproductive ODF v OOXML stalemate?

Just because something as powerful as CDF is unfamiliar doesn't mean it's going to hurt you.

Burton Group Report on ODF | OOXML

Peter O'Kelly & Guy Creese's summary of the ODF | OOXML situation is a uniquely vendor-neutral summary that is both detached and accurate in its coverage -- though strangely weighted toward Microsoft when both sides have glaring sins.

"What's Up .DOC?" (PDF, requires e-mail address registration)

A weakness is its omission of any detailed coverage of the meaning of OOXML throughout the rest of Microsoft software products, those existing outside the office suite domain. Covering OOXML as a document format alone is like covering the tip of an iceberg.

Additionally, the conclusion that either of the new XML formats can help mitigate vendor lock-in is simply a good reflection of writer's fatigue.

Apart from these flaws, and its kid-glove treatment of Microsoft's incongruous and unexplainable machinations, O'Kelly's & Creese's effort is a worthwhile 37-page read.

Document Formats at GOSCON 2007

The video for the ODF (OpenDocument Format) Panel at GOSCON 2007 (Government Open Source Conference) was just kindly posted by Deb Bryant. The panel took place in Portland Oregon on 16 October 2007.

Panelists included ...

• Buck Martin (OpenDocument Foundation)
• Jason Matusow (Microsoft)
• Doug Johnson (Sun Microsystems)
• Arnaud Le Hors (IBM)
• James King (Adobe)

DOWNLOAD the video (Real Media format | 59:01)

Panel management was disciplined by Andy Stein, CTO of Newport News Virginia, and the second half of the video contains a Q&A with uncommonly apt questions -- reflecting the importance of the document formats issues to the audience.

An interesting moment comes in the 44th minute when Microsoft's Jason Matusow answers a question directed to him about interoperability ...

"You don't want interop to be an innovation killer. You want to be able to have that balance  where ... maybe interop is less important as something emerges but over time it becomes more important in that particular space."

Coming from Microsoft this non-sense -- complete with sophistry -- justifies the current Microsoft strategy.

'Don't stop innovation' -- this is certainly a point where the office suite vendors concur and it is a statement which has in this past year repeatedly come from the ODF development community, too -- particularly Sun's leadership of the ODF TC at OASIS, who have made ODF just about as dependent on its mother application as OOXML is upon MS Office 2007.

This is the central point of the whole situation. In the two areas out in the glorious World Wide Web where near perfect interop exists with diverse sets of authoring tools and readers -- e-mail and Web pages -- innovation was historically superceded and even frozen by the primary concern for interop. It leads me to conclude there is no solution to the document interoperability problem that is rigorous enough to enforce functional interop for editable documents coming from the vendor community; that the governments of the world are going to have to create, enforce and protect their own single document format standard and only against strong opposition from all vendors except Adobe.

I believe the requirements are clear but the IT sell-side and the IT government buy-side are talking past each other.

The separate set of frequently-asked-questions on the GOSCON 2007 site is also worth a moment -- particularly useful for catching up to the substance of this long, noisy -- often nauseating -- debate.

Dutch Format: Define Success, Get Success

A familiar electric feeling of excitement courses the spine unbidden when I hear the improving news about the Dutch national policy for ODF. The Dutch determination is faintly familiar. It's nearly like Massachusetts in August of 2005, when I heard through backchannels about the surprising contents of the final draft ETRM 3.5 and, chewing half a pizza, I called the scoop in to the Financial Times' San Francisico-based technology editor, Richard Waters.

Let's not let this one go, like Massachusetts, Denmark or Belgium.

What are the metrics of success here?

In a rodeo calf-roping competition, the iron gate crashes open and the calf bolts from his pen. The cowboy and his mustang, a single silhouette together in the corner, bolt in pursuit, hoofs gulping dirt, muscular haunches straining to speed from a standstill. The man-horse, gaining, swings the stiff loop once overhead and punches it downward. With talent, the noose catches a foreleg and the calf flips, heals over head, oblivious to what force of the sky could raise the ground so quickly. The pursuer comes down from his horse. The clock -- less than eight seconds has elapsed -- will not stop until he winds the rope around three awkward calf-legs and stabs the air with his gloved hand. The time is best; the crowd roars.

In Holland, success means a timely, inexpensive & painless migration to OpenOffice.org on existing largely Windows XP machines. Period. Stab the air. If it's a careless, blind grope, we all go home and the name 'ODF' is murder in our ears.

The ODF case worldwide hangs on a Dutch fulcrum: is it possible, as well as easy, to implement the software necessary to fulfill the ODF policy? That's the question enterprise IT executives want to know. And if the ODF vendors, Sun & IBM, are sincere for a win, their professionals will be descending like leaves in Autumn from now through next summer upon Arnhem, Amsterdam, Den Haag, Eindoven, Groningen, Heerenveen, Nijmegen, Rotterdam & Utrecht.

In those places, success means wrestling with business processes, identifying what people do with documents and spreadsheets, analyzing their behavior and re-routing their actions onto new & unfamiliar systems. Details count. People's time matters to them.

Perhaps many IT people worldwide have already given up hope; but this second spasm in the Netherlands cranes necks from Capetown to Cape Horn to St Petersburg to St Paul.

They are watching (if not holding their breath).

Yale Whitepaper on Documents & Democracy

Laura DeNardis & Eric Tam of Yale Law School's Information Society Project published last month their whitepaper -- "Open Documents and Democracy: A Political Basis for Open Standards" (PDF).

This paper will come to represent a touchstone of the case for a good de jure standard document format.

How we get there is a world of questions in a separate sphere, but we've been waiting for an authoritative, clear & definitive expression of the basic common reasons why software markets should give way to democratic principles & process with respect to the common editable document format.

OpenDocument Foundation is Closing

People have wondered why I haven't blogged about the Foundation closing. It is. And I haven't said anything here because corporate housekeeping takes a little time: board votes, blah blah blah.

We will issue a press release when we have thought through and fully understand what we are going to do next.

Thanks for your patience.