... in response to discussions on ZDNet ...
2.2.1.1. 
da Vinci here
gary.edwards - 11/03/07
Thank you John. Just because we are garage challenged doesn't mean we can't find the back door to the big house :)
The larger issue at stake here is not whether or not we have a garage, or what our contribution to ODF has been over the course of five years as active members of OASIS ODF. What it really comes down to is the implementation of ODF in the real world.
The chickens came home to roost when Massachusetts started a year long pilot study regarding the implementation of ODF. The study began shortly after the OASIS approval of ODf 1.0, and ended in May of 2006. The results were nothing short of a disaster for ODF.
Not wanting to give up on ODF, Massachusetts came up with another plan; a hail Mary cry for help known as the RFi. The Request for Information concerning the feasibility of an ODF plug-in for MSOffice.
They couldn't implement ODF using the many OpenOffice variations that participated in the year long pilot study, so they came up with the idea of an ODF clone of the MS-OOXML Compatibility Pack plug-in for MSOffice. The thinking being that if Microsoft could convert documents, applications and processes to MS-OOXML using a plug-in, then maybe it's possible to do the same with ODF?
The ODf vendors had of course advised Massachusetts that such a plug-in could not be done. But then CIO, Louis Gutierrez, and his staff persisted, releasing the public plea for help. We responded to the RFi based on an interest myself and some other Foundation members had dating back to the initial release of the MSOffice 2003 beta when we were first amazed that an XML conversion could actually be done using the plug-in architecture.
The one thing about the Massachusetts ODf implementation problem that people refuse to admit is that the hard barrier was not political. Sure, the politics were nasty. But Louis and his staff were determined, even though their entire budget was cut out from under them. The hard barrier that was uncovered during the pilot study was that of a wide proliferation of MSOffice bound workgroup-workflow business processes. Processes demanding the high fidelity exchange of documents commonly demanding perfect application version synchronization throughout the chain. It was too disruptive to rip out and replace MSOffice because of these bound business processes. And it was too costly to re engineer the business processes for ODf ready alternatives. Hence the idea of an ODf plug-in for MSOffice.
Even though we proved it was possible to write an ODf clone of the MS-OOXML Compatibility Pack plug-in, and demonstrated this on June 19th, 2006 at a day long event hosted by IBM, there still remained the problem of adapting ODf to a situation it was not designed to deal with. Because we were dealing with business processes, there was an absolute need for a high fidelity “round trip” capable conversion process.
This statement that ODf was not designed to meet the Massachusetts market requirements comes as a shock to the faithful. Yet, most would admit that if Microsoft were to join the OASIS ODf TC, there would no doubt be a massive wave of compatibility eXtensions and application specific eXtensions required for MSOffice apps to implement ODf. From our point of view though, we only needed five generic eXtensions to get the job done.
Doesn't sound like a big deal does it? But it is. The issue of eXtending ODf to meet the compatibility – interoperability needs of MSOffice desktops was destined from day one to tear the ODf community to shreds. It's kind of like the issue of slavery was to the original framers of the Constitution. They couldn't agree as to how to deal with the issue when the country was founded, and it later came back to tear the Republic apart in a horrific civil war. I'll try to explain, so bear with me.
To start with, this much must be understood. The core Massachusetts market requirements are expressed as compatibility with existing file formats (including Microsoft binary documents), and interoperability with existing applications (including MSOffice apps). There is a third requirement called grand convergence, but we'll save that for later.
Others, like Sun with their ANSI – ISO vote in favor of MS-OOXML expressed this same issue as: ”We wish to make it completely clear that we support DIS 29500 becoming an ISO Standard and are in complete agreement with its stated purposes of enabling interoperability among different implementations and providing interoperable access to the legacy of Microsoft Office documents.
The ODf compatibility-interoperability with Microsoft documents and applications has a long and troubling history at OASIS. The very first instance of this issue came on December 16th, 2002, at the first OASIS Open Office XML TC meeting, when the proposed charter came up for vote. Keep in mind that the first phase OASIS ODf group was overwhelmingly comprised of enterprise publication, content, and archive management systems representatives. Representatives from Stellent, ArborText, Boeing, Corel, SpeedLegal, the Australian National Archives, and the Society of Biblical Literature were prominent members. The desktop office suites were a minority.
Throughout those first 15 months of phase one work, the interests, influence and persuasive arguments of this “enterprise” side of the original OASIS ODF group greatly influenced the Foundation's goals of a single universal file format. A vision that went beyond the limits of desktop office productivity suites, and into the realm of what has been called the ”grand convergence” of desktop, server, device and web systems.
So what happened at that first meeting that is so significant to the later events in Massachusetts? When the proposed charter was brought up for discussion, Phil Boutros, the legendary conversion - reverse engineering - Über document expert from Stellent, proposed that the charter be amended to include as a primary objective, “compatibility with existing file formats”. Including Microsoft file formats.
Phil argued that if we didn't include in the charter this important objective, in the end our work might prove to be irrelevant to the needs of the marketplace. Compatibility with existing file formats would be absolutely essential to the process of converting those documents to ODf XML. So spoke the legend.
There was wide agreement on this. But being a phone conference, all i can say is that only one member attending put forward an objection. Sun's Michael Brauer, who was also the Chairman, postured that we had to be careful with the wording of this “compatibility” objective. His concern being that people might think we were specifically referring to Microsoft binary documents, and that would compromise our goal of providing an application independent specification.
Phil aughed that of course we were referencing MS documents because those are exactly the documents that the world would be most interested in converting!
The issue of amending the charter was tabled pending future review. A review that never came even though it was proposed and discussed on many occasions. In later years, whenever the issue of compatibility with Microsoft file formats or interoperability with Microsoft applications came up, which was often, the final word ending discussion was invariably, ”That's outside the charter and out of scope”.
And there you have it. So we knew full well, as Louis and his staff explained the facts of life, that we would have a very difficult time getting any compatibility-interoperability with Microsoft eXtensions through the OASIS ODf TC. Louis took the position that the ODf community and vendors would not allow ODf to fail in Massachusetts. He fully expected everyone to support what came to be called the ODf iX proposals.
Louis of course was wrong. The ODf community didn't even show up. Massachusetts was hung out to dry. But he did spend that summer of 2006 doing everything he possibly could to successfully implement ODf using our da Vinci plug-in. Including signing off on the first three ODf iX proposals submitted to OASIS for discussion in July-August of 2006. Between July of 2006 and February of 2007, there were a total of five major iX proposals submitted to OASIS for discussion. Another very important, but alternative set of RDF metadata iX initiatives was approved by the OASIS ODf Metadata SC as part of the metadata requirements in August of 2006.
The long and the short of it is that ODf could not meet the Massachusetts requirements without the five generic eXtensions known as ODf iX. The five generics addressed the OpenOffice – MSOffice application specific differentials concerning lists, tables, fields, sections, and page dynamics. It is the application unique implementation models for these document structures that accounted for most of the problematic conversion fidelity loss.
Many people believe that the only way ODf can establish high fidelity “round trip” conversion
compatibility – interoperability with Microsoft documents, applications
and processes is to have the secret binary blueprints. The thing is,
our da Vinci plug-in was designed as a clone of the MS-OOXML
Compatibility Pack plug-in, and, as such, leveraged the internal
conversion process native to MSOffice apps. We don't need the secret
blueprints. Microsoft does a good job of converting the binaries for us
We even released ACME 376 as proof that we could in fact hit a conversion fidelity the equivalent of the MS-OOXML plug-in.
The important thing to understand is that an internal conversion process differs greatly from trying to import and break an external stand alone MS binary document. There are two aspect to internal conversion – both of which are critical to conversion fidelity. The first aspect is that of triggering, capturing and decoding the internal application conversion of in-memory-binary-representations. The second is that of piping the decoded structure into the target file format.
For short we call these two aspects ”conversion” and piping. And we have cute names for the actual components that perform these tasks within da Vinci.
Acme 376 is the first aspect of the da Vinci process. The second aspect is called ”InfoSet”, and it is here that the piping into a target file format occurs.
ACME 376 can do the job it was intended to do. But without ODf iX, we are unable to pipe the results into ODf without loss in the areas of our five document structures. Trying to pipe into ODf as it's currently written is the equivalent of trying to pour 100 gallons of jet fuel into an 85 gallon tank.
The amazing thing is that we can pour that 100 gallons of jet fuel into the W3C's CDF, with plenty of room to spare. We don't need any eXtensions or special exceptions to do this. As this dawned on us after months of testing, we we're speechless. If you've spent any time converting MSOffice binaries and xml, CDF will take your breath a way. All i can say is that the W3C CDF Workgroup has done an amazing job of writing a flexible and expansive framework able to handle everything we know MSOffice has to throw at it. Even with the years of business development on the MSOffice platform that must be accommodated.
And oh yeah, the 130 critical workgroup test documents that ODf failed with in Massachusetts are not a problem for the W3C's CDF. Wow. If only we knew this in July of 2006.
The failure of ODf in Massachusetts has had a devastating world wide impact. People accuse of us of failing in Massachusetts. We did. No question about it. Yet one has to wonder, where were the faithful when ODf hung by a thread?
On October 4th, 2006, with the resignation of CIO Louis Gutierrez, ODf was branded in those hushed CIO conversations where pragmatic solutions trump both ISO and legislative mandate efforts every time, as difficult and perhaps impossible to implement. Just as the legend Phil Boutros had warned near four years earlier.
Following the resignation, we stopped all work on ODf da Vinci, and went back into the OASIS ODf process to finish the iX work we had started. The da Vinci plug-in and ODf iX were inextricably tied. Neither could move without the other. But the subsequent events at OASIS turned out to be one disaster after another. By April of 2007 we had failed with the “List Enhancement Proposal”, our metadata requirements were dropped and discarded, our interoperability issues brushed aside, and even the OpenDocument Foundation's purpose for being had been totally gutted.
This ended our work on ODf (April of 2007). And the search for a worthy alternative to MS-OOXML began.
Our feeling was that the marketplace of 550 million MSOffice bound workgroup desktops would migrate to MS-OOXML unless we were able to offer them an alternative. The thing is, the world is going to migrate to XML, regardless of what happens at ISO. It's a pragmatic reality. The only question is, “Which XML?”
The hard truth is that ODf was not designed for the conversion of existing MSOffice documents, applications and processes. MS-OOXML on the other hand, was designed exactly for this purpose. And that's the challenge we're up against. We need an alternative to MS-OOXML that meets the same critical market requirements. Otherwise, we might as well all pack it in as we stand by and watch Microsoft migrate those MSOffice bound business processes over to the core of the emerging MS Stack, the Exchange/SharePoint developers hub.
We believe the W3C's CDF can fill that role as an effective, easy to implement and transition to alternative. We know for certain that ODf can not full fill this task, as events in Massachusetts proved.
We also believe that we need to neutralize and re purpose MSOffice with CDF for another reason. The Exchange/SharePoint juggernaut is now at 65% marketshare and accelerating. The advantage E/S has over all other competitive alternatives is superior integration into existing MSOffice bound business processes through the systems level binary <> MS-OOXML conversion function. If E/S has 65% marketshare, so does MS-OOXML.
Furthermore, it's becoming ever more obvious that W3C Web platform technologies are being relegated to the bottom by whole raft of higher level MS Stack proprietary “foundation” replacements. ODf isn't the target of MS-OOXML. HTML is. With the prize being the future of the open Internet.
We might not be able to rip out and replace MSOffice, but at least we can neutralize and re purpose the business process mainstay using CDF and the available plug-in architecture.
My last point is that there were three primary market requirements in Massachusetts: compatibility with existing file formats, including Microsoft documents; interoperability with existing applications, including MSOffice; and the grand convergence, of desktop, server, device, and web systems.
Of these requirements, the first two relate to the legacy situation much of the world finds themselves having to deal with. The third requirement of grand convergence however represents the Massachusetts vision of the future. And it is here that the real strength of CDF comes into play. Grand convergence is the sweet spot for CDF, exactly what it's designed for. What we have to now do is enable the great herd of 550 million desktops so that they can easily , effortlessly and without disruption, consider the transition to CDF as a fully capable alternative to MS-OOXML.
Because so much of the world's attention was focused on Massachusetts, we were privileged to have had many conversations with CIO's the world over. We know the reality of these requirements is undeniable and valid worldwide. So much so that we often have to answer the question as to how it came to be that ODf was not designed to meet these requirements? In recent conversations concerning our work with the W3C's CDF, one analyst commented that, “ODf is a fine format for an alternative universe where MSOffice doesn't exist”. And while many wonder how it came to be, we really don't have the luxury of time to figure out how it is that five years of work on ODF could have missed the mark for 550 million desktops. But miss we did.
IMHO, the reasonable course of action going forward has to be an all out effort to provide pragmatist, who find themselves bound to MSOffice workgroups and business processes, with an alternative to MS-OOXML. Otherwise, it doesn't matter what happens at ISO. MS-OOXML will be the default file format for the majority of businesses and organizations going forward. If the W3C's CDF can be used as an effective alternative to MS-OOXML, this ought to be done yesterday. If those who disagree with Massachusetts and the market requirements continue to insist that ODf can be implemented as a non disruptive alternative to MS-OOXML, then they really need to get out of the blogosphere and into the trenches where pragmatic solutions rule, and blather walks.
Hope this helps.
~ge~
Recent Comments