WebServices dot org

Introduction

Web services are somewhere around the crest of their hype cycle and currently the darling of the prevalent media. This cresting is like that of other technologies in that it precedes full development and maturity. Web services, an undoubtedly important technology regardless of media interest, have a good deal of development ahead of them. Those who find success using Web services will be those who understand the technology fundamentally: its motivations, the reasons why some components are winning out over others, and the likely course of maturity.

For this reason, I start with the history of Web services. This is no mere nostalgic side-trip: the business and technical environment into which Web services was conceived, and the various players that have waxed and waned in prominence in their history to date are likely to have a strong effect on the future of Web services. You can already see this happening with developments such as the emerging role of Organization for the Advancement of Structured Information Standards ( OASIS ) as incubator of security, workflow and transaction standards for Web services. OASIS was once seen as the very opposition to mainstream Web services.

The stage set for Web services

Distributed application development has been an important field ever since typical computing moved from encapsulated jobs on centralized mainframe computers to peer-networked minicomputers and workstations. As this developed, the IT manager was still usually master of all he or she surveyed. The strategic concerns of distributed development were confined to whatever issues allowed the IT manager to accept the necessary inputs and deliver the necessary reports to management. Keeping a well-integrated data center was key, and supporting heterogeneous platforms and environments not so important. The areas where such things were important, say the electronic data interchange (EDI) shop were usually completely separate entities within the organization.

For some reason the later minicomputer era is often omitted from histories of distributed computing, even though it is the scion from which almost all the fundamentals of Web services sprang. This is probably because it overlapped, and was overshadowed by the PC revolution, which didn't really figure significantly in distributed computing until the commoditization of the Internet. The distribution of services across multiple workstations, which was encouraged in minicomputers as a way to scale and organize the data center, required a variety of approaches to communications, both synchronous and asynchronous, and low-level facilities for integration with arbitrary applications. As a result, the prevalent architectures and operating systems of the period, such as DEC VMS, Tandem and HP and Sun UNIX variants developed distributed messaging technologies that would astonish today's self-styled pioneers with their sophistication.

IBM came a bit late to all this with MQSeries. Seeing requests for applications to work across minicomputer platforms and its mainframe platforms, it purchased the popular ezBridge software which could already do a lot of this and grew it into MQ, which is the main technology from this era to flourish to this day. But messaging often required novel ways of thinking for programmers bred on procedural languages, who wanted to simply have familiar function invocation systems for remote procedures. At the very end of the 80s, the Distributed Computing Environment (DCE) emerged with an initiative to standardize the various competing remote procedure call technologies. This effort consciously omitted messaging technologies, and for a variety of reasons, mostly political, never attained widespread industry support. By this time, a new (though not necessarily better) generation of distributed computing was emerging.

At this time the IT manager was moving into the executive ranks: information systems had become fundamental parts of a firm's strategy and a prodigiously growing part of firms' budgets. The realities of organization meant that computing in most companies stopped being concentrated in a single datacenter, and instead exploded into a variety of departmental systems. Systems integration became one of the most important considerations in the choice of technology and instead of standardization on a platform, IT managers looked for standardization on a networking technology. IT shops adopted object systems as the standard for software development because of the promised benefits of maintainability and code reuse which promised to slim IT budgets and improve return on investments. As a result the emerging distributed technologies took on a strong object flavor.

Common Object Request Broker Architecture ( CORBA ), like DCE, also came out of an industry consortium effort to standardize on distributed procedure technology: this time the large Object Management Group ( OMG ) which focused on requests on remote objects for cross-platform distributed applications. Because of its attempt to make issues such as object state and lifecycle management transparent, CORBA proved less scalable than messaging technologies and DCE. The same problem plagued Microsoft's answers to CORBA: the Component Object Model (COM) and the DCOM remote object protocol. At the same time, e-mail and the Web were proving themselves the most successful distributed architectures ever, and designers sought distributed development technologies that provided the looser coupling of messaging technologies and the ubiquity of the Internet. Standards that were embraced by all major vendors were another wish-list item as they would reduce the risk associated with choosing such technologies.

During this period other more specialized distributed technologies were emerging. The popularity of Java led to its specialized RMI protocol, and the niche successes of MQSeries led to Microsoft and Java messaging flavors.

So, as the new millennium approached, the stage was set for a new generation of distributed computing, based on information-systems needs and experience with network technologies to date. The prevalent needs are:

• Suitability both for distributed operation within an application, and the use of generic services across applications. In otherwords: the ability to support both software developers and systems integrators.

• Suitability for exchanges within an organization and between organizations, requiring cross-platform support and a data-driven focus.

• Concordance with existing Internet infrastructure as much as possible.

• Ability to scale as the number of nodes, heterogeneity of nodes and the complexity of each node's needs increase.

• Solid internationalization.

• Tolerance of failure. Networks where nodes are very tightly coupled together often suffer catastrophic failure when one node goes down. This is a serious problem for heterogeneous networks.

• Strong support in general software development and business workflow management tools from a rich choice of vendors.

• Suitability for the most trivial request/response exchanges as well as handling the most sophisticated orchestration, transaction and security concerns where necessary.

Web services emerge

In its own high-end UNIX platforms, HP trod a remarkable path directly from the minicomputer era of distributed computing to modern Web services. In the early to mid 90s, HP Labs began to explore how to reduce the well-known technical and cost problems of distributed systems. The resulting design principles which led to HP's e-Speak, released in late 1999, addressed most of the needs outlined at the end of the last section, and e-Speak emerged as probably the first Web services technology, and certainly the first commercially marketed Web services technology. It used generic protocols such as HTTP and an XML data representation to treat all manner of networked systems as "e-services" into which one could rapidly plug data streams. Unfortunately, because its vision is far more coherent than the current state of Web services, HP has recently suppressed e-Speak in favor of a more mainstream Web services offering.

The same insight into how to use HTTP and XML technologies to meet the above needs came to the less corporate UserLand community, whose leader, Dave Winer, led the development of XML-RPC , a very simple (the specification is only a few pages) system for calling functions on a remote server. XML-RPC is still very popular, especially in the open-source community where peer pressure and the development of many open and license-free implementations has led to high levels of interoperability, and low cost. Perhaps inevitably, given its shoe-string roots and early arrival, XML-RPC has some glaring deficiencies in meeting the needs I outline. For one thing, its bewildering insistence on ASCII strings alone (despite its using the well-internationalized XML) means it is really not suitable except in English-only contexts. It also has a rather haphazard choice of data types. But most importantly, it is confined to simple request/response services with very uniform and highly structured message types.

There were also more general messaging formats for XML emerging at about the same time as XML-RPC. Many of them were really more geared to the serialization of programming language constructs and database dumps than document-oriented messaging, but they were still more flexible than the strict RPC approach. The most prominent of these is Web Distributed Data Exchange ( WDDX ), originally by Allaire, which is now an open specification with a significant community of its own.

Meanwhile, XML and Internet protocols were revolutionizing the thinking of a separate group as well: the EDI industry. Firstly EDI over SMTP (e-mail) and HTTP (Web) emerged as a way to dodge the expensive transaction fees of Value-Added Networks (VANs). Next, groups such as CEN/ISSS in Europe and the XML/EDI group in the U.S. (largely) started working on ways to encode EDI transactions, which are notorious for looking like line noise, as XML. These trends led to early forms of Web services that were inspired not by the need for enterprise application integration (EAI) but rather for business-to-business transactions. These early XML/EDI systems were also designed to take advantage of the stable and well-understood EDI mechanisms for orchestration, security and other such matters that are still fuzzy in Web services space. The effort to turn this into a formal standard became e-business XML (ebXML), which, though mostly comprising the usual EDI stalwarts, was quickly joined by a vendor more usually associated with procedures and objects: Sun. ebXML, which started its self-imposed eighteen-month development period at the end of 1999, is a joint project of OASIS, an SGML/XML pioneer, and the United Nations Centre for Trade Facilitation and Electronic Business (UN/CEFACT), a key organization in the development of traditional EDI.

Also, back in 1998, a little specification for structured exchange of XML documents began brewing among a small group of kindred organizations (including the titan Microsoft): Simple Object Access Protocol was for various political reasons not released to the public in its original forms, and the SOAP we knew took the stage in late 1999.

SOAP springs forth, and supporting languages proliferate

SOAP was born into controversy. From worries about the specialized type system it incorporated to the jockeying for position between the SOAP camp (publicly led by Microsoft) and the ebXML camp (publicly led by Sun), it was clear that the major players knew that Web services were the next big thing. SOAP also spread through the grass-roots Web services camp through advocacy that started in the same community as XML-RPC: Userland's . It was not as simple as XML-RPC, but it did address some of the earlier specification's shortcomings, and open-source developers were amazed to see all the major commercial vendors backing such an open specification (not all the surprise had been used up by the success of XML). Seeing the opportunity to wrench distributed development from the strict control of ponderous industry giants and consortia, a vibrant SOAP community sprang up.

SOAP is just a communications protocol. It became clear pretty early on that there would be significant value in a standardized form for Web services metadata, that is, the information that informs the processing of Web services. There were many initiatives for creating such a specialized language. Web Interface Definition Language ( WIDL ), by WebMethods, was one of the earliest ones, targeted as it was towards first-generation Web services systems such as XML-RPC and WDDX. WIDL is designed to mimic Interface Definition Languages that are the basis of CORBA and COM, but in XML form. Microsoft developed Service Description Language (SDL) and SOAP Contract Language (SCL) in 2000 for service descriptions of Web services end points. Microsoft also developed a system for syndicating information about Web services: Discovery of Web Services (DISCO), which would allow users to find SCL descriptions of services.

IBM created competing specifications: the Network Accessible Service Specification Language (NASSL), which paralleled SDL and SCL, and Advertisement and Discovery of Services (ADS), which paralleled DISCO (and was incorporated into the IBM alphaWorks toolkits for Web services. These, along with similar efforts from Ariba and other companies, coalesced into the Web Services Description Language ( WSDL ), originally by IBM, Microsoft and Ariba.

The same trio was also merging their various proprietary discovery systems into Universal Description, Discovery and Integration ( UDDI ), a system for directories (white pages, yellow pages and "green" pages) of Web services. UDDI was released as the product of a consortium of 36 companies, which soon grew to over a hundred. Its products were very elaborate and highly formalized, as distinct from the original SOAP and WSDL specifications, which were very simple and obviously written for straightforward understanding and implementation. This difference is a reflection of the realities of Web services: the low-level technical details such as the core communications and technical service descriptions were expected to grow from the bottom up, by incorporation into a variety of freely-available languages and toolkits, while discovery and business/legal description would have to grow from the top down, according to centralized authorities of information and certification of such information.

This has become a fundamental split in the "personality" of Web services. Firstly, there is grass roots Web services: the scrappy, "desperate Perl hacker" facet which has flourished within departments and separate from overall corporate strategies. Then there is Corporate Web services: a cornerstone of e-commerce strategy, which is chartered in boardrooms. Many of Web services successes have been at the grass roots level, and UDDI has been put forth as an example of how Corporate Web services is yet to find its central value proposition (and how it has in some cases refused to learn the long and hard lessons of the EDI community before it).

Vendors take over the stage

The W3C Workshop on Web services (WSWS), in April of 2001, was a grand exercise in planning the future of Web services. Web services had so far been developed outside the W3C, either by the grass roots or by separate consortia such as UDDI.org . It seemed a natural move to have the W3C bring Web services under the aegis of other Web standards such as HTML and XML. The aim of the WSWS was to charter the form and goals of Web services activity in the W3C. The XML Protocol working group (XMLPWG) had already been formed in the W3C, although it had done little substantive work besides cataloguing the myriad of proposals and candidates for XML protocols. By the WSWS, it was already understood that the XMLPWG would be beefed up for focusing on the task of working SOAP into a full W3C recommendation.

True to the opportunities Web services was providing for reviving all manner of technology strategies, the WSWS plumbed a dizzying array of considerations to build upon the base protocol:

• IBM looked to emphasize network management issues such as quality of service (QoS) with its Web Services Endpoint Language (WSEL), as well as orchestration of Web services (formal description of the business processes underlying service exchanges) with Web Services Flow Language (WSFL).

• Hewlett-Packard offered the broader and more ambitious approach to Web services that characterized e-Speak, discussing the need for ontologies (formal, machine-readable descriptions) of the core concepts underlying any particular service, as well as the need to keep a variety of service environments in mind, especially the embedded space and small-device computing.

• Microsoft also looked to emphasize QoS, and along with IBM presented the vision of Web services that the two companies had been jointly pursuing.

• Jamcracker and BEA emphasized the importance of Web services in systems integration.

• Verisign emphasized the importance of security and the role of digital certificates. Novell emphasized the management of such authorization and certificates in directory services.

• In addition, many vendors were merely showing off their own proprietary frameworks for developing and deploying Web services as future models for any standards to emerge.

There was token representation of the grass roots arm of Web services (Dave Winer of UserLand, early developer of both XML-RPC and SOAP) as well as liaisons from the OASIS and ebXML. The latter had recently ended a major dispute in the Web services community by adopting a variation of SOAP over e-mail with attachments rather than a more elaborate specialized messaging protocol they had earlier developed. I myself was invited to present a paper on how the W3C's Resource Description Framework (RDF) can and should be used for integrating the expression of Web services metadata with other metadata systems.

The struggle for the stack

The main theme of the WSWS was the Web services "stack", or architecture. Position papers suggested where various facilities, such as QoS, security, orchestration and transactions might properly fit into the stack. SOAP and WSDL were firmly entrenched as the core of Web services, so the battle to fill this stack with technologies favorable to the various major players took center stage in the latter half of 2002.

Security has been an area of especially concentrated work. It is well understood that before Web services become prevalent in communications outside corporate firewalls, strong security will have to be in place. There are many aspects of security, and one is just making sure the payload has not been tampered with. The W3C's XML Signature working group works on a system for digital signatures of XML documents, supported by specifications such as canonicalization (c14n). c14n converts XML documents to a standardized form so that the order of attributes and such things that do not affect the meaning of XML documents do not cause false positives for altered payloads. The XML Encryption working group works on encryption, including the XML Encryption Syntax and Processing specification, for encrypting data so that the result is a well-formed XML document. The XML Key Management working group works on simple Web services specifications for management of digital key and trust certification tokens. There are newer developments in 2002 such as Security Assertion Markup Language ( SAML ) and the WS-Security framework. I shall cover these in the next part of this article.

Business process and workflow are another part of the stack that have seen a lot of activity. ebXML is the early player here with its Business Process Specification Schema ( BPSS ) and its registry and repository standards for managing capability and agreement profiles for partners. It also provides long-lived transaction management support, security and QoS. Also early was the Transaction Authority Markup Language (XAML), by IBM, HP, Oracle, Sun and others, which has now melded into other Web services transactions efforts. IBM's WSFL and WSEL covered workflow and QoS respectively. Microsoft's entry, XLANG, covered workflow. WSFL and XLANG are both extensions to WSDL and, as expected, have been pitched in to a generalized business-process flow language WS-Coordination , which, along with WS-Transaction and BPELWS , I shall cover in the next part of this article.

And so to 2002

As we rounded into 2002, Web services were evolving at all levels. At the grass-roots level, where various developer interest groups had set up to sort out the notorious interoperability problems of early Web services. At the corporate level, the Web services stack was coming into place, and vendors were becoming impatient for the next wave of Web services specifications to give the final push to respectability. In part 2 of this article (published at webservices.org next week), I shall map out the present (year 2002) of Web services, and a look at what the future might bring.

------------------------------------
Please use this feedback form to make private comments to WebServices.Org and the author. If you wish a response, we would prefer you used the open forum below, since everyone can learn and benefit from the comments and responses.

-----------------------------------------------
Uche Ogbuji is a consultant and co-founder of Fourthought Inc ., a software vendor and consultancy specializing in XML solutions for enterprise knowledge management. Fourthought develops 4Suite , an open source platform for XML, RDF, and knowledge-management applications. Mr. Ogbuji is a computer engineer and writer born in Nigeria, living and working in Boulder, Colorado, USA. You can contact him at uche.ogbuji@fourthought.com .