Are Server-Side XForms Engines the Future of XForms?

• Erik Bruchez, Orbeon, Inc. http://www.orbeon.com/

Abstract

XForms, the next generation web forms specification from the W3C, promises to make the life of web developers easier by replacing HTML forms. But while some web browser plugins are available, and while the Mozilla project has announced a project to implement XForms, Microsoft Internet Explorer appears to shun the specification. In such conditions, is there hope for XForms to ever take off outside the intranet?

In this paper, we present the result of two years of experience implementing a server-side XForms engine now part of the open source Orbeon PresentationServer. The benefit of such an approach is that it brings XForms transparently to currently deployed web browsers, including Internet Explorer, without requiring the use of plugins or other upgrades.

We consider the parts of the XForms specification that can be easily implemented server-side, and the challenges posed by implementing the remainder of the specification. We also look at how server-side XForms impacts the relationship between XForms and XHTML and between client and server.

Quick Introduction to XForms

XForms is the next generation web forms specification from the W3C. It is a recommendation since October 2003. It aims at replacing HTML forms, as well as providing other host languages such as SVG the ability to handle forms. There is no restriction on what host language XForms is used with, but it is expected to be mostly used in conjunction with XHTML.

XForms provides many benefits over HTML forms, including:

• Clear separation between the data captured and the presentation (separation of concerns).
• Flexible, structured XML-based data model. Define your data model as an XML document, then create forms accessing the data.
• Built-in facilities for validation and data integrity based on XML Schema and/or assertions.
• XForms's approach is very declarative and therefore requires much less, or no scripting at all for many common scenarios that do require scripting with HTML forms.
• Ease of internationalization.
• Support for a variety of devices.

Compared to most other forms technologies, XForms provides the following benefits:

• It is a standard.
• It is not tied to a particular platform or language such as Java or .NET.

XForms was intended as a client-side technology. This means that an XForms engine, like a Cascaded Style Sheets (CSS) engine, for example, was designed to be implemented within a web browser. This was the idea of the specification all along, but if proof is needed, consider for example that the specification has provisions for submitting an XForms instance to a server.

The Problem

After Netscape succumbed to Internet Explorer in the late 90s, the development of client-side web browser technologies has largely stalled. With a market share of over 90% of the installed web browsers base, Microsoft has not seen any point in making any improvement at all and after IE 5, Internet Explorer hasn't had any serious upgrade. Following this trend, Microsoft has said it wouldn't support XForms. In fact, Microsoft has its own competing proprietary technology to push forward: InfoPath.

On the bright side, the Mozilla project has produced an ever more solid browser, culminating with the recent release of Firefox 1.0, and has announced that it was working on an XForms 1.0 implementation. There are also commercial third-party plugins available for Internet Explorer that provide XForms support.

XForms plugins and XForms support in Firefox and possibly other browsers will allow XForms deployment for applications in controlled environments such as corporate intranet applications. However the perspectives for mass-deployment of XForms outside the intranet appear very dim:

• Unless Microsoft changes its mind, Internet Explorer will not ship with XForms any time soon.
• Assuming full XForms 1.0 support in Firefox in the near future, this browser still has a market share of under 10%. Even assuming the most optimistic growth strategy, it is unlikely to relegate Internet Explorer to a negligible market share.

So is XForms dead as the next generation of web forms?

You Said Server-Side?

If XForms on the client cannot be a reality tomorrow, is there a way around it? Can XForms be more than a toy for technology enthusiasts? We believe that the answer is yes.

One of the benefits of XForms is that it provides a high level of functionality to the web application developer. What matters most is what the developers or its programming tools sees, not what the web browser sees. Following this logic, the solution consists in building a server-side emulation layer for XForms. This is what is meant by "server-side XForms".

In such an architecture, the developer writes XHTML and XForms code, as he would with an XForms-aware web browser. A server-side piece of software, the server-side XForms engine, translates this into languages understood by the web browser, namely (X)HTML, CSS, and JavaScript. The following figures illustrate the overall architecture of a server-side XForms implementation:

Server-Side XForms Architecture

The benefits of server-side XForms are clear:

• Currently deployed browsers, including Internet Explorer, can benefit of the XForms technology without the installation of plugins or other upgrades.
• Developers can use XForms without worrying more about differences between web browsers than they currently do.
• XForms has a chance to be used in a mainstream fashion and to grow as a specification.
• Developing XForms extensions such as creating new components, compound components, supporting XPath 2.0, etc. does not require upgrading web browsers.
• Optimizations such as keeping large XForms instances on the server are possible, potentially improving latency, saving bandwidth, and providing enhanced confidentiality.

A Simple Example

Let's consider an example that illustrates how XForms emulation can be achieved. Let's assume an XForms instance as follows:

<credit-card>
    <type/>
    <number/>
    <expiration-month/>
    <expiration-year/>
    <verification-code/>
    <valid/>
</credit-card>

This form may have associated controls looking as follows:

Examples of Credit Card Selection Form

An XForms input control, which is meant to display a text field that captures information into the number element, is embedded within XHTML:

<xforms:input ref="/credit-card/number"/>

A simple approach consists in mapping the XForms control to an HTML form element, for example as follows:

<input name="%2fcredit-card%2fnumber">

When the HTML form is submitted back to the server, the server-side XForms engine receives a parameter name (%2fcredit-card%2fnumber) and a value (the value entered by the user in the text field). The XForms engine can then decode the parameter name to /credit-card/number, associated it with an XForms instance XML document, and store the associated value into the XForms instance.

In reality, the strategy is a little more complex, because features of XML such as attributes and namespaces must be taken into account. The gist of it however is that there must be a mapping between the name of an HTML form control and a certain element or attribute of and XForms instance. How cleverly this mapping can be done is left as an exercise for the reader.

Techniques

There are several techniques useful to emulate an XForms engine server-side:

• Markup rewriting
• Full client-server round-trips
• "Ajax"

We explore these techniques in the following sections.

Markup Rewriting

This technique is mostly useful to render XForms controls As the simple example above illustrates, converting XForms controls to HTML controls is fairly simple. The technique mainly consists in converting XHTML and XForms into (X)HTML and possibly scripting. Most XForms controls have already an equivalent in HTML. There are exceptions, for example:

• The XForms range control, which is missing from HTML.
• Open list selection, which allows entering free text in addition to a list of choices such as a dropdown list.

Such controls can however be emulated in HTML:

• A range control can be composed of images and JavaScript code to increment and decrement values on a scale.
• An open list selection may be composed of a text field and a drop-down list, which the server-side XForms engine then combines together.

Examples of Controls with Open List Selection

Differences in Processing Model

HTML forms mandate a very simple processing model:

• The user fills-out data in a form.
• The user submits the form to the server.

There is no level of interaction between the user and the form other than filling-out data. Additional interactivity may be provided through scripting, but that is above what simple HTML forms do in standard.

XForms on the other hand provides for much more advanced interaction. For example:

• The XForms switch module provides for a "construct that allows the creation of user interfaces where the user interface can be varied based on user actions and events".
• XForms supports actions and events based on XML Events.
• XForms controls can be validated by XML Schema or by model item properties, and display adequate alert messages when controls fail validation.
• The XForms repeat construct allows insertion and deletion of rows within repeated structures, such as a list of line items in a purchase order.

Example of XForms Repeat

Such advanced features cannot be emulated directly by HTML forms. A little more work must be performed, with the following aspects:

• Processing: obtain the information necessary to realize the update
• Rendering: present the updated information to the user

Consider the XForms repeat example: when a user presses an "Add" button, a new row must be displayed. Following the distinction above, some code must:

• Determine what new information must be presented, i.e. what fields and other HTML constructs must be repeated, and where.
• Actually perform the update of the page based on such information, which will result in a new row being added.

From the client's perspective, the information can be obtained in one of two ways:

• Locally: using scripting, based on information present in the web page itself, whether as HTML markup or JavaScript variables.
• Remotely: on the server, where information can be calculated based on state information provided by the client, or state kept on the server.

Still from the client's perspective, rendering can be done in one of two ways:

• Regenerate the entire HTML page
• Modify locally the HTML page, using the DOM APIs and scripting, to reflect the change

Full Client-Server Round-Trips

The full client-server round-trip strategy is a simple way of achieving complex XForms behavior. Following-up on the XForms repeat example, let's assume the user clicks on the "Add" button. The following sequence of events takes place:

• The HTML form is submitted to the server.
• The server-side XForms engine detects, based on the fact that the "Add" button was pressed, that a new row must be inserted.
• The server-side XForms engine identifies the elements that must be duplicated in the XForms instance, and updates it on the server.
• The server-side XForms engine regenerates the page. The xforms:repeat control, based on the new elements in the XForms instance, generates one more row.
• The client renders a complete updated HTML page.

While the HTML form is submitted to the server, the form is not submitted to the application from the XForms programmer's point of view: the submission is intercepted by the XForms engine first.

XForms Repeat with Newly Added Row

This solution is straightforward and does not require large amount of client-side scripting, if any. It can be used in such situations as:

• The client does not support scripting, or only limited scripting.
• Calculations are complex, for example XML Schema validation must be performed, and it is easier to perform it on the server because all the validation tools are available there.
• Calculations involve state information that is confidential and must not be tampered with.

However, full round-trips imply latency, consume network and server resources, cause web pages to refresh, and therefore provide a suboptimal user experience.

Keeping State

An XForms engine works with one or more XForms models, which may each contain an XForms instance. Such an instance is simply an XML document. The life cycle of XForms processing operates on one or more XForms instances. In addition, XForms submission requires that a complete XForms instance must be submitted to a server.

This means that the server-side XForms engine must keep at least one entire XForms instance as part of the state associated with a page containing XForms controls. As with most web application frameworks, there are a few strategies to keep state, mainly:

• Server-side, stored in association with a session. Note that programming tricks can be used to make sure that pages do not use the same state information, even if it is stored in association with the same session.
• Client-side, stored in a hidden HTML form element or in JavaScript variables.

Storing state client-side has the following advantages:

• Less server resources are consumed.
• Implementation can arguably be easier.
• State information does not expire with the server-side session.

Storing state client-side has some disadvantages:

• Pages are larger, and more so when XForms instances grow large. This can be alleviated by compressing XForms instance data before storing it in HTML hidden fields.
• The integrity and confidentiality of XForms instance data has to be guaranteed. This can be solved by encrypting and signing XForms instance data before storing it on the client.

Whether state is stored client-side or server-side can be an option of the server-side XForms engine.

Supporting Client-Side and Server-Side XForms

A server-side XForms engine may also support user agents that implement XForms. This requires that the server-side XForms engine be either configurable by the developer, and/or that the XForms engine be able to detect user agents that support XForms. A server-side XForms engine can this way make transparent to the developers several types of user agents, including the following main categories:

• User agents supporting client-side XForms: the engine forwards mostly unmodified XHTML + XForms to the user agent.
• User agents supporting limited scripting functionality: the engine rewrites XHTML + XForms to include limited client-side scripting, and is able to perform full client-server round-trips.
• User agents supporting enhanced scripting functionality, including Ajax: the engine rewrites XHTML + XForms to include more advanced scripting and Ajax support.

Impact on the Client - Server Relationship

Server-side XForms splits the XForms engine into two parts: one part acts on the client, and is composed of script and state information; the second part acts on the server. Between the time an application produces XHTML + XForms, until the time the form is submitted to the application zero or more interactions may occurs between the client-side part of the XForms engine and the server-side part, whether producing full client-server round-trips, or whether using Ajax techniques. This has an impact on debugging. Documents that would be observable simply by viewing a page source within a web browser must now be observed at the server-side XForms engine level.

Relationship with Web Forms 2.0

A recent W3C submission by Opera Software introduces Web Forms 2.0, an incremental improvement over HTML forms. As mentioned in that submission, it does not compete directly with XForms 1.0. Instead, it can be a facilitator for implementing server-side XForms, by providing for example:

• Validation features such as a type attribute on inputs fields allowing user agents to present specialized controls and regular expression matching.
• A repetition model that can be helpful to implement the XForms repeat module.

Whether Web Forms 2.0 will be adopted and become a de facto standard remains to be seen. At the moment, XForms 1.0 has the benefit of being an actual W3C recommendation.

Current Implementations

There are several server-side XForms implementations on the market. None covers totally the XForms 1.0 specification yet, but they are getting close. Implementations include:

• The Orbeon PresentationServer (OPS) XForms engine
• Chiba

The OPS implementation started with using very simple rewriting techniques, then progressively included more advanced techniques to keep state and exchange information between client and server. It aims at supporting Ajax in its next release.

Conclusion

We believe that server-side XForms is the most promising solution today to give XForms a real chance in the industry. From a technical standpoint, the technology to implement it is adequate, and implementations are getting more mature as months pass.

Acknowledgements

Thanks to Alessandro Vernet for his input.

Bibliography

[XFORMS10] XForms 1.0 W3C Recommendation.

[AJAX] Ajax: A New Approach to Web Applications.

[WEBFORMS20] Web Forms 2.0.

[OPS] Orbeon PresentationServer.

[CHIBA] Chiba.

Biography

Erik Bruchez

Software Architect, Orbeon, Inc., Orbeon, Inc. http://www.orbeon.com/

Erik has extensive experience in the software industry as an architect and consultant. He was involved with World Wide Web and Java technologies since their inception in 1992 and 1995 respectively. He has worked in all the areas of Java development, including constrained environments (J2ME), desktop applications (J2SE) and enterprise software (J2EE). A former employee of Symantec Corp. in Cupertino, he contributed to the VisualCafe for Java product line, before co-founding Orbeon, Inc., a company specialized in J2EE and XML consulting. He consulted and developed applications for companies such as WorldGate Communications (J2ME), Manugistics (supply-chain management), Sun (development tools), Volkswagen Financial Services (financial applications), and the City of San Diego (administrative applications). He is currently a specialist of web applications architecture and the architect of Orbeon PresentationServer (OPS), an open source J2EE-based platform for XML-centric web applications developed. Erik is the co-author of several articles on J2EE and web application development. Erik holds a MS/CS from the Swiss Institute of Technology (EPFL) in Lausanne, Switzerland.