The Existing Perl Modules (Cont.)

The DataSource Module

As you can see, the DBI interface is an extremely powerful tool. However, we realized right away that the DBI architecture was far too limited to really cover the needs of web applications.

Specifically, data should be able to be drawn from many more sources besides standard databases. For example, in the real-world, many web sites do not have access to databases. These sites, hosted on ISPs, have only a CGI-flatfile capability at best. Their ISPs will not provide them database services. And even if they did, such services may well be more costly than the typical web site owner can afford.

Further, the administration of a database may be more work than the typical website owner wants to deal with.

Finally, for many data storage needs, a full-fledged database may be an overkill. For typical mailing lists for example, a site may get 20 or 30 signups per week. Why bring in Oracle for a mere 20 or 30 additions per week?

Further, data should be accessible across the network. Data should be accessible via HTTP or via another CGI script entirely.

Thus, products for an "outlet" web store might contain products from 10 or 12 other web stores across the net. The outlet store must be able to grab product information from a dozen networked locations and be able to present them in a single interface.

For this reason, we devised the DataSource module for accessing data in a manner that is suitable for web applications.

The DataSource module wraps around and incorporates the functionality of DBI but adds on the ability to think of a data source not as a database but any data structure such as a local flat file, a CGI script, a URL-accessed flatfile, or even hierarchical/object-based systems such as entire file systems, XML documents, and Object-Oriented databases.

It is important to note that DBI is not the primary interface to Datasource. DBI is merely a mechanism for implementing a Datasource. The reason for this is that DBI is relatively heavyweight unless you need the power of a database such as mySQL, Sybase, or Oracle. However, for doing simple file-based things, we wanted a simple, lightweight interface without worrying about SQL parsing. Furthermore, we wanted to support non-SQL oriented Datasources such as XML.

The following is a summary of the design goals of Datasource:

• Provide a lightweight way of abstracting simple files such as log files, database flat files, to message files.
• Provide an easy migration path to SQL databases so the core CGI applications can scale.
• Provide flexibility in the types of data to be accessed and include distributed files as well as hierarchical datatypes in the mix.
• Provide an interface to common tasks such as querying, updating, deleting, adding elements on a Datasource.
• Provide an automatic mapping for certain common fields.
  • For example, modification_time is a field in the file-based data source which automatically checks the -M flag of the file in Perl. However, in the DBI Data source, modification_time would be a field that can optionally be auto updated by the data source when a record gets added or changed in the database.

    This will allow a true plug-n-play system where objects that you would not normally think of as a Datasource can be transferred easily to a database. For example, Message files in a Forum subdirectory could be mapped to Message records inside of a Forum table in a relational database.

Inside an application, DataSources become everything from logs, session files, and shopping carts to temp files, data files, and user files.

The Mail Module

We also saw a need to utilize the interface architecture to handle emailing on multiple platforms. In this new methodology we have created a single Mail module with a send() method. The send() method is implemented on several underlying mail dependent modules including Sendmail (UNIX), SMTP mail (Mac), Postie (Windows) and Blat (Windows). The benefit of this architecture is that modules can quickly be written for every email package available to a site administrator if for some reason they do not wish to use the basic set we provide.

The Encrypt Module

Similarly, encryption will be handled as an interface with an API of encrypt() and compare(). Underlying dependent modules will include PGP, crypt and MD5 but can be added to in the future to include many more encryption technologies, especially those for e-commerce.

The Authentication Module

We will also rewrite the current authentication libraries to take advantage of the DataSource, Encryption, and Session modules.

The Session Module

We will provide services for dealing with the storage and maintenance of session information including the use of hidden fields or cookies to maintain state.

The Search Module

The capability of searching HTML and other data files recursively live or indexed will be incorporated as a module for applications that require this. For example, a BBS allows you to search BBS messages and the HTML template based web store allows you to search the HTML in the Web Store for a product.

The DataValidation Module

We will provide a module for basic validation of user input that is a frequently requested service for many applications. The module will include validation services for:

• Date (US and European)
• Price
• Credit Card Numbers
• Email Addresses
• Censored Data
• Bad Characters
• Integers
• Floats
• Letter characters only

The Application Modules

Finally, we will provide application-level objects that will allow the current suite of web applications to plug-n-play with each other.

Currently, our applications are relatively standalone except for libraries that deal with infrastructure such as authentication and mail libraries. However, adding BBS functionality to a web store is currently difficult without cutting and pasting a lot of code. In some cases, because we are using procedural code in the current applications, there may be naming conflicts when the cut and paste technique is used.

The solution is that the next generation of scripts will be made up of loosely coupled application components. For example, a BBS would have some of the following major components:

• Message
  • Post
  • Reply
  • Retrieve
• Forum
  • RetrieveMessages
  • SearchMessages
  • SortMessages

Further, a generic web store might have the following application-specific components:

• ShoppingCart
  • RetrieveCartContents
  • DeleteItem
  • AddItem
  • UpdateItem
• Item
  • Get/SetItemID
  • Get/SetItemPrice
  • Get/SetItemQuantity
• Order
  • CalculateShipping
  • CalculateDiscount
  • CalculateSalesTax

These models are OK for standalone applications. But the power will come when a developer can integrate these objects together to create hybrid applications. For example, the Forum object could be placed into the Web Store in order to allow users to chat about the products that they are looking for. In fact, a virtual salesperson could be on call watching for messages to appear, and thus increase the interactive experience of shopping on the Net.

In addition, loosely coupled objects will allow us to change the view of how an application gets developed. For our suite of applications, this will mean that applications that are very similar will get merged. For example, the Chat script is really a specialization of the BBS. With the next generation of Application objects, a different mixture of objects will form a Chat script with minimal coding once the main BBS application is glued together.

After these core applications have been constructed as examples of how to use the components, the new concept of releasing web applications will be to release them as different Views of data. For example, a calendar script will be easily modified to give month, day, week, quarterly, and yearly views. Applications will simply consist of logic that different views will use to glue the core application logic objects together.

The Existing Perl Modules
Extropia WebWare Suite 2.0: Towards a New Application Development Framework for Server-Side Web-Based Applications in Perl and Java
Suggested Areas for Community Development