Introduction to Webware: Developing Web Applications in Python

Jason Hildebrand



Acknowledgements

Thanks to the Webware developers. Much of the information in this paper was drawn from


What is Webware?


A Collection of Kits

Webware is structured as a collection of components or modules, called kits in Webware-speak. These kits are designed to work well with each other, but also to be useful independently as much as possible. This modular design allows alternate implementations of existing kits to be written and allows new kits to extend Webware's functionality.

Anyone interested in developing a web application will certainly want to make use of the WebKit, which provides a multithreaded application server with session support and the ability to execute servlets.

The MiddleKit is used to implement the "business objects" or "domain objects" as they are often called, which make up the middle layer in a typical three tier architecture, between the user interface (i.e. web page) and the storage (i.e. relational database). MiddleKit allows the developer to create a hierarchy of classes, and provides automatic persistence of these objects to a relational database. This will be discussed in more detail below.

There are several additional kits which this paper does not address. The best way to become familiar with them is to visit the Webware Home Page.

  • The PSP kit allows the user to write HTML pages with embedded Python code in a fashion similar to PHP, but with some key advantages.

  • UserKit supports users stored in files or using MiddleKit, and operations such as login, logout, listing logged-in users and check user roles/permissions. This kit is still in alpha, but may be used by early adopters.

  • TaskKit provides a framework for the scheduling and management of tasks which can be triggered periodically or at specific times.

  • FunFormKit is a package for generating and validating HTML forms

  • FormKit is a newly released framework which offers some improvements over the FunFormKit

  • Cheetah is a python-powered template engine which integrates tightly with Webware


Installation


Servlets


Behind the Scenes


A look at Page

Page implements respond by calling the method writeHTML, which in turn calls writeHead and writeBody, which in turn call some other functions to generate the page output. These method calls are represented in the diagram below.

From this diagram it is clear how our writeContent method was invoked to generate the page. A servlet can add or override any of Page's methods to affect how the page is generated. It is a good idea to look through the source for Page (found in Webware/WebKit/Page.py) to become familiar with the methods and their intended use.


Using Inheritance

Typically, a web developer will implement a SitePage class which all the other servlets subclass. The SitePage may define the general look and feel of a page, such as menus, borders and colors (perhaps by referencing an external stylesheet), and may also provide any convenience methods which its subclasses will find useful. This is an excellent way to reduce maintenance costs, as the look and feel of your entire website can be changed by modifying SitePage. Of course, you can extend the idea by having two or more layers of base classes, so that different sections of your site can use different layouts and can be modified independently.

For example, let's say we'd like to implement a menu bar along the left side of our pages. A nice way to do this would be to encapsulate this into a MenuPage class, from which other servlets can inherit. In MenuPage we'd override the writeBodyParts method to call writePreContent and writePostContent functions before and after the content is written (see diagram below). These two new functions can set up an HTML table (or perhaps two nested HTML tables) so that output written from within writeSidebar will appear within a menu on the left side of the page, and output from writeContent will appear within the main area of the page.

Such an implementation should separate layout from content as much as possible. For example, the writeSidebar and writeContent methods should not assume that their output is within a certain HTML table cell, or within any table at all. If done in this way, the layout functions may be changed later to change the menu's size or location without necessitating changes to the content.

Note: This assumes that the person doing the site design is familiar with programming, or can at least work closely with the programmer to divide sections of the HTML page into logical units (menu, content, etc.). If this is not the case, you may want to investigate templating technologies such as PSP and/or Cheetah, possibly in combination with servlets.


Actions and Request Fields

You will want your application to be able to respond to data that the user has entered. Although you may do this "by hand" by examining the request object in writeContent or elsewhere, Webware applications typically use actions to keep things a bit more structured.

Here is a simple example which demonstrates the use of the request fields and actions:

from WebKit.Page import Page

ValidationError = 'ValidationError'

class Test(Page): 
    def writeContent(self,msg=''):
        self.writeln('''
            %s<BR>
            <form method="Post" action="Test">
            <input type="text" name="value1">
            <input type="text" name="value2">
            <input type="submit" name="_action_add" value="Add">               (1)
            <input type="submit" name="_action_multiply" value="Multiply">     (1)
        ''' % msg )

    def actions(self):
        return Page.actions(self) + ["add", "multiply"]    (2)

    def validate(self):
        req = self.request()
        if not req.field('value1') or not req.field('value1'):(3)
            raise ValidationError, "Please enter two numbers."
        try:
            value1 = float(req.field('value1'))            (4)
            value2 = float(req.field('value2'))
        except ValueError:
            raise ValidationError, "Only numbers may be entered."
        return ( value1, value2 )

    def add(self):                                         (5)
        try:
            value1, value2 = self.validate()
            self.write( "<body>The sum is %f<BR>" % ( value1 + value2 ))       (6)
            self.write( '<a href="Test">Play again</a></body>')
        except ValidationError, e:
            self.writeContent(e)

    def multiply(self):                                    (5)
        try:
            value1, value2 = self.validate()
            self.write( "<body>The product is %f<BR>" % ( value1 + value2 ))
            self.write( '<a href="Test">Play again</a></body>')
        except ValidationError, e:
            self.writeContent(e)
(1)
When using actions, the name attribute of the submit button must be the string _action_ followed by the action name.
(2)
The actions method must be defined which enumerates the actions that this servlet supports. This is necessary for security reasons. It is usually correct to include any actions supported by the parent class (imagine a parent class which displays a login box in a side menu).
(3)
Request fields are accessed through the request object using the methods field(fieldname), which returns the value of the field, hasField(fieldname), which returns true if the field exists in the request, and fields(), which returns a dictionary-style object of all request fields. Cookies can be accessed using cookie(cookiename), hasCookie(cookiename) and cookies(). The value(valuename) and hasValue(valuename) functions can be used when you are looking for either a field or a cookie, but don't care which.
(4)
Text fields are received by the application server as strings, and must therefore be converted to floats. We catch the ValueError exception to recognize conversion errors and display an error message.
(5)
This method name corresponds to the name attribute of the submit button.
(6)
The Servlet.preAction(actionname) and Servlet.postAction(actionname) methods are called before and after the action method is invoked. The default implementations are written such that the action method must output the body of the HTML page (the header and closing HTML tag are written automatically). You can override these methods if necessary.

Note: Both GET and POST variables are accessed through the same functions.


Using Sessions

Sessions are used to maintain user-specific data between requests. In Webware, they can be used in three modes: file, memory and dynamic. In file mode, session data is serialized to disk after every transaction. In memory mode all sessions are kept in memory between requests. In dynamic mode (the default) recently used sessions are kept in memory and other sessions are saved to disk after a user-defined amount of time, so that only active sessions use up memory. You can also limit the number of sessions which are held in memory. These settings are located in the workdir/Configs/Application.config file.

There are two ways that the application server can link a specific browser to its session. The default method is to store the session id within a cookie (the application server does this automatically). If your application should work without requiring the user to enable cookies, you might want to instead use the second method, which embeds the session id within the URL. The setting UseAutomaticPathSessions enables this mode.

Warning

There is a security consideration if you enable UseAutomaticPathSessions. If you provide external links from your website to other websites, these websites will be able to see your session ids in the HTTP_REFERER field (this field is passed by most browsers and contains the URL from which the link was activated). One way around this is to avoid using any direct external links. Replace external links with links to a redirect servlet, passing the target URL in the query string. The redirect servlet should then use an HTTP redirect to cause the user's browser to go to the external URL.

Sessions are easy to use. Use

self.session().setValue(name,value)
to set a value,
self.session().hasValue(name)
to test if a value has been set, and
self.session().value(name)
to retrieve the value of a session variable.

Tip: Session variables are not limited to simple variables. You can store sequences and dictionaries, too, or complicated nested data structures. In general, using the current session implementation, you can store any data which can be pickled by the standard Python module cPickle.


MiddleKit

Overview

The MiddleKit is a framework for implementing a hierarchy of business or domain objects. These are the objects that are created, organized and manipulated and stored by the application. Traditionally such information is stored in relational database, and web applications execute queries to retrieve the data and display it.

Even though some databases support object-oriented extensions, some "glue" is usually needed to make the information in the database available to the rest of the application as true objects. The MiddleKit is an object-relational mapping (ORM), a layer of software which attempts to bridge the gap between object-oriented Python and relational databases.

Some of the benefits of using MiddleKit are that writing code to manipulate database objects becomes less cumbersome and less error-prone, and that since the database access is isolated in one layer, it is reasonably straightforward to migrate to a different database in the future. It should also be noted that MiddleKit objects can be accessed from stand-alone Python (i.e. not through the WebKit application server). This makes it convenient for debugging, running automated tests on Middle Objects, or running maintenance scripts which manipulate the objects in some way.

MiddleKit currently supports MySQL and MS SQL Server. Adding support for other databases is straightforward, and work is being done to support PostgreSQL.

Although MiddleKit is still considered beta, it has already been used in several commercial websites. You may run into a few rough edges, but with source code in hand you have the means to help polish these smooth.


The Design Stage

The Model

MiddleKit allows you to design a model, a hierarchy of classes. This model is specified in a comma-separated file (CSV), which is most easily manipulated with the help of a spreadsheet program. I recommend using Gnumeric to create this file.

MiddleKit can use this blueprint to generate SQL for creating the database schema, and Python classes corresponding to each class you defined in the model.

As an example, we will design a system for managing a bookstore. It will keep track of employees, customers, purchases, the items the store sells, and the shelves on which the items are displayed.

Assume the following directory structure. It makes sense to put this directory somewhere in your PYTHONPATH, so that you can import the modules from your web application. Don't put it under your servlet directories, though, since you don't want to make it externally accessible.

Middle/     BookStore.mkmodel/       contains model information
                Classes.csv          class definitions
                Samples.csv          sample data to be inserted into database
                Settings.config      config settings (see MiddleKit docs)
            __init__.py              create this empty file so that the directory
                                     can be used as a Python package

In this directory structure, BookStore is the name I chose for the model, but this name is only really used in a few places.

We start with a few classes to represent employees and customers:

The Person class is an abstract class. We will never instantiate it directly; it is only useful as a base class for Customer and Employee, because those two classes will have many common attributes (name and address fields). We set isAbstract=1 in the Extras column so that MiddleKit knows not to create a database table to hold Person instances.

Parentheses are used to specify the base class, if any. Multiple inheritance is not supported. You are free to name the attributes however you like, but keep in mind that accessor methods will be generated for each attribute. Because of this, attributes are conventionally namedLikeThis. For an attribute fooBar, MiddleKit will generate the accessor method object.fooBar() for retrieving the value and object.setFooBar() for setting the value.

A browse through the source code shows support string, int, bool, float, date, datetime, decimal, enum and long types. Personally I've only used string, int and bool extensively, so I don't know if support for the other types is complete or only partially implemented. We will see below that you can also create attributes whose types are user-defined classes.

The isRequired field specifies whether the attribute may be left blank (i.e. None in Python or NULL in SQL). The min and max fields, if given, specify constraints for numerical types (int,float, etc.). For strings, max specifies the maximum field width.

Employee extends Person by adding an employee id. Customer is a subclass of Person which keeps track of the items a customer has purchased. We will define the Bought class below.

The Bought class represents a transaction in which a Customer buys an Item from the store. The customer attribute is a reference to an instance of the Customer class which we just defined. The item attribute is a reference to an instance of the Item class, which we define below.

Look again at the purchased attribute of the Customer class above. In MiddleKit, these list attributes are implemented using back references. Those familiar with SQL know that it is not possible to store a list of entities in a single column. Lists in SQL are normally implemented using foreign keys and/or cross-reference tables. MiddleKit hides some of this from the user, but it is still helpful to know what is going on.

MiddleKit will not create a column corresponding to the purchased attribute in the Customer class. Rather, when you execute cust.purchased() (for some instance cust), MiddleKit will query the database for instances of Bought where the customer attribute corresponds to cust. In MiddleKit speak, Bought.customer is a back reference for Customer.purchased.

The onDeleteOther='cascade' specifies that if the referenced object (i.e. the customer) is deleted, we would like the Bought object to be deleted as well. By default, MiddleKit will not allow a referenced object to be deleted.

Warning

MiddleKit does not create back reference attributes automatically; it is up to you to add them.

The Shelf class represents a shelf in the store upon which the items are displayed (we assume simplistically that all items of a certain kind will be on the same shelf). The Shelf.items attribute will give us a list of Items on that shelf. Note that the back reference Item.shelf is required for this to work.

Item is an abstract class; it provides a base class for the Book and CompactDisc classes.

In general, a 1 to N relationship, such as that between items and shelves, is implemented by a list attribute in one entity and a back reference in the other. Remember that a list attribute does not correspond to a SQL column; it is more like a macro which relies on the corresponding back reference to produce its result.

An N to N relationship, such as that between customers and items, is implemented by an intermediate object (in our example, the Bought class). Each of the primary objects contains a list attribute, and the intermediate class contains a back reference to each. Note that the intermediate object may contain other attributes as well, if this makes sense (consider the Bought.date attribute).


Inserting Data

You may provide the file Middle/BookStore.mkmodel/Samples.csv in the format as described below. If this file exists, MiddleKit will generate the file MiddleKit/GeneratedSQL/Insert.sql containing SQL insert statements. This is convenient for inserting sample data or migrating data to MiddleKit. Here is an example:

Note: Objects are assigned ids in order starting from 1. Use these numbers if you need to refer to another object. If you need to refer to a subclass, use the form classname.nn, as in the example.


At Runtime


Further Information