8—
Adding a Macro Language

Python made it easy to write the core of our application, but just about everything so far could have been done in one of many different languages. In this chapter, we move to one of Python's greatest strengths: making applications extensible. In the process, we'll learn a lot about how Python works internally.

Many popular applications (notably the Microsoft Office family) have a macro language that allows users to customize the way the application behaves. The kind of things users should be able to do are:

Write scripts
Users can write arbitrary scripts in Python that work with BookSets and Transactions. These run from within our GUI application or independently.

Handle events
Adding, editing, and deleting transactions are good candidates for events users can hook into.

Create validation rules
Validation rules can get you closer to a robust accounting system. With the right events to trap, users can produce their own rules to ensure the validity of their data.

Create user-defined queries
Some queries have been hardcoded into our system. You can generalize this concept to specify the inputs and outputs to a query and allow users to write their own. These would need some limitations, e.g., return a 2D array suitable for display in a grid.

If you start writing your application in a compiled language such as Visual Basic, Delphi, or Visual C++, you'll find it hard to add the macro capability. Writing your own macro language is not a small task, and there is little point when so many exist. Even if you decide to use Python as your macro language, you still need to write a huge amount of code to bridge between, for example, your Delphi objects and methods and your Python macros. However, we've written the core of the application in Python already. This makes it easy to create an extensible application. In the next few sections we'll run through some of the capabilities you can add to open up the application to users.

Dynamic Code Evaluation

With the right documentation, your users can already write Python scripts that create and manipulate BookSets and Transactions. However, these scripts are totally detached from your browser application. What would be smart to do is to provide a command window within the browser that works on the current BookSet. If a user creates a custom import function and runs it, he could hit an update key and see the new records on the screen immediately.

Executing Commands and Evaluating Expressions

Python has built-in functions and statements to facilitate this dynamic code evaluation: eval (expression, [globals[, locals]]) is a built-in function that evaluates a string, and exec expression, [globals[, locals]] is a statement (not a function: no parentheses needed) that executes a string. The following clarifies how it works:

>>> exec "print 'this expression was compiled on the fly' "
this expression was compiled on the fly
>>> exec "x = 3.14"
>>> eval("x + 1")
4.14

Let's pause for a moment and consider the implications of this code. You could pass a chunk of text to a running Python application, and it's parsed, compiled, and executed on the fly. In Python, the interpreter is always available. Few languages offer this capability, and it's what makes Python a good macro language.

You may not have consciously absorbed this until a few lines back, but Python distinguishes between expressions and statements. What happens if you aren't sure what the user wants and get it wrong? Here are two more examples:

>>> exec "x+2"         # try to execute an expression - nothing happens
>>> eval("print x+3")   # and evaluate a command - causes an exception
Traceback (innermost last):
  File "<interactive input>", line 0, in ?
  File "<string>", line 1
     print x+3
        ^
  SyntaxError: invalid syntax
>>>
Case Study: Formula Evaluation

In 1997, one of the authors was consulting for a large packaging company building a database of its packaging designs. A cardboard box has a certain geometry, which may be thought of as a collection of flat panels joined at the edges, with a series of folding instructions. The overwhelming majority of designs were for cuboid boxes. This allowed the length of a certain edge of a panel to be expressed as a formula such as ''2L + 3.5W + T,'' where L is the length of the product to be packed, W is the width, and T is the thickness of the board. Packaging designers all over Europe had been patiently entering formulae for each panel of a design into a database for months. An urgent need arose to verify these formulae. Using Python, it was a simple exercise to load a dictionary with the standard variables (L, W, H, T, and a few more for various flap lengths), plug in some numbers, and evaluate the expressions. Those cases that generated exceptions could be identified easily. A Python script was rapidly produced that queried the database and verified the integrity of all the strings. In addition to checking for errors, it produced a report listing standard panel sizes for a generic 400×400×400mm box. Meanwhile, the main development team spent a great deal of time looking for and eventually writing a library of their own to evaluate simple numeric expressions in PL/SQL.


Executing an expression is generally safe, but evaluating a statement causes a syntax error. If the user gives a single line of input to process, and you don't know what it is, you can try to evaluate, then execute it if an exception occurs.

The exec function accepts not only single-line statements such as the previous example, but multiline ones as well. These can range from a two-line loop command to print the numbers 1?0, to a function definition, or even a 500-line class definition.

You may be wondering where the variable x was stored in the previous examples. To understand this, you need to delve into some Python internals. The previous console commands are executed in the global namespace, which means that x becomes a global variable. There is a function called globals() that allows you to examine this namespace directly; let's see what it returns:

>>> # where is the 'x' kept?
>>> for item in globals().items():
?nbsp;    print item
?BR> ('__doc__', None)
('pywin', <module 'pywin'>)
('x', 3.14)
('__name__', '__main__')
('__builtins__', <module '__builtin__'>)
>>>

This seems scary, but look at the third line. The global namespace is just a dictionary, and it has an entry called x with value 3.14. It also has a few other bits and pieces you don't need to worry about at this point.

At this point we touch on a significant fact about Python: almost everything is built out of dictionaries. Objects are implemented using dictionaries; their keys are the method or attribute names stored as strings, and the values are the attribute values and function objects themselves. Namespaces are dictionaries, too. And when a chunk of code is evaluated, it's internally handed two dictionaries: one containing the global variables at that point in time, the other containing the local variables to the current function or method.

This detail is interesting to language lawyers, but it also has an immediate practical payoff: you can design a namespace to suit yourself and execute the user's code in it. Specifically, you can modify the global namespace of the Python process in your browser to include a variable called TheBookSet, which refers to the currently running BookSet, or you can create an entirely new namespace in a fresh dictionary of your own.

The simplest demonstration of a COM server, which is included in PythonWin in the file Python\win32com\servers\interp.py, creates a COM object called Python. Interpreter. This exposes two methods to execute and evaluate expressions, which can easily be grafted onto any COM server. We want to build a console that lets the user do both and returns any output, so we'll merge them into one method called interpretString(). This either executes or evaluates the expression; if there is a return value, you hand a string representation of it back to the user.

You also need to extend the __init__() method of the BookServer to add a namespace with the needed global variable. Here's the new initialization code and the new method to interpret a string:

def __init__(self):
    self.__BookSet = doubletalk.bookset.BookSet()

    # create a custom namespace for the user to work with
    # with one variable name already defined
    self.userNameSpace = {'TheBookServer', self.__BookSet}

def interpretString(self, exp):
    """Makes it easier to build consoles.
    """
    if type(exp) not in [type("), UnicodeType]:
        raise Exception(desc="Must be a string", \
                    scode=winerror.DISP_E_TYPEMISMATCH)
    try:
        # first, we assume it's an expression
        result_object = eval(str(exp), self.userNameSpace)

    if result_object == None:
        return "
    else:
        return str(result_object)
except:
    #failing that, try to execute it
exec str(exp) in self.userNameSpace
return "

It's necessary to add the statement from pywintypes import UnicodeType at the beginning of the module. Note that the code accepts normal and Unicode strings and raises helpful exceptions if the wrong kind of object is passed in. Try to evaluate the string as an expression, then try to execute it as a statement. If it causes an error, leave it unhandled so that the error message can filter through to the VB user. Note that as we discuss in Chapter 12, Advanced Python and COM, leaving an unhandled Python exception to propagate to the user of the object is not considered good design, but is suitable for the purposes of this demonstration.

If there is a return value, convert it as a string. You could return it raw, allowing eval() to potentially return numbers and arrays, but there is a risk of a user expression returning something VB doesn't expect. The intention in this example is to get back a printable string to show the user, so make sure the return type is always either a string representation of the data or an empty string.

Grabbing Python's Output

You now have the hooks to execute arbitrary strings of Python code, but you can't necessarily see the output. You need to implement one more feature first, to capture Python's standard output, so that print statements in your users' code show up properly. You might think this would involve low-level Windows process control, but actually, Python knows how to redirect its own output. If you enter the following statements in a Python source file or the Python DOS prompt, any subsequent output (for example, print statements) are redirected to a file:

>>> import sys
>>> mylogfile = open('c:\\temp\\mylog.txt', 'w')
>>> sys.stdout = mylogfile
>>>

Output can be redirected to any Python object that offers a write() method. The easiest way to grab the output is to add just such a write() method to our COMBookSet class, which stores the standard output internally; provide another method to grab this data from VB on demand; and start trapping the output when our instance of COMBookSet starts. Here are the needed extra methods:

def beginTrappingOutput(self):
    self.outputBuffer = []
    self.old_output = sys.stdout
    sys.stdout = self

def write(self, expr):
    """ this is an internal utility used to trap the output.
    add it to a list of strings - this is more efficient
    than adding to a possibly very long string."""
    self.outputBuffer.append(str(expr))

def getStandardOutput(self):
    "Hand over output so far, and empty the buffer"
    text = string.join(self.outputBuffer, ")
    self.outputBuffer = []
    return text

def endTrappingOutput(self):
    sys.stdout = self.old_output
    # return any more output
    return self.getStandardOutput()

Everything but write() is exposed as a COM public method. When VB creates the server, add a line to call TheBookServer.beginTrappingOutput().

A word of warning at this point: you aren't the only person interested in Python's standard output. In Chapter 12, you'll learn about the Trace Collector debugging tool. This is a feature of PythonWin that enables you to debug your COM server while calling it from VB; we used it quite a bit in writing this chapter. If you've registered your COM server for debugging, all the output that should have gone to the Trace Collector shows up in your console window. That's why we've provided some explicit methods to start and stop trapping, rather than just to start trapping when the COMBookSet initializes and leaving it on forever.

Building an Interactive Console

Now we have everything needed to create a basic interactive console. We implemented this as an extra child window with a one-line text box for input and a multiline, uneditable text box for output. When the user inputs an expression, the VB console form executes the commands TheBookServer.interpretString (expression) to get the return value and TheBookServer.getStandardOutput to retrieve any output that was generated. It then assembles these together into one chunk of text and appends this to the output text box. Figure 8-1 is an example of our console in action.

Note you have full access to the data of your running server and can modify its data. You can also create your own variables and generally do anything you can from a regular Python console.

Industrial-Strength Consoles

The previous console is extremely simple and allows only one statement at a time. Ideally, something like the interactive prompt in PythonWin would be preferable.

0127-01.gif
Figure 8-1.
A crude Python console

There is actually quite a lot of work involved in writing such a window, and we won't go through the code to do it here. As text is entered a line at a time, your interpreter window needs to decide if it's a complete expression and when to process the input, as well as suggest indent levels for subsequent lines, and so on. The event-handling depends a great deal on the type of GUI widget used to build the console.

If you want to build such a console, look at the file code.py in the standard Python library. This contains a function called interact() that defines an interactive console written in Python. It needs adapting, but shows the general approach; as each line is entered, it tries to evaluate the current text buffer, and assumes the statement is incomplete if it gets an error. You could modify this to provide hooks for your VB console. Code.py is used by both IDLE and Pythonwin to emulate the Python interactive mode.

Executing Scripts

Quite often the user wants to execute simple scripts. If the user has created a simple script in a file on disk, it can be run with the built-in function execfile (file[, globals[, locals]]). This is broadly equivalent to the exec statement discussed earlier, except that it's a function, and it takes filename as an argument, processing the entire contents. To expose this, we've implemented a new method of COMBookSet, which takes the filename as a single argument, and calls execfile (filename, self.userNameSpace):

def execFile(self, filename):
    if type(filename) not in [type("), UnicodeType]:
        raise Exception(desc="Must be a string", \
                          scode=winerror.DISP_E_TYPEMISMATCH)
    execfile(str(filename), self.userNameSpace)

VB provides a rich-text editor component that makes it easy to create an editor, so we've added yet another form to our application called frmScriptEditor. This has a single editing region and a menu. We provided menu options to open and save files, and it keeps track of the filename and whether the text has changed. We won't cover those functions here. It also has a menu option to run a script, which is straightforward to implement:

Private Sub mnuScriptRun_Click()
    mnuScriptSave_Click
    If Saved Then
        On Error GoTo mnuScriptRun_Error
        frmMain.BookServer.execFile FileName
        On Error GoTo 0
        frmConsole.UpdateOutput
    End If
    Exit Sub
mnuScriptRun_Error:
    MsgBox "Error running script:" + vbCrLf + vbCrLf + Err.Description
End Sub

We handle errors, since the user is bound to make some at some stage, and ask the console window to display any new output afterwards. The user interface checks that any script is saved before running it, since we need to execute a file on disk. Figure 8-2 is a basic script in action, querying the running BookSet and producing some console output.

Importing a Module

The difference between importing a module and running a script is that a module object is created in memory with a name you can access. All the functions and classes defined in the module go into the newly created namespace. When executing a script, all code is executed in the global namespace. You could let the user

0129-01.gif
Figure 8-2.
A script running under our control

import modules by typing import foo in a console, or even in a script executed in this manner. However, there are some benefits to exposing this in the user interface. For example, you could save a list of standard imports for each user in the registry as a configuration variable and import those modules every time the application starts up. To do this, let's take a look at Python's import mechanism, which gives fine-grained control of how modules are created.

The library module imp exposes Python's import mechanism. This includes functions to search the Python path for modules, and to load modules once located. Let's say you create a simple module in the file c:\temp\import\temp.py that defines one function called func (x). If you want to use this from PythonWin, ensure it's on the path and type import temp. Within a custom application, you can often drop to a lower level of detail and customize the details of this process. For example, the text for a module might be a compiled resource in the program rather than a file on disk, and you might want to swap different source files in and out under the same module name. Let's look at what happens behind the scenes of an import by recreating the steps manually.

First, Python looks for it with the find_module function:

>>> import imp
>>> found = imp.find_module('temp', ['c:\\temp\\import'])
>>> found
(<open file 'c:\temp\import\temp.py', mode 'r' at 1078200>, 'c:\\temp\\import\\temp.py', ('.py', 'r', 1))
>>>

find_module takes a list of locations to search as an optional second argument; if this is omitted, it searches sys.path. Thus, by taking control of the import mechanism, you can keep a separation between your Doubletalk code locations and your general Python code; such a separation is useful in a production application. If successful, it returns an open file handle to the module file, the full pathname, and some background information about the file.

The next step is to call load_module, which lets you control what the module gets called. The arguments to this are the name to give the module in your namespace, and the three return values from the previous function. This returns a module object you can manipulate further:

>>> mymodule = imp.load_module('temp', found[0], found[1], found[2])
>>> mymodule
<module 'temp'>
>>>

If the module contains errors, it raises an exception, and the file handle in found[0] is left open. Your code should use a try?nbsp;finally?/TT> block that closes the file afterwards.

A module object is like any other Python object. Let's put this one in the global namespace as usual:

>>> globals()['temp'] = mymodule
>>> temp.func('blah')
'blahblah'
>>>

You now have a module object in memory and can call its functions.

Armed with this knowledge, you can add a menu option to your script editor to import a script and make sure it's available in the right namespace for users: the dictionary userNameSpace in COMBookSet. As usual, expose a public method in COMBookSet:

def importFile(self, fullpathname):
    #import as the filename
    import imp
    path, filename = os.path.split(str(fullpathname)
    root, ext = os.path.splitext(filename)
    found = imp.find_module(root, [path]) #takes a list of files
    if found:

    (file, pathname, description) = found
    try:
        module = imp.load_module(root, file, pathname, description)
        # ensure it's visible in our namespace
        self.userNameSpace[root] = module

        print 'loaded module', root
    finally:
        file.close()
else:
    print 'file not found'

Note that this takes the name from the filename and adds it to userNameSpace The VB script import command looks like the earlier one for execFile, but calls importFile instead. Users can now edit scripts and choose both Script ® Run and Script ® Import from the menu as they can in PythonWin and access the functions created from the console.

Providing a Startup Script

A useful customization is to allow an option for a startup script. The script could be a standard, hardcoded filename or a configuration variable. Users can do almost anything with this script; they can put in a series of standard import statements, go off and import commonly used data files from elsewhere, and (as we'll see) set up the BookSet as they want with validation rules and custom views. The script executes after the BookSet has been initialized, so it can't control the way BookSet initializes.

This feature is easy to provide with the tools we've just built. In Figure 8-3, we've gone for a user-defined script name in an Options dialog.

0131-01.gif
Figure 8-3.
Specifying a startup script from the client

Earlier on we put two public methods in frmMain called InitCOMServer and CloseCOMServer, and never used the latter. Here there's a good use for it.

InitCOMServer has been expanded as follows (ignoring error trapping to save space):

Sub InitCOMServer()
    Dim startupScript As String

    'called when the program starts
    On Error GoTo InitCOMServer_error
    Set BookServer = CreateObject("Doubletalk.BookServer")
    On Error GoTo 0

    'tell it to capture output for the console
    BookServer.beginTrappingOutput

    'if there is an init script, run it
    If frmOptions.txtStartupScript.text <>"" Then
        On Error GoTo InitCOMServer_StartupScriptError
        BookServer.execFile frmOptions.txtStartupScript.text
        On Error GoTo 0
    End If

    'grab any standard output for the console
    frmConsole.UpdateOutput
    Exit Sub

Test to see if there is a startup script and run it under an error handler if there is. Then tell the console to fetch any standard output.

We've also provided a button titled Reload BookServer Now. This one just shuts down and restarts the server (losing any running data):

Private Sub cmdReload_Click()
    Dim Proceed As Boolean
    If frmMain.BookServer.count > 0 Then
        If MsgBox("You have data in the BookServer which will be " + _
          "lost. Proceed?", vbOKCancel, "Warning") = vbCancel Then
            Exit Sub
        End If
    End If
    frmConsole.Clear
    frmMain.CloseCOMServer
    frmMain.InitCOMServer 'this calls the script
    frmMain.UpdateAllViews
    Beep

End Sub

The user now has a startup script that allows almost limitless customization. We haven't implemented a place to save this script; the choices are generally the registry or an INI file.

owl.gif
A Reload button is extremely useful. During development, we frequently switched between Python and Visual Basic. After changing any Python code, it was originally necessary to shut down and restart the VB application. After implementing this button, it took just one click to start exploring any new Python code.

Defining User and System-Code Directories

This section doesn't discuss a feature, it's just a recommendation. Your application and documentation should define clearly where the main source package lives (e.g., C:\Program Files\Doubletalk\Source) and where users' code should go (e.g., C:\Program Files\Doubletalk\UserSource). The application should add the latter directory to the path when starting, and suggest it as the default location for any user scripts if you provide a script editor.

Making an Application Extensible

We've built all the tools you need to provide the user with a macro language. We'll now look at ways to let users extend the object model. Exactly what you choose to make extensible depends a great deal on the application.

In the case of the Doubletalk browser, we'd like to add two new capabilities. We want to trap certain events occurring in the BookSet and allow users to write code to respond, and we'd like to let users write their own views.

Bear in mind that from now on we are talking about power users, who are presumed to have some programming experience or at least aptitude. With a well-documented object model, their task should be easier in Python than in other languages; but they still have the ability to create bugs and damage data. We'll try to structure the application in a way that keeps users away from critical code and keeps it simple.

Changing the Delegate Class

It's useful to specify in another option what class to use in the place of the BookSet. Imagine variations of BookSet that fetch data from a relational database and commit each record as it's edited, or that perform some degree of caching to answer certain queries quickly.

Changing the base class involves a lot of development and testing and is not easy to do on the fly. However, it's possible to arrange things so that a user-written module is consulted to determine the delegate class at startup.

If you distribute your core application as source, the users can always create a subclass to do what they want. However, subclassing involves a lot of work; users must ensure that their new BookSet subclass still does everything the COMBookSet class expects. While possible, this pattern is not really recommended for a complex class like BookSet.

A Delegation Framework for Responding to Events

There are four events in the BookSet API that allow modification: adding, editing, and deleting transactions, and renaming accounts. For each of these, you should provide a hook called before the event that gives the user a chance to modify the data or cancel the action altogether and another hook called after the event that allows the user to update other variables elsewhere.

Rather than have the user write numerous disconnected functions or subclass the entire BookSet, you can use a pattern known as a delegate. A delegate is a helper object a class informs of various events: it delegates certain responsibilities. The delegation mechanism was a cornerstone of the almost legendary NeXTStep development environment and is widely used behind the scenes in Delphi, where each component delegates the task of responding to events to its parent form. A delegate is typically much simpler than the class it's supporting. Users will find it far less work to write their own delegates to achieve a task than to rewrite or subclass the entire BookSet.

Views and Validators

The notifications before the event are intended to validate data, and the notifications after the event can maintain custom views. Accordingly, we define two types of delegate:

?A Validator is an object a BookSet notifies before changing data, asking for permission to proceed.

?A View is an object the BookSet notifies after changes have been made.* It also has a method to return a 2D array of data on demand, which contains whatever users wish.

It was traditional until recently to have just one delegate for an object. Some Java development environments allow a list of delegates that can be added and removed at runtime, and we've borrowed this pattern. We could also have built a more complex delegate that combined the functions of Validator and View, but this seemed a better fit to our goals for the users.

* Design-pattern aficionados might also argue that this is an instance of the observer pattern. Call it what you will.

For each delegate, youshould provide a base class users can subclass. You should also define a subclass of BookSet that can use them. All this code can be found in the module doubletalk.userhooks, which also includes examples of Validators and Views. Here's the definition of a View:

class View:
    """This delegate is informed of all changes after they occur,
    and returns a 2d array of data when asked."""
    def setBookSet(self, aBookSet):
        self.BookSet = aBookSet
        self.recalc()

    def getDescription(self):
        return 'abstract base class for Views'

    # hooks for notification after the event
    def didAdd(self, aTransaction):
        pass
    def didEdit(self, index, newTransaction):
        pass
    def didRemove(self, index):
        pass
    def didRenameAccount(self, oldname, newname):
        pass
    def didChangeDrastically(self):
        #can be used to notify of major changes such as file/open
        self.recalc()

    def recalc(self):
        #override this to work out the data
        pass

    def getData(self):
        return [()] # simple 2D array for display

The View receives a SetBookset call when hooked up, triggering a recalculation. At this point it probably walks through the entire BookSet, gathering the data it needs, in the same way the existing query methods did in BookSet.

The View provides five notification methods for the BookSet to call with changes; the user won't call these directly. Define the four changes identified earlier and allow for one more (didChangeDrastically) that can be called after, for example, opening a new data file, which triggers a full recalculation. These allow the View to update its data intelligently and efficiently in response to changes.

Validators look similar, but respond to calls such as self.mayAdd (transaction). If the call returns zero (false), the action is rejected. Views are just notified of changes and don't have to return anything.

Let's look at our new UserBookSet class, which knows what to do with Views and Validators. Here's how to initialize it, add Views, and fetch their data later:

class UserBookSet(BookSet):
    def __init__(self):
        BookSet.__init__(self)
        self.validators = []
        self.validator_lookup = {}
        self.views = []
        self.view_lookup = {}

    def listDelegates(self):
        # utility to tell us what's hooked up
        [details omitted to save space]

    def addView(self, aView):
        #put it in both a list and a dictionary
        # join them together
        self.views.append(aView)
        self.view_lookup[aName] = aView
        aView.setBookSet(self)

    def getViewData(self, aName):
        return self.view_lookup[aName].getData()

Views are added with a name the user specifies. The View goes in both a list and a dictionary, allowing you to iterate over the list of Views and to quickly access individual Views by name. You can then ask the UserBookSet to return the data for any of its Views. There is a broadly similar method to add a Validator.

Now we'll override the methods of BookSet that may modify data. Here's the new method in UserBookSet to add a transaction:

def add(self, tran):
    for v in self.validators:
        if not v.mayAdd(tran):
            # rejected, stop
            return

    #call the inherited method
    BookSet.add(self, tran)

    # notify them all
    for v in self.views:
        v.didAdd(tran)

This code says, ''Ask all the loaded Validators for permission before adding the transaction to the BookSet. Then after adding it, tell each View." Similar methods have been written for edit, remove, and renameAccount.

Finally, if you want to commit to this new architecture, change the __init__ method for COMBookSet to create a UserBookSet instead of a BookSet.

A User-Defined View: The Backend

Now we can write a new View, a simple one that keeps track of the month-end balances of an account. The array has two columns; the first entry to show year and month, and the second the month-end balance. For a two-year data file, you thus get back about 24 rows:

class MonthlyAccountActivity(View):
    """Keeps track of activity in an account. Does
    smart recalculations."""

    def __init__(self, anAccount):
        self.account = anAccount
        self.balances = doubletalk.datastruct.NumDict()

    def getDescription(self):
        return 'Month end balances for ' + self.account

    def didAdd(self, tran):
        effect = tran.effectOn(self.account)
        if effect ==0:
            return
        else:
            #year and month as the key
            yymm = time.gmtime(tran.date) [0:2]
            self.balances.inc(yymm, effect)
            print 'added %s, %0.2f' % (yymm, effect)

    def didRemove(self, index):
        tran = self.BookSet[index]
        self.didAdd(-tran) #invert and add

    def didEdit(self, index, newTran):
        oldTran = self.BookSet[index]
        self.didAdd(-oldTran)
        self.didAdd(newTran)

    def didChangeDrastically(self):
        self.recalc()

    def recalc(self):
        "Do.it all quickly in one pass"
        self.balances.clear()
        for tran in self.BookSet:
           yymm = time.gmtime(tran.date)[0:2]
            for (acct, amount, etc) in tran.lines:
                if acct == self.account:
                    self.balances.inc(yymm, amount)

    def getData(self):
        # numdict returns it all sorted; just need to format
        # the date column
        formatted = []

for (period, balance) in self.balances.items():
    (year, month) = period #unpack tuple?BR>     monthname = doubletalk.dates.SHORT_MONTHS[month-1]
    displayDate = monthname = '-' + str(year)
    formatted.append ( (displayDate,balance))
return formatted

This should be fairly straightforward for users to produce. The recalc() method works it all out in five lines, using the NumDict utility class to categorize the numbers. When a single transaction is added, recalc() tests if the transaction affects the account; if not, no work is needed. If so, it just changes one entry in the NumDict. The methods for the other events are repetitive but similar. When the user requests the data from the GUI, the only work needed is to retrieve and sort a list of 24 items, which should happen almost instantaneously.

userhooks.py contains a test routine that can be called from a Python console to verify that the UserBookSet and View are working correctly.

A User-Defined View: The Front End

Now how do you look at View in the interface? Once again, you need to extend COMBookSet. First, change its initializer so that it creates a UserBookSet instead of a BookSet. Second, expose a method called getViewData() that calls the underlying method of BookSet. It's easiest to create and add the View with a short chunk of Python script:

from doubletalk.userhooks import MonthlyAccountActivity
view = MonthlyAccountActivity('MyCo.Assets.NCA.CurAss.Cash')
TheBookSet.addView(v, 'CashBalances')

The Rolls Royce approach would be to build this on the fly after letting users select view types and parameters from a menu; however, this means having some sort of configuration dialog for each View. At a more basic level, power users writing their own Views could write this code themselves and put it in a start-up script.

Having done this, all you have to do is build a generic form with a grid. This keeps track of its name and fetches the View data (and a description) when asked. The next section presents a user-defined View of the monthly cash balances. See Figure 8-4.

User-Defined Validations

Imagine you're building a set of year-end accounts, repeatedly importing data from several sources and making manual edits. It all has to be perfect, and it's 2:00 a.m. You think you have the first half of the year sorted out and accidentally enter a transaction dated October 1998 instead of October 1999. It doesn't show up in the

0139-01.gif
Figure 8-4.
A user-defined View

View you are looking at, so you enter it again and make other corrections elsewhere based on your erroneous account balances. If you are unlucky, you could waste hours finding the error and unravelling all the dependent changes (One author knows this all too well). The userhooks file also defines a sample Validator that puts a time lock on the BookSet; this prevents any changes before a cutoff date. Here's the code:

class DateWindowValidator(Validator):
    """An example. Prevents changes on or before a certain date
    locking the bookset"""

    def __init__(self, aDescription, aDate):
        Validator.__init__(self, aDescription)
        self.cutoff = aDate

    def mayAdd(self, aTransaction):
        return (aTransaction.date > self.cutoff)

    def mayEdit(self, index, newTransaction):
        oldtran = self.BookSet[index]

    if oldtran.date <= self.cutoff:
        return 0
    elif newTransaction.date <= self.cutoff:
        return 0
    else:
        return 1

def mayRemove(self, index):
    tran = self.BookSet[index]
 -  return (tran.date > self.cutoff)

# renameAccount will not break anything

More sophisticated Validators might have beginning and ending time windows, a list of accounts not to touch, or even a user-related permissions mechanism. All these can be implemented without the users needing to touch the core BookSet code.

More Ways to Extend the Application

Views and Validators can be used for other jobs as well as displaying interactive data:

?If you want to implement an error-recovery system, you could create a View that writes every change to a log file and then can roll backward and forward through the log (edits are reversible; renaming is not). This provides a full audit trail, invaluable when making lots of minor corrections at year-end.

?If the system were holding real data (e.g., as part of an executive information system), you could build a system of alerts to warm people or generate certain reports if accounts fell below certain levels, or if creditors were more than a certain amount of time overdue.

A Note on Performance

Extending BookSet to UserBookSet and adding in Views and Validators changes the performance characteristics enormously. If our goal is a simple, general-purpose class around which to write scripts, it may not be worth doing. The original BookSet can add, edit, and remove transactions quickly as it keeps them in a list, but most useful queries involve a loop over the entire set of data. A running UserBookSet in an interactive application might have 10,000 transactions in memory, five Views open, and two Validators. This means that any addition involves talking to seven other objects, and an edit involves 14 objects. Naturally, this dramatically slows bulk operations. However, it dramatically enhances query performance; a view on the screen displaying month-end balances of every account might need to redraw only one or two cells of the grid after an edit, rather than recalculate completely. Think of each running View as an extra database index, and you won't go far wrong.

Conclusion

This chapter has taken us beyond standard Windows development and into an area that is one of Python's greatest strengths: extensibility. We have taken an application that had a useful object model but a limited feature set and opened it to users. They have full access to the object model and can interact with the data. In the course of this, we've learned more about Python internals.

In addition, we have refined the object model using delegation to make it easy for users to customize. With a little training and documentation, they can build farreaching extensions precisely tailored to the nature of their business.

This type of development would be prohibitively expensive and difficult without a dynamic environment such as Python. Python lets you create extensible applications with ease.


Back