Blackboards

BLACKBOARDS 165

There’s a massive reporting job that goes through looking for overbooked or full flights to suggest when additional flights might be scheduled. It works fine, but it takes hours to run.

We’d like to have a little more flexibility in processing wait-list passengers, and we’ve got to do something about that big report—it takes too long to run. Use the ideas from this section to redesign this interface.

Figure 5.6. Someone found a connection between Humpty’s gambling debts and the phone logs. Perhaps he was getting threatening phone calls.

H. Dumpty (Male, Egg): Accident or Murder?

Photos Shell fragments Phone logs King’s men Gambling debts

Eyewitnesses Graffiti Wife’s alibi

Detective 1 Detective 2 Detective 3

Different detectives may come and go during the course of the pro- cess, and may work different shifts.

There are no restrictions on what may be placed on the blackboard.

It may be pictures, sentences, physical evidence, and so on.

We’ve worked on a number of projects that involved a workflow or distributed data gathering process. With each, designing a solution around a simple blackboard model gave us a solid metaphor to work with: all of the features listed above using detectives are just as appli- cable to objects and code modules.

A blackboard system lets us decouple our objects from each other com- pletely, providing a forum where knowledge consumers and producers can exchange data anonymously and asynchronously. As you might guess, it also cuts down on the amount of code we have to write.

Blackboard Implementations

Computer-based blackboard systems were originally invented for use in artificial intelligence applications where the problems to be solved were large and complex—speech recognition, knowledge-based reason- ing systems, and so on.

Modern distributed blackboard-like systems such as JavaSpaces and T Spaces [URL 50, URL 25] are based on a model of key/value pairs first

BLACKBOARDS 167

popularized in Linda [CG90], where the concept was known as tuple space.

With these systems, you can store active Java objects—not just data—

on the blackboard, and retrieve them by partial matching of fields (via templates and wildcards) or by subtypes. For example, suppose you had a type Author, which is a subtype of Person. You could search a blackboard containing Personobjects by using an Authortemplate with a lastName value of “Shakespeare.” You’d get Bill Shakespeare the author, but not Fred Shakespeare the gardener.

The main operations in JavaSpaces are:

Name Function

read Search for and retrieve data from the space.

write Put an item into the space.

take Similar to read, but removes the item from the space as well.

notify Set up a notification to occur whenever an object is written that matches the template.

T Spaces supports a similar set of operations, but with different names and slightly different semantics. Both systems are built like a database product; they provide atomic operations and distributed transactions to ensure data integrity.

Since we can store objects, we can use a blackboard to design algo- rithms based on a flow of objects, not just data. It’s as if our detec- tives could pin people to the blackboard—witnesses themselves, not just their statements. Anyone can ask a witness questions in the pur- suit of the case, post the transcript, and move that witness to another area of the blackboard, where he might respond differently (if you allow the witness to read the blackboard too).

A big advantage of systems such as these is that you have a single, con- sistent interface to the blackboard. When building a conventional dis- tributed application, you can spend a great deal of time crafting unique API calls for every distributed transaction and interaction in the sys- tem. With the combinatorial explosion of interfaces and interactions, the project can quickly become a nightmare.

Organizing Your Blackboard

When the detectives work on large cases, the blackboard may be- come cluttered, and it may become difficult to locate data on the board. The solution is topartition the blackboard and start to orga- nize the data on the blackboard somehow.

Different software systems handle this partitioning in different ways;

some use fairly flat zones or interest groups, while others adopt a more hierarchical treelike structure.

The blackboard style of programming removes the need for so many interfaces, making for a more elegant and consistent system.

Application Example

Suppose we are writing a program to accept and process mortgage or loan applications. The laws that govern this area are odiously com- plex, with federal, state, and local governments all having their say.

The lender must prove they have disclosed certain things, and must ask for certain information—but mustnot ask certain other questions, and so on, and so on.

Beyond the miasma of applicable law, we also have the following prob- lems to contend with.

There is no guarantee on the order in which data arrives. For instance, queries for a credit check or title search may take a sub- stantial amount of time, while items such as name and address may be available immediately.

Data gathering may be done by different people, distributed across different offices, in different time zones.

Some data gathering may be done automatically by other systems.

This data may arrive asynchronously as well.

Nonetheless, certain data may still be dependent on other data. For instance, you may not be able to start the title search for a car until you get proof of ownership or insurance.

BLACKBOARDS 169

Arrival of new data may raise new questions and policies. Suppose the credit check comes back with a less than glowing report; now you need these five extra forms and perhaps a blood sample.

You can try to handle every possible combination and circumstance using a workflow system. Many such systems exist, but they can be complex and programmer intensive. As regulations change, the work- flow must be reorganized: people may have to change their procedures and hard-wired code may have to be rewritten.

A blackboard, in combination with a rules engine that encapsulates the legal requirements, is an elegant solution to the difficulties found here.

Order of data arrival is irrelevant: when a fact is posted it can trigger the appropriate rules. Feedback is easily handled as well: the output of any set of rules can post to the blackboard and cause the triggering of yet more applicable rules.

TIP43

Use Blackboards to Coordinate Workflow

We can use the blackboard to coordinate disparate facts and agents, while still maintaining independence and even isolation among partici- pants.

You can accomplish the same results with more brute-force methods, of course, but you’ll have a more brittle system. When it breaks, all the king’s horses and all the king’s men might not get your program working again.

Related sections include:

The Power of Plain Text, page 73 It’s Just a View, page 157

Challenges

Do you use blackboard systems in the real world—the message board by the refrigerator, or the big whiteboard at work? What makes them effective?

Are messages ever posted with a consistent format? Does it matter?

Exercises

30. For each of the following applications, would a blackboard system be ap- Answer

on p. 297 propriate or not? Why?

1. Image processing. You’d like to have a number of parallel processes grab chunks of an image, process them, and put the completed chunk back.

2. Group calendaring. You’ve got people scattered across the globe, in different time zones, and speaking different languages, trying to schedule a meeting.

3. Network monitoring tool. The system gathers performance statistics and collects trouble reports. You’d like to implement some agents to use this information to look for trouble in the system.

Chapter 6

While You Are Coding

Conventional wisdom says that once a project is in the coding phase, the work is mostly mechanical, transcribing the design into executable statements. We think that this attitude is the single biggest reason that many programs are ugly, inefficient, poorly structured, unmaintain- able, and just plain wrong.

Coding is not mechanical. If it were, all the CASE tools that people pinned their hopes on in the early 1980s would have replaced program- mers long ago. There are decisions to be made every minute—decisions that require careful thought and judgment if the resulting program is to enjoy a long, accurate, and productive life.

Developers who don’t actively think about their code are programming by coincidence—the code might work, but there’s no particular rea- son why. In Programming by Coincidence, we advocate a more positive involvement with the coding process.

While most of the code we write executes quickly, we occasionally develop algorithms that have the potential to bog down even the fastest processors. InAlgorithm Speed, we discuss ways to estimate the speed of code, and we give some tips on how to spot potential problems before they happen.

Pragmatic Programmers think critically about all code, including our own. We constantly see room for improvement in our programs and our designs. In Refactoring, we look at techniques that help us fix up existing code even while we’re in the midst of a project.

Something that should be in the back of your mind whenever you’re producing code is that you’ll someday have to test it. Make code easy

171

to test, and you’ll increase the likelihood that it will actually get tested, a thought we develop inCode That’s Easy to Test.

Finally, inEvil Wizards, we suggest that you should be careful of tools that write reams of code on your behalf unless you understand what they’re doing.

Most of us can drive a car largely on autopilot—we don’t explicitly com- mand our foot to press a pedal, or our arm to turn the wheel—we just think “slow down and turn right.” However, good, safe drivers are con- stantly reviewing the situation, checking for potential problems, and putting themselves into good positions in case the unexpected happens.

The same is true of coding—it may be largely routine, but keeping your wits about you could well prevent a disaster.

31

Programming by Coincidence

Do you ever watch old black-and-white war movies? The weary sol- dier advances cautiously out of the brush. There’s a clearing ahead:

are there any land mines, or is it safe to cross? There aren’t any indica- tions that it’s a minefield—no signs, barbed wire, or craters. The soldier pokes the ground ahead of him with his bayonet and winces, expecting an explosion. There isn’t one. So he proceeds painstakingly through the field for a while, prodding and poking as he goes. Eventually, convinced that the field is safe, he straightens up and marches proudly forward, only to be blown to pieces.

The soldier’s initial probes for mines revealed nothing, but this was merely lucky. He was led to a false conclusion—with disastrous results.

As developers, we also work in minefields. There are hundreds of traps just waiting to catch us each day. Remembering the soldier’s tale, we should be wary of drawing false conclusions. We should avoid pro- gramming by coincidence—relying on luck and accidental successes—

in favor ofprogramming deliberately.

PROGRAMMING BYCOINCIDENCE 173

How to Program by Coincidence

Suppose Fred is given a programming assignment. Fred types in some code, tries it, and it seems to work. Fred types in some more code, tries it, and it still seems to work. After several weeks of coding this way, the program suddenly stops working, and after hours of trying to fix it, he still doesn’t know why. Fred may well spend a significant amount of time chasing this piece of code around without ever being able to fix it.

No matter what he does, it just doesn’t ever seem to work right.

Fred doesn’t know why the code is failing becausehe didn’t know why it worked in the first place. It seemed to work, given the limited “testing”

that Fred did, but that was just a coincidence. Buoyed by false confi- dence, Fred charged ahead into oblivion. Now, most intelligent people may know someone like Fred, but we know better. We don’t rely on coincidences—do we?

Sometimes we might. Sometimes it can be pretty easy to confuse a happy coincidence with a purposeful plan. Let’s look at a few examples.

Accidents of Implementation

Accidents of implementation are things that happen simply because that’s the way the code is currently written. You end up relying on undocumented error or boundary conditions.

Suppose you call a routine with bad data. The routine responds in a particular way, and you code based on that response. But the author didn’t intend for the routine to work that way—it was never even con- sidered. When the routine gets “fixed,” your code may break. In the most extreme case, the routine you called may not even be designed to do what you want, but it seemsto work okay. Calling things in the wrong order, or in the wrong context, is a related problem.

paint(g);

invalidate();

validate();

revalidate();

repaint();

paintImmediately(r);

Here it looks like Fred is desperately trying to get something out on the screen. But these routines were never designed to be called this way;

although they seem to work, that’s really just a coincidence.

To add insult to injury, when the component finally does get drawn, Fred won’t try to go back and take out the spurious calls. “It works now, better leave well enough alone. . . .”

It’s easy to be fooled by this line of thought. Why should you take the risk of messing with something that’s working? Well, we can think of several reasons:

It may not really be working—it might just look like it is.

The boundary condition you rely on may be just an accident. In different circumstances (a different screen resolution, perhaps), it might behave differently.

Undocumented behavior may change with the next release of the library.

Additional and unnecessary calls make your code slower.

Additional calls also increase the risk of introducing new bugs of their own.

For code you write that others will call, the basic principles of good modularization and of hiding implementation behind small, well-docu- mented interfaces can all help. A well-specified contract (seeDesign by Contract, page 109) can help eliminate misunderstandings.

For routines you call, rely only on documented behavior. If you can’t, for whatever reason, then document your assumption well.

Accidents of Context

You can have “accidents of context” as well. Suppose you are writing a utility module. Just because you are currently coding for aGUI envi- ronment, does the module have to rely on aGUIbeing present? Are you relying on English-speaking users? Literate users? What else are you relying on that isn’t guaranteed?

Implicit Assumptions

Coincidences can mislead at all levels—from generating requirements through to testing. Testing is particularly fraught with false causalities and coincidental outcomes. It’s easy to assume thatX causesY, but as we said inDebugging, page 90: don’t assume it, prove it.