Richard's BLOG

Start, Ruby, and me

2009-04-12T13:44:00.010Z

Category: Programming Topic: Building tools

Ever since I heard Dave Thomas talk about it at a Spa (un-)conference in 2006, I've been a big fan of the programming language Ruby.

Perhaps that's partly because the first time I 'got' object-oriented programming was when I played around with Smalltalk, and Ruby reminds me of Smalltalk. But it's partly because Ruby comes with a library that makes writing small tools so easy. I'm thinking of the kind of tools that help me get computer work done more quickly and more easily.

Here's an example. Because of who my clients are, I use Windows. I'm a big fan of its start menu. I don't just have shortcuts to programs such as Firefox and Thunderbird; I have shortcuts to folders I often want to open, such as the folder called Aadvark, which is where I instruct applications to save files, so that I know where to find them.

(Aside: It's called Aadvark so that it appears near the beginning of where I keep almost all of my documents, in a folder called Filestore, which sits inside My Documents. I don't work directly in My Documents. I get too irritated. Windows gives me the facility to select an entry in Explorer by its first letter, but then names the folders within My Documents as My This and My That, so they all begin with 'm'. Aaaaagh!)

On the start menu, I also have shortcuts to Ruby programs that open several things, ready for me to start work on a task. For instance, when I have saved yet another movie or TV program to DVD, and want to add it to my database of DVDs, I want to open:

- the folder '.../Filestore/.../Lists/DVDs

- the Ruby application that allows me to add a DVD to the database, and then builds a new HTML index from it, so that family members don't have to browse the database directly

- 'My Computer', so that I can see what name I gave the DVD when I burned it

- the Internet Movie Database website, in case I want to look up details of a movie.

The Ruby program that does the opening for me uses Ruby's Win32OLE module, and is based around a two-line function:


def open(pathname)
  shell = WIN32OLE.new('Shell.Application')
  shell.ShellExecute(pathname, '', '', 'open', '')
end

For instance, the call open('My Computer') opens an Explorer window on My Computer. How easy is that?

If you are going to build tools to help you work, the effort you put into tool-building must be repaid (otherwise you're building the tools as a hobby). And with Ruby, the effort required to build tools is so low that I always find the effort is repaid.

Trading the Turtle way

2009-03-24T09:45:00.012Z

Category: Trading Topic: About a trading book

I've just finished reading The Turtle Way by Curtis Faith, and I'm going to re-read it, now.

In the 1980s, one famous trader, Rich Dennis, bet another, Bill Eckhardt, that traders are made, not born, and he could make anyone into a successful trader. Curtis Faith was one of those selected to be trained by Rich Dennis, and became the most successful of all the Turtles, as they were known.

The book presents the trading rules the Turtles were taught. For me, the book is much more valuable for describing how to trade successfully. In brief, this is what you do:

1. Develop the rules for a trading system
2. Backtest the system
3. If the backtest shows the system to be profitable, apply the rules.

As Curtis Faith observes, beginners think that step 1 is the hard part; experienced traders know that step 3 is what separates successful traders from unsuccessful ones.

Rich Dennis knew this, too. He's quoted in the book as saying that he could put his rules in a newspaper but most people still wouldn't follow them.

The book points the reader to someone who clearly understands the importance of step 3 - Mike Taylor. I had a fascinating time browsing Mike's pages.

Once the book on domain modeling is out of my hands, I'll be restarting work on my backtester!

Book: progress report

2009-02-27T14:21:00.002Z

Category: IT Topic: Book on domain modeling

My co-authors and I have reached an important milestone in the writing of our book. We've now got a complete draft of all chapters.

My tasks for next week will be:

- find a displacement activity

- tell myself off for engaging in displacement activity, and stop it

- find a new displacement activity

- reward myself for being so busy

- notice I haven't actually achieved anything useful

- find a new displacement activity

- finally admit that displacement activities aren't the same as writing the book

- patiently check the details of the case study in every chapter

- find a displacement activity

- tell myself off for engaging in displacement activity, and stop it

- find a new displacement activity

- admit again that displacement activities aren't the same as writing the book

- go through whole book, filling in the few remaining missing bits.

You can see where the phrase "struggling artist" comes from !

You might already be on our list of people we plan to ask to review the complete manuscript but, if you'd like to review the book, email me just to be sure (if you don't already have an email address for me, please use richard at inferdata dot com).

The precision gap

2009-02-03T10:58:00.020Z

Category: IT Topic: The dangers of imprecision

Software development proceeds from a high-level, vague project vision to low-level, precise code. Most development methods, especially those based on the common mis-understanding of top-down I talked about in my previous blog, imagine the route from vision to code like this:

If, however, you know how to build precise models at a high level of abstraction, you can choose this route, instead, in which a vague project vision is turned into a precise model, which is then implemented:

Another way to see the difference between the two approaches is to remember the two kinds of detail I mentioned in an earlier blog: there's the kind of detail involved in adding precision at the current level of abstraction, and there's the kind of detail involved in changing to a lower level of abstraction.

If you do see development as the one-dimensional line in the first figure, you'll need to choose some level of abstraction at which to write your functional specifications (or whatever you call the specifications of what functionality the next release of a system is to provide).

Whatever level you choose, you'll face the following problem: your functional spec will necessarily be lacking some of the precision that the programmers will need in order to write code.

Consequences of this "precision gap" can include:

programmers keep going back to business experts for clarification

the system as built incorporates different rules from the ones the business experts imagined

programmers invent the business rules

To overcome some of these negative consequences, IT departments are prone to add yet lower and lower levels of detail in their functional specs. As long as you believe development is a one-dimensional path, moving to a lower level is the only way to add precision to your functional specs. There's a downward pull on the level of abstraction at which you write functional specs.

Of course, if you understand that there are two kinds of detail, and that development is two-dimensional, you can choose to add precision to your functional specs without moving to a lower-level of abstraction, following the development path illustrated in the second figure near the beginning of this blog.

Top-down development

2009-01-09T15:44:00.006Z

Category: IT Topic: Adding detail to descriptions

The 1980s saw a big rise in the number of methods for developing systems (consultants called them methodologies, so they could charge more for them). Most of these methods incorporated a notion of top-down development, a notion that first appeared in earlier methods for program development.

But many of the system methods did not use top-down as envisaged by the pioneers of program development methods. To understand the difference you have to understand two different kinds of detail.

There's the kind of detail you add when you get nearer to finishing a description at a given level of abstraction. Think, for example, of the detail you add when you write the second half of a method in an object-oriented program. It's detail of the same kind as in the first half of the method. But the method's not complete without it. (You'll see that I am using description in a very general sense.)

By contrast, there's the kind of detail you add when you change to a lower level of abstraction. Think, for example, of the kind of detail to be found in a specification of a method, and the different kind of detail to be found in the code for that method.

The pioneers of program development methods were clear about the difference. In an important paper entitled Program Development by Stepwise Refinement (CACM, Vol 14, No 4, April 1971), Niklaus Wirth says, "In each step, one or several instructions of the given program are decomposed into more detailed instructions."

Developers of system development methods took this to mean that a description of a system could be built by writing a vague, high-level description, then explaining some of the vagueness at a lower level of abstraction, and explaining the vagueness in that description at yet a lower level, and so on. The problem with this approach is that none of the descriptions at any of the levels is, in any useful sense, complete (unless you continue down until you have a working program, but system development methods were meant to produce system specs, not programs).

Now read what Wirth says next in his sentence: "This successive refinement of specifications terminates when all instructions are expressed in terms of an underlying ... programming language." The word specifications is key.

At each level of abstraction we have, not a vague description to be made more precise at a lower level, but a specification appropriate to the current level of abstraction, capable of being implemented (i.e., described) at a lower level.

In a future post, I'll introduce what I call the precision gap that arises if you try to avoid being precise at higher levels of abstraction, and why the gap can make your development process less effective.

An interesting way to teach

2008-12-08T10:43:00.005Z

Category: Sailing Topic: Course on diesel engines

I recently attended an RYA couse on diesel engine maintenance, at the Lewes campus of Sussex Downs College.

From the brochure, I expected six students and an instructor clustered around a diesel engine. It wasn't like that at all.

Our instructor, Graham Whittington of Southern Marine, turned up with half-a-dozen crates containing bits scavenged from lots of different engines. Eleven students sat around a large, rectangular table, and Graham built parts of engines at one end, before our very eyes.

When Graham wanted to explain the fuel supply subsystem, for example, he got out a can of the kind you might have in your car, and using a variety of bits of piping connected it to the various filters and pumps you'd find on a real engine. Then he could demonstrate topics such as how air gets trapped, and how to bleed the system.

There were two immediate benefits of this approach. First, we could see everything. When we were learning about crankshafts and camshafts, we could handle actual examples of these parts. Second, our attention was automatically drawn to the parts that were important to the topic under discussion. While learning about the fuel system, the bits concerned with air supply were back in their crate.

Of course, we never saw a complete engine running, but what would that have added? Overall, I found the approach refreshing. My interest was held all day. And I came away confident to handle the various parts of an engine that need regular checking and maintenance.

Domain modelling - making observations

2008-11-21T12:23:00.007Z

Category: IT Topic: Business Modelling

In my post of Oct 21st, I said that key to our approach to domain modeling is the idea of observational semantics. When we're trying to uncover the business rules for a domain, we don't ask questions such as "when actor A1 starts use case U1, what do the actor and the system do?" Instead, we shift our thinking and ask "when I observe the business running, what changes can I see taking place that I need to describe?"

(Aside: the observing can be actual, if we're modeling a business as-is, or imagined, if we're modeling a to-be business.)

Here's an example of how the shift in thinking can help. Imagine a large commercial bank that can lend the kinds of sums of money you'd need to build a new factory. The contract for such a loan would involve many parties. There would be other banks who were contributing to the loan. There would be the manufacturer. There would be the prime building contractor, and the major subcontractors. And so on.

Each party would need to be involved in drafting, amending, and approving the contract, so lots of documents would circulate between the parties. Eventually, representatives of all the parties would meet in one place at one time to sign copies of the contract. The bank wanted a computer-based system to help them during the process of gathering together all the paperwork needed for a signing meeting.

When we joined the team, they hadn't modeled a change you can imagine happening - "a document has gone missing." The team were focusing on what was going on inside the system when actors invoked use cases, and they knew they had databases that didn't lose data. Once a document was logged in the database, it couldn't go missing. But someone who could imagine observing the business running (without worrying what was IT and what wasn't) could easily see that a real physical document could go missing, independently of what was recorded in a database.

It takes a while to get used to thinking just in terms of observable changes. The hard part is stopping thinking about how change is caused (e.g., which actor invokes a system function). But we've found that thinking about observable changes has helped us build better business models.

Precise business modelling

2008-11-09T13:47:00.006Z

Category: IT Topic: Business Modelling

In the previous two entries, I talked about using observational semantics when modelling a business domain, and about modelling large domains from a number of different points of view.

Here, I want to talk about a third key ingredient of the approach to domain modelling developed by staff of Mitchell Horvath Ltd and InferData Ltd. That ingredient is precision.

Businesses run according to rules. These rules are for business people to prescribe. Yet, in many businesses, the only people who work with precise ways of expressing the business's rules are the developers writing program code.

Let me take quite an extreme example. When a bank converts a sum of money from one currency to another, the result will depend on how many places of decimals are used to express the exchange rate and any intermediate calculations. Because the result depends on the number of places of decimals, that number is something the business should decide. In the banks I've worked with, there are experts in the foreign exchange part of the business who understand these issues, and who employ mathematicians to help them make the necessary decisions.

But in many kinds of businesses, it's common for such precise details to be omitted from business models. There's a widespread opinion amongst business and IT professionals that precise details are somehow "low-level," and only a matter for developers. We think that opinion needs challenging. Much of the precision captured in program code is also a concern for business professionals.

In the book we're writing, we show how to make domain models precise. We're not going to argue that all parts of all domain must be mathematically precise. But we would argue that most business models suffer from too much imprecision.

More on business modelling

2008-10-28T15:42:00.008Z

Category: IT Topic: Business Modelling

In the previous blog entry, I talked about using observational semantics to help focus on modelling the domain of a problem, rather than software.

When the domain to be modelled is big, we bring another technique to bear, too. We build a number of smaller models, each with its own point of view, and then combine these smaller models into a larger model of the whole domain. Each smaller model is built from a certain viewpoint, and captures some aspect of the business.

For example, a business that rents out DVDs and a business that sells health food supplements will both have a point-of-sale aspect, characterized by the exchange of goods or services for money. Both businesses will have rules about adding a sale item to an ongoing sale, for instance, and a point-of-sale model can capture these rules.

By modelling the point-of-sale aspect of your domain separately from other aspects, you can exploit expertise. If someone has become something of an expert in point-of-sale issues on one project, they can carry it over to another project more easily if the point-of-sale aspect is initially kept separate from other aspects of the problem domain.

There are, of course, other ways to partition a big model. We've found modelling different views to be extremely useful in practice, and we think it's underused. That's why it'll make its way into our book.

Business Modelling

2008-10-21T14:39:00.009Z

Category: IT Topic: Business Modelling

I'm working on a book describing how Mitchell Horvath and InferData consultants build business models. It's 80% finished, so there's only another 80% to go :-)

Key to the approach is the idea of observational semantics. Instead of asking, for instance, "when a customer withdraws cash, what are the business rules to constrain this piece of behaviour?" we ask "when we observe such-and-such a piece of behaviour, what does the business call what has happened?"

Adopting the observational approach helps to keep you focused on the business to be modelled, and away from software-oriented ideas of the system(s) to be built. We like to be able to focus on the business, rather than the software, whenever we're trying to uncover (or help to design) the rules by which the business runs.

The goal is to have a complete draft of the book to send to a publisher by the end of the (northern hemisphere) winter.