On Gideon and experimentation

In reaching the final stages of our usability project, I've started to think about the kind of change which this process forces upon the system manager/developer. It's not just about working your way through a list of software modifications; it's about identifying which ones are the most important and prioritising them.

Knowing that they are going to be re-tested by the same users as before binds the organisation to the user. If some of this testing is anonymous (for instance, through remote click tests), then the pressure to deliver a great service builds because users can feed back on the product as they find it, and there is no opportunity for political interference of these results.

But that pressure requires a specific ability on the part of the system developer: experimentation. Functions need to be tested over and over again before release, often using other team members from within the organisation. To function effectively, all the team members need to appreciate the value of this iterative but ultimately repetitive behaviour. For a certain period of time, they have to reject what is customary or an accepted standard for an information service and place themselves in a research context (much like how the usability standard persona tool works).

This isn't an easy piece of intellectual gymnastics to perform and some might say it's too exposed to the mundane organisational risks of political interference. However, the levelling factor will be the end user's response – no amount of influence will tell a humanities researcher what to think. And if that response is well documented and capable of being compared across time, as the usability process gives us the framework to do, then that voice will be heard.

So it looks as if running experiments such as surveys and click tests, standard tools within the social sciences, is finally here to stay for humanities products, as the results can easily communicate across all levels of an organisation how specific investments can provide a return. That's why the term usability applies as much to the methodology of this process (i.e. interview & focus group transcriptions/notes, survey results, click records) as it does to the system under investigation. Approaching the process in this way allows it to form not only the basis for modification of the system under review, but also for the very processes used to examine the system.

PS: you may be wondering why Gideon was in the title to this post. Gideon, as you may know, was unsure of whether he was actually speaking to God when he prayed; so he asked him to put water on his fleece but keep the ground dry. This was duly done, but being a sceptical fellow, the next day Gideon asked God to make the ground wet but keep the fleece dry. Now, instead, he could have simply asked someone if God existed and accepted their answer...but he didn't. Not only did he experiment, but crucially, he made that experiment easy to understand.