Last Wednesday, we had a JISC programme meeting which all of the projects attended.
There were some really high-profile projects there but I was suprised that so few 'researchers' attended - it was dominated by software professionals (I should probably include myself in that description!). Therein lies the first observation - one of the current perception of usability, that it's an engineering discipline. This situation really isn't healthy - usability affects the success of the whole project, initially and throughout its life.
Now I consider myself (correctly or otherwise) a proficient user of systems, meaning I can work my way around most things and am perfectly content to read help files to that aim. However, during the presentatons, I got the distinct feeling that I'd be pretty much lost if I had to take a usability test on these other websites and products. So my second observation is to be really careful about who we invite to take tests. We need intelligence not statistics - that means being able to segment the feedback along research classifications. Are we, as engineers, the right people to be left doing this function?
This programme is looking at taking usability and proving its worth to a set of people (researchers) who may not even know it exists as a field. At the moment, they don't even speak the language, and monitoring how that changes will be an interesting secondary benefit of undertaking this research.