One of the things that I understand as a user experience designer is that it is often better to observe how users behave than it is to listen to them tell you how they behave. This was driven home for me quite vividly on a visit to a customer site several years ago.
A colleague and I were meeting with truck drivers to better understand how they thought about getting directions for routes they needed to take. The drivers that we met were all thoughtful and understood their work well, even if they had different ideas about what worked and why.
One driver was especially articulate in explaining how he worked and what issues were important to them. He provided a lot of insights, and was generally engaging and fun to talk to. One question that I was particularly interested in was whether he found landmarks useful in directions. He was quick to say that he didn’t like them, and provided some quite rational reasons as to why. For example, landmarks aren’t always visible at night, or they may have disappeared. Great points, and we were happy to hear his insights.
As it turned out, this driver was our last meeting of the day. While packing up, I asked him for directions back to our hotel. He was happy to help, and started to sketch a simple map on a piece of paper while providing verbal directions. His directions included the memorable instruction to “then take the next right after the giant chicken at the side of the road”. I waited for him to complete his directions, and then asked him why he had included a landmark (the chicken) in his directions when he had previously indicated that he didn’t like landmarks. He was a little surprised himself, and we discussed it a little. What it came down to was that he liked landmarks when he knew that they were accurate and visible, and could thus be relied upon.
How he behaved when providing directions was different from what he had reported to us in an interview. Users aren’t always the most reliable reporters, even of their own actions or preferences. Observation is a great tool for getting around that.