A good intranet can only be built on the foundation of solid user research. James Robertson talks about this in his excellent article Conducting intranet needs analysis . James discusses several methodologies that can be used but the one I want to focus on is what James calls ‘contextual enquiry’ .
To quote James directly –
‘(Contextual enquiry) is a combination of staff interviews and workplace observation that involves exploring issues with a staff member, while situated within their normal working environment. By conducting the interview ‘in context’, it becomes possible to see the resources used by staff when conducting work activities.
The interviewer can also ask the staff member to show them how they complete specific activities, for example, showing how they find a piece of information on the intranet.’
Over the years I’ve found contextual enquiry to be by far the most illuminating, innovative and motivating experience associated with intranet user research.
Its illuminating because it can really throw some light on how other people think. People successfully access information and carry out tasks in weird and wonderful ways all of which, where possible, should be supported by the intranet.
Its innovative because people are generally more comfortable within their own familiar workspace and will be more willing to volunteer ideas. OK some of the ideas you’ll hear may be things you have already thought of or may not be practically or technically possible. But occasionally an idea will come from left field that will completely knock you out. I would say that at least 50% of the ideas that led to improvements in my last intranet came directly from contextual enquiry.
Its motivating simply because you get to meet your users face to face. When I think of a change in design or when problems are reported I don’t think of my users as a faceless mass. I think of the real people I’ ve met – people who are just trying to do a good days work, people who may face real problems and pain if the intranet is not up to scratch.
The approach to contextual enquiry I describe here won’t tell you what questions to ask, although you can find some sample questions and critical success factors (which are echoed in this post) in another of James Robertson’s articles Stakeholder interviews as simple knowledge mapping. This post is about how the contextual enquiry process might be carried out using an approach thats worked well for me for some years.
Preparation is the key. At the heart of the approach is the script. You will need to define and document the aims of the research and then write the questions supporting the aims in the form of a script. The script will be both a memory aid in ensuring that you don’t miss out anything and will also be used to record responses.
As it is not practical to interview everyone, contextual enquiry inevitably means a sample group must be used. At this point you may also want to consider the make up of the sample group so that it is representative of the user groups related to the research. I normally put together a table and enter in the heading for each space what type of staff member I needed to interview, (e.g. men, women, levels of responsibility, work roles etc), and how many of each type I have decided to interview. If, say, I need to interview five women I would enter ‘5, 4, 3, 2, 1’ in the appropriate space and simply cross out a numeral at the end of each interview. Once the last numeral has been crossed out I know that the sample set is complete.
The script should contain a mix of closed and open questions. Closed questions are good when the answer is likely to be ‘yes’ or ‘no’, or, for example, when using a Likert scale. Closed questions address specific information points and generally take up less time than open questions. However it’s by asking the correct open questions that you will generally hit pay dirt. Open questions allow for a less constrained response so ensure that there are open questions for all of the key data points you want to cover in the research.
Always pilot the script. Lack of time should never be an excuse. Even if you sit down and do dry runs with two or three colleagues you will find out very quickly where the weak points are and you will also get a good idea of how long an average session might take. More than an hour might suggest that some trimming of the script is required.
Choose people randomly and ask permission first. As discussed in Content Value Analysis ensuring the sample is random is crucial in ensuring that the results are robust. A sort of randomicity can be achieved by booking an appointment not with individuals but with the manager of a work area. Arrange to turn up on a given day and just interview whoever happens to be in. The manager involved may introduce you and try and tell the staff member what the enquiry is about. Don’t pitch straight in. You need to ask the person if they are happy to continue with the session. Staff will be more likely to contribute to the enquiry if they feel that they have given their permission rather than having been ordered to do it by their boss. If they answer ‘No’ (its never happened yet to me) then chat for a bit and go on to the next person. An unwilling participant is just a waste of your time.
Chat for a while and then explain what you are doing. Don’t be too formal. Before you start asking work related questions see if you can find something else to discuss first. Look closely (without looking like you are) at the person’s work space. Do they have they photos of family, books or obvious interests you can comment on? I always used to love the guys who had their favourite football club’s crest on their coffee cup. I was guaranteed a good discussion on the relative merits of his team and my team and it was great for relaxing the interviewee who might by then have formed the opinion that I might just be a normal person after all.
Then fully explain the reasons for the research and what is going to happen in the interview to come. Spend a little time answering questions and ensure the interviewee is clear about the whole process. That way you will be sure to get more useful and more relevant data out of the session.
Use a laptop. There are a few reasons why you should do this. It helps immensely if you can make the complete process transparent. Doodling in a notebook after a response leaves the interviewee in limbo and they may get a crick in their necks from trying to crane over and see whats being written about them. Place the laptop in a postion so that both you and the interviewee can comfortably see the screen. You can really involve the interviewee by listening closely to what they are saying, without the distraction of writing in a notebook or entering data in a computer, and generally say it back as a bullet point. If they agree that the bullet point adequately covers what they have said then enter the bullet point as a response against the scripted question. Though always take care to ensure that if the comment is a really telling one that you enter it verbatim. As the interviewee can see what you are inserting you can then ask them to read through the response to ensure you have noted it correctly.
There are several benefits in doing this –
You ensure that you are capturing the correct and complete data for each question
You are involving the interviewee throughout the whole process
The interviewee is usually quite impressed that you think their comments are so valuable that you will spend time ensuring every word is correct. This may lead them into being even more forthcoming and co-operative as the session goes along
At the end of the session the electronic information set is complete. You don’t need to transfer your notes from paper which you would need to do if you used a notebook.
Let your interviewees speak and forget the time. During the session don’t cut your interviewees off. Let them speak as long as they want and follow up interesting ideas even if they are not in the script. Never look at your watch or the clock on the wall, the session will last as long as it takes and never look bored (even if you are). I know that the fourth interview in a day might become somewhat tedious but always bear in mind that this might be the one where you nail that killer idea!
Show me. If the interviewee starts describing a particular task, if possible, get them to show you the actual task step by step. You’ll learn more that way and you’ll probably find its also quicker than having the interviewee try to remember and describe each step.
At the end of the session thank the interviewee nicely and ensure them that they will get sight of the final report when it is ready. I have always found it incredibly galling when I’ve taken part in research and nobody has bothered to let me know what the outcome was.
Spend time analysing your data. As a general rule it may take as long to carry out a complete analysis of the data as it did to carry out the research so ensure beforehand that enough time is made available. Analysing the closed questions should be relatively simple as the results can be calculated quantitively as e.g. a percentage.
Open questions are a little more difficult to quantify. One technique I learnt from a professor of HCI was a simple sort of textual anaysis and is a close cousin of the card sort. Print all comments off in a table, one comment per cell and then cut along the lines with a pair of scissors so that you end up with a pile of slips with just one comment on each slip of paper. Then, on a large table top, start grouping similar comments together and assign a category to each group. Carry on until all comments sit under a category. Categories might, for instance, include navigation, content, technical issues etc. All that is needed then is to calculate what percentage of the total comments lie in each category and this can then be presented as a quantitive result.
To make the final report more compelling insert one or two comments that best reflect each category. Looking at dry figures only tells you so much. Allowing your users to speak in their own voice by inserting their comments will make for a much more powerful report.
Good luck with your contextual enquiries!
(Thanks to markuz for his workplace Flickr CC photo and obLiterated for the laptop Flickr CC photo)