Content value analysis for intranets

September 14, 2008

It might go something like this –

– Sampling would have to represent a fair percentage of the content and samples would have to be taken from several areas of a site or sub-site. Areas must be selected without bias. In an intranet with 1,000 pages a fair sample might be at a minimum 5%, that is 50 pages, or you could use statistical tables for sample sizes.

– Each page would be examined and scored for its value against a set of heuristics e.g. if content is obsolete, outside of a review date, incorrect, incomplete or irrelevant the score would be 0. If there is some value on the page it could be marked as 0.5 and if the page content generally has value the score would be 1. Perhaps screen dumps of representative non-value adding pages might form part of a results document. 

– The scores would then be aggregated and presented as a percentage e.g. if after appraisal of our sample 50 pages the score is 28.5, then we could present the site as providing only 57% of the possible content value to its users

This might encourage content owners to improve their scores by ensuring that non-value adding pages are deleted, reviewed or that good content is added. I think that the trick in making this approach work will be in defining the heuristics and finding a way of removing any bias from the sample.

Using this approach (I call it ‘Content Value Analysis’ as it has to be called something) it should be possible to benchmark sites and parts of sites against each other by using the same heuristics and, if done periodically, to chart improvement trends in the value a site offers its users.

Hopefully this approach will provide intranet managers with a tool that allows them to assess content in a structured way, produce a number that will accurately reflect the state of the content within the site evaluated and then to repeat the process over time to ensure that the intranet is going in the right direction – towards the lean intranet. Results can then be presented graphically as bar charts or graphs – a form of information that most senior managers are comfortable with and understand.

For a more complete description of the methodology see Part 2

11 Responses to “Content value analysis for intranets”

  1. Interesting post Patrick. Have you ever been able to apply it to a site you manage?

  2. patrick c walsh Says:

    Not yet but I am going to give it a try. As I say the heuristics have to be right. I’ll do a post on the blog once I’ve carried out a round and I’ll let you know what heuristics I used to support the scoring regime.

  3. Great to see this approach.

    In my experience with information usage surveys, sampling of the production side of the equation (i.e., the physical information) is difficult to tie to the usage side (e.g., benefits received from using, cost of accessing, etc.) without some kind of profiling of content or meaning. Sometimes this can be related to the goals and functions of the sponsor (e.g., this is why our organization exists, let’s see if the web site supports it).

    I agree that issues such as frequency of update and obsolescence are important but they are secondary to how and why the web site is being used. Very old information may actually be of very high value.

  4. Kim Ashley Says:

    Our intranet team tried something similar, with the evaulation work being distributed among individual intranet editors within the company. While I liked the idea, I found the results nearly useless because the evaluators didn’t know enough about all aspects of the business to be able to evaluate how appropriate, complete, relevant, and useful the content is. Some content that I know to be highly used and valuable was rated low, and some content that I know to be incomplete or incorrect was rated high. Our intranet team also doesn’t have the breadth/depth of knowledge to do the rating.

    Do you have any suggestions for overcoming this problem? We’re about to do this exercise again, and I’d like to suggest ways to improve the quality of our quality ratings 😉

  5. patrick c walsh Says:

    Thanks for your thoughtful comment. You are absolutely right that old information may be as valid as new information. However reviewing information is important. Its a fast moving world we live in and things can change quickly. Also many CMS display the review dates at the foot of the page and users may not trust information that has not been reviewed on time.
    I also totally agree that its difficult to tie the benfits gained by the user to the content on the page. Didn’t say I had it all worked out yet! If you have any ideas on that I’d be delighted to hear them

  6. patrick c walsh Says:

    You have brought up a very important point. It goes to the heart of a concept very dear to my heart – content centred design. The paradigm to date with intranets seems to be that basically intranet teams are there to provide the container (hardware, software, structure) while the content is left to people who have no idea what the structure is or should be or what other people are throwing into the intranet from other areas.
    IMHO an intranet is its content. Its what give benefits to its users and what an intranet team is judged on. Intranet teams need to know and understand all content – else how can they get the best out of the intranet by designing around the content?
    In short the intranet team must manage and approve not only the infrastructure but the content as well. That way they should be conversant with the content and able to make a valid judgment. In the short term I would use a my common sense for the majority of pages and content experts for those pages that I wasn’t sure of. However the thought occurs that if content is well written and structured shouldn’t it be understandable to all?

  7. Kim Ashley Says:

    Thanks Patrick, I hadn’t considered it that way, but I agree. It’s the easy way out for the intranet team to maintain the infrastructure structure and farm out the content. While our intranet team does try to provide a consistent interaction experience, they don’t provide a consistent content experience and it creates an overall patchwork-y feel that I hadn’t quite figured out how to verbalize. This is helpful!

  8. patrick c walsh Says:

    Thanks for that. I think you have hit a very important nail on the head!
    A great source of frustration with many intranet staff is that they have a problem but they can’t name it and if they can’t name it they can’t get a handle on it. Looking at the problem from the content end and using the content as well as the users to inform design will, IMHO, remove many of these problems.
    Sites often have problems with navigation due to too much mediocre content and, rather than examine the content and try to reduce it so that only top grade content is left, they embark on ever more complex and unwieldy navigational structures that add to the problem rather than solving it.

    Best of luck with your intranet

  9. Patrick, I didn’t mean to suggest that it isn’t necessary to review digital media periodically to make sure it is current, just that doing so without attending to how the info is used that is gleaned from the pages would be insufficient. As far as linking physical measures to usage and benefit measures, I don’t think there’s any substitute for asking people about why they looked for the information and how they used it. The trick is to do so in a way that is representative of actual usage, and that gets us into the (potentially expensive) realm of sampling, statistics, questionnaire wording, etc.

  10. patrick c walsh Says:

    I agree totally. Unfortunately many of the people responsible for the content don’t appreciate the point. To be honest the main point of doing Content Value Analysis is to try and get a measurement that can be used to convince content owners, management etc that their content is basically not up to scratch. If they really listened to their users there would be no need for CVA at all.
    I think that CVA should be only an interim approach – once intranet content is validated centrally and managed as in the ‘lean intranet’ approach there will be little need for the technique

  11. […] Content value analysis for intranets I’m not suggesting that IAs become statisticians (although using statistics really give your results some gravitas) but you can’t evaluate every intranet page. Sampling techniques have been used with success in the manufacturing sector for nearly 90 years. […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: