A quasi-experiment measuring how organization of content affects the usability of web sites.

In this quasi-experiment the authors measured the effects of the organization of web sites on users' efficiency in using them. Four university web sites were selected for comparison. Subjects were given the task of finding a specific piece of information on these sites. The time spent and the number of links clicked were measured. The results were suggestive of some general trends and provided partial support for our hypotheses. Implications for future research on the same topic are discussed.

Internet is among the fastest growing and most widely embraced media of today. It is a strong candidate for being the most popular medium of all time with users reaching hundreds of millions worldwide. While the content of radio and TV programs has stayed the same over time, the content users are free to explore on the Internet is quite diverse and rapidly changing. An Internet user can contact other people via chat and e-mail, visit commercial web sites to purchase products, do some of the routine daily activities on-line (e.g., using on-line bank services), collect information about virtually any subject, or contribute his/her own content.

Web sites differ from each other in terms of basic organization of knowledge, page layout, and the nature and the extent of interaction possible. These and many other factors determine whether a web site is user-friendly. The time it takes to find a specific knowledge at a web site is one of the most valid criteria to determine the user-friendliness of that web site (at least at the face-validity level). The organization of a web site should include cues that guide the user correctly in navigation and should be able to do this in the least possible time (assuming a constant speed of Internet connection). At this point, psychology comes into play. However, the application of psychological findings to the improvement of web design has been mostly neglected. This study attempts to do just that.

Although our main criterion is categorizational, research from the cognitive psychology literature has shown that brain does not operate on the basis of strict hierarchical categorization. Quillian (1966) proposed that we store information about various categories such as canaries, robins, fish, etc. in a hierarchical network structure, and named those as "semantic networks". Collins and Quillian (1969) showed that subjects were faster in answering questions related to superordinate category members. However subsequent research findings failed to find such a linear relationship (e.g. Conrad 1972). Anderson (1995) nicely summarized the conclusions:

1. If a fact about a concept is frequently encountered, it will be stored with that concept even if it could be inferred from a more superordinate concept.
2. The more frequently encountered a fact about a concept is, the more strongly that fact will be associated with the concept. And the more strongly associated facts are with concept facts, the more rapidly they are verified.
3. Verifying facts that are not directly stored with a concept but that must be inferred takes a relatively long time. (p. 153)

Recently it has become apparent that semantic networks are not sufficient to explain categories, and schemas started to be used in explaining conceptual knowledge. A schema basically involves a configuration of features holding specific values on an attribute. Schemas provide much more predictable information than semantic networks (Anderson, 1995) since they do not obey hypothetical constructs as much as semantic networks. For instance, although Homo Sapiens is listed deep in the Animal category in the scientific classification of species, most people would have some trouble identifying this link. Instead most would initially think that humans have a rather special standing within that classification. Even if we all know the classification of species very well, prior knowledge and environmental factors would distort the individual response patterns to some extent. The implications are more obvious in real life since we all filter and store information according to our unique tastes and interests. So, it is expected that the contents of individual schemas of a concept would vary from person to person. To exploit these findings further we considered several cognitively helpful web design issues such as the use of rollovers, pop-up menus, and search capabilities.

The term rollover is used for the dynamic elements of a web page, which are sensitive to changes in the position of the mouse pointer. For instance, when the user points on one link (but does not click), as in the case of UNC's web site, additional information about that link appears on the screen. Pop-up menus are similar to rollovers but are much more interactive. When you move the mouse pointer over a link, a small menu including a number of clickable options related to the main link is displayed on the screen (The DU web site includes pop-up menus). Finally, search capabilities (typically involving a small box where you can enter keywords) give the user the option of bypassing the interface and engaging in a direct search covering all the contents of a web site. All of the four universities have a search tool but as we will soon see, UCI employs a different and somewhat superior search technique.

To test our hypotheses about the usefulness of these techniques and how much they contribute to the user-friendliness of a web site, we provided subjects with the homepages of one of the four universities. Using all features of the web site they were given, subjects were required to find the homepage of the Department of Physics and Astronomy under that university as fast as they could. The four universities each have a number of different paths from the homepage to the Department of Physics and Astronomy's homepage. We examined the web sites and found the optimal path (intended to be so by the designer) for each. Among all possible paths, the optimal path represents the shortest way to reach the intended location, involving the least number of clicks. The labels of the links for the optimal paths are as follows:

UCI: Instruction & Research > Physical Sciences > Department of Physics and Astronomy (3 clicks)

DU: Academics \ Academic Programs (from pop-up menu) > Physics and Astronomy (2 clicks)

UNC: Departments > Physics and Astronomy (2 clicks)

USM: Academics > Colleges, Schools and Departments > College of Science and Technology > Academics > Department of Physics and Astronomy (5 clicks)

Subjects can, of course, reach the departmental homepage through completely different routes and we take that as a successful navigation, too. What we want to observe, however, is in which university subjects are more likely to discover the optimal path. If such a difference occurs and is meaningful, then by carefully examining the university or universities in which the subjects are most successful in navigating through, we will try to identify elements of web design that contribute to this success. Our criteria of user success in navigation are the time spent and the number of links followed to the target.

Based on experience (one of the authors is a professional web designer), intuition, and findings from the cognitive psychology literature, we had some prior beliefs about elements of web design. First of all, since the lesser the number of links, the faster the access (to the intended location, departmental homepage in this study), we hypothesized that redundant links would reduce efficiency and might cause confusion by increasing the time spent and the number of links followed. We chose the web site of USM as an inferior example because it contained redundant links. There was a point where subjects had to click a link labeled "Academics" for the second time.

From our experience with the Internet, we are also aware that there is not a well-established labeling convention for links that serve similar purposes. For instance, UCI has labeled what most other universities (in fact, all others in our study), conventionally labeled "Academics" as "Instruction and Research". In order to follow the optimal path in UCI, one must click on this link. We hypothesize that this inability to provide common labels to the user will result in confusion and therefore increase the time spent. The standardization of labels, both within a site and between sites designed for similar purposes (e.g., educational, commercial, etc.) should be very helpful for visitors. We also believe that rollovers increase efficiency by summarizing information and giving the user helpful hints about the effects of clicking on a link. The same is true for pop-up menus. They provide users cues about category membership of links. For instance, when one moves the mouse pointer over a link, say academics, a menu pops up and it includes a number of other links, say one of them is called staff. The user can infer that "staff" here refers to academic staff whereas he/she cannot make such an inference if staff is a separate link. Pop-up menus also help the user save time by hiding links. At DU, there are seven links with pop-up menus. Each of these menus includes six links on average. That makes close to fifty links. Imagine trying to find a link among fifty links listed on one page. It should be a lot more time consuming than identifying a major category first (such as academics, admission, libraries, etc.) and doing a refinement by choosing a more specific link that appears in the pop-up menu.

Searching by words is sometimes a better option than following links. Our subjects, although not explicitly encouraged, were free to perform a search. The UCI is noteworthy here because of its "QuickLink" feature which allows one-click access to the target (entering the word "physics" and clicking on the QuickLink button brings the homepage of the Department of Physics and Astronomy -astonishingly more efficient than other alternatives in terms of both time and number of links). By placing the search box on the top of its homepage, UCI also encourages its visitors to commit a search more than other universities.

To Page 2

Rating: Not rated yet