Because of the open character of Wikipedia readers should always be aware of the possibility of false information. WikiTrust aims at helping readers to judge the trustworthiness of articles by coloring the background of less trustworthy words in a shade of orange. In this study we look into the effects of such coloring on reading behavior and trust evaluation by means of an eye-tracking experiment. The results show that readers had more difficulties reading the articles with coloring than without coloring. Trust in heavily colored articles was lower. The main concern is that the participants in our experiment rated usefulness of WikiTrust low.
WikiTrust seems to be slightly more useful when only small parts are colored, but even then usefulness is limited. Participants in our experiment were not sure what to do with the information on the age of words in the text. Further development of WikiTrust could benefit from knowledge about the (heuristic) strategies of Wikipedia users when assessing trustworthiness.
WikiTrust is a promising support tool and in fact the only one that made it to the stage where it is actually available to the Wikipedia public. This study has shown that the decision to present trust information by a separate tab was right since reading behavior is affected by its coloring. However, more effort should be put into the usability of the system. This study showed that users are having problems to see how they can benefit from it, even though a clear explanation was provided and the participants were highly educated Master's students.
