We Probably Know What You Will Read Next

If you think about it, the choice of what you read online is not random. You probably do a quick visual scan of the headlines and possibly the first paragraph. If the content interests you, you might read on to find out more. In practice, most of us end up looking out for those few keywords that indicate that we might find that particular article appealing because of its relevance to our current interests. In other words, a few small pieces of information can determine what and whether we read an article or not.

People we mix with also influence what we read. We usually mix with like-minded people, those who share similar interests or hobbies. So it is perhaps not surprising to learn that our friends tend to read what we read. We turn to our friends for advice because we can’t be experts in everything and these self-selected domain experts in turn, influence how we find out about things. The same principle it seems applies when it comes to our choice of reading material. We may choose slightly different literary topics and have our particular likes or dislikes, but our peer group of companions can still have an impact on our choices.

To explore this phenomenon further, I, together with Professors Wedel and Rust from the University of Maryland, ran a series of field experiments to test out a “News Personalisation” algorithm that works on the logic outlined above to predict what people will or won’t read.

As news articles are effectively collections of words, we designed a novel text mining technique based on a statistical method called the Naïve Bayes algorithm. Articles that a reader chooses to read or not results in the generation of collections of words that either interest the reader or not, respectively. The text mining technique basically predicts whether an individual will choose to read an article by looking at how many of the keywords that interest the reader exists in the article. Specifically, our application of this algorithm predicts that when the article contains more keywords that interest the reader, the more likely will the article be read. In other words, we can derive the probability that an article will be read or not based on the evaluation of a very small but relevant amount of its content.

However, not all keywords are created equal - just as not all topics interest us equally. So part of our procedure actually calculates the usefulness of the different keywords and uses only those highly predictive keywords to make the overall prediction. The trimming and prioritising of predictive keywords is important as the number of such keywords in text mining grows exponentially. We found that by using a modified Naïve Bayes algorithm, our personalisation procedure does an excellent job of screening out news articles which individuals find irrelevant.

There is also a need, however, to cater to the constantly changing tastes of individuals as well as external social influences. We know that what we choose to read is influenced by the people around us and we also know that our (and their) reading preferences may change over time. The algorithm therefore needed to incorporate homophily (the tendency of individuals to bond with those similar to themselves) and induction effects (the influence of like-minded individuals to influence each other) into the news personalisation procedure. In other words, if an individual is made aware of what his friends are reading, the chances are higher that he/she will read it too, rather than another article chosen at random. In addition, if an article is highly popular, the probability of any other individual reading it is also increased.

To be an effective news personalisation algorithm, the system needs to account for individual preferences and also how much the individual accepts recommendations via social influences. We tested for this in our field studies and showed that a news personalisation system that accounts for individual preferences using individualised keywords, together with the incorporation of social influences, significantly improves the proportion of personalised news articles read. While there is obviously an increased risk that pertinent articles may be erroneously screened out, this is clearly balanced by the appropriateness of the articles selected for presentation to each individual. Over time, the system is able to adapt to changing individual preferences, thereby increasing individual satisfaction about what they are presented with. We believe that such algorithms hold much promise in helping refine and focus the way in which news articles are personalised in the future.

About the author

Chung Tuck Siong is the Assistant Research & Development Director of ACI, and Assistant Professor in Nanyang Business School’s Division of Marketing and International Business. His research expertise spans the areas of services and digital marketing, and he has published, among other works, a well-cited article titled “Marketing Models of Service and Relationships” in Marketing Science, a top-tier marketing journal. He has taught Services Marketing for a few years and is now teaching Market Relationships at the Nanyang Business School. He is also the chair of the Marketing and International Business division's PhD committee.