IVIR

News recommender systems aim to predict which news items their users would like to read based on their past reading behavior. However, rather than only catering to a readers' preferences, a diverse recommender system could also be used to expand a reader's world view, to help them be more informed, or to expose them to events and ideas they were not aware of before. This dissertation therefore aims to answer the question: “How can we evaluate news recommender systems on their normative diversity?” The dissertation takes an interdisciplinary approach towards answering this question. It contains interviews with practitioners from public service media organizations in the Netherlands on how they conceptualized diversity in their recommender systems (Chapter 2); proposes new diversity evaluation metrics founded in democratic theory (Chapter 3); generalizes these evaluation metrics into a rank-aware divergence-based formalization (Chapter 4); analyzes the public datasets available for news recommendation on their suitability to diversity-based research (Chapter 5); and describes workshop sessions with a national news organization to collaboratively define evaluation metrics for their recommender systems (Chapter 6). The work shows that there is no one-size-fits-all solution to implementing diversity. Furthermore, it notes that it is fundamentally unlikely that abstract theoretical concepts can be perfectly captured in technical applications. Instead, it argues that we should aim for consciously imperfect solutions that are understood and accepted by all different stakeholders within an organization; to look for workable simplifications, rather than reductive generalizations.