publications

2024

1. SIGIR

   Can We Trust Recommender System Fairness Evaluation? The Role of Fairness and Relevance

   Theresia Veronika Rampisela, Tuukka Ruotsalo , Maria Maistro , and Christina Lioma

   In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , 2024

   Abs arXiv Code Slides

   Relevance and fairness are two major objectives of recommender systems (RSs). Recent work proposes measures of RS fairness that are either independent from relevance (fairness-only) or conditioned on relevance (joint measures). While fairness-only measures have been studied extensively, we look into whether joint measures can be trusted. We collect all joint evaluation measures of RS relevance and fairness, and ask: How much do they agree with each other? To what extent do they agree with relevance/fairness measures? How sensitive are they to changes in rank position, or to increasingly fair and relevant recommendations? We eempirically study for the first time the behaviour of these measures across 4 real-world datasets and 4 recommenders. We find that most of these measures: i) correlate weakly with one another and even contradict each other at times; ii) are less sensitive to rank position changes than relevance- and fairness-only measures, meaning that they are less granular than traditional RS measures; and iii) tend to compress scores at the low end of their range, meaning that they are not very expressive. We counter the above limitations with a set of guidelines on the appropriate usage of such measures, i.e., they should be used with caution due to their tendency to contradict each other and of having a very small empirical range.

2023

1. ACM TORS

   Evaluation Measures of Individual Item Fairness for Recommender Systems: A Critical Study

   Theresia Veronika Rampisela, Maria Maistro , Tuukka Ruotsalo , and Christina Lioma

   ACM Trans. Recomm. Syst., Nov 2023

   Abs arXiv Code

   Fairness is an emerging and challenging topic in recommender systems. In recent years, various ways of evaluating and therefore improving fairness have emerged. In this study, we examine existing evaluation measures of fairness in recommender systems. Specifically, we focus solely on exposure-based fairness measures of individual items that aim to quantify the disparity in how individual items are recommended to users, separate from item relevance to users. We gather all such measures and we critically analyse their theoretical properties. We identify a series of limitations in each of them, which collectively may render the affected measures hard or impossible to interpret, to compute, or to use for comparing recommendations. We resolve these limitations by redefining or correcting the affected measures, or we argue why certain limitations cannot be resolved. We further perform a comprehensive empirical analysis of both the original and our corrected versions of these fairness measures, using real-world and synthetic datasets. Our analysis provides novel insights into the relationship between measures based on different fairness concepts, and different levels of measure sensitivity and strictness. We conclude with practical suggestions of which fairness measures should be used and when. Our code is publicly available. To our knowledge, this is the first critical comparison of individual item fairness measures in recommender systems.

2021

1. ISMART

   Classification of the likelihood of Indonesian Facebook users in spreading hoaxes using Support Vector Machine (SVM)

   Theresia Veronika Rampisela, Hanna Tiara Andarlia , and Zuherman Rustam

   In Journal of Physics: Conference Series , Nov 2021

   Abs

   Social media is the most commonly accessed internet content by Indonesian netizens; 97.4% Indonesians access social media while using the internet. In 2016, Facebook was the most commonly used social media format for Indonesian internet users. While many are benefited by the features offered by Facebook, many also use Facebook for things that they are not supposed to, such as sharing hoaxes, either intentionally or unintentionally. This clearly puts Facebook users at a disadvantage, considering the increasing trend of hoax-sharing nowadays. Therefore, this research aims to prevent the spread of hoax through Facebook by analyzing the pattern of how people use Facebook. This pattern is obtained from a survey on 200 samples that use Facebook, chosen by purposive sampling. Using the Support Vector Machine method, an application of the collaboration between mathematics and computer science, the acquired data is used to predict whether or not someone has the potential of spreading hoaxes. Simulation results show that the average of the prediction accuracy of this binary classification problem is 86 percent. Hence, it is hoped that Facebook could prevent the sharing of hoaxes by making use of the results from this research.

2020

1. ICoICT

   Academic Expert Finding in Indonesia using Word Embedding and Document Embedding: A Case Study of Fasilkom UI

   Theresia Veronika Rampisela, and Evi Yulianti

   In 2020 8th International Conference on Information and Communication Technology, ICoICT 2020 , Nov 2020

   Abs

   Expertise retrieval covers the problems of expert and expertise finding. In academia, expert finding can be beneficial in finding a research partner or a potential thesis supervisor. This research finds the experts in the Faculty of Computer Science in Universitas Indonesia (Fasilkom UI) using the thesis abstract and metadata of Fasilkom UI students. The methods that are used to represent the query and expertise of the lecturers are the combination of word2vec and doc2vec, which are word embedding and document embedding, respectively. Both embeddings are able to model semantic information, which is necessary for solving the problem of vocabulary mismatch in search problems. Our result shows that representing the expertise query with word2vec leads to better performance than using doc2vec. In addition, we also found that generally, the performance of the embedding models is comparable to the standard retrieval model BM25 in retrieving experts using expertise queries in both Indonesian and English languages.

2. ISITIA

   Characteristics of Expertise Locator System in Academia: A Systematic Literature Review

   Theresia Veronika Rampisela, Damayanti Elisabeth , and Dana Indra Sensuse

   In Proceedings - 2020 International Seminar on Intelligent Technology and Its Application: Humanification of Reliable Intelligent Systems, ISITIA 2020 , Nov 2020

   Abs

   The ever-increasing need to find experts in academia contributes to the rise of Expertise Locator System (ELS) in academia. ELS can exist in the form of an expert search system or expert recommender system. Albeit the systems’ common purpose of finding an expert, the existing ELS are different from each other in characteristics. There is a consideration, reason, and purpose why an ELS is created with specific characteristics. It is therefore essential to assess and review the characteristics of the system, before creating an ELS. By systematically reviewing recent articles using the Kitchenham method, this research identifies various type of characteristics, so that in the future, ELS in academia can be developed with these characteristics. This study has also identified and proposed seven new characteristics of ELS.

3. ICEMT

   Expert locator system for finding co-authors using snowball network method: A case study of Fasilkom UI lecturers

   Theresia Veronika Rampisela, Dana Indra Sensuse , Damayanti Elisabeth , and Nurma Ayu W. Subroto

   In Proceedings of the 4th International Conference on Education and Multimedia Technology , Nov 2020

   Abs

   One of the essential activities for academics is research. Research is typically done through collaborations. One way to find research collaborators is through existing connections, for example, via the co-author relationships. However, existing systems only provide limited information on collaborators. Furthermore, previous works have not fully explored the details of co-author relationships. Hence, we propose an Expert Locator System (ELS) that can identify collaborators and provide information on the location, publication keywords, as well as their impact and productivity. We can obtain the relationship using the snowball network method, a type of Social Network Analysis (SNA). This ELS can serve to obtain collaborators’ information that can be useful in conducting research.

4. IALP

   Semantic-Based Query Expansion for Academic Expert Finding

   Theresia Veronika Rampisela, and Evi Yulianti

   In 2020 International Conference on Asian Language Processing, IALP 2020 , Nov 2020

   Abs

   Expert finding in academic domain is useful for many purposes, such as: to find research collaborators, article reviewers, thesis advisors, thesis examiners, etc. This work examines the use of semantic information, i.e. word embedding and document embedding, for query expansion to enhance the effectiveness of expert finding system. This information is utilized to bridge the lexical gap between the query and the expertise evidence of the experts. This semantic-based query expansion approach is then combined with a BM25 retrieval method to find relevant experts to the given query. The results show that our methods consistently outperform the strong retrieval method BM25, the semantic-based retrieval, and query expansion using pseudo relevance feedback method according to all recall- and precision-based measures used in this work. This indicates the effectiveness of our methods in improving the number and the accuracy of relevant experts retrieved.

2018

1. MISEIC

   Classification of Schizophrenia Data Using Support Vector Machine (SVM)

   Theresia Veronika Rampisela, and Zuherman Rustam

   In Journal of Physics: Conference Series , Nov 2018

   Abs

   Schizophrenia is a severe and chronic mental disorder. This disorder is marked with disturbances in thoughts, perceptions, and behaviours. Due to these disturbances that can trigger Schizophrenics to commit suicide or attempt to do so, Schizophrenics have a lower life expectancy than the general population. Schizophrenia is also difficult to diagnose as there is no physical test to diagnose it yet and its symptoms are very similar to several other mental disorders. Using Northwestern University Schizophrenia Data, this research aims to distinguish people who are Schizophrenics and people who are not. The data consists of 392 observations and 65 variables that are demographic data and clinician-filled Scale for the Assessment of Positive and Negative Symptoms questionnaires. Classification method used is machine learning with Support Vector Machines (SVM). Simulations are done with different data and percentage of training data. In each simulation, accuracy is measured. Model performance validation and evaluation are done by averaging ten times Hold-Out Validations that were done. In conclusion, SVM successfully classified Schizophrenia data with final accuracy of 90.1%. Furthermore, SVM with linear kernel and Gaussian kernel reached an accuracy of 95.0% in at least one simulation in classifying Schizophrenia data.