Personalizing Type-Based Facet Ranking

. In Faceted Search Systems (FSS), users navigate the information space through facets, which are attributes or meta-data that describe the underlying content of the collection. Type-based facets (aka t-facets) help explore the categories associated with the searched objects in structured information space. This work investigates how personalizing t-facet ranking can minimize user effort to reach the intended search target. We propose a lightweight personalisation method based on Vector Space Model (VSM) for ranking the t-facet hierarchy in two steps. The ﬁrst step scores each individual leaf-node t-facet by computing the similarity between the t-facet BERT embedding and the user proﬁle vector. In this model, the user’s pro-ﬁle is expressed in a category space through vectors that capture the users’ past preferences. In the second step, this score is used to re-order and select the sub-tree to present to the user. The ﬁnal ranked tree reﬂects the t-facet relevance both to the query and the user proﬁle. Through the use of embeddings, the proposed method effectively handles unseen facets without adding extra processing to the FSS. The effectiveness of the proposed approach is measured by the user effort required to retrieve the sought item when using the ranked facets. The approach outperformed existing personalization baselines.


Introduction
Faceted Search Systems (FSS) have become one of the main search interfaces used in vertical search systems, offering users meaningful facets to refine their search query and narrow down the results. As the magnitude of data in a collection increases, the number of facets becomes impractical to display on a single page. Ranking the top facets is required as it assists the searcher in locating the target document with minimum effort.
When the information space is structured, type-based facets (t-facets) are extracted from the types of objects (e.g. values of isA or rdfs:type relationships) [1]. Hierarchical taxonomies of types are derived from ontologies by exploiting the subClassOf relationships. When the t-facets belong to a large multilevel hierarchy of categories, it is particularly important to prioritize the most relevant t-facets that can help the user filtering the results by type.
In this work, we analyse the role of personalization in t-facet ranking in isolation from other FSS aspects. Existing facet ranking methods rely on attribute frequencies, navigation cost models, textual queries or click logs to order the facets [1]. This study contributes to the research in this area by introducing a light and effective ranking algorithm for type-based facets.
The algorithm exploits the user's past preferences to build a vector which represents the user profile. T-facets are scored according to their similarity with user profile by exploiting their BERT embeddings. The proposed approach works in two steps. First, it generates personalized relevance scores for each t-facet in the last level of the taxonomy. Then, the second stage aggregates these scores to re-arrange the ancestor t-facets and rebuild the final t-facet tree to be rendered to the user. Our experiment aims at answering the following research question: RQ: Does personalizing the t-facet ranking using BERT embeddings minimize users effort to fulfil their search needs?
The implemented approach is evaluated using the TREC Contextual Suggestion (TREC-CS) track dataset [2]. TREC-CS is a personalized Point-Of-Interest (POI) recommendation task. We solve the POI suggestion problem by ranking the types of venues as t-facets. In our evaluation, we measure the extent to which this ranked tree minimizes the user effort to reach the first relevant POI.

Personalized Facet Ranking Related Research
Several approaches have been proposed in the literature to solve the problem of personalized facet ranking that make use of individual user models, collaborative filtering (CF), or a mixture between the two. Factic [3] is a FSS that personalizes by building models from semantic usage logs. Several layers of user adaption are implemented and integrated with different weights to enhance the facet relevance model . Koren et al. [4] suggested a CF approach by leveraging explicit user feedback about the facets, which is used to build a facet relevance model for individuals. They also use the aggregated facet ratings to build a collaborative model for the new users in order to provide initial good facets in absence of a user profile. The Adaptive Twitter search system generates user models from Twitter to personalize facet ordering [5]. The user model contains entities extracted from the user's tweets. The facets are weighted higher if they exist in the user profile. Le et al. [6] also collects user profile from social networks. The profile is learned from user activities and preferences using a tf-idf feature vector model. Important facets are then highlighted through a matching with the model. A personalized ranking based on CF features was suggested by Chantamunee et. al [7,8].They used user ratings and Matrix Factorization via SVM and Autoencoders to learn facet ranks.
All the aforementioned approaches do not address the special case of t-facets. Sah and Wade [9] employ session-based user interaction to personalize t-facets. When the searcher selects a t-facet, the system re-organizes the other t-facets according to their similarity to the selected one. Neither the user interests nor the hierarchical nature of the t-facets are considered by this method. Our approach exploits VSM to build user profile, which employs users' historical ratings to infer their preferred t-facets, an unexplored area in literature.

Proposed Approach
Our method works in the context of personalized venue search. When a user submits a query, the underlying search engine retrieves a relevant set of venues for it 3 . In order to achieve query relevance, our method works on this set by collecting the t-facets associated with the retrieved venues. The proposed t-facet ranking approach consists of two steps. The first step assigns a relevance score to each t-facet leaf node. The input to this step are the retrieved venues with their relevancy score, the t-facets to which they belong, and the user profile. The second step constructs the final t-facet tree to be displayed to the user. The input to this step are both the score for each t-facet (from the first step) and the original hierarchical taxonomy from which the t-facets are derived. The output of the t-facet ranking is a sub-tree, which contains the ordered set of relevant t-facets.
Step 1: T-Facet Scoring using BERT vectors. At this step, we establish relevance between the t-facet and the user profile through their vector representation. Each t-facet f i is represented by a BERT embedding − → f i . We employ the t-facet's label to generate its corresponding BERT embedding using pre-trained BERT model 4 . BERT provides a meaningful vector representation of text and has been proven effective in many IR tasks. It is particularly useful in this task as it semantically embeds the meaning of the type in the vector. This is important in handling new and unseen t-facet which did not exist in the user profile before. In this case the model will handle them based on their similarity with existing t-facets. In our early experiments we also experimented word2vec models but BERT-based models gave better results. However we acknowledge that other semantic-based embedding can be used in this approach and improve the results.
The vector representing the user profile is obtained using the Rocchio formula, which is a classical approach to relevance feedback in IR. Since in our case we only have the user feedback (i.e. ratings) at POI level, we assume that the rating can also be transferred to the types associated with the POI. Based on this assumption, we represent the user profile through a vector combining positive, negative and neutral preferences (i.e. rated POI types). We define the positive (pos.) user vector − → u i pos as the average of t-facet vectors rated positively by the user u i : Where F pos u is the set of t-facets rated positively by the user. Each vector is weighted by w f pos j (see eq 2). The weight averages the probability that this specific user will rate the t-facet as positive Pr( f pos j |u i ) and the probability that any user in the system will rate the t-facet as positive Pr( f pos j |U). In absence of any personal preference only the general probability will be used. The value w f pos j affects how each t-facet contributes to the final positive profile vector.
The parameter γ ∈ [0, 1], is used to balance between individual user feedback versus general population feedback. In a similar manner, the negative (neg.) and the neutral (neu.) user vectors are computed. In order to model the final vector representation of the user's interest, we experimented with two versions of the Rocchio formula. The first (Rocchio-BERT) is the traditional version computed according to equation 3.
The contribution of each user vector is regulated through the weights α, β , λ ∈ [0, 1]. The second (Ortho-BERT) is a modified version of Rocchio formula proposed by [10], in which instead of subtracting the negative vector they add the user negative vector's orthogonal complement ( − → u i neg ⊥ ). Equation 4 illustrates the computation.
The orthogonal complement is obtained using Gram-Schmidt process. The weights are the same as equation 3. Finally, the query t-facets are ranked according to the cosine similarity score between the user profile vector − → u i and the the t-facet vector − → f j .
Step 2: T-Facet Tree Building The tree construction algorithm re-orders the original taxonomy tree by using the generated scores from the previous step. It follows a bottomup approach where the t-facets at the lower level in the taxonomy are sorted first, then it proceeds by sorting all the ancestors of those t-facets, and so on up to the root of the hierarchy. To build a final t-facet tree with v levels, we adopted a fixed level strategy [11]. The strategy respects the original taxonomy hierarchy and uses a predefined fixed page size for each t-facet level. It starts by grouping t-facets at level-v by their parent. Then, it sorts the parent nodes at level-(v − 1) by aggregating the scores of their top k children, the children are ordered by their relevance score generated in step 1, and so on up to level-1. We use Max. aggregation function to keep the top ranked t-facet at the top of the final tree.

Experimental Results
Experimental setup. Our approach is evaluated on TREC-CS 2016 dataset [2]. The tfacet taxonomy is derived from the Foursquare venue category hierarchy 5 . To link as much Foursquare venues to TREC-CS POIs as possible, we complement the original data with three Foursquare supplementary datasets from [12,13] and our own crawled POIs.
The final dataset has 58 requests and an average of 208 t-facets per request to be ranked.
Only the first two levels of the taxonomy are included. The document search engine implements BM25 with NDCG value of 0.4023, the query is formed by combining user tags weighed by their most common rating. The existence of relevance judgments makes it possible to evaluate our approach against a well established ground-truth. We follow the strategy used in Faceted Search task of INEX 2011 Data-Centric Track [14]. We report two metrics suggested by the organizers. The number of actions (#Actions) counts how many clicks the user has to perform on the ranked facets in order to reach the first relevant document in the top 5 results. The faceted scan (F-Scan) measures the user's effort to scan facets and documents until user reach the same document. It is calculated as the sum of the t-facet rank in the tree plus the document position in the list after being filtered using this t-facet. We focus on these metrics as a proxy for user's effort, which will help in answering our research question. To chose the optimal weights for equations 4 and 3, we used hyper-parameter tuning with range from 0 to 1 and step of 0.25. We report results for the Rocchio-BERT and Ortho-BERT methods in addition to KNN-BERT, which uses a trained KNN model for each user to predict the user's preference to a given t-facet, with k = 1. All scoring results are reported by adopting the Fixed Level-Max strategy, with level-1 and level-2 page size set to three. , and λ = 0, which means that the positive vector profile is the key component in the formula. γ = 0, which means that the person probability is favoured over the global probability in the t-facets weighting(2. The second part of Table 1 reports the results of other existing personalization methods. As these methods do not handle the hierarchical nature of the t-facets, we use them as scoring methods with the Fixed Level-Max strategy. Rocchio-BERT still results in the minimum #Actions and F-scan when compared with these models. In addition, the proposed VSM approach has other advantages. It provides a light yet effective personalized ranking. The user profile vectors can be pre-computed and stored offline, reducing the computation at retrieval time of the cosine distance between the query t-facets and the user profile. This can be further optimized by pre-computing a distance matrix for the user and all t-facets in the FSS. User profiles on the other hand, can be updated offline as the user rates new venues. Optimizing this step will reduce the total time needed by FSS to populate its final results page.

Conclusions
This work presented a two-step t-facet ranking approach employing BERT embeddings to personalize the t-facet ranking. The first step assigns score to t-facets. The second step uses the score to re-arrange and build the final t-facet tree. To personalize the scores, we explored several methods based on vector space model. They showed promising results when combined with the personal preferences of the users. We plan to experiment with additional POI suggestion datasets and explore learning to rank methods for the t-facet scoring step.