Contrasting Impact of Start State on Performance of AReinforcement Learning Recommender System

A recommendation problem and RL problem are very similar, as both tryto increase user satisfaction in a certain environment. Typical recommender systems mainly rely on history of the user to give future recommendations and doesn’t adapt well tocurrent changing user demands. RL can be used to evolve with currently changing user demands by considering a reward function as feedback. In this paper,recommendation problem is modeled as an RL problem using a squared grid environment, with each grid cell representing a unique stategenerated by a biclusteringalgorithm Bibit. These biclusters are sorted according to their overlapping and then mapped to a squared grid. An RL agent then moveson this grid to obtain recommendations. However, the agent hasto decide themost pertinent start state that can give best recommendations. To decide the start state of the agent, a contrastingimpact of different start states on the performance of RL agent-based RSs is required. For this purpose, we applied seven different similarity measures to determine the start state ofthe RL agent. These similarity measures are diverse, attributed tothe fact that some may not use rating values, some may only use rating values, or some may use global parameters like average rating value or standard deviation in rating values. Evaluation is performed on ML-100K and FilmTrust datasets under different environment settings. Results provedthat careful selection of start state can greatly improve the performance of RL-based recommender systems.