Recommender systems have become a core component of various online platforms, helping users get relevant information from the abundant digital data. Traditional RSs often generate static recommendations, which may not adapt well to changing user preferences. To address this problem, we propose a novel reinforcement learning (RL) recommendation algorithm that can give personalized recommendations by adapting to changing user preferences. However, a significant drawback of RL-based recommendation systems is that they are computationally expensive. Moreover, these systems often fail to extract local patterns residing within dataset which may result in generation of low quality recommendations. The proposed work utilizes biclustering technique to create an efficient environment for RL agents, thus, reducing computation cost and enabling the generation of dynamic recommendations. Additionally, biclustering is used to find locally associated patterns in the dataset, which further improves the efficiency of the RL agent's learning process. The proposed work experiments eight state-of-the-art biclustering algorithms to identify the appropriate biclustering algorithm for the given recommendation task. This innovative integration of biclustering and reinforcement learning addresses key gaps in existing literature. Moreover, we introduced a novel strategy to predict item ratings within the RL framework. The validity of the proposed algorithm is evaluated on three datasets of movies domain, namely, ML100K, ML-latest-small and FilmTrust. These diverse datasets were chosen to ensure reliable examination across various scenarios. As per the dynamic nature of RL, some specific evaluation metrics like personalization, diversity, intra-list similarity and novelty are used to measure the diversity of recommendations. This investigation is motivated by the need for recommender systems that can dynamically adjust to changes in customer preferences. Results show that our proposed algorithm showed promising results when compared with existing state-of-the-art recommendation techniques.