Log messages record important system runtime information and are useful for detecting anomalous behaviors and managing modern software systems. Many supervised and unsupervised learning methods have been proposed recently for log-based anomaly detection. State-of-the-art unsupervised methods predict the next log event given a log sequence and apply fixed configurations that use the same filter condition (i.e. k, the top k predicted log events will be regarded as normal next events) which leads to inferior performance in the detection stage because it sets one fixed k for all log sequences, which ignores the dynamic nature and variance in different log sequences. Recently, deep reinforcement learning (DRL) are widely applied to make intelligent decisions in a dynamic environment. In this work, we contend that it is necessary to apply adaptive filters for different log sequences. To achieve this, we propose a novel approach based on DRL to construct a learned adaptive filter and apply different normal/abnormal filter thresholds for different log sequences. We define the Markov Decision Process (MDP) and formulate the learned adaptive filter as a problem that can be solved by DRL. We evaluate the learned adaptive filter on two state-of-the-art log-based anomaly detection unsupervised approaches DeepLog and LogAnomaly in two datasets HDFS and BGL. Extensive experiments show that our approach outperforms the fixed configurations and achieves significantly better performance in log-based anomaly detection.