Emerging wireless services with extremely high data rate requirements, such as real-time extended reality applications, mandate novel solutions to further increase the capacity of future wireless networks. In this regard, leveraging large available bandwidth at terahertz frequency bands is seen as a key enabler. To overcome the large propagation loss at these very high frequencies, it is inevitable to manage transmissions over highly directional links. However, uncoordinated directional transmissions by a large number of users can cause substantial interference in terahertz networks. While such interference will be received over short random time intervals, the received power can be large. In this work, a new framework based on reinforcement learning is proposed that uses an adaptive multi-thresholding strategy to efficiently detect and mitigate the intermittent interference from directional links in the time domain. To find the optimal thresholds, the problem is formulated as a multidimensional multi-armed bandit system. Then, an algorithm is proposed that allows the receiver to learn the optimal thresholds with very low complexity. Another key advantage of the proposed approach is that it does not rely on any prior knowledge about the interference statistics, and hence, it is suitable for interference mitigation in dynamic scenarios. Simulation results confirm the superior bit-error-rate performance of the proposed method compared with two traditional time-domain interference mitigation approaches.