The concept of cognitive radar (CR) enables radar systems to achieve intelligent adaption to a changeable environment with feedback facility from receiver to transmitter. However, the implementation of CR in a fast-changing environment usually requires a well-known environmental model. In our work, we stress the learning ability of CR in an unknown environment using a combination of CR and reinforcement learning (RL), called RL-CR. Less or no model of the environment is required. We also apply the general RL-CR to a specific problem of automotive radar spectrum allocation to mitigate mutual interference. Using RL-CR, each vehicle can autonomously choose a frequency subband according to its own observation of the environment. Since radar's single observation is quite limited compared to the overall information of the environment, a long short-term memory (LSTM) network is utilized so that radar can decide the next transmitted subband by aggregating its observations over time. Compared with centralized spectrum allocation approaches, our approach has the advantage of reducing communication between vehicles and the control center. It also outperforms some other distributive frequency subband selecting policies in reducing interference under certain circumstances.