Owing to the enhanced flexibility in deployment and decreasing costs of manufacturing, the demand for unmanned aerial vehicles (UAVs) is expected to soar in the upcoming years. In this paper, we explore a UAV aided search and rescue~(SAR) operation in indoor environments, where the GPS signals might not be reliable. We consider a SAR scenario where the UAV tries to locate a victim trapped in an indoor environment by sensing the RF signals emitted from a smart device owned by the victim. To locate the victim as fast as possible, we leverage tools from reinforcement learning~(RL). Received signal strength~(RSS) at the UAV depends on the distance from the source, indoor shadowing, and fading parameters, and antenna radiation pattern of the receiver mounted on the UAV. To make our analysis more realistic, we model two indoor scenarios with different dimensions using commercial ray-tracing software. Then, the corresponding RSS values at each possible discrete UAV location are extracted and used in a Q-learning framework. Unlike the traditional location-based navigation approach that exploits GPS coordinates, our method uses the RSS to define the states and rewards of the RL algorithm. We compare the performance of the proposed method where directional and omnidirectional antennas are used. The results reveal that the use of directional antennas provides faster convergence rates than the omnidirectional antennas.