The materials of surfaces in a room play an important room in shaping the auditory experience within them. Different materials absorb energy at different levels. The level of absorption also varies across frequencies. This paper investigates how cues from a measured impulse response in the room can be exploited by machines to detect the materials present. With this motivation, this paper proposes a method for estimating the probability of presence of 10 material categories, based on their frequency-dependent absorption characteristics. The method is based on a CNN-RNN, trained as a multi-task classifier. The network is trained using a priori knowledge about the absorption characteristics of materials from the literature. In the experiments shown, the network is tested on over 5,00 impulse responses and 167 materials. The F1 score of the detections was 98%, with an even precision and recall. The method finds direct applications in architectural acoustics and in creating more parsimonious models for acoustic reflections.