The ever-increasing number of Internet of Things (IoT) devices has created a new computing paradigm, called edge computing, where most of the computations are performed at the edge devices, rather than on centralized servers. An edge device is an electronic device that provides connections to service providers and other edge devices; typically, such devices have limited resources. Since edge devices are resource-constrained, the task of launching algorithms, methods, and applications onto edge devices is considered to be a significant challenge. In this paper, we discuss one of the most widely used machine learning methods, namely, Deep Learning (DL) and offer a short survey on the recent approaches used to map DL onto the edge computing paradigm. We also provide relevant discussions about selected applications that would greatly benefit from DL at the edge.