The Internet of Medical Things (IoMT) paradigm is becoming mainstream in multiple clinical trials and healthcare procedures. Cardiovascular diseases monitoring, usually involving electrocardiogram (ECG) traces analysis, is one of the most promising and high-impact applications. Nevertheless, to fully exploit the potential of IoMT in this domain, some steps forward are needed. First, the edge-computing paradigm must be added to the picture. A certain level of near-sensor processing has to be enabled, to improve the scalability, portability, reliability, responsiveness of the IoMT nodes. Second, novel, increasingly accurate, data analysis algorithms, such as those based on artificial intelligence and Deep Learning, must be exploited. To reach these objectives, designers and programmers of IoMT nodes, have to face challenging optimization tasks, in order to execute fairly complex computing tasks on low-power wearable and portable processing systems, with tight power and battery lifetime budgets. In this work, we explore the implementation of a cognitive data analysis algorithm, based on a convolutional neural network trained to classify ECG waveforms, on a resource-constrained microcontroller-based computing platform. To minimize power consumption, we add an adaptivity layer that dynamically manages the hardware and software configuration of the device to adapt it at runtime to the required operating mode. Our experimental results show that adapting the node setup to the workload at runtime can save up to 50% power consumption. Our optimized and quantized neural network reaches an accuracy value higher than 97% for arrhythmia disorders detection on MIT-BIH Arrhythmia dataset.