We consider an Internet-of-Things (IoT) system in which an energy harvesting powered sensor node monitors the phenomenon of interest and transmits its observations to a remote monitor over a Gaussian channel. We measure the timeliness of the signals recovered by the monitor using age of information (AoI), which could be reduced by transmitting more observations to the monitor. We evaluate the corresponding distortion with the mean-squared error (MSE) metric, which would be reduced if a larger transmit power and a larger source coding rate were used. Since the energy harvested by the sensor node is random and limited, however, the timeliness and the distortion of the received signals cannot be optimized at the same time. Thus, we shall investigate the timeliness-distortion trade-off of the system by minimizing the average weighted-sum AoI and distortion over all possible transmit powers and transmission intervals. First, we explicitly present the optimal transmit powers for the performance limit achieving save-and-transmit policy and the easy-implementing fixed power transmission policy. Second, we propose a backward water-filling based offline power allocation algorithm and a genetic based offline algorithm to jointly optimize the transmission interval and transmit power. Third, we formulate the online power control as an Markov Decision Process (MDP) and solve the problem with an iterative algorithm, which closely approach the trade-off limit of the system. Also, we show that the optimal transmit power is a monotonic and bi-valued function of current AoI and distortion. Finally, we present our results via numerical simulations and extend results on the save-and-transmit policy to fading sensing systems.