A point-to-point integrated sensing and communication (ISAC) system is considered where a transmitter conveys a message to a receiver over a discrete memoryless channel (DMC) and simultaneously estimates the state of the channel through the backscattered signals of the emitted waveform. We derive achievability and converse bounds on the rate-distortion-error tradeoff in the finite blocklength regime, and also characterize the second-order rate-distortion-error region for the proposed setup. Numerical analysis shows that our proposed joint ISAC scheme significantly outperforms traditional time-sharing based schemes where the available resources are split between the sensing and communication tasks.