Full-duplex transmission comprises the ability to transmit and receive at the same time on the same frequency band. It allows for more efficient utilization of spectral resources, but raises the challenge of strong self-interference (SI). Cancellation of SI is generally implemented as a multi-stage approach. This work proposes a novel adaptive SI cancellation algorithm in the digital domain and a comprehensive analysis of state-of-the-art adaptive cancellation techniques. Inspired by recent progress in acoustic echo control, we introduce a composite state-space model of the nonlinear SI channel in cascade structure. We derive a SI cancellation algorithm that decouples the identification of linear and nonlinear elements of the composite state. They are estimated separately and consecutively in each adaptation cycle by a Kalman filter in DFT domain. We show that this adaptation can be supported by a-priori signal orthogonalization and decoding of the signal-of-interest (SoI). In our simulation results, we analyze the performance by evaluating residual interference, system identification accuracy and communication rate. Based on the results, we provide recommendations for system design. In case of input orthogonalization, our Kalman filter solution in cascade structure delivers best performance with low computational complexity. In this configuration, the performance lines up with that of the monolithic (parallel) Kalman filter or the recursive-least squares (RLS) algorithms. We show that the Kalman-based algorithm is superior over the RLS under time-variant conditions if the SoI is decoded and in this way the covariance information required by the Kalman filter can be provided to it.