This paper is motivated by the observation that the average queueing delay can be decreased by sacrificing power efficiency in wireless communications. In this sense, we naturally wonder what is the minimum queueing delay when the available power is limited and how to achieve the minimum queueing delay. To answer these two questions in the scenario where randomly arriving packets are transmitted over multi-state wireless fading channel, a probabilistic cross-layer scheduling policy is proposed in this paper, and characterized by a constrained Markov Decision Process (MDP). Using the steady-state probability of the underlying Markov chain, we are able to derive the mathematical expressions of the concerned metrics, namely, the average queueing delay and the average power consumption. To describe the delay-power tradeoff, we formulate a non-linear programming problem, which, however, is very challenging to solve. By analyzing its structure, this optimization problem can be converted into an equivalent Linear Programming (LP) problem via variable substitution, which allows us to derive the optimal delay-power tradeoff as well as the optimal scheduling policy. The optimal scheduling policy turns out to be dual-threshold-based, which means transmission decisions should be made based on the optimal thresholds imposed on the queue length and the channel state.