We prove a combinatorial formula for the alternating descent polynomials of type A and B. Combining with Josuat-Vergès' combinatorial interpretation for Hoffman's derivative polynomials for tangent and secant functions, we obtain a unified combinatorial proof of Ma-Yeh's two formulae linking these two families of polynomials.