Voice control is a popular way to operate mobile devices, enabling users to communicate requests to their devices. However, adversaries can leverage voice control to trick mobile devices into executing commands to leak secrets or to modify critical information. Contemporary mobile operating systems fail to prevent such attacks because they do not control access to the speaker at all and fail to control when untrusted apps may use the microphone, enabling authorized apps to create exploitable communication channels. In this paper, we propose a security mechanism that tracks the creation of audio communication channels explicitly and controls the information flows over these channels to prevent several types of attacks.We design and implement AuDroid, an extension to the SELinux reference monitor integrated into the Android operating system for enforcing lattice security policies over the dynamically changing use of system audio resources. To enhance flexibility, when information flow errors are detected, the device owner, system apps and services are given the opportunity to resolve information flow errors using known methods, enabling AuDroid to run many configurations safely. We evaluate our approach on 17 widely-used apps that make extensive use of the microphone and speaker, finding that AuDroid prevents six types of attack scenarios on audio channels while permitting all 17 apps to run effectively. AuDroid shows that it is possible to prevent attacks using audio channels without compromising functionality or introducing significant performance overhead.