Gaze detection and head orientation are an important part of many advanced human-machine interaction applications. Many systems have been proposed for gaze detection. Typically, they require some form of user cooperation and calibration. Additionally, they may require multiple cameras and/or restricted head positions. We present a new approach for inference of both face orientation and gaze direction from a single image with no restrictions on the head position. Our algorithm is based on a face and eye model, deduced from anthropometric data. This approach allows us to use a single camera and requires no cooperation from the user. Using a single image avoids the complexities associated with of a multi-camera system. Evaluation tests show that our system is accurate, fast and can be used in a variety of applications, including ones where the user is unaware of the system.