This paper discusses a vision based health monitoring system which would be very easy in use and deployment. Elder and sick people who are not able to talk or walk they are dependent on other human beings for their daily needs and need continuous monitoring. The developed system provides facility to the sick or elder person to describe his or her need to their caretaker in lingual description by showing particular hand gesture with the developed system. This system uses fingertip detection technique for gesture extraction and artificial neural network for gesture classification and recognition. The system is able to work in different light conditions and can be connected to different devices to announce users need on a distant location.