Recent studies in different fields of science caused emergence of needs for high performance computing systems like Cloud. A critical issue in design and implementation of such systems is resource allocation which is directly affected by internal and external factors like the number of nodes, geographical distance and communication latencies. Many optimizations took place in resource allocation methods in order to achieve better performance by concentrating on computing, network and energy resources. Communication latencies as a limitation of network resources have always been playing an important role in parallel processing (especially in fine-grained programs). In this paper, we are going to have a survey on the resource allocation issue in Cloud and then do an optimization on common resource allocation method based on the latencies of communications. Due to it, we added a table to Resource Agent (entity that allocates resources to the applicants) to hold the history of previous allocations. Then, a probability matrix was constructed for allocation of resources partially based on the history of latencies. Response time was considered as a metric for evaluation of proposed method. Results indicated the better response time, especially by increasing the number of tasks. Besides, the proposed method is inherently capable for detecting the unavailable resources through measuring the communication latencies. It assists other issues in cloud systems like migration, resource replication and fault tolerance.