Device-to-Device (D2D) communication has been recognized as a promising technique to offload the traffic for the evolved Node B (eNB). However, the D2D transmission as an underlay causes severe interference to both the cellular and other D2D links, which imposes a great technical challenge to radio resource allocation. Conventional graph based resource allocation methods typically consider the interference between two user equipments (UEs), but they cannot model the interference from multiple UEs to completely characterize the interference. In this paper, we study channel allocation using hypergraph theory to coordinate the interference between D2D pairs and cellular UEs, where an arbitrary number of D2D pairs are allowed to share the uplink channels with the cellular UEs. Hypergraph coloring is used to model the cumulative interference from multiple D2D pairs, and thus, eliminate the mutual interference. Simulation results show that the system capacity is significantly improved using the proposed hypergraph method in comparison to the conventional graph based one.