The novel concept of simultaneously transmitting and reflecting (STAR) reconfigurable intelligent surfaces (RISs) is investigated, where the incident wireless signal is divided into transmitted and reflected signals passing into both sides of the space surrounding the surface, thus facilitating a full-space manipulation of signal propagation. Based on the introduced basic signal model of `STAR', three practical operating protocols for STAR-RISs are proposed, namely energy splitting (ES), mode switching (MS), and time switching (TS). Moreover, a STAR-RIS aided downlink communication system is considered for both unicast and multicast transmission, where a multi-antenna base station (BS) sends information to two users, i.e., one on each side of the STAR-RIS. A power consumption minimization problem for the joint optimization of the active beamforming at the BS and the passive transmission and reflection beamforming at the STAR-RIS is formulated for each of the proposed operating protocols, subject to communication rate constraints of the users. For ES, the resulting highly-coupled non-convex optimization problem is solved by an iterative algorithm, which exploits the penalty method and successive convex approximation. Then, the proposed penalty-based iterative algorithm is extended to solve the mixed-integer non-convex optimization problem for MS. For TS, the optimization problem is decomposed into two subproblems, which can be consecutively solved using state-of-the-art algorithms and convex optimization techniques. Finally, our numerical results reveal that: 1) the TS and ES operating protocols are generally preferable for unicast and multicast transmission, respectively; and 2) the required power consumption for both scenarios is significantly reduced by employing the proposed STAR-RIS instead of conventional reflecting/transmiting-only RISs.