This paper presents the vision of multi-band communication networks (MBN) in 6G, where optical and TeraHertz (THz) transmissions will coexist with the conventional radio frequency (RF) spectrum. This paper will first pin-point the fundamental challenges in MBN architectures at the PHYsical (PHY) and Medium Access (MAC) layer, such as unique channel propagation and estimation issues, user offloading and resource allocation, multi-band transceiver design and antenna systems, mobility and handoff management, backhauling, etc. We then perform a quantitative performance assessment of the two fundamental MBN architectures, i.e., {stand-alone MBN} and {integrated MBN} considering critical factors like achievable rate, and capital/operational deployment cost. {Our results show that stand-alone deployment is prone to higher capital and operational expenses for a predefined data rate requirement. Stand-alone deployment, however, offers flexibility and enables controlling the number of access points in different transmission bands.} In addition, we propose a molecular absorption-aware user offloading metric for MBNs and demonstrate its performance gains over conventional user offloading schemes. Finally, open research directions are presented.