Recent developments in machine learning (ML) techniques enable users to extract, transmit, and reproduce information semantics via ML-based semantic communication (SemCom). This significantly increases network spectral efficiency and transmission robustness. In the network, the semantic encoders and decoders among various users, based on ML, however, require collaborative updating according to new transmission tasks. The various heterogeneous characteristics of most networks in turn introduce emerging but unique challenges for semantic codec updating that are different from other general ML model updating. In this article, we first overview the key components of the SemCom system. We then discuss the unique challenges associated with semantic codec updates in heterogeneous networks. Accordingly, we point out a potential framework and discuss the pros and cons thereof. Finally, several future research directions are also discussed.