The paper presents a binarization scheme that converts non-binary data into a set of binary strings. At present, there are many binarization algorithms, but they are optimal for only specific probability distributions of the data source. Overcoming the problem, it is shown in this paper that the presented binarization scheme conserves the entropy of the original data having any probability distribution of $m$-ary source. The major advantages of this scheme are that it conserves entropy without the knowledge of the source and the probability distribution of the source symbols. The scheme has linear complexity in terms of the length of the input data. The binarization scheme can be implemented in Context-based Adaptive Binary Arithmetic Coding (CABAC) for video and image compression. It can also be utilized by various universal data compression algorithms that have high complexity in compressing non-binary data, and by binary data compression algorithms to optimally compress non-binary data.