In blockchain-based order book systems, buyers and sellers trade assets, while it is miners to match them and include their transactions in the blockchain. It is found that many miners behave selfishly and myopically, prioritizing transactions with high fees and ignoring many desirable matches that could enhance social welfare. Existing blockchain mechanisms fail to address this issue by overlooking miners' selfish behaviors. To our best knowledge, this work presents the first analytical study to quantify and understand buyer and seller transaction fee choices and selfish miners' transaction matching strategies, proving an infinitely large price of anarchy (PoA) for social welfare loss. To mitigate this, we propose an adjustable block size mechanism that is easy to implement without altering the existing decentralized protocols and still allows buyers and sellers to freely decide transaction fees and miners to selfishly match. The analysis is challenging, as pure strategy Nash equilibria do not always exist, requiring the analysis of many buyers' or sellers' interactive mixed-strategy distributions. Moreover, the system designer may even lack information about each buyer's or seller's bid/ask prices and trading quantities. Nevertheless, our mechanism achieves a well-bounded PoA, and under the homogeneous-quantity trading for non-fungible tokens (NFT), it attains a PoA of 1 with no social welfare loss. We implement our mechanism on a local instance of Ethereum to demonstrate the feasibility of our approach. Experiments based on the realistic dataset demonstrate that our mechanism achieves social optimum for homogeneous-quantity trading like NFT. It can enhance social welfare up to 3.7 times compared to the existing order book benchmarks for heterogeneous-quantity trading of Bitcoin tokens. It exhibits robustness against random variations in buyers and sellers.