The TV set feedback feature standardized in the next generation TV system, ATSC 3.0, would enable opportunistic access of active TV channels in future Cognitive Radio Networks. This new dynamic spectrum access approach is named as black-space access, as it is complementary of current TV white space, which stands for inactive TV channels. TV black-space access can significantly increase the available spectrum of Cognitive Radio Networks in populated urban markets, where spectrum shortage is most severe while TV whitespace is very limited. However, to enable TV black-space access, secondary user has to evacuate a TV channel in a timely manner when TV user comes in. Such strict real-time constraint is an unique challenge of spectrum management infrastructure of Cognitive Radio Networks. In this paper, the real-time performance of spectrum management with regard to the degree of centralization of infrastructure is modeled and tested. Based on collected empirical network latency and database response time, we analyze the average evacuation time under four structures of spectrum management infrastructure: fully distribution, city-wide centralization, national-wide centralization, and semi-national centralization. The results show that national wide centralization may not meet the real-time requirement, while semi-national centralization that use multiple co-located independent spectrum manager can achieve real-time performance while keep most of the operational advantage of fully centralized structure.