In this paper we study data from financial markets using an information-theory tool that we call the normalised Mutual Information Rate and show how to use it to infer the underlying network structure of interrelations in foreign currency exchange rates and stock indices of 14 countries world-wide and the European Union. We first present the mathematical method and discuss about its computational aspects, and then apply it to artificial data from chaotic dynamics and to correlated random variates. Next, we apply the method to infer the network structure of the financial data. Particularly, we study and reveal the interrelations among the various foreign currency exchange rates and stock indices in two separate networks for which we also perform an analysis to identify their structural properties. Our results show that both are small-world networks sharing similar properties but also having distinct differences in terms of assortativity. Finally, the consistent relationships depicted among the 15 economies are further supported by a discussion from the economics view point.