In the hidden Markov process, there is a possibility that two different transition matrices for hidden and observed variables yield the same stochastic behavior for the observed variables. Since such two transition matrices cannot be distinguished, we need to identify them and consider that they are equivalent, in practice. We address the equivalence problem of hidden Markov process in a local neighborhood by using the geometrical structure of hidden Markov process. For this aim, we introduce a mathematical concept to express Markov process, and formulate its exponential family by using generators. Then, the above equivalence problem is formulated as the equivalence problem of generators. Taking this equivalence problem into account, we derive several concrete parametrizations in several natural cases.