In this paper, we study the dual of generalized bent functions $f: V_{n}\rightarrow \mathbb{Z}_{p^k}$ where $V_{n}$ is an $n$-dimensional vector space over $\mathbb{F}_{p}$ and $p$ is an odd prime, $k$ is a positive integer. It is known that weakly regular generalized bent functions always appear in pairs since the dual of a weakly regular generalized bent function is also a weakly regular generalized bent function. The dual of non-weakly regular generalized bent functions can be generalized bent or not generalized bent. By generalizing the construction of \cite{Cesmelioglu5}, we obtain an explicit construction of generalized bent functions for which the dual can be generalized bent or not generalized bent. We show that the generalized indirect sum construction method given in \cite{Wang} can provide a secondary construction of generalized bent functions for which the dual can be generalized bent or not generalized bent. By using the knowledge on ideal decomposition in cyclotomic field, we prove that $f^{**}(x)=f(-x)$ if $f$ is a generalized bent function and its dual $f^{*}$ is also a generalized bent function. For any non-weakly regular generalized bent function $f$ which satisfies that $f(x)=f(-x)$ and its dual $f^{*}$ is generalized bent, we give a property and as a consequence, we prove that there is no self-dual generalized bent function $f: V_{n}\rightarrow \mathbb{Z}_{p^k}$ if $p\equiv 3 \ (mod \ 4)$ and $n$ is odd. For $p \equiv 1 \ (mod \ 4)$ or $p\equiv 3 \ (mod \ 4)$ and $n$ is even, we give a secondary construction of self-dual generalized bent functions. In the end, we characterize the relations between the generalized bentness of the dual of generalized bent functions and the bentness of the dual of bent functions, as well as the self-duality relations between generalized bent functions and bent functions by the decomposition of generalized bent functions.