In this article, for the finite field $\mathbb{F}_q$, we show that the $\mathbb{F}_q$-algebra $\mathbb{F}_q[x]/\langle f(x) \rangle$ is isomorphic to the product ring $\mathbb{F}_q^{°f(x)}$ if and only if $f(x)$ splits over $\mathbb{F}_q$ into distinct factors. We generalize this result to the quotient of the polynomial algebra $\mathbb{F}_q[x_1, x_2,\dots, x_k]$ by the ideal $\langle f_1(x_1), f_2(x_2),\dots, f_k(x_k)\rangle.$ On the other hand, we establish that every finite-dimensional $\mathbb{F}_q$-algebra $\mathcal{S}$ has an orthogonal basis of idempotents with their sum equal to $1_{\mathcal{S}}$ if and only if $\mathcal{S}\cong\mathbb{F}_q^l$ as $\mathbb{F}_q$-algebras, where $l=\dim_{\mathbb{F}_q} \mathcal{S}$. Instead of studying polycyclic codes over $\mathbb{F}_q$-algebras $\mathbb{F}_q[x_1, x_2,\dots, x_k]/\langle f_1(x_1), f_2(x_2),\dots, f_k(x_k)\rangle$ where $f_i(x_i)$ splits into distinct linear factors over $\mathbb{F}_q,$ which is a subclass of $\mathbb{F}_q^l,$ we study polycyclic codes over $\mathbb{F}_q^l$ and obtain their unique decomposition into polycyclic codes over $\mathbb{F}_q$ for every such orthogonal basis of $\mathbb{F}_q^l$. We refer to it as an $\mathbb{F}_q$-decomposition. An $\mathbb{F}_q$-decomposition enables us to use results of polycyclic codes over $\mathbb{F}_q$ to study polycyclic codes over $\mathbb{F}_q^l$; for instance, we show that the annihilator dual of a polycyclic code over $\mathbb{F}_q^l$ is a polycyclic code over $\mathbb{F}_q^l$. Furthermore, with the help of different Gray maps, we produce a good number of examples of MDS or almost-MDS or/and optimal codes; some of them are LCD over $\mathbb{F}_q$. Finally, we study Gray maps from $(\mathbb{F}_q^l)^n$ to $\mathbb{F}_q^{nl},$ and use it to construct quantum codes with the help of CSS construction.