Let $k \in \mathbb{N}$ and let $H_1, H_2, \ldots, H_k$ be simple graphs such that for each $j \in \{ 1, 2, \ldots, k \}$, the vertex set of $H_j$ is $\{ 0, 1, 2, \ldots, n_j - 1 \}$ for some $n_j \in \mathbb{N}$. The ordered Ramsey number $R_\mathrm{ord}(H_1, H_2, \ldots, H_k)$ is the smallest $n \in \mathbb{N}$ for which every $k$-edge-coloring of the complete graph on the vertex set $\{ 0, 1, 2, \ldots, n - 1 \}$ contains $H_j$ as a monochromatic subgraph of color $j$ for some $j \in \{ 1, 2, \ldots, k \}$, with the vertices appearing in the same order as in $H_j$. Inspired by the work of Poljak, we apply the Kissat SAT solver to determine new small two-color ordered Ramsey numbers of various classes of graphs: monotone paths, monotone cycles, alternating paths, stars, complete graphs and nested matchings. In addition, we introduce the cyclic Ramsey numbers $R_\mathrm{cyc}(H_1, H_2, \ldots, H_k)$ as a natural relaxation of the ordered Ramsey numbers, and once again use Kissat to determine various such numbers for the two-color case. By observing structural patterns in the computational results, we determine all ordered or cyclic Ramsey numbers for several pairs of classes of graphs. Furthermore, we obtain some bounds on ordered and cyclic Ramsey numbers where one argument is a connected graph, while the other is a monotone path or a monotone cycle. We also explore how reinforcement learning can be used through the recently developed Reinforcement Learning for Graph Theory (RLGT) framework to obtain lower bounds on ordered and cyclic Ramsey numbers. Finally, we introduce the permutational Ramsey numbers to show how the different Ramsey-type formulations involving standard, ordered and cyclic Ramsey numbers can be unified within a group-theoretic framework.