In this work, we develop low complexity, optimal power allocation algorithms that would allow ultra reliable operation at any outage probability target with minimum power consumption in the finite blocklength regime by utilizing Karush-Kuhn-Tucker (KKT) conditions. In our setup, we assume that the transmitter does not know the channel state information (CSI). First, we show that achieving a very low packet outage probability by using an open loop setup requires extremely high power consumption. Thus, we resort to retransmission schemes as a solution, namely Automatic Repeat Request (ARQ), Chase Combining Hybrid ARQ (CC-HARQ) and Incremental Redundancy (IR) HARQ. Countrary to classical approaches, where it is optimal to allocate equal power with each transmission, we show that for operation in the ultra reliable regime (URR), the optimal strategy suggests transmission with incremental power in each round. Numerically, we evaluate the power gains of the proposed protocol. We show that the best power saving is given by IR-HARQ protocol. Further, we show that when compared to the one shot transmission, these protocols enable large average and maximum power gains. Finally, we show that the larger the number of transmissions is, the larger power gains will be attained.