As many blog entries in the past have been written on Oobabooga/text-generation-webui, we would be remiss if we failed to mention there was another much-loved frontend available for use on Runpod that may be of significant value to anyone interested in writing or roleplaying with an AI. KoboldAI comes with its own set of instructions, functions, and quirks, and let's take a fast look at it and show you how to get a new story up and running in no time at all.

Spinning up a KoboldAI Pod on Runpod

Starting up a pod is as easy as ever. Under the Community templates section, find the KoboldAI template and click Deploy, and within a few minutes you're up and running.

Once you load up the pod, if you've used Oobabooga in the past, you may find that the KoboldAI UI is a bit busier. No worries, we will get through it!

Installing a model

One of the nice things about KoboldAI is that rather than having to download files from Huggingface directly, you have a list of models that you can click and install easily. You'll need to get a model loaded to use the front end, of course, and there are a few options to choose from:

Adventure models are models that simulate the look and feel of old MUDs or Infocom style adventure games. Interacting with these models generally involves the model giving you a block of text to work with, and then you declare an action, after which the model will then continue the story based on your action.

Novel models, on the other hand, are more directed at writing third-person prose, as if you were writing a novel cooperatively with another individual; where the AI will supply a section, then you supply a section, and then the AI does so, and so on.

NSFW models.. well, that's out of the context of this blog, and you can use your imagination for those. 🥴

Using the Memory Function

Much like Oobabooga,  KoboldAI has a Memory pad where you can give the bot persistent context to keep in mind throughout the scene. This is very similar to the Character tab within Oobabooga, in that you can provide things that the AI will keep in mind with every prompt, and you can add and edit this dynamically as the scene progresses to keep it apprised of any new pertinent info that it needs to know.

Advantages and Disadvantages of KoboldAI over Oobabooga

There's a few compelling reasons that you may prefer KoboldAI, depending on what your use case is.

Context Matters

Like all text generation models, KoboldAI has a token context limit (2048, the same as Oobabooga before their most recent update.) How these two front ends use that context allotment differs, though. After considering all of the persistent fields on the Character tab, Ooba will then start with the most recent log entry, and then travel backwards to incorporate as much text as it can until it fills up its buffer, at which point it essentially forgets everything prior. This can manifest itself in rather silly ways, like when the model seems to forget what a character is wearing even though you told it explicitly a few pages back. So, it essentially remembers everything, for awhile, until it then remembers nothing.  

KoboldAI, on the other hand, uses "smart context" in which it will search the entire text buffer for things that it believes are related to your recently entered text. Although it has its own room for improvement, it's generally more likely to be able to search and find details in what you've written so far. (If you've ever used the long-term-memory extension in Oobabooga, I believe it behaves somewhat akin to this.)

More freedom - and responsibility - to edit output

Unlike Oobabooga, which only allows you to edit the most recent output (and takes more clicks to do it), KoboldAI allows you to edit any part of the output at any time, as if you were sharing a word processor with another person. Because it relies more on the whole of your log than Ooba does, it requires you to routinely edit whatever it outputs until it is exactly what you want it to be. This makes it a bit more involved than Ooba, which you can generally treat more like a roleplay partner with their own sense of agency. KoboldAI expects a bit more handholding, but also gives you more power to do it, with the knowledge that it will also be able to incorporate more of your past history in future outputs.  

Output length

Based on my experimentation, models such as Pygmalion tend to be much more verbose in Oobabooga than in KoboldAI. I tended to get paragraphs and paragraphs with some of the more expressive models, whereas I never got quite so detailed responses within KoboldAI. However, balanced against that is the fact that the long responses in Oobabooga tended to be very overloaded and repetitious, while KoboldAI's tended to be more succinct and to the point. If you want output that's more "punchy" for lack of a better word, then KoboldAI may be your thing.

Conclusion

So which one is right for you?

Having used both, they do different things in ways that are easy and not so easy to describe. But the primary use case between  KoboldAI and Oobabooga is how long you intend to use it for a particular scene.

The big problem I've noticed with Oobabooga is that due to the always on/off nature of its context window, it can be very hard to keep an AI-driven character consistent over a long period of time, since its personality will be largely driven only by its most recent actions. KoboldAI seems to be a lot better at retaining details over a long period of time.

On the other hand, for short scenes, Oobabooga is really good at working within that short period of time. But it does feel like a lot of my interactions and stories within Oobabooga have a very noticeable shelf life, with a large dropoff in enjoyment after it "goes off" for lack of a better term.

So my eventual suggestion would be to try both, with the knowledge that if you plan on writing much longer-term works, the decision shifts towards KoboldAI the longer your story goes – since at least for my taste, Ooba gets extremely immersion breaking when it there's so little long-term stability over how it manages the character.

Have any questions? Feel free to pop into our Discord and ask!

‍

Author profile: Brendan McKeag