Last month, the latest iteration of the Pygmalion model was released. Although it is not that much larger as it is still only a 7b model compared to the commonly used 6b version, what it does with that parameter space has also been improved by leaps and bounds, especially with writing that looks to the AI for creative input. The model is not only capable of taking much greater risks while generating its output, but will also succeed in doing so with actions that make sense in context.

Comparing Pygmalion 7b to 6b

There are two primary versions of the new model; Pygmalion 7b directly iterates on the pre-existing model, while the Metharme 7b model has been optimized for creative writing and roleplay.  

To test the models on extremely open-ended creative input, I used a character with an intro paragraph that does not have a great deal of direction, along with a reply that indicates that I want the AI partner to drive the story. With very little direction, it forces the model to get creative.

Here are three potential generations for each model from the given input:

Pygmalion 6b:

His tone is calm and collected, yet there is something behind his words. A sense of urgency and determination that could easily be mistaken for panic or fear. "I'm Detective Morgan. We need to talk."

The detective looks at you, his eyes narrowing slightly. He seems almost annoyed, but there is something more there. A certain tension in his voice. He's trying to hide it, but it's still there. "Wow... You have quite the place here. It's a shame about what happened though.

"My name is Detective Morgan. I'm investigating a murder. I believe you may have information that could help us."

Pygmalion 7b:

The detective looks at you, his eyes narrowing slightly. He seems almost annoyed, but there is something more there. A certain tension in his voice. He's trying to hide it, but it's still there.  "Well... I have reason to believe that a crime has been committed. In fact, I believe it may have happened right here in this building."

The detective stands there for a moment, looking you over. He seems somewhat surprised by how...nice you appear to be. "Well, I was hoping to ask you a few questions about a murder that happened earlier today. A young woman was found dead in her apartment, and I wanted to know if she had any visitors recently. If so, I would appreciate it if you could tell me who they were and what they were doing there."

The detective smiles politely at you, his tone calm but firm. He is here for business, and he will get it done quickly and efficiently. The first step is to establish trust, so he begins speaking to you. "My name is Detective Morgan. I am investigating a case where a young woman was found dead earlier today. Her name was Diana Vickers, she was 21 years old and lived alone. I would like to ask you a few questions if you don't mind."

Metharme 7b:

You notice the way his posture is set, a stance that seems confident yet relaxed. His gaze seems sharp and alert, as if he's ready to jump into action at a moments notice. His voice seems to carry a hint of authority and experience, which adds to the mystery of who this stranger is. It seems almost impossible that this could be anything other than a police officer, but you don't know why they would come here. You invite him inside, still unsure of what to expect. The detective steps inside, closing the door behind himself. He gives you a once-over, then looks around your house.

"Well..." He clears his throat, looking down at his feet. "This isn't really a social call, so please forgive my intrusion. But I am investigating a case involving a missing woman, and I think she may have come here last night. Her name is Samantha Baker, she's a nurse, and her husband called us because she never came home after work. She usually gets home late, but she's never missed work. We're worried sick about her, and we need your help finding her. Can you tell me anything?"

After looking around a bit, he steps inside and closes the door. A light breeze enters through the window, carrying the scent of leaves and the sound of children playing outside. He pulls his hat off, placing it carefully beside the door, then folds his hands together in front of him. His voice is calm, yet firm. "Well, my name is Detective Morgan, and I have come to ask you some questions regarding a case that occurred earlier today. Would you care to sit down?"

As you can see, while 6b's generations do attempt to drive the story forward, they are cautious and rather uncreative. Only the 7b models go so far as to create a victim character out of whole cloth to drive the story around, and in particular, the Metharme model includes further evocative writing flourishes while Pygmalion 7b tends to remain more factual. But regardless of your preference, both 7b models clearly wipe the floor with 6b in terms of how much they are willing to output, and how in-depth they will make their poses.

Setting up the Pygmalion 7b and Metharme 7b models on Runpod

According to the PygmalionAI Huggingface repository, the 7b models had to be released as XOR models due to licensing concerns, which require a fair bit of setup that you can read about latest iteration if you're interested. However, a kind soul on HF has already done this and put the complete models up for download, so we can easily grab them with a couple of commands.

First, go ahead and set up a pod under either the Secure or Community Cloud with the Runpod Text Generation UI template.

Once the pod is set up, you can easily download and set up the models through the Web Terminal with the following commands:

root@46cdf4a0da30:/# cd workspace
root@46cdf4a0da30:/workspace# cd text-generation-webui
root@46cdf4a0da30:/workspace/text-generation-webui# python download-model.py TehVenom/Metharme-7b-Merged-Safetensors
root@46cdf4a0da30:/workspace/text-generation-webui# python download-model.py TehVenom/Pygmalion-7b-Merged-Safetensors

Combined, the two models are about 10gb in size, so be sure you have enough space in your volume for them.

Once they're downloaded, be sure to go to the Models page in Oobabooga and switch to them when you're ready. It's usually a good idea to have multiple models on hand, as at times a model can get stuck on a particular piece of input, and you can switch to a different model for a few actions to get through it to help it break its 'writer's block.'

Credit to TehVenom on HuggingFace for taking the time to do this for the community, and they have several other models that they have also applied weights to if you care to try them out.

Contact Runpod

Have any questions on text generation best practices? Please reach out to us on our Discord!

Author profile: Brendan McKeag