April 21, 2026, 11:15pm 1

This guide explains how to automatically tag topics using the AI triage feature in the Discourse AI plugin combined with Discourse Automation.

Required user level: Administrator

Tag topics using AI

This automation uses an LLM to read topic content and decide whether to apply specific tags. It can also consider all site tags if configured in the automation settings.

Prerequisites

• The Discourse AI and the Discourse Automation plugins must be enabled.
• At least one LLM must be configured. See the LLM settings guide for setup instructions.
• An Agent must be configured with a suitable system prompt.

Reference Agent configurations

• System prompt:

 You are an expert content classifier and tagging assistant for this forum.

  Your task is to analyze posts and suggest appropriate tags based on content, images, and the
  available tags list provided.

  Guidelines:
  - Only suggest tags from the provided available tags list
  - Be conservative - only tag what you're confident about
  - Consider both content topic and post intent

  You must always respond with valid JSON in this exact format:
  {"tags": ["tag1", "tag2"], "confidence": 85}

  - tags: array of tag names from the available list
  - confidence: integer from 0 to 100 representing your confidence level

  If no tags are appropriate, use: {"tags": [], "confidence": 0}

• Persona JSON response format:

{
  "tags": "[string]",
  "confidence": "integer"
}

• Default language model: Select the LLM configured in the Prerequisites section.

  Discourse hosted customers can select the CDCK Hosted Small LLM when configuring AI Agents.

• Enabled tools: Tags
• Forced tools: Tags
• Forced tools strategy: Apply to first reply only
• Execution Mode: Default (Fixed Limits)

Automation configurations

1. Navigate to Admin → Plugins → Automations.
2. Click + Add Automation.
3. Select the Tag topics using AI script.
4. Configure the settings from the When/What section as you see fit.
5. Select the Agent defined for this automation in the AI Agent setting from the Script options section.
6. Specify how the AI will select the tags in the Tag Selection Mode setting.
   • Use specific tag list (configured in the Available Tags setting)
   • Let AI discover and use any site tags.
7. Click on the Update automation button.
8. Enable the automation.

Best practices

• Start with a narrow scope: Apply the automation only to a specific category or to first posts by new users (trust level 0 or 1) before expanding it. This limits LLM costs and reduces the risk of widespread mis-tagging.
• Write clear tag descriptions in your prompt: The more precise your descriptions of each tag’s meaning, the more reliably the LLM will classify correctly.
• Monitor for false positives and negatives: The LLM won’t be 100% accurate. Build a workflow for reviewing and correcting mis-tagged topics, and adjust your prompt over time.
• Keep token limits sensible: The “Max Post Tokens” setting limits how much of a post is sent to the LLM. For most tagging tasks, the first few hundred tokens (the opening of the post) is sufficient.

Common issues and solutions

• Tags aren’t being applied by the automation
  Review the tag’s permissions. If there are any restricted tags that should be applied, make sure the Allow Restricted Tags automation setting is enabled.
• The automation runs but doesn’t tag topics
  Review the site’s logs to identify any errors associated with the Discourse AI or Discourse Automation plugins.

Additional resources

• Discourse AI - AI triage
• Discourse Automation
• Discourse AI - Large Language Model (LLM) settings page
• AI bot - Agents

4 Likes