惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

S
Security Affairs
S
Schneier on Security
N
News | PayPal Newsroom
T
Threatpost
Cloudbric
Cloudbric
H
Heimdal Security Blog
Recent Commits to openclaw:main
Recent Commits to openclaw:main
Google Online Security Blog
Google Online Security Blog
D
Darknet – Hacking Tools, Hacker News & Cyber Security
Spread Privacy
Spread Privacy
V
Vulnerabilities – Threatpost
The Last Watchdog
The Last Watchdog
CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
L
LINUX DO - 最新话题
P
Proofpoint News Feed
C
CXSECURITY Database RSS Feed - CXSecurity.com
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
Apple Machine Learning Research
Apple Machine Learning Research
NISL@THU
NISL@THU
Application and Cybersecurity Blog
Application and Cybersecurity Blog
The Hacker News
The Hacker News
O
OpenAI News
人人都是产品经理
人人都是产品经理
C
Cyber Attacks, Cyber Crime and Cyber Security
C
Check Point Blog
C
Cisco Blogs
GbyAI
GbyAI
J
Java Code Geeks
L
LangChain Blog
I
Intezer
T
Tailwind CSS Blog
有赞技术团队
有赞技术团队
MyScale Blog
MyScale Blog
美团技术团队
The Register - Security
The Register - Security
Help Net Security
Help Net Security
WordPress大学
WordPress大学
Y
Y Combinator Blog
T
Tor Project blog
M
MIT News - Artificial intelligence
爱范儿
爱范儿
TaoSecurity Blog
TaoSecurity Blog
V
Visual Studio Blog
T
Threat Research - Cisco Blogs
P
Palo Alto Networks Blog
月光博客
月光博客
T
Tenable Blog
S
Securelist
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
D
DataBreaches.Net

Recent Commits to openclaw:main

test: merge chat side-result checks · openclaw/openclaw@ddd2c2a test: merge cron history checks · openclaw/openclaw@f7eb746 test: merge responsive navigation shell checks · openclaw/openclaw@c2e4b47 docs(changelog): add codex oauth fixes · openclaw/openclaw@628e6cd test: merge navigation routing cases · openclaw/openclaw@5d8cecb Tests: mock channel registry bundled fallback · openclaw/openclaw@2b08233 Secrets: avoid broad web search discovery for single plugin config · openclaw/openclaw@a464f59 test: merge config view browser checks · openclaw/openclaw@20cf511 fix(status): align oauth health with runtime · openclaw/openclaw@eed7116 feat: add macOS screen snapshots for monitor preview (#67954) thanks … · openclaw/openclaw@f377db1 fix: report shared auth scopes in hello-ok (#67810) thanks @BunsDev · openclaw/openclaw@0b6c39b Auto-reply: avoid eager bundled route fallback · openclaw/openclaw@3ea1bf4 Tests: narrow session binding contract setup · openclaw/openclaw@54e4e16 fix(macOS): enable undo/redo in webchat composer text input (#34962) · openclaw/openclaw@00951dc Tests: speed up channel setup promotion · openclaw/openclaw@82b529a Docs: refresh agent instructions · openclaw/openclaw@5775fe2 fix(auth): serialize OAuth refresh across agents to fix #26322 (#67876) · openclaw/openclaw@8e79080 test: allow ollama public surface boundary test · openclaw/openclaw@7d4f1a6 Docs: add test performance guardrails · openclaw/openclaw@89706d3 Tests: restore context-engine usage proof · openclaw/openclaw@e4c4f95 Tests: slim context engine runtime coverage · openclaw/openclaw@74c198f ci: retry failed custom checkouts · openclaw/openclaw@0ee5baf test: trim duplicate provider auth onboarding cases · openclaw/openclaw@1ffc02e matrix: fix sessions_spawn --thread subagent session spawning (#67643) · openclaw/openclaw@1ce2596 test: reduce auth choice fixture churn · openclaw/openclaw@857b9cd test: mock health status config boundaries · openclaw/openclaw@9d5ab4a test: mock onboard config io boundary · openclaw/openclaw@299694d test: mock legacy state plugin boundaries · openclaw/openclaw@2713089 test: mock channel install boundaries · openclaw/openclaw@b945248 test: mock doctor preview channel boundaries · openclaw/openclaw@b1a3ad4 test: trim doctor command hotspots · openclaw/openclaw@c66f16a test: isolate agent auth and spawn hotspots · openclaw/openclaw@9285935 test: stabilize MCP startup disposal race · openclaw/openclaw@dd9d2eb test: merge browser contract server suites · openclaw/openclaw@5817a76 test: narrow ollama provider discovery setup · openclaw/openclaw@a0d9598 build: declare qa-lab aimock runtime dependency · openclaw/openclaw@24431e5 test: speed up safe-bins exec harness · openclaw/openclaw@ee856ab test: preserve tool helpers in embedded runner mocks · openclaw/openclaw@acd86a0 refactor: move memory embeddings into provider plugins · openclaw/openclaw@77e6e4c test: reuse system-run temp fixtures · openclaw/openclaw@7e9ff0f test: trim hotspot wait overhead · openclaw/openclaw@12a59b0 Check: avoid duplicate boundary prep · openclaw/openclaw@baf11b8 test: reduce hotspot fixture overhead · openclaw/openclaw@3a59edd feat(ui): overhaul settings and slash command UX (#67819) thanks @Bun… · openclaw/openclaw@2cfb660 QA Matrix: exit cleanly on failure · openclaw/openclaw@42805d2 QA Matrix: isolate scenario coverage · openclaw/openclaw@7e659e1 Matrix: refresh crypto bootstrap state · openclaw/openclaw@94081d8 QA Lab: add provider registry · openclaw/openclaw@bb7e982 Matrix: add plugin changelog · openclaw/openclaw@4acab55 test: trim more hotspot overhead · openclaw/openclaw@f485311 test: trim remaining hotspot tests · openclaw/openclaw@6ba8626 test: narrow hotspot mocks · openclaw/openclaw@dbc8179 test: isolate gemini embedding request helpers · openclaw/openclaw@cd330f5 test: trim memory and mcp hotspots · openclaw/openclaw@fd48dfa test: slim provider registry mocks · openclaw/openclaw@2e08c77 test: harden Parallels update smoke · openclaw/openclaw@1a98090 feat: default Anthropic to Opus 4.7 · openclaw/openclaw@628b454 fix: harden node-host shell payload mutability checks · openclaw/openclaw@75c551e fix: land node-host approval binding for native binaries (#66731) (th… · openclaw/openclaw@29919bb CI: add daily schedule to CodeQL workflow (#67645) · openclaw/openclaw@69d25f5 fix(gateway): capture config hash after plugin auto-enable to prevent… · openclaw/openclaw@8c11210 fix: repair sanitized replay tool results before send (#67620) (thank… · openclaw/openclaw@c3c7a99 fix: restrict HTML timeout short-circuit to transient statuses · openclaw/openclaw@de129a6 fix: keep TUI watchdog bound to active run (#67401) (thanks @xantorres) · openclaw/openclaw@3525273 Gateway/skills: dedupe skills prefix-match + drop dead fallback on log · openclaw/openclaw@d7f489f Extensions/lmstudio: back off inference preload after consecutive fai… · openclaw/openclaw@b555214 TUI/streaming: add watchdog that resets the activity indicator after … · openclaw/openclaw@f44ab20 Agents/tool-loop: enable unknown-tool stream guard by default · openclaw/openclaw@36ed367 Gateway/skills: invalidate session skills snapshot on config write · openclaw/openclaw@b23d59a fix: classify HTML provider error pages correctly (#67642) (thanks @s… · openclaw/openclaw@e588e90 fix(skills): remove unused model-usage import (#67641) · openclaw/openclaw@55f05df docs(changelog): credit codex fix superseded PRs · openclaw/openclaw@e485f24 fix(openai-codex): normalize stale transport metadata in resolution a… · openclaw/openclaw@90801ba CI: pin Docker-related GitHub Actions (#67632) · openclaw/openclaw@f697b01 Android: modernize WebView and discovery API usage (#67627) · openclaw/openclaw@44a6e50 fix(deps): bump hono to 4.12.14 and @hono/node-server to 1.19.14 (GHS… · openclaw/openclaw@fbccc18 fix(deps): bump dompurify to 3.4.0 (#67614) · openclaw/openclaw@2c2dc00 CI: add explicit permissions to all workflow jobs (fixes code-scannin… · openclaw/openclaw@01b7516 fix: register bundled TTS providers and route overrides correctly (#6… · openclaw/openclaw@6ea3cdd fix: align host tilde paths with OS home (#62804) (thanks @stainlu) · openclaw/openclaw@ecfaf64 fix: flush creds queue before reconnect socket open (#67464) (thanks … · openclaw/openclaw@405c63f fix: strip standalone <function> tool call tags from visible text (#6… · openclaw/openclaw@78df859 fix(agents): preserve cli session metadata before transcript persist … · openclaw/openclaw@898fd04 docs(changelog): move cli transcript entry · openclaw/openclaw@c1817c6 fix(agents): normalize cli transcript api field · openclaw/openclaw@3a3fae0 docs(changelog): note cli transcript persistence · openclaw/openclaw@6c343f1 fix(agents): persist cli transcript turns · openclaw/openclaw@b8ef507 fix(msteams): harden security-sensitive flows (#65841) · openclaw/openclaw@c56b56e [Dashboard] Fix exec approval modal overflow for long command content… · openclaw/openclaw@053c5b0 Docs: remove QA changelog entry · openclaw/openclaw@7fd5771 QA: fix private runtime source loading (#67428) · openclaw/openclaw@d5933af docs(gateway): correct protocol.md schema path, hello-ok example, aut… · openclaw/openclaw@489404d CI: pin Node 22 runners to 22.18.0 · openclaw/openclaw@4ffa621 models.authStatus: normalize provider ids + tighten env-backed escape… · openclaw/openclaw@f2fdb9d Update CHANGELOG.md · openclaw/openclaw@7694a92 test(parallels): clean up npm update guard jobs · openclaw/openclaw@045ea7b Plugins: prefer scanDir override paths · openclaw/openclaw@b2974da fix(dreaming): default storage.mode to "separate" so phase blocks sto… · openclaw/openclaw@8c392f0 fix(memory-core): skip dreaming transcript ingestion via session stor… · openclaw/openclaw@a1b01f0 fix: dedupe replayed exec.finished node events (#67281) · openclaw/openclaw@5dcf526
docs(prometheus): rewrite with Steps quick start, Tabs for enable met… · openclaw/openclaw@893f070
vincentkoc · 2026-04-26 · via Recent Commits to openclaw:main

@@ -1,47 +1,84 @@

11

---

22

summary: "Expose OpenClaw diagnostics as Prometheus text metrics through the diagnostics-prometheus plugin"

33

title: "Prometheus metrics"

4+

sidebarTitle: "Prometheus"

45

read_when:

56

- You want Prometheus, Grafana, VictoriaMetrics, or another scraper to collect OpenClaw Gateway metrics

67

- You need the Prometheus metric names and label policy for dashboards or alerts

78

- You want metrics without running an OpenTelemetry collector

89

---

91010-

OpenClaw can expose diagnostics metrics through the bundled

11-

`diagnostics-prometheus` plugin. It listens to trusted internal diagnostics and

12-

renders a Prometheus text endpoint at:

11+

OpenClaw can expose diagnostics metrics through the bundled `diagnostics-prometheus` plugin. It listens to trusted internal diagnostics and renders a Prometheus text endpoint at:

13121413

```text

15-

/api/diagnostics/prometheus

14+

GET /api/diagnostics/prometheus

1615

```

171618-

The route uses Gateway authentication. Do not expose it as a public

19-

unauthenticated `/metrics` endpoint.

17+

Content type is `text/plain; version=0.0.4; charset=utf-8`, the standard Prometheus exposition format.

18+19+

<Warning>

20+

The route uses Gateway authentication (operator scope). Do not expose it as a public unauthenticated `/metrics` endpoint. Scrape it through the same auth path you use for other operator APIs.

21+

</Warning>

22+23+

For traces, logs, OTLP push, and OpenTelemetry GenAI semantic attributes, see [OpenTelemetry export](/gateway/opentelemetry).

20242125

## Quick start

222623-

```json5

24-

{

25-

plugins: {

26-

allow: ["diagnostics-prometheus"],

27-

entries: {

28-

"diagnostics-prometheus": { enabled: true },

29-

},

30-

},

31-

diagnostics: {

32-

enabled: true,

33-

},

34-

}

35-

```

27+

<Steps>

28+

<Step title="Enable the plugin">

29+

<Tabs>

30+

<Tab title="Config">

31+

```json5

32+

{

33+

plugins: {

34+

allow: ["diagnostics-prometheus"],

35+

entries: {

36+

"diagnostics-prometheus": { enabled: true },

37+

},

38+

},

39+

diagnostics: {

40+

enabled: true,

41+

},

42+

}

43+

```

44+

</Tab>

45+

<Tab title="CLI">

46+

```bash

47+

openclaw plugins enable diagnostics-prometheus

48+

```

49+

</Tab>

50+

</Tabs>

51+

</Step>

52+

<Step title="Restart the Gateway">

53+

The HTTP route is registered at plugin startup, so reload after enabling.

54+

</Step>

55+

<Step title="Scrape the protected route">

56+

Send the same gateway auth your operator clients use:

365737-

You can also enable the plugin from the CLI:

58+

```bash

59+

curl -H "Authorization: Bearer $OPENCLAW_GATEWAY_TOKEN" \

60+

http://127.0.0.1:18789/api/diagnostics/prometheus

61+

```

386239-

```bash

40-

openclaw plugins enable diagnostics-prometheus

41-

```

63+

</Step>

64+

<Step title="Wire Prometheus">

65+

```yaml

66+

# prometheus.yml

67+

scrape_configs:

68+

- job_name: openclaw

69+

scrape_interval: 30s

70+

metrics_path: /api/diagnostics/prometheus

71+

authorization:

72+

credentials_file: /etc/prometheus/openclaw-gateway-token

73+

static_configs:

74+

- targets: ["openclaw-gateway:18789"]

75+

```

76+

</Step>

77+

</Steps>

427843-

Then scrape the protected Gateway route with the same Gateway authentication you

44-

use for operator APIs.

79+

<Note>

80+

`diagnostics.enabled: true` is required. Without it, the plugin still registers the HTTP route but no diagnostic events flow into the exporter, so the response is empty.

81+

</Note>

45824683

## Metrics exported

4784

@@ -74,16 +111,99 @@ use for operator APIs.

7411175112

## Label policy

7611377-

Prometheus labels stay bounded and low-cardinality. The exporter does not emit

78-

raw diagnostic identifiers such as `runId`, `sessionKey`, `sessionId`, `callId`,

79-

`toolCallId`, message IDs, chat IDs, or provider request IDs.

114+

<AccordionGroup>

115+

<Accordion title="Bounded, low-cardinality labels">

116+

Prometheus labels stay bounded and low-cardinality. The exporter does not emit raw diagnostic identifiers such as `runId`, `sessionKey`, `sessionId`, `callId`, `toolCallId`, message IDs, chat IDs, or provider request IDs.

117+118+

Label values are redacted and must match OpenClaw's low-cardinality character policy. Values that fail the policy are replaced with `unknown`, `other`, or `none`, depending on the metric.

119+120+

</Accordion>

121+

<Accordion title="Series cap and overflow accounting">

122+

The exporter caps retained time series in memory at **2048** series across counters, gauges, and histograms combined. New series beyond that cap are dropped, and `openclaw_prometheus_series_dropped_total` increments by one each time.

123+124+

Watch this counter as a hard signal that an attribute upstream is leaking high-cardinality values. The exporter never lifts the cap automatically; if it climbs, fix the source rather than disabling the cap.

125+126+

</Accordion>

127+

<Accordion title="What never appears in Prometheus output">

128+

- prompt text, response text, tool inputs, tool outputs, system prompts

129+

- raw provider request IDs (only bounded hashes, where applicable, on spans — never on metrics)

130+

- session keys and session IDs

131+

- hostnames, file paths, secret values

132+

</Accordion>

133+

</AccordionGroup>

134+135+

## PromQL recipes

136+137+

```promql

138+

# Tokens per minute, split by provider

139+

sum by (provider) (rate(openclaw_model_tokens_total[1m]))

140+141+

# Spend (USD) over the last hour, by model

142+

sum by (model) (increase(openclaw_model_cost_usd_total[1h]))

143+144+

# 95th percentile model run duration

145+

histogram_quantile(

146+

0.95,

147+

sum by (le, provider, model)

148+

(rate(openclaw_run_duration_seconds_bucket[5m]))

149+

)

150+151+

# Queue wait time SLO (95p under 2s)

152+

histogram_quantile(

153+

0.95,

154+

sum by (le, lane) (rate(openclaw_queue_lane_wait_seconds_bucket[5m]))

155+

) < 2

156+157+

# Dropped Prometheus series (cardinality alarm)

158+

increase(openclaw_prometheus_series_dropped_total[15m]) > 0

159+

```

160+161+

<Tip>

162+

Prefer `gen_ai_client_token_usage` for cross-provider dashboards: it follows the OpenTelemetry GenAI semantic conventions and is consistent with metrics from non-OpenClaw GenAI services.

163+

</Tip>

164+165+

## Choosing between Prometheus and OpenTelemetry export

166+167+

OpenClaw supports both surfaces independently. You can run either, both, or neither.

168+169+

<Tabs>

170+

<Tab title="diagnostics-prometheus">

171+

- **Pull** model: Prometheus scrapes `/api/diagnostics/prometheus`.

172+

- No external collector required.

173+

- Authenticated through normal Gateway auth.

174+

- Surface is metrics only (no traces or logs).

175+

- Best for stacks already standardized on Prometheus + Grafana.

176+

</Tab>

177+

<Tab title="diagnostics-otel">

178+

- **Push** model: OpenClaw sends OTLP/HTTP to a collector or OTLP-compatible backend.

179+

- Surface includes metrics, traces, and logs.

180+

- Bridges to Prometheus through an OpenTelemetry Collector (`prometheus` or `prometheusremotewrite` exporter) when you need both.

181+

- See [OpenTelemetry export](/gateway/opentelemetry) for the full catalog.

182+

</Tab>

183+

</Tabs>

184+185+

## Troubleshooting

8018681-

Label values are redacted and must match OpenClaw's low-cardinality character

82-

policy. Values that fail the policy are replaced with `unknown`, `other`, or

83-

`none`, depending on the metric.

187+

<AccordionGroup>

188+

<Accordion title="Empty response body">

189+

- Check `diagnostics.enabled: true` in config.

190+

- Confirm the plugin is enabled and loaded with `openclaw plugins list --enabled`.

191+

- Generate some traffic; counters and histograms only emit lines after at least one event.

192+

</Accordion>

193+

<Accordion title="401 / unauthorized">

194+

The endpoint requires the Gateway operator scope (`auth: "gateway"` with `gatewayRuntimeScopeSurface: "trusted-operator"`). Use the same token or password Prometheus uses for any other Gateway operator route. There is no public unauthenticated mode.

195+

</Accordion>

196+

<Accordion title="`openclaw_prometheus_series_dropped_total` is climbing">

197+

A new attribute is exceeding the **2048**-series cap. Inspect recent metrics for an unexpectedly high-cardinality label and fix it at the source. The exporter intentionally drops new series instead of silently rewriting labels.

198+

</Accordion>

199+

<Accordion title="Prometheus shows stale series after a restart">

200+

The plugin keeps state in memory only. After a Gateway restart, counters reset to zero and gauges restart at their next reported value. Use PromQL `rate()` and `increase()` to handle resets cleanly.

201+

</Accordion>

202+

</AccordionGroup>

8420385-

The exporter caps retained time series in memory. If the cap is reached, new

86-

series are dropped and `openclaw_prometheus_series_dropped_total` increments.

204+

## Related

8720588-

For full traces, logs, OTLP export, and OpenTelemetry GenAI semantic attributes,

89-

use [OpenTelemetry export](/gateway/opentelemetry).

206+

- [Diagnostics export](/gateway/diagnostics) — local diagnostics zip for support bundles

207+

- [Health and readiness](/gateway/health)`/healthz` and `/readyz` probes

208+

- [Logging](/logging) — file-based logging

209+

- [OpenTelemetry export](/gateway/opentelemetry) — OTLP push for traces, metrics, and logs