惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

U
Unit 42
T
The Blog of Author Tim Ferriss
H
Help Net Security
博客园 - 叶小钗
云风的 BLOG
云风的 BLOG
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
D
DataBreaches.Net
博客园 - 聂微东
A
About on SuperTechFans
大猫的无限游戏
大猫的无限游戏
P
Proofpoint News Feed
Martin Fowler
Martin Fowler
博客园 - 【当耐特】
S
SegmentFault 最新的问题
Blog — PlanetScale
Blog — PlanetScale
酷 壳 – CoolShell
酷 壳 – CoolShell
G
Google Developers Blog
I
InfoQ
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
GbyAI
GbyAI
B
Blog
Engineering at Meta
Engineering at Meta
V
V2EX
Hugging Face - Blog
Hugging Face - Blog

Recent Commits to openclaw:main

test: merge chat side-result checks · openclaw/openclaw@ddd2c2a test: merge cron history checks · openclaw/openclaw@f7eb746 test: merge responsive navigation shell checks · openclaw/openclaw@c2e4b47 docs(changelog): add codex oauth fixes · openclaw/openclaw@628e6cd test: merge navigation routing cases · openclaw/openclaw@5d8cecb Tests: mock channel registry bundled fallback · openclaw/openclaw@2b08233 Secrets: avoid broad web search discovery for single plugin config · openclaw/openclaw@a464f59 test: merge config view browser checks · openclaw/openclaw@20cf511 fix(status): align oauth health with runtime · openclaw/openclaw@eed7116 feat: add macOS screen snapshots for monitor preview (#67954) thanks … · openclaw/openclaw@f377db1 fix: report shared auth scopes in hello-ok (#67810) thanks @BunsDev · openclaw/openclaw@0b6c39b Auto-reply: avoid eager bundled route fallback · openclaw/openclaw@3ea1bf4 Tests: narrow session binding contract setup · openclaw/openclaw@54e4e16 fix(macOS): enable undo/redo in webchat composer text input (#34962) · openclaw/openclaw@00951dc Tests: speed up channel setup promotion · openclaw/openclaw@82b529a Docs: refresh agent instructions · openclaw/openclaw@5775fe2 fix(auth): serialize OAuth refresh across agents to fix #26322 (#67876) · openclaw/openclaw@8e79080 test: allow ollama public surface boundary test · openclaw/openclaw@7d4f1a6 Docs: add test performance guardrails · openclaw/openclaw@89706d3 Tests: restore context-engine usage proof · openclaw/openclaw@e4c4f95 Tests: slim context engine runtime coverage · openclaw/openclaw@74c198f ci: retry failed custom checkouts · openclaw/openclaw@0ee5baf test: trim duplicate provider auth onboarding cases · openclaw/openclaw@1ffc02e matrix: fix sessions_spawn --thread subagent session spawning (#67643) · openclaw/openclaw@1ce2596 test: reduce auth choice fixture churn · openclaw/openclaw@857b9cd test: mock health status config boundaries · openclaw/openclaw@9d5ab4a test: mock onboard config io boundary · openclaw/openclaw@299694d test: mock legacy state plugin boundaries · openclaw/openclaw@2713089 test: mock channel install boundaries · openclaw/openclaw@b945248 test: mock doctor preview channel boundaries · openclaw/openclaw@b1a3ad4
refactor: share agent truncate utilities · openclaw/openc...
vincentkoc · 2026-05-30 · via Recent Commits to openclaw:main
1-

/**

2-

* Shared truncation utilities for tool outputs.

3-

*

4-

* Truncation is based on two independent limits - whichever is hit first wins:

5-

* - Line limit (default: 2000 lines)

6-

* - Byte limit (default: 50KB)

7-

*

8-

* Never returns partial lines (except bash tail truncation edge case).

9-

*/

10-11-

export const DEFAULT_MAX_LINES = 2000;

12-

export const DEFAULT_MAX_BYTES = 50 * 1024; // 50KB

13-

export const GREP_MAX_LINE_LENGTH = 500; // Max chars per grep match line

14-15-

export interface TruncationResult {

16-

/** The truncated content */

17-

content: string;

18-

/** Whether truncation occurred */

19-

truncated: boolean;

20-

/** Which limit was hit: "lines", "bytes", or null if not truncated */

21-

truncatedBy: "lines" | "bytes" | null;

22-

/** Total number of lines in the original content */

23-

totalLines: number;

24-

/** Total number of bytes in the original content */

25-

totalBytes: number;

26-

/** Number of complete lines in the truncated output */

27-

outputLines: number;

28-

/** Number of bytes in the truncated output */

29-

outputBytes: number;

30-

/** Whether the last line was partially truncated (only for tail truncation edge case) */

31-

lastLinePartial: boolean;

32-

/** Whether the first line exceeded the byte limit (for head truncation) */

33-

firstLineExceedsLimit: boolean;

34-

/** The max lines limit that was applied */

35-

maxLines: number;

36-

/** The max bytes limit that was applied */

37-

maxBytes: number;

38-

}

39-40-

export interface TruncationOptions {

41-

/** Maximum number of lines (default: 2000) */

42-

maxLines?: number;

43-

/** Maximum number of bytes (default: 50KB) */

44-

maxBytes?: number;

45-

}

46-47-

function splitLinesForCounting(content: string): string[] {

48-

if (content.length === 0) {

49-

return [];

50-

}

51-

const lines = content.split("\n");

52-

if (content.endsWith("\n")) {

53-

lines.pop();

54-

}

55-

return lines;

56-

}

57-58-

/**

59-

* Format bytes as human-readable size.

60-

*/

61-

export function formatSize(bytes: number): string {

62-

if (bytes < 1024) {

63-

return `${bytes}B`;

64-

} else if (bytes < 1024 * 1024) {

65-

return `${(bytes / 1024).toFixed(1)}KB`;

66-

}

67-

return `${(bytes / (1024 * 1024)).toFixed(1)}MB`;

68-

}

69-70-

/**

71-

* Truncate content from the head (keep first N lines/bytes).

72-

* Suitable for file reads where you want to see the beginning.

73-

*

74-

* Never returns partial lines. If first line exceeds byte limit,

75-

* returns empty content with firstLineExceedsLimit=true.

76-

*/

77-

export function truncateHead(content: string, options: TruncationOptions = {}): TruncationResult {

78-

const maxLines = options.maxLines ?? DEFAULT_MAX_LINES;

79-

const maxBytes = options.maxBytes ?? DEFAULT_MAX_BYTES;

80-81-

const totalBytes = Buffer.byteLength(content, "utf-8");

82-

const lines = splitLinesForCounting(content);

83-

const totalLines = lines.length;

84-85-

// Check if no truncation needed

86-

if (totalLines <= maxLines && totalBytes <= maxBytes) {

87-

return {

88-

content,

89-

truncated: false,

90-

truncatedBy: null,

91-

totalLines,

92-

totalBytes,

93-

outputLines: totalLines,

94-

outputBytes: totalBytes,

95-

lastLinePartial: false,

96-

firstLineExceedsLimit: false,

97-

maxLines,

98-

maxBytes,

99-

};

100-

}

101-102-

// Check if first line alone exceeds byte limit

103-

const firstLineBytes = Buffer.byteLength(lines[0], "utf-8");

104-

if (firstLineBytes > maxBytes) {

105-

return {

106-

content: "",

107-

truncated: true,

108-

truncatedBy: "bytes",

109-

totalLines,

110-

totalBytes,

111-

outputLines: 0,

112-

outputBytes: 0,

113-

lastLinePartial: false,

114-

firstLineExceedsLimit: true,

115-

maxLines,

116-

maxBytes,

117-

};

118-

}

119-120-

// Collect complete lines that fit

121-

const outputLinesArr: string[] = [];

122-

let outputBytesCount = 0;

123-

let truncatedBy: "lines" | "bytes" = "lines";

124-125-

for (let i = 0; i < lines.length && i < maxLines; i++) {

126-

const line = lines[i];

127-

const lineBytes = Buffer.byteLength(line, "utf-8") + (i > 0 ? 1 : 0); // +1 for newline

128-129-

if (outputBytesCount + lineBytes > maxBytes) {

130-

truncatedBy = "bytes";

131-

break;

132-

}

133-134-

outputLinesArr.push(line);

135-

outputBytesCount += lineBytes;

136-

}

137-138-

// If we exited due to line limit

139-

if (outputLinesArr.length >= maxLines && outputBytesCount <= maxBytes) {

140-

truncatedBy = "lines";

141-

}

142-143-

const outputContent = outputLinesArr.join("\n");

144-

const finalOutputBytes = Buffer.byteLength(outputContent, "utf-8");

145-146-

return {

147-

content: outputContent,

148-

truncated: true,

149-

truncatedBy,

150-

totalLines,

151-

totalBytes,

152-

outputLines: outputLinesArr.length,

153-

outputBytes: finalOutputBytes,

154-

lastLinePartial: false,

155-

firstLineExceedsLimit: false,

156-

maxLines,

157-

maxBytes,

158-

};

159-

}

160-161-

/**

162-

* Truncate content from the tail (keep last N lines/bytes).

163-

* Suitable for bash output where you want to see the end (errors, final results).

164-

*

165-

* May return partial first line if the last line of original content exceeds byte limit.

166-

*/

167-

export function truncateTail(content: string, options: TruncationOptions = {}): TruncationResult {

168-

const maxLines = options.maxLines ?? DEFAULT_MAX_LINES;

169-

const maxBytes = options.maxBytes ?? DEFAULT_MAX_BYTES;

170-171-

const totalBytes = Buffer.byteLength(content, "utf-8");

172-

const lines = splitLinesForCounting(content);

173-

const totalLines = lines.length;

174-175-

// Check if no truncation needed

176-

if (totalLines <= maxLines && totalBytes <= maxBytes) {

177-

return {

178-

content,

179-

truncated: false,

180-

truncatedBy: null,

181-

totalLines,

182-

totalBytes,

183-

outputLines: totalLines,

184-

outputBytes: totalBytes,

185-

lastLinePartial: false,

186-

firstLineExceedsLimit: false,

187-

maxLines,

188-

maxBytes,

189-

};

190-

}

191-192-

// Work backwards from the end

193-

const outputLinesArr: string[] = [];

194-

let outputBytesCount = 0;

195-

let truncatedBy: "lines" | "bytes" = "lines";

196-

let lastLinePartial = false;

197-198-

for (let i = lines.length - 1; i >= 0 && outputLinesArr.length < maxLines; i--) {

199-

const line = lines[i];

200-

const lineBytes = Buffer.byteLength(line, "utf-8") + (outputLinesArr.length > 0 ? 1 : 0); // +1 for newline

201-202-

if (outputBytesCount + lineBytes > maxBytes) {

203-

truncatedBy = "bytes";

204-

// Edge case: if we haven't added ANY lines yet and this line exceeds maxBytes,

205-

// take the end of the line (partial)

206-

if (outputLinesArr.length === 0) {

207-

const truncatedLine = truncateStringToBytesFromEnd(line, maxBytes);

208-

outputLinesArr.unshift(truncatedLine);

209-

outputBytesCount = Buffer.byteLength(truncatedLine, "utf-8");

210-

lastLinePartial = true;

211-

}

212-

break;

213-

}

214-215-

outputLinesArr.unshift(line);

216-

outputBytesCount += lineBytes;

217-

}

218-219-

// If we exited due to line limit

220-

if (outputLinesArr.length >= maxLines && outputBytesCount <= maxBytes) {

221-

truncatedBy = "lines";

222-

}

223-224-

const outputContent = outputLinesArr.join("\n");

225-

const finalOutputBytes = Buffer.byteLength(outputContent, "utf-8");

226-227-

return {

228-

content: outputContent,

229-

truncated: true,

230-

truncatedBy,

231-

totalLines,

232-

totalBytes,

233-

outputLines: outputLinesArr.length,

234-

outputBytes: finalOutputBytes,

235-

lastLinePartial,

236-

firstLineExceedsLimit: false,

237-

maxLines,

238-

maxBytes,

239-

};

240-

}

241-242-

/**

243-

* Truncate a string to fit within a byte limit (from the end).

244-

* Handles multi-byte UTF-8 characters correctly.

245-

*/

246-

function truncateStringToBytesFromEnd(str: string, maxBytes: number): string {

247-

const buf = Buffer.from(str, "utf-8");

248-

if (buf.length <= maxBytes) {

249-

return str;

250-

}

251-252-

// Start from the end, skip maxBytes back

253-

let start = buf.length - maxBytes;

254-255-

// Find a valid UTF-8 boundary (start of a character)

256-

while (start < buf.length && (buf[start] & 0xc0) === 0x80) {

257-

start++;

258-

}

259-260-

return buf.slice(start).toString("utf-8");

261-

}

262-263-

/**

264-

* Truncate a single line to max characters, adding [truncated] suffix.

265-

* Used for grep match lines.

266-

*/

267-

export function truncateLine(

268-

line: string,

269-

maxChars: number = GREP_MAX_LINE_LENGTH,

270-

): { text: string; wasTruncated: boolean } {

271-

if (line.length <= maxChars) {

272-

return { text: line, wasTruncated: false };

273-

}

274-

return { text: `${line.slice(0, maxChars)}... [truncated]`, wasTruncated: true };

275-

}

1+

export {

2+

DEFAULT_MAX_BYTES,

3+

DEFAULT_MAX_LINES,

4+

GREP_MAX_LINE_LENGTH,

5+

formatSize,

6+

truncateHead,

7+

truncateLine,

8+

truncateTail,

9+

type TruncationOptions,

10+

type TruncationResult,

11+

} from "../../../../packages/agent-core/src/harness/utils/truncate.js";