惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

J
Java Code Geeks
月光博客
月光博客
aimingoo的专栏
aimingoo的专栏
Google DeepMind News
Google DeepMind News
Recent Announcements
Recent Announcements
MyScale Blog
MyScale Blog
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
S
SegmentFault 最新的问题
Hugging Face - Blog
Hugging Face - Blog
Martin Fowler
Martin Fowler
WordPress大学
WordPress大学
F
Fortinet All Blogs
小众软件
小众软件
D
Docker
U
Unit 42
博客园 - 聂微东
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
爱范儿
爱范儿
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
IT之家
IT之家
云风的 BLOG
云风的 BLOG
博客园 - 司徒正美
有赞技术团队
有赞技术团队
腾讯CDC

Recent Commits to openclaw:main

test: merge chat side-result checks · openclaw/openclaw@ddd2c2a test: merge cron history checks · openclaw/openclaw@f7eb746 test: merge responsive navigation shell checks · openclaw/openclaw@c2e4b47 docs(changelog): add codex oauth fixes · openclaw/openclaw@628e6cd test: merge navigation routing cases · openclaw/openclaw@5d8cecb Tests: mock channel registry bundled fallback · openclaw/openclaw@2b08233 Secrets: avoid broad web search discovery for single plugin config · openclaw/openclaw@a464f59 test: merge config view browser checks · openclaw/openclaw@20cf511 fix(status): align oauth health with runtime · openclaw/openclaw@eed7116 feat: add macOS screen snapshots for monitor preview (#67954) thanks … · openclaw/openclaw@f377db1 fix: report shared auth scopes in hello-ok (#67810) thanks @BunsDev · openclaw/openclaw@0b6c39b Auto-reply: avoid eager bundled route fallback · openclaw/openclaw@3ea1bf4 Tests: narrow session binding contract setup · openclaw/openclaw@54e4e16 fix(macOS): enable undo/redo in webchat composer text input (#34962) · openclaw/openclaw@00951dc Tests: speed up channel setup promotion · openclaw/openclaw@82b529a Docs: refresh agent instructions · openclaw/openclaw@5775fe2 fix(auth): serialize OAuth refresh across agents to fix #26322 (#67876) · openclaw/openclaw@8e79080 test: allow ollama public surface boundary test · openclaw/openclaw@7d4f1a6 Docs: add test performance guardrails · openclaw/openclaw@89706d3 Tests: restore context-engine usage proof · openclaw/openclaw@e4c4f95 Tests: slim context engine runtime coverage · openclaw/openclaw@74c198f ci: retry failed custom checkouts · openclaw/openclaw@0ee5baf test: trim duplicate provider auth onboarding cases · openclaw/openclaw@1ffc02e matrix: fix sessions_spawn --thread subagent session spawning (#67643) · openclaw/openclaw@1ce2596 test: reduce auth choice fixture churn · openclaw/openclaw@857b9cd test: mock health status config boundaries · openclaw/openclaw@9d5ab4a test: mock onboard config io boundary · openclaw/openclaw@299694d test: mock legacy state plugin boundaries · openclaw/openclaw@2713089 test: mock channel install boundaries · openclaw/openclaw@b945248 test: mock doctor preview channel boundaries · openclaw/openclaw@b1a3ad4
fix(memory): yield while parsing session transcripts · op...
steipete · 2026-05-10 · via Recent Commits to openclaw:main

@@ -27,6 +27,7 @@ const DREAMING_NARRATIVE_RUN_PREFIX = "dreaming-narrative-";

2727

// toxic line. Wrapped continuation lines still map back to the same JSONL line.

2828

// This limit applies to content only; the role label adds up to 11 chars.

2929

const SESSION_EXPORT_CONTENT_WRAP_CHARS = 800;

30+

const SESSION_ENTRY_PARSE_YIELD_LINES = 250;

3031

const DIRECT_CRON_PROMPT_RE = /^\[cron:[^\]]+\]\s*/;

31323233

export type SessionFileEntry = {

@@ -51,6 +52,8 @@ export type BuildSessionEntryOptions = {

5152

generatedByDreamingNarrative?: boolean;

5253

/** Optional preclassification from a caller-managed cron transcript lookup. */

5354

generatedByCronRun?: boolean;

55+

/** Override for tests or specialized callers that need a tighter parse yield cadence. */

56+

parseYieldEveryLines?: number;

5457

};

55585659

export type SessionTranscriptClassification = {

@@ -520,6 +523,25 @@ function parseSessionTimestampMs(

520523

return 0;

521524

}

522525526+

function resolveSessionEntryParseYieldLines(opts: BuildSessionEntryOptions): number {

527+

const configured = opts.parseYieldEveryLines;

528+

if (typeof configured === "number" && Number.isFinite(configured)) {

529+

return Math.max(1, Math.floor(configured));

530+

}

531+

return SESSION_ENTRY_PARSE_YIELD_LINES;

532+

}

533+534+

async function yieldSessionEntryParseIfNeeded(

535+

lineIndex: number,

536+

everyLines: number,

537+

): Promise<void> {

538+

if (lineIndex > 0 && lineIndex % everyLines === 0) {

539+

await new Promise<void>((resolve) => {

540+

setImmediate(resolve);

541+

});

542+

}

543+

}

544+523545

export async function buildSessionEntry(

524546

absPath: string,

525547

opts: BuildSessionEntryOptions = {},

@@ -543,10 +565,10 @@ export async function buildSessionEntry(

543565

};

544566

}

545567

const raw = (await readRegularFile({ filePath: absPath })).buffer.toString("utf-8");

546-

const lines = raw.split("\n");

547568

const collected: string[] = [];

548569

const lineMap: number[] = [];

549570

const messageTimestampsMs: number[] = [];

571+

const parseYieldEveryLines = resolveSessionEntryParseYieldLines(opts);

550572

const sessionStoreClassification =

551573

opts.generatedByDreamingNarrative === undefined || opts.generatedByCronRun === undefined

552574

? classifySessionTranscriptFromSessionStore(absPath)

@@ -559,8 +581,12 @@ export async function buildSessionEntry(

559581

opts.generatedByCronRun ?? sessionStoreClassification?.generatedByCronRun ?? false;

560582

const allowArchiveContentCronClassification =

561583

isUsageCountedSessionArchiveTranscriptPath(absPath);

562-

for (let jsonlIdx = 0; jsonlIdx < lines.length; jsonlIdx++) {

563-

const line = lines[jsonlIdx];

584+

for (let jsonlIdx = 0, lineStart = 0; lineStart <= raw.length; jsonlIdx++) {

585+

await yieldSessionEntryParseIfNeeded(jsonlIdx, parseYieldEveryLines);

586+

const newlineIndex = raw.indexOf("\n", lineStart);

587+

const lineEnd = newlineIndex === -1 ? raw.length : newlineIndex;

588+

const line = raw.slice(lineStart, lineEnd);

589+

lineStart = newlineIndex === -1 ? raw.length + 1 : newlineIndex + 1;

564590

if (!line.trim()) {

565591

continue;

566592

}