fix(title): generate thread titles asynchronously by tda1017 · Pull Request #922 · bytedance/deer-flow

tda1017 · 2026-02-27T19:03:00Z

Summary

decouple title generation from TitleMiddleware.after_agent so main run completion is no longer blocked by title model latency
add an async title queue/updater with timeout + retry + fallback logic and guarded write-back that skips non-empty/manual titles
add backend tests covering async enqueue behavior and updater fallback/non-overwrite semantics

Testing

make test (fails: /bin/sh: 1: uv: not found)
python3 -m pytest backend/tests/test_title_generation.py backend/tests/test_title_async.py -v (fails: No module named pytest)
pnpm -C frontend check (fails: next: not found, node_modules missing)
Playwright manual check attempted on http://localhost:2026 and http://localhost:3000 (both ERR_CONNECTION_REFUSED)

Fixes #887

…ce#887)

CLAassistant · 2026-02-27T19:03:08Z

All committers have signed the CLA.

WillemJiang · 2026-02-28T01:43:18Z

@tda1017 Please check the .gitconfig file user setting in your commit box. It looks like you just use root as the user.name.

tda1017 · 2026-02-28T07:26:15Z

Hi @WillemJiang, thanks for the review! I've checked the commits and they show tda1017 <jiaxinchen007@gmail.com> as the author:

c691ac2 tda1017 <jiaxinchen007@gmail.com> - chore(title): replace async title prints with logger
035f76f tda1017 <jiaxinchen007@gmail.com> - test(title): add async title middleware and updater coverage
297ec76 tda1017 <jiaxinchen007@gmail.com> - fix(title): move thread title generation to async worker

The CLA has also been signed now. Is there anything else that needs to be addressed?

WillemJiang · 2026-02-28T14:09:43Z

@tda1017 Please check the Ready for review button if the PR is ready.

WillemJiang · 2026-03-01T14:25:29Z

@tda1017 Do you mind fixing the lint error?

Copilot

Pull request overview

This PR addresses #887 by decoupling thread-title generation from the main agent run so the UI can complete a conversation without waiting on title-model latency. It introduces an asynchronous title generation queue/updater with timeout/retry/fallback behavior and adds test coverage for the new async flow.

Changes:

Move title generation out of TitleMiddleware.after_agent into an async queue + updater that writes back titles after the main run completes.
Add timeout_seconds and max_retries to title generation config.
Add backend tests covering enqueue behavior, timeout fallback, and non-overwrite semantics.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
config.example.yaml	Adds new title timeout/retry configuration knobs.
backend/src/config/title_config.py	Extends `TitleConfig` with timeout and retry fields + validation.
backend/src/agents/middlewares/title_middleware.py	Switches from synchronous title generation to enqueueing async work.
backend/src/agents/title/queue.py	Introduces a singleton queue that dispatches background workers.
backend/src/agents/title/updater.py	Implements title generation with timeout/retry/fallback and guarded write-back via LangGraph SDK.
backend/src/agents/title/init.py	Exposes queue/updater symbols at the package level.
backend/tests/test_title_generation.py	Updates config tests to cover the new config fields/validation.
backend/tests/test_title_async.py	Adds tests for async enqueueing and updater behavior (fallback + non-overwrite).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-01T14:27:05Z

backend/tests/test_title_async.py

+    set_title_config(TitleConfig(timeout_seconds=0.01, max_retries=0, max_chars=20))
+    monkeypatch.setattr("src.agents.title.updater.create_chat_model", lambda **kwargs: SlowModel())
+
+    updater = TitleGenerationUpdater(client_factory=lambda url: None)
+    title = updater.generate_title([_message("human", "A long first message for fallback title")])
+
+    set_title_config(original)
+    assert title.startswith("A long first message")


This test mutates the global title config and restores it afterwards, but restoration isn’t protected if the test fails mid-way (assertion error/exception), which can leak config into later tests. Use try/finally or a fixture to guarantee set_title_config(original) always runs.

Suggested change

set_title_config(TitleConfig(timeout_seconds=0.01, max_retries=0, max_chars=20))

monkeypatch.setattr("src.agents.title.updater.create_chat_model", lambda **kwargs: SlowModel())

updater = TitleGenerationUpdater(client_factory=lambda url: None)

title = updater.generate_title([_message("human", "A long first message for fallback title")])

set_title_config(original)

assert title.startswith("A long first message")

try:

set_title_config(TitleConfig(timeout_seconds=0.01, max_retries=0, max_chars=20))

monkeypatch.setattr("src.agents.title.updater.create_chat_model", lambda **kwargs: SlowModel())

updater = TitleGenerationUpdater(client_factory=lambda url: None)

title = updater.generate_title([_message("human", "A long first message for fallback title")])

assert title.startswith("A long first message")

finally:

set_title_config(original)

Copilot · 2026-03-01T14:27:05Z

backend/tests/test_title_async.py

+    updater = TitleGenerationUpdater(client_factory=lambda url: ClientStub())
+    updater.process(TitleGenerationTask(thread_id="thread-2", messages=[_message("human", "x")]))
+


test_updater_does_not_override_manual_title calls updater.process(...), which currently runs generate_title() (and may try to load config.yaml / hit model creation) even though the thread title is already manual and no update should occur. To keep this unit test fast/deterministic, stub create_chat_model or TitleGenerationUpdater.generate_title in this test as well.

Copilot · 2026-03-01T14:27:05Z

backend/src/agents/title/updater.py

+    def process(self, task: TitleGenerationTask) -> None:
+        title = self.generate_title(task.messages)
+        self._update_thread_title_if_needed(task.thread_id, title)
+


process() generates a title before checking whether the thread already has a non-default/manual title. If the user manually renames the thread before the async worker runs, this will still spend an LLM call (and potentially retries/timeouts) even though write-back is skipped. Consider fetching the current thread title first (or passing the known title into the task) and returning early before calling generate_title() when the title is already set.

Suggested change

def process(self, task: TitleGenerationTask) -> None:

title = self.generate_title(task.messages)

self._update_thread_title_if_needed(task.thread_id, title)

def process(self, task: TitleGenerationTask) -> None:

# Avoid unnecessary title generation if the thread already has a non-default/manual title.

if self._has_non_default_title(task.thread_id):

return

title = self.generate_title(task.messages)

self._update_thread_title_if_needed(task.thread_id, title)

def _has_non_default_title(self, thread_id: str) -> bool:

"""

Returns True if the thread already has a non-empty title, indicating that

a manual or non-default title has been set and we should skip generation.

"""

# If no client is available, we cannot check the existing title; fall back to generation.

if self._client_factory is None and _get_sync_client is None:

return False

try:

if self._client_factory is not None:

client = self._client_factory(self._langgraph_url)

else:

client = _get_sync_client(self._langgraph_url) # type: ignore[misc]

thread = client.threads.get(thread_id)

except Exception as exc:

logger.warning("Failed to fetch thread %s while checking existing title: %s", thread_id, exc)

return False

existing_title = getattr(thread, "title", None)

if not isinstance(existing_title, str):

return False

return existing_title.strip() != ""

Copilot · 2026-03-01T14:27:05Z

backend/src/agents/title/updater.py

+    def _invoke_with_timeout(self, prompt: str, config: TitleConfig) -> str:
+        result: dict[str, Any] = {}
+        error: dict[str, BaseException] = {}
+
+        def _invoke() -> None:
+            try:
+                model = create_chat_model(name=config.model_name, thinking_enabled=False)
+                response = model.invoke(prompt)
+                result["content"] = str(getattr(response, "content", "") or "")
+            except Exception as exc:
+                error["exception"] = exc
+
+        worker = threading.Thread(target=_invoke, daemon=True)
+        worker.start()
+        worker.join(timeout=config.timeout_seconds)
+        if worker.is_alive():
+            raise TimeoutError("title generation timed out")
+        if "exception" in error:


The timeout implementation starts a daemon thread to run model.invoke() and raises TimeoutError if join() times out, but the underlying invoke thread continues running. With retries enabled, multiple timed-out invocations can run concurrently and accumulate, causing unbounded thread growth and lingering network calls. Prefer enforcing timeouts at the model/client layer (e.g., request timeout settings) or using a cancellable execution mechanism so timed-out attempts don’t keep running.

Copilot · 2026-03-01T14:27:06Z

backend/src/agents/title/queue.py

+    def add(self, thread_id: str, messages: list[Any]) -> None:
+        with self._lock:
+            if thread_id in self._pending:
+                return
+            self._pending.add(thread_id)
+
+        worker = threading.Thread(target=self._process, args=(thread_id, messages), daemon=True)
+        worker.start()
+


TitleGenerationQueue.add() spawns a new daemon thread per enqueued task. Under load (many new threads completing around the same time), this can create a large number of concurrent threads and increase memory/CPU overhead. Consider using a bounded worker pool (e.g., ThreadPoolExecutor) or a single background worker thread consuming from a queue to cap concurrency.

tda1017 added 2 commits February 28, 2026 02:59

fix(title): move thread title generation to async worker (bytedance#887)

297ec76

test(title): add async title middleware and updater coverage (bytedan…

035f76f

…ce#887)

chore(title): replace async title prints with logger

c691ac2

tda1017 force-pushed the fix/async-title-generation branch from 0bdc88b to c691ac2 Compare February 28, 2026 06:59

Merge branch 'main' into fix/async-title-generation

552fbfe

tda1017 marked this pull request as ready for review February 28, 2026 14:55

Merge branch 'main' into fix/async-title-generation

221da46

WillemJiang requested a review from Copilot March 1, 2026 14:22

Copilot started reviewing on behalf of WillemJiang March 1, 2026 14:22 View session

Copilot AI reviewed Mar 1, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(title): generate thread titles asynchronously#922

fix(title): generate thread titles asynchronously#922
tda1017 wants to merge 5 commits intobytedance:mainfrom
tda1017:fix/async-title-generation

tda1017 commented Feb 27, 2026

CLAassistant commented Feb 27, 2026 •

edited

Loading

WillemJiang commented Feb 28, 2026

tda1017 commented Feb 28, 2026

WillemJiang commented Feb 28, 2026

WillemJiang commented Mar 1, 2026

Copilot AI left a comment

Copilot AI Mar 1, 2026

Copilot AI Mar 1, 2026

Copilot AI Mar 1, 2026

Copilot AI Mar 1, 2026

Copilot AI Mar 1, 2026

Labels

4 participants

		updater = TitleGenerationUpdater(client_factory=lambda url: ClientStub())
		updater.process(TitleGenerationTask(thread_id="thread-2", messages=[_message("human", "x")]))

-    def process(self, task: TitleGenerationTask) -> None:
-        title = self.generate_title(task.messages)
-        self._update_thread_title_if_needed(task.thread_id, title)
+    def process(self, task: TitleGenerationTask) -> None:
+        # Avoid unnecessary title generation if the thread already has a non-default/manual title.
+        if self._has_non_default_title(task.thread_id):
+            return
+        title = self.generate_title(task.messages)
+        self._update_thread_title_if_needed(task.thread_id, title)
+    def _has_non_default_title(self, thread_id: str) -> bool:
+        """
+        Returns True if the thread already has a non-empty title, indicating that
+        a manual or non-default title has been set and we should skip generation.
+        """
+        # If no client is available, we cannot check the existing title; fall back to generation.
+        if self._client_factory is None and _get_sync_client is None:
+            return False
+        try:
+            if self._client_factory is not None:
+                client = self._client_factory(self._langgraph_url)
+            else:
+                client = _get_sync_client(self._langgraph_url)  # type: ignore[misc]
+            thread = client.threads.get(thread_id)
+        except Exception as exc:
+            logger.warning("Failed to fetch thread %s while checking existing title: %s", thread_id, exc)
+            return False
+        existing_title = getattr(thread, "title", None)
+        if not isinstance(existing_title, str):
+            return False
+        return existing_title.strip() != ""

Conversation

tda1017 commented Feb 27, 2026

Summary

Testing

CLAassistant commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

WillemJiang commented Feb 28, 2026

tda1017 commented Feb 28, 2026

WillemJiang commented Feb 28, 2026

WillemJiang commented Mar 1, 2026

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Copilot AI Mar 1, 2026

Choose a reason for hiding this comment

Copilot AI Mar 1, 2026

Choose a reason for hiding this comment

Copilot AI Mar 1, 2026

Choose a reason for hiding this comment

Copilot AI Mar 1, 2026

Choose a reason for hiding this comment

Copilot AI Mar 1, 2026

Choose a reason for hiding this comment

Labels

4 participants

CLAassistant commented Feb 27, 2026 •

edited

Loading