Skip to content

bug: cora brain FTS-only mode returns 0 results for common words present in code (e.g. "registry"), no stemming #451

Description

@ajianaz

Describe the bug

Without an LLM/embedding provider configured, cora brain falls back to FTS5-only mode (results show signals: ["fts"]). It returns 0 results for common words that clearly appear in filenames, import paths, and code, while other words work fine. Stemming is also not applied.

To reproduce

Project: hompimpah (SvelteKit + Phaser + Rust/Axum), 582 symbols indexed.

cora index
for q in registry game games GameMeta "get game" scene; do
  printf '%-12s -> %s results\n' "$q" "$(cora brain "$q" --json | jq length)"
done

Actual results

Query Results Note
registry 0 ❌ filename registry.ts, present in imports/comments
game 12
games 2 ❌ no stemming — differs from game
GameMeta 15 camelCase kept as single token
get game 0 phrase query, no match
scene 19

Expected

  • registry should match (it is a filename and appears in import paths).
  • game and games should converge (stemming or query expansion).

Likely cause

  1. The FTS5 index appears to cover the symbol signature/snippet only — not file path, imports, or comments. So words that live primarily in paths/comments are unreachable.
  2. No stemming enabled → game and games are two distinct tokens.
  3. camelCase identifiers are indexed as a single token → space-separated or sub-word queries cannot reach them.

Suggested fix

  • Index file_path and comment/doc tokens into the FTS table (or a separate lower-weighted column).
  • Enable the FTS5 porter tokenizer (or unicode61 + stemming) so gamegames.
  • Optionally split camelCase identifiers into sub-tokens for matching.

Environment

  • cora 0.11.1 (released 2026-08-01)
  • No provider configured (FTS fallback path)
  • macOS arm64

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingscope:code-intelCode intelligence — indexing, graph, impact analysis

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions