A new dataset cannot be queried the instant it is imported. FastGPT 4.17.2 split index state into three stages: parsed, indexing, and indexed. Data that has not been indexed does not participate in recall, and rebuilding an index covers only data whose indexing has completed.
That single fact explains a large share of failed demonstrations, where a question receives a weak answer minutes after an import and the model takes the blame.
Three stages, three waits
Parsed means content is available and awaiting indexing, indexing means index processing is under way, and indexed means that processing has completed. File extraction progress is a separate dimension. Check both the parsing result and the data index state before interpreting an empty search result.
Importing by format batch makes the progress easier to observe. Mixing everything into one import produces a single long queue whose stage is hard to read.
Build the test set before the demo
A workable retrieval test needs three parts: twenty to thirty real questions, the source passage each question should return, and a list of questions the corpus cannot answer.
Real questions come from support tickets and internal search logs rather than from imagination. The source passage is recorded with file and paragraph so that a citation can be checked against a fixed target. The unanswerable list shows whether the system invents content when the corpus lacks it.
Freeze the set once built. Changing questions between rounds makes results incomparable, and retrieval tuning then proceeds on impression.
Run recall and citation as two steps
First, check whether the retrieved passages contain the expected paragraph. Second, check whether the answer cites those passages. The two failures have different causes: a retrieval miss points at chunking, synonyms, or the embedding model, while a citation mismatch points at generation and prompt structure.
Separating the steps prevents the common habit of treating every weak answer as a model problem. It also makes the tuning order obvious: fix recall before rewriting prompts.
Recheck on a schedule
Rerun the frozen question set after relevant source, index, or application changes. Choose the ongoing review schedule from actual usage, content updates, and known failure patterns, and record the reason for that schedule. Watch three things: whether newly imported material is recalled, whether citations still point at the source, and whether the refusal behaviour for unanswerable questions has degraded.
Keep the records. They become the baseline for the next retrieval change, and the only way to show that a change improved something rather than moved it. Index state definitions are in the 4.17.2 release notes.
Separate parsing failure from retrieval failure
Three failure modes look identical from the outside: nothing relevant comes back. Parsing can fail on a document, indexing can be incomplete, and retrieval can miss a relevant chunk.
Each has a distinct check. For parsing, inspect the extracted text and embedded images. For indexing, read the training progress. For retrieval, run a keyword that appears verbatim in the source.
Running these checks in sequence preserves evidence for each stage and prevents the most expensive mistake in this area, which is tuning prompts to fix a parsing problem.
Record the test set version
A frozen question set is only useful if its version is recorded. Each test round writes the set version, the configuration under test, and the result summary.
When two rounds disagree, the version tells whether the question set changed or the system did. Without it, every disagreement becomes an argument about method.
The same record shows progress over months, which is the only credible way to justify a configuration change to a reviewer who was not present for the earlier rounds.
Tell users what the wait means
A short note in the interface, stating that newly imported files become searchable after processing, removes a class of support ticket before it is filed.
The note should name the three states and where to see progress. Users who can see progress report problems with a stage attached, which shortens diagnosis considerably.
Review the note after an interface or processing change, so its state descriptions continue to match the application users actually see.
What to do when the counts disagree
If recall hits rise while citation accuracy falls, the retrieval change is working and the answer layer needs attention. The reverse pattern points at the opposite end. Recording both numbers per round makes that judgement mechanical rather than a matter of opinion, which is the entire reason for measuring them separately.
