The useful part, up front

What you’ll take away

  • One source can produce several indexed children; writing its new output does not itself remove omitted old keys.
  • An idempotent update can repeatedly preserve an incomplete replacement.
  • Compare desired children with records owned by the same parent, and preserve unrelated parents.
  • Absence from a partial update is not an authoritative whole-corpus deletion signal.
  • A completed write and a search-visible change are different observation stages.
  • Verify exact membership, values and visibility without treating freshness as proof of source truth or answer accuracy.
01

The new document is present, but an old piece is still searchable

A knowledge page changes from “East Hall” to “West Hall.” The new text reaches the search index, yet a retrieval still returns the old location as well. The update did not necessarily fail to write the new text. It may have failed to remove a search record that the new document no longer needs. This distinction becomes important when one source produces several separately stored chunks.

Microsoft describes an indexer as a component that reads a source and populates a search index; optional transformations can split content and generate vectors [1]. A source file and the records derived from it therefore need not have a one-to-one relationship. Saying that a file was updated leaves the state of its former child records unanswered.

We will follow an original two-document example through replacement, cleanup and search visibility. Its retrieval rule only looks for two literal words. No embedding model, language model or search product was run. That small boundary lets us inspect exactly how obsolete text survives without guessing whether a model invented the old answer.

One source changes from two derived records to one. Original fictional sentences, shortened here to their location-bearing meaning. J is outside the replacement scope.
Original fictional sentences, shortened here to their location-bearing meaning. J is outside the replacement scope. · Ameer Journal — original explanatory diagramOriginal editorial illustration; no third-party artwork reused.
02

Keep the source identity and its child identities separate

Call our first source document K and the unrelated second document J. A parent identity tells us which source a record belongs to. A child identity distinguishes the separate pieces produced from that source. In this example, K:0 and K:1 mean K's first and second nonempty lines. J:0 belongs to J. These are teaching labels, not a recommendation for every indexing system.

Each child also records its source revision, the exact source-text hash and the name of the splitting rule. Those fields answer different questions. The child name says which stored entry a write addresses. The parent name says which group a replacement owns. The revision and text tell us whether that entry represents the intended input. A reused child name can hold new text while a different child keeps old text.

Our splitter is intentionally simple: one nonempty line becomes one child. The article does not compare splitting quality. Keeping the rule fixed allows us to examine record replacement without also changing chunk length, overlap or an embedding model.

03

The replacement turns two old chunks into one new chunk

K at revision r1 contains two lines. The first says, “The loan kiosk is in East Hall.” The second says, “Map returns use the loan kiosk in East Hall.” They produce K:0 and K:1. J's one line says, “The reading room opens at nine,” producing J:0. All places and statements in this exercise are fictional.

Replace K with revision r2: “The loan kiosk and map returns are in West Hall.” It contains one nonempty line, so its entire desired child set is now just K:0. There should no longer be a K:1 under this recipe. J has not changed. The desired complete index contains the new K:0 and the original J:0.

For a visible query, require both words “map” and “returns,” ignoring capitalization. Initially only K:1 matches. The new K:0 should match after a complete replacement becomes visible. This rule has no ranking or top-result cutoff, so every matching record remains available for inspection.

04

Writing the desired key leaves the omitted key untouched

An upsert in our toy updater means “replace this key if it exists, or add it if it does not.” Submit the new K:0. That entry now contains revision r2 and West Hall. But the operation says nothing about K:1, so K:1 still contains revision r1 and East Hall. J:0 also remains, as it should.

Once this mixed state is made visible to the query, both K:0 and K:1 match “map returns.” One points to West Hall; the other still points to East Hall. A check that searches only for the new sentence would succeed. A check that inspects all matching child identities would reveal the leftover record.

Nothing in this result requires the search system to remember a deleted sentence mysteriously. The sentence is still in an ordinary indexed record. Before blaming answer generation, inspect the actual candidates and their parent revisions. In this constructed case, the inconsistency is already present before any answer is written.

Repeated upserts do not remove an omitted child. Original dictionary operation. Every row is made visible before its query is shown; literal matching requires both map and returns.
Original dictionary operation. Every row is made visible before its query is shown; literal matching requires both map and returns. · Ameer Journal — original explanatory diagramOriginal editorial illustration; no third-party artwork reused.
05

An operation can be repeatable and still leave the wrong set

Run the same upsert again. K:0 receives the same new value, while K:1 remains untouched. Run it five times and the result is still the same mixed set. Repetition has not supplied an instruction to remove the obsolete child.

This illustrates a useful distinction in the word “idempotent.” An operation is idempotent when repeating it does not keep changing its result. That property can help make a retry predictable. It does not prove that the result satisfies every requirement of a larger task. Our incomplete update is repeatable precisely because it keeps writing the same one key and keeps ignoring the same other key.

The right completion question is therefore larger than “did the retry return successfully?” Ask whether K has exactly the children its current source should produce. A successful repeat can confirm that one write is stable while leaving the parent replacement incomplete. Increasing retry counts cannot repair an operation whose defined scope omits the necessary removal.

06

Compare the old and desired children within the same parent

Before cleanup, the indexed children owned by K are K:0 and K:1. The desired children for K are just K:0. The old-owned set minus the desired set is therefore K:1. Removing that one obsolete child leaves the new K:0 in place and leaves J:0 untouched.

The ownership boundary is doing essential work. We are replacing K, not rebuilding every source in the collection. A record can be absent from K's output because it belongs to J, not because it is obsolete. Keeping the parent identity on each child makes that distinction explicit in the example.

After this reconciliation, check more than the number two. The complete write view should contain K:0 at r2 and J:0 at r1, with their expected texts. A different pair of records could have the same count. Exact membership, revision and content checks explain what the count represents and expose errors that a total alone would hide.

07

Absence from a partial batch is not a whole-corpus deletion signal

Now imagine passing only K's replacement records into a batch. That batch contains K:0. If a cleanup wrongly compares it with every indexed key, it sees J:0 missing and may decide to remove J. But J was never part of this update. Its absence says nothing about whether its source still exists.

In our reconciled write view, the whole-index difference against the K-only batch is exactly J:0. The correctly scoped difference for K is empty, because K already matches its desired set. The same input batch therefore produces two very different interpretations depending on which population it claims to describe.

A replacement operation should make that scope explicit: one named parent, a complete named subset, or an authoritative inventory of the whole source collection. Those are different contracts. Our example supplies a complete replacement for K and no deletion instruction for J. It cannot justify a collection-wide cleanup merely because the batch finished without an error.

A parent-scoped difference protects the unrelated document. Original replacement-set calculation. Desired output covers K only; it is not a complete inventory of every source.
Original replacement-set calculation. Desired output covers K only; it is not a complete inventory of every source. · Ameer Journal — original explanatory diagramOriginal editorial illustration; no third-party artwork reused.
08

Correct stored records may not yet be the records a query sees

Our example keeps two views: the records receiving writes and the records currently visible to its query. We copy the write view into the visible view only at named steps. This is a teaching device, not a claim that a real search engine copies a whole dictionary whenever it updates.

After K:1 is removed from the write view, the older visible view still contains both K records. The query continues to return both locations until the next visibility step. At that step it returns only K:0 at r2. Thus a correct cleanup and an observed search result can refer to different completed stages.

Elasticsearch documents refresh as the process that exposes indexed changes to search. Its request options distinguish making changes visible from simply returning after a write; wait_for waits for visibility and has a documented listener-limit case that can force a refresh [3][4]. These are search-index semantics. They do not fetch a newly edited source file or remove an omitted child that no operation addressed.

09

A cache hit and a document-management record answer different questions

Ingestion systems can keep several kinds of bookkeeping. LlamaIndex documents caching a node together with its transformation. It separately describes document management enabled by a document store, using document identifiers and hashes; the documented changed-hash reprocessing behavior depends on the attached vector-store setup [5]. Those descriptions should be read with their configuration conditions.

For our example, the source hash tells us which exact version of K was supplied, while the recipe name tells us how its children were produced. A cached transformation result could help avoid repeating the same work. It would not, by itself, state which old child identifiers still belong in the destination collection. That requires the replacement membership question we have made explicit.

Do not read the toy upsert failure as a finding about LlamaIndex's implementation. We have not run its pipeline or tested its deletion behavior. The useful comparison is between evidence roles: identifying an input, reusing a transformation and checking destination membership are related tasks with different completion conditions.

The write view can be correct before the query view changes. Original two-map teaching model. The visibility step is a local copy, not Lucene internals, a cluster transaction or a latency promise.
Original two-map teaching model. The visibility step is a local copy, not Lucene internals, a cluster transaction or a latency promise. · Ameer Journal — original explanatory diagramOriginal editorial illustration; no third-party artwork reused.
10

Removing the source parent is another operation to account for

After a correct replacement becomes visible, remove K from our source map. The indexed K:0 still exists because changing that map does not automatically change either indexed view. Only an explicit operation on K's owned children removes it from the write view; the following visibility step removes it from our query results. J remains throughout.

Microsoft's Azure Storage indexing guidance distinguishes change detection from deletion detection. It also explicitly says that the native and custom-metadata soft-delete approaches discussed on that page do not apply to one-to-many indexing [2]. That limitation matters here: one source producing several search records cannot inherit an automatic cleanup promise from a different documented case.

Our local deletion sequence proves only that K is absent from the declared final search view. It does not prove that every historical copy, log or retained source has been erased. Keep the conclusion tied to the operation and view actually inspected. A source disappearance and a searchable-record disappearance are separate observations.

11

The final set does not describe every intermediate state

There are two simple ways to order our local replacement. Remove K's old children first, then write the new child, and an observer between those steps can find no K result. Write the new child first, then remove the obsolete one, and an observer between those steps can find both locations. Both orders can finish with the same correct pair: new K:0 and unchanged J:0.

The difference concerns what becomes visible during the transition. A final-state check alone cannot tell you whether a temporary gap or mixed revision was observable earlier. Our script records both intermediate states explicitly, without claiming that a particular hosted service exposes them in this way.

If a product requires a stronger replacement guarantee, its actual publication, transaction or version-selection mechanism needs to be assessed. A sequence of correct dictionary operations is not evidence of a distributed atomic replacement. The example helps define the required observation before selecting a mechanism that promises to provide it.

12

Verify the complete child set and the search view separately

A useful completion record for this example is specific. K's current source is r2; the declared recipe produces only K:0; that stored child contains the West Hall sentence; K:1 is absent; and J:0 is unchanged. After the named visibility step, the literal query returns exactly the new K:0. Each statement answers a question that “update succeeded” leaves open.

The later parent-deletion record is different: K is explicitly removed from the covered source set, its indexed children are removed, and the visible query has no K result while J remains. Keeping those outcomes separate avoids treating one successful source operation as proof that every downstream representation followed it.

This is a freshness and membership explanation, not a complete RAG evaluation. Correctly replacing records does not establish that the source is true, that a semantic retriever will choose the best evidence, or that an answer will use it correctly. It gives the next stage a clearly defined collection whose old and new records can be accounted for.

Two local operation orders reach the same final records. Original intermediate-state comparison. These snapshots show possible local views; no hosted search transaction or concurrent update is simulated.
Original intermediate-state comparison. These snapshots show possible local views; no hosted search transaction or concurrent update is simulated. · Ameer Journal — original explanatory diagramOriginal editorial illustration; no third-party artwork reused.
Follow the evidence

Sources & further reading

Explore the references behind this guide. External pages may change after our review.

  1. Indexer Overview - Azure AI Search | Microsoft Learn
    Microsoft Learn · Accessed 7 Nov 2026
  2. Changed and Deleted Blobs - Azure AI Search | Microsoft Learn
    Microsoft Learn · Accessed 7 Nov 2026
  3. Near real-time search
    Elastic · Accessed 7 Nov 2026
  4. The refresh parameter
    Elastic · Accessed 7 Nov 2026
  5. Ingestion Pipeline | Developer Documentation
    LlamaIndex · Accessed 7 Nov 2026

Article updated 7 Nov 2026.

Found something we should revisit?

Good explanations get better with thoughtful feedback.

Suggest a correction