Imagine a small, hypothetical set of household notes kept on a fridge door. One note says, "Buy oat milk for tea." A second says, "Oat milk is in the blue carton." A third, unrelated note says, "Water the basil on the windowsill." We will use only these three invented notes throughout, so every later step can be checked against the same tiny collection[3].

The question is simple to state and harder to answer: can a computer turn each note into numbers so that the two milk notes are treated as close, while the basil note is treated as farther away? The sources describe a retrieval system in which documents are split into chunks, each chunk is embedded, and a query is embedded by the same model so that a similarity search can return the closest stored chunks[3]. That is the mechanism we will follow, without pretending that closeness in numbers is the same as understanding in every sense.

A bounded example matters because it prevents vague claims. If the milk notes end up near each other, we can ask what part of the process made that happen: the splitting, the token limit, the embedding model, or the distance measure. If they do not, we can also ask which step failed. Keeping the example small and hypothetical makes the limits visible instead of hiding them behind a large, impressive-sounding dataset[2].

Consider a deliberately bounded hypothetical corpus consisting of three short household notes: two concerning oat milk and one concerning watering basil. The corpus is not evidence about any real embedding model; it is a controlled illustration for tracing the pipeline described in the sources, in which a knowledge base is divided into chunks, chunks are embedded and stored, and a query embedded by the same model is used for similarity search[3]. Holding the corpus fixed allows each transformation to be examined independently.

The explanatory target is the mapping from variable-length text to a fixed-length numeric vector, and from vector geometry to retrieval ranking. The sources support the existence of this pipeline, including chunking before embedding, storage in a vector database, and query-time similarity search[3]. They do not, by themselves, establish that any particular model will place the two milk notes close together; that outcome depends on the model, its training, its input limits, and the chosen distance[2].

This distinction between pipeline structure and model-specific behaviour is essential. Chunking and storage are engineering choices documented across the sources[1][2][3]. The geometry of a particular embedding space, including whether negation or word order is preserved, is a property of a specific model and evaluation setup. The hypothetical notes therefore serve as a test case for reasoning, not as a measured result[3].

A whole document is often too long to embed as one piece. Splitting it into smaller chunks helps each piece fit within the input limits of an embedding model and makes it possible to retrieve one relevant piece instead of an entire document[2]. In our fridge-door example the notes are already short, but the same idea applies: each note, or each part of a longer note, becomes a separate unit that can be stored and found on its own[3].

There are several ways to split text, and none is best for every dataset. One approach uses a fixed size, such as a set number of words or characters, sometimes with a small overlap between neighbouring chunks so that context is not cut too abruptly[2]. Another approach splits at natural boundaries such as sentences, paragraphs, or headings, which can preserve the structure of the original text[1]. A third approach tries to keep meaningful units together, so that a chunk represents one idea rather than an arbitrary slice[2].

For the milk notes, a sentence-based split would keep "Buy oat milk for tea" together as one chunk. A fixed-size split that cut in the middle of that sentence could separate "Buy oat" from "milk for tea," which would be unhelpful. This is why the sources warn that chunks that are too small can split sentences and lose meaning, while chunks that are too large can mix several ideas together[3]. The right balance depends on the content and on how the chunks will be used[2].

Chunking is required for at least two distinct reasons documented in the sources. First, embedding and chat models impose maximum token input lengths; the Azure documentation gives 8,191 tokens as the maximum for one named model, equivalent to roughly 6,000 words, and warns that input must remain under the limit[2]. Second, retrieval operates on chunks rather than whole documents, so chunk boundaries determine the granularity at which relevance can be expressed[3].

The sources describe fixed-size, variable-size, semantic, and custom-combination strategies[2]. Recursive splitting attempts to preserve larger units such as paragraphs and only descends to sentences or words when a unit exceeds the target size[1]. Structure-based splitting uses headings or other markup to group semantically related text[1]. Each strategy encodes a different assumption about where meaning resides: in length regularity, in punctuation, in document structure, or in semantic coherence[2].

Overlap is a further design variable. The Azure guidance recommends starting with 512 tokens and 25 percent overlap, while noting that the optimal value varies with content type and use case[2]. In the hypothetical milk example, overlap would be unnecessary because each note is a single sentence. In a longer recipe, however, overlap could preserve a transition between an ingredient list and a method step that a hard boundary would otherwise sever[2].

Models do not count words in the way people do. They count tokens, which are smaller pieces of text, and the number of tokens in a piece of text depends on the tokenizer being used[2]. A short note such as "Buy oat milk for tea" will contain only a handful of tokens, but a long document can contain thousands. That is why chunk size is often measured in tokens rather than only in characters or words[3].

Token counting also explains why two chunks with the same number of characters can have different token lengths. The Azure documentation notes that character counts do not align exactly with token definitions, and that the token length of a chunk varies with its contents[2]. In practical terms, a chunk that looks short on the page might still be too long for a model with a small input limit, or it might be much shorter than expected[2].

For our hypothetical notes, token limits are not a problem because the notes are tiny. The lesson is still useful: before embedding any text, it is worth checking how many tokens it contains and whether it fits the model being used[3]. The sources show token counting being used to inform chunk-size decisions, rather than guessing from character length alone[2].

Tokenization mediates between character-level text and model input. The sources treat token count as the operative length measure for embedding models, distinct from character count and word count[2][3]. One tutorial explicitly changes a splitter to count length in tokens instead of characters so that chunk size can be compared with the embedding model's maximum sequence length[3]. This is not a cosmetic change: a character-based limit can silently produce chunks that exceed a token-based model limit[2].

The relationship between characters and tokens is content-dependent. The Azure documentation states that each token is around four characters for common models, but also that measured token length varies with chunk contents[2]. Consequently, a fixed character budget does not guarantee a fixed token budget. Systems that need strict compliance with a model limit should measure tokens with the relevant tokenizer, as illustrated by the token-counting example in the sources[2].

In the bounded hypothetical example, tokenization has a further implication for word order and negation. Tokens preserve the sequence of pieces presented to the model, but whether the resulting vector is sensitive to that sequence depends on the model's architecture and training. The sources document token limits and counting, not a universal guarantee about order sensitivity[3]. That limitation must be kept separate from the well-supported claim that token count constrains chunk size[2].

An embedding model turns a chunk of text into a list of numbers called a vector. The list has a fixed length for a given model, even though the original text can be short or long[3]. One tutorial in the sources works with embeddings of 384 dimensions, meaning each chunk is represented by 384 numbers[3]. The exact number depends on the model, but the idea is the same: text in, fixed-length list of numbers out.

The same model must be used for the stored chunks and for the query. When a person asks a question, that question is embedded by the same model, and the system searches for stored vectors that are closest to the query vector[3]. In our example, the question "Where is the oat milk?" would be turned into a vector, and the system would compare it with the vectors for the milk notes and the basil note[3].

It is important not to overstate what the numbers mean. The sources say that embeddings represent a document's meaning for the purpose of retrieval, and that closeness in meaning should be reflected in closeness of the vectors[3]. That is a design goal, not a promise that every nuance of meaning is preserved. A vector is a compressed numeric summary, and compression always leaves something out[3].

Embedding is a function from a token sequence of variable length to a point in a fixed-dimensional real vector space. The sources illustrate this with a 384-dimensional embedding space, projected to two dimensions for visualisation[3]. The dimensionality is a property of the chosen model, not a universal constant. What is invariant across the documented pipeline is that the same embedding function is applied to stored chunks and to the query, enabling a common space for comparison[3].

The sources describe normalisation as part of this mapping when cosine similarity is used: vectors are rescaled to unit norm so that comparison depends on direction rather than magnitude[3]. This preprocessing step is mathematically consequential. Two vectors pointing in the same direction but with different lengths become identical under cosine similarity after normalisation, whereas they would differ under a magnitude-sensitive measure[3].

Crucially, the sources support the claim that embeddings are intended to represent meaning for retrieval, not that they encode meaning losslessly[3]. A fixed-length vector is a lossy compression of a token sequence. Information that the model does not encode, or that is discarded during pooling or normalisation, cannot be recovered from the vector alone. This lossiness is the root of several failure modes discussed later[3].

The basic intuition is that words and phrases that appear in similar contexts tend to receive similar vectors. If "oat milk" often appears near words about tea, cartons, and shopping, its vector will reflect those associations. Another phrase that appears in similar contexts, such as "soy milk" in a shopping list, may therefore lie nearby in the vector space[3]. Closeness is a record of shared context, not a dictionary definition.

In our hypothetical notes, "Buy oat milk for tea" and "Oat milk is in the blue carton" share the words "oat milk" and a shopping or kitchen context. If the embedding model captures that shared context, the two vectors should be closer to each other than either is to "Water the basil on the windowsill," which shares neither the key words nor the topic[3]. The sources describe exactly this expectation: as embeddings represent meaning, closeness in meaning should be reflected in closeness of the vectors[3].

This tendency has limits. Similar context does not always mean similar meaning, and the sources do not provide a universal test that guarantees the milk notes will be nearest neighbours for every model[3]. The outcome depends on the model, the chunking, and the distance measure. Treating closeness as a useful signal for retrieval is reasonable; treating it as proof of identical meaning is not[3].

The distributional hypothesis underlying embedding geometry is that linguistic context provides a learnable signal for semantic similarity. The sources do not present a formal proof of this hypothesis; they present a retrieval pipeline in which embedded chunks are compared by similarity search and in which spatial proximity in a projected embedding space is interpreted as reflecting closeness in meaning[3]. The claim is therefore best understood as an empirical design principle for a specific class of models, not as a theorem about language[3].

Mechanistically, a contextual encoder produces token representations that depend on surrounding tokens, and a pooling or aggregation step combines them into a single vector. The sources document the input-output behaviour of this process, including maximum sequence length and normalisation, rather than the internal training dynamics[3]. It is therefore accurate to say that the model is trained or constructed so that contextually similar inputs map to nearby vectors, but inaccurate to claim that the sources demonstrate why any particular pair of words must be close[3].

For the hypothetical milk notes, shared tokens and shared topical context provide a plausible basis for proximity. The basil note, by contrast, shares neither the salient tokens nor the topic. Whether that plausibility is realised depends on the specific model and on whether chunking preserves the relevant context[2][3]. The sources support the pipeline and the interpretive goal; they do not support a model-independent guarantee[3].

Once text has become vectors, the system needs a way to measure closeness. One common measure is cosine similarity, which compares the direction of two vectors rather than their length[3]. Imagine two arrows pointing in almost the same direction: their cosine similarity is high, even if one arrow is longer than the other. If the arrows point in different directions, the similarity is lower[3].

Cosine similarity requires the vectors to be normalised, meaning they are rescaled to a standard length before comparison[3]. The sources show this being set explicitly when cosine similarity is chosen, with embeddings normalised to unit length[3]. Without normalisation, a longer vector could appear more similar to many others simply because of its size, which is usually not what a search system wants[3].

A different measure, the dot product, takes vector length into account. The sources note that this can have the undesirable effect of making a longer vector seem more similar to everything else[3]. For our milk example, cosine similarity would focus on whether the query vector points in a similar direction to the milk-note vectors, rather than on which vector happens to be longest[3].

Cosine similarity is defined as the cosine of the angle between two vectors and is invariant to positive rescaling after normalisation[3]. The sources state that it allows comparison of vector directions regardless of magnitude and that it requires normalising all vectors to unit norm[3]. This invariance is a deliberate modelling choice: it discards magnitude information in favour of directional alignment. Whether magnitude carries useful semantic signal is therefore decided by the choice of distance, not by the embedding alone[3].

Dot product, by contrast, conflates direction and magnitude. The sources warn that increasing a vector's length makes it more similar to all others under dot product[3]. In a retrieval system, this can bias results toward long or high-norm embeddings irrespective of topical relevance. The tutorial in the sources selects cosine similarity because the particular model works well with it, and sets normalisation accordingly in both the embedding model and the index[3].

The choice of distance is thus not neutral. It determines which geometric property counts as similarity. For the hypothetical query about oat milk, cosine similarity would rank chunks by directional alignment with the query vector. A dot-product system might rank a chunk highly because its vector is long, even if its direction is less aligned. The sources support this distinction and its practical consequences[3].

After chunks are embedded, their vectors are stored in a vector database together with the original text and any useful metadata, such as where the chunk came from[3]. When a query arrives, it is embedded by the same model and compared with the stored vectors. The system returns the closest chunks, often a small number set by a parameter called top-k[3]. In our example, a query about oat milk should return the milk notes before the basil note if the vectors are well behaved[3].

The number of results matters. Retrieving more chunks, a larger top-k, increases the chance that a relevant chunk is included, but it also brings in more noise[3]. Retrieving too much text can overwhelm the next stage of the system, which has to read the retrieved chunks and produce an answer[3]. The sources therefore treat top-k and chunk size as tuning choices that balance relevance against overload[3].

Metadata is also important. Keeping track of the source of each chunk makes it possible to show where a retrieved note came from and to filter results[3]. For the fridge-door example, metadata might record which note each chunk came from, so that the system can present the original wording rather than only a list of numbers[3].

The documented architecture consists of an embedding step, a vector store, and a similarity search operation[3]. The tutorial describes the retriever as an internal search engine that returns relevant snippets for a query, with top-k controlling how many snippets are returned[3]. Chunk size and top-k interact: smaller chunks increase the number of candidates and the importance of ranking, while larger top-k values increase recall at the cost of precision and downstream context load[3].

The sources also note that the summed length of retrieved documents should not be excessive, citing the lost-in-the-middle phenomenon as a reason to limit total retrieved length[3]. This is a constraint on the reader stage, not on the embedding stage, but it feeds back into retrieval design. A system that retrieves many long chunks may satisfy the embedding model's input limit while still overwhelming the model that must use the results[3].

Metadata preservation is a further architectural requirement. The tutorial emphasises that the chosen library allows document metadata to be kept throughout processing[3]. Without metadata, a retrieved vector cannot be traced back to its source chunk, which undermines both user trust and debugging. In the hypothetical example, source metadata is what allows the system to display the milk note text rather than an opaque vector[3].

Let us walk through the hypothetical example with explicit assumptions. Assume, purely for illustration, that each note is one chunk, that the embedding model accepts short inputs, and that cosine similarity with normalised vectors is used[3]. The three chunks are therefore the three notes themselves. Each is embedded into a fixed-length vector, and the three vectors are stored with their original text[3].

Now assume a query: "Where is the oat milk?" The query is embedded by the same model[3]. Under our hypothetical assumption that the model captures shared context, the query vector should point in a direction closer to the two milk-note vectors than to the basil-note vector. A similarity search with top-k set to two would then return the two milk notes[3]. This is a hypothetical outcome, not a measured result from any real model.

The example also shows what could go wrong. If the notes had been split badly, or if a different distance measure had been used, the ranking could change[2][3]. If the query were "What should I water?" the basil note should rank first instead. The same pipeline can therefore produce different rankings for different queries, which is exactly what a retrieval system is supposed to do[3].

The worked example proceeds as follows under stated hypothetical assumptions: chunking is trivial because each note is a single sentence; embedding uses a single model for both corpus and query; vectors are normalised; and cosine similarity defines the ranking[3]. The corpus vectors are stored in a vector database, the query vector is computed at search time, and similarity search returns the nearest neighbours[3]. The expected ranking under these assumptions is milk notes first for a milk query and the basil note first for a watering query[3].

This example is illustrative rather than evidential. The sources do not report an experiment on these invented notes, and no claim is made that any particular model would produce this ranking[3]. The value of the example is that it makes the pipeline's dependencies explicit: chunk boundaries, model choice, normalisation, distance, and top-k each affect the result[2][3]. Changing any one of them could change the ranking, which is why the sources treat these parameters as tuning decisions rather than fixed constants[3].

A further assumption deserves emphasis: the example assumes that the embedding model encodes topical similarity in vector direction. The sources support this as the intended behaviour of the pipeline[3], but they also document that chunk size, overlap, and model limits affect whether relevant context is preserved[2]. If chunking severed the phrase "oat milk" from its context, the resulting vectors could be misleading even with a well-chosen distance measure[2].

Closeness in vector space is not the same as sameness of meaning. Consider word order: "The cat sat on the mat" and "The mat sat on the cat" contain the same words but describe different scenes. Whether an embedding model distinguishes them depends on the model, and the sources do not guarantee that every model will[3]. A retrieval system might return both for a query about cats and mats, even though only one is correct[3].

Negation is a sharper problem. "Buy oat milk" and "Do not buy oat milk" share most of their words and topic, so their vectors may be close even though the required actions are opposite[3]. The sources describe embeddings as representing meaning for retrieval, but they do not show that a small word such as "not" will always move a vector far enough to change the ranking[3]. A system that relies only on closeness could therefore retrieve a note that says the opposite of what the user wants.

A new topic creates a different kind of problem. If a query introduces a topic that is not represented in the stored chunks, the closest vector may still be returned even though none of the chunks is truly relevant[3]. The system always returns the nearest neighbours, whether or not they are near in an absolute sense. For the fridge-door example, a query about car repairs would still return one of the three notes as the closest, even though none of them answers the question[3].

Word order sensitivity is a property of the encoder's architecture and training objective. The sources document that token sequences are embedded and that maximum sequence length constrains input[3], but they do not establish that any given model preserves order information in a way that reliably separates anagrams or role reversals. It is therefore unsafe to infer from the existence of an embedding pipeline that order-dependent meaning will be preserved[3].

Negation presents a related but distinct challenge. Because negation often involves a small number of tokens with large semantic consequences, a model that weights topical content heavily may place affirmative and negative statements close together. The sources support the general claim that embeddings represent meaning for retrieval[3], but they provide no evidence that negation is always encoded with sufficient magnitude to affect ranking. The correct conclusion is conditional: negation may or may not be preserved, and this must be tested for the specific model in use[3].

Topic shift exposes the limits of nearest-neighbour retrieval. Similarity search returns the closest stored vectors regardless of whether any vector is close in an absolute sense[3]. The sources describe top-k as controlling how many snippets are returned, not as a relevance threshold[3]. A system that always returns top-k results will therefore produce results even for out-of-scope queries. Mitigations such as relevance thresholds or reranking are mentioned in the sources as possible enhancements, but they are not guaranteed by the basic pipeline[3].

There is no single best chunk size, overlap, or top-k value for every dataset. The sources repeatedly emphasise that these choices depend on the content, the models being used, and the task[2][3]. A system that stores short notes needs different settings from one that stores long reports. The sensible approach is to start with a reasonable default, test it on representative examples, and adjust based on what the retrieval actually returns[2].

A useful comparison is between fixed-size chunking and structure-based chunking. Fixed-size chunking is simple and predictable, and it guarantees that chunks stay within a size limit[2]. Structure-based chunking respects paragraphs, sentences, or headings, which can preserve meaning better when the text has clear structure[1]. For the milk notes, structure-based chunking is natural because each note is a sentence. For a long document with headings, structure-based chunking may be preferable, while for unstructured text a fixed size may be simpler[1][2].

Here is a practical checklist for the hypothetical fridge-door system. First, confirm that each chunk fits the embedding model's token limit[2]. Second, use the same embedding model and normalisation for stored chunks and queries[3]. Third, choose cosine similarity deliberately and normalise vectors if that measure is used[3]. Fourth, set top-k small enough to avoid overwhelming the reader, but large enough to include relevant chunks[3]. Fifth, test with negation, word-order, and out-of-topic queries, and inspect the actual rankings rather than assuming closeness means correctness[3].

Parameter selection must be grounded in model constraints and task requirements. Chunk size must respect the embedding model's maximum token input, which the sources illustrate with a specific limit of 8,191 tokens for one model[2]. Overlap, chunking strategy, and top-k should be tuned against representative queries, because the sources state that optimal values vary with content type and use case[2][3]. The tutorial's warning that too-small chunks split sentences and lose meaning, while too-large chunks mix ideas, captures the central trade-off[3].

A comparison of chunking strategies clarifies the trade-off. Fixed-size chunking offers predictable length control and simple implementation[2]. Variable-size and structure-based chunking preserve sentence, paragraph, or heading boundaries, which can maintain semantic coherence[1][2]. Semantic chunking aims to preserve context across sentences and paragraphs[2]. None of these strategies is universally superior; the sources present them as options whose suitability depends on document structure and downstream use[1][2].

The checklist for a defensible implementation is therefore as follows. Measure token counts with the relevant tokenizer and ensure chunks fit the model limit[2]. Apply identical embedding and normalisation to corpus and query[3]. Select the distance measure explicitly and understand its invariance properties[3]. Tune chunk size, overlap, and top-k on representative data, including adversarial cases involving negation, word order, and topic shift[3]. Finally, retain source metadata for every chunk so that retrieved results can be inspected and audited[3].