A SHARED VOCABULARY

Words behind the words.

Short definitions to get your bearings. Follow a term into a complete explanation.

Find a term

254 terms

Absolute position #: A location measured from a sequence’s beginning, such as position 12. It differs from the distance between two tokens.
Abstention #: Declining to supply an unsupported answer when the available evidence is insufficient. Its usefulness depends on when the system abstains and when it can answer.
Activation #: A value computed while processing an input. Activations depend on the current sequence; parameters persist across ordinary requests.
Activation function #: A function applied within a network to transform a computed value. Nonlinear activation functions let stacked layers represent more than a single linear mapping.
API #: Application programming interface: a defined way for software components to request operations or exchange data. Having a language model does not automatically provide access to an API.
Architecture #: The arrangement and rules of a model’s computation, including its layers and connections. A checkpoint supplies particular parameter values for that arrangement.
Arithmetic parser #: A program that reads a restricted mathematical expression according to explicit rules. The calculator lab parses numbers and operators instead of executing user text as program code.
Artificial intelligence #: The broad field of building systems that perform tasks such as recognizing patterns, using language, or making decisions. The label does not establish human-like understanding.
Assistant message #: A message assigned to the assistant role in a conversation format, often containing a previous response or the response being generated.
Attention #: A mechanism that forms weighted combinations of value vectors using compatibility scores between queries and keys.
Attention matrix #: A table relating query positions to key positions. Its entries may be raw scores or normalized weights, depending on the table’s label.
Attention score #: A compatibility score between a query and key before softmax. It is not yet a normalized attention weight or a measure of factual truth.
Attention weight #: A coefficient describing how much an available value vector contributes to an attention mixture. These input-dependent coefficients are different from the model’s persistent learned weights.
Automatic differentiation #: Calculating derivatives by applying differentiation rules to the operations a program executes. Backpropagation is a reverse-mode form of this process.
Autoregressive #: Generating a sequence by conditioning each next prediction on the tokens available before it.
AWQ #: Activation-aware Weight Quantization: a weight-only method that uses activation statistics to guide which weight channels need protection from quantization error.
Backpropagation #: An efficient use of the chain rule to calculate how a loss changes with the model’s parameters.
Backward pass #: Propagating derivatives backward through a computation to obtain gradients. An optimizer must still apply an update to change the parameters.
Base model #: A pretrained model before a particular assistant-oriented post-training stage; it is not synonymous with a complete chat product.
Batch #: A set of training examples whose losses or gradients are combined for an update.
Batch size #: The number of examples or requests processed together in a batch. Its effect on speed and memory depends on the workload and implementation.
Batching #: Processing several examples or requests together. It can improve hardware utilization while changing memory use and how long an individual request waits.
Benchmark #: A specified evaluation dataset and procedure. Its scope determines what a score can support.
Bias #: An adjustable offset added to a weighted sum in a neural-network calculation. This mathematical meaning differs from social or statistical bias in a model’s behavior.
Bidirectional attention #: Attention that can use positions on both sides of the current position. Without another restriction, it is unsuitable for hiding future targets in ordinary next-token training.
Bigram #: A pair of consecutive tokens. A bigram language model predicts the next token using only the immediately preceding token.
Bit #: A binary unit with two possible values. Bit width describes how many bits a representation uses, not how many facts a model knows.
Bit width #: The number of bits used for a stored numeric value or representation. Lower bit width does not by itself specify total memory use or model quality.
Byte #: A unit of eight bits used to represent data. A written character can require more than one byte in a text encoding.
Byte fallback #: Representing an otherwise unsupported text piece using its encoded bytes. This is a tokenizer design choice, not a rule that all tokenizers behave identically.
Byte-pair encoding #: A family of tokenization methods that learns merges of frequently adjacent symbols. Exact preprocessing and vocabulary vary.
Causal attention #: Self-attention restricted to the current and earlier input positions. This prevents future input tokens from supplying an answer during next-token training.
Causal mask #: A restriction preventing a position from attending to later positions in a causal decoder.
Chain rule #: A rule for differentiating a composition: combine derivatives through the intermediate operations. Backpropagation applies it efficiently across a computation graph.
Chat template #: Rules for converting role-labeled messages into the token sequence a chat model expects, including separators and special markers.
Checkpoint #: Saved model parameters and sometimes optimizer state, allowing a run or model to be resumed or used.
Computation graph #: A record of numerical operations and their dependencies. It shows how outputs depend on intermediate values, inputs, and parameters.
Compute #: The computational work or resources used to train or run a model. Arithmetic, memory capacity, and data movement are different constraints.
Compute-optimal training #: Allocating a fixed training-compute budget among choices such as parameter count and training tokens to optimize a stated training objective. This need not minimize deployment cost.
Conditional probability #: The probability of an outcome given specified information. Next-token probabilities are conditioned on the available prefix.
Context #: The information available in the current model input and generated prefix. It is different from information encoded in the model’s learned parameters.
Context management #: Choosing which instructions, messages, and supporting material to supply within the available context budget. Removal, summarization, and retrieval have different tradeoffs.
Context window #: The bounded sequence available to a model computation. It is distinct from knowledge encoded in parameters.
Coordinate #: One component of a vector, such as the first number in [0.2, 0.8]. A coordinate does not automatically correspond to a human-readable concept.
Corpus #: A collection of text or other material assembled for training, evaluation, or analysis.
Cosine #: A function of an angle. For two nonzero vectors, the cosine of the angle between them measures directional alignment: 1 for the same direction, 0 for perpendicular directions, and −1 for opposite directions.
Cosine similarity #: The dot product of two nonzero vectors divided by the product of their lengths; a measure of directional alignment.
Cross-entropy #: A loss that, for a single observed target token, is minus the logarithm of its predicted probability.
Data contamination #: Evaluation information appearing in training or other unintended parts of the evaluation setup.
Data leakage #: Information reaching a training or evaluation step where it should be unavailable, such as test answers appearing in training data. It can make a result misleading.
Data mixture #: The proportions of different sources or categories encountered during training. Sampling rules can change the mixture without changing the stored files.
Dataset #: A defined collection of examples used for training, evaluation, or another task. Its selection and organization affect what a result can establish.
Decode phase #: The phase that repeatedly generates additional tokens from the available context. Its timing differs from processing the initial prompt.
Decoder-only transformer #: A transformer design that uses causal attention to generate continuations from preceding tokens, without a separate encoder stack in the basic architecture.
Decoding #: The procedure that selects output tokens from model scores, such as greedy selection or sampling.
Deduplication #: Removing repeated or sufficiently similar data according to a chosen procedure.
Deep learning #: Machine learning with neural networks that compose many layers of transformations. More layers alone do not guarantee better performance.
Derivative #: A measure of how a function changes with a small change to an input. Training uses derivatives to calculate how parameters influence loss.
Dimension #: For a vector, the number of coordinates it contains. Model and head dimensions specify how many numbers each representation uses.
Distribution shift #: A change between the examples or conditions used during development and those encountered later. Performance can change even when the model weights stay fixed.
Document index #: A searchable structure built from a document collection. Updating it can change retrieved evidence without retraining the language model.
Dot product #: Multiply corresponding coordinates of two equal-length vectors, then add the results. It depends on both their lengths and how their directions align.
DPO #: Direct Preference Optimization: a preference-based objective that can train a policy without a separate explicit reward-model stage.
Embedding #: A numerical vector associated with an item, such as a token, and learned or supplied for a task.
Embedding retrieval #: Finding candidate material by comparing learned vector representations of the query and documents. Similarity alone does not prove that a passage supports an answer.
Embedding table #: A matrix with an embedding vector for each vocabulary entry. A token ID selects the corresponding row.
End token #: A special token used to signal that generation should stop for a sequence. A separate length limit can also stop generation.
End-to-end evaluation #: Testing the complete workflow from the request through retrieval or tools to the final answer, rather than testing only one component with ideal inputs.
Epoch #: One pass through a defined training dataset; large training runs do not always organize all data into repeated complete passes.
Evaluation #: Measuring a model or complete system against defined examples, tasks, and criteria. Results support claims about the measured setting rather than every possible use.
Exponential #: The function exp(x), or e raised to the power x. Softmax exponentiates scores before normalizing them into probabilities.
External memory #: Information retained outside the model’s weights, such as saved conversation records or documents retrieved for a later request. The application must explicitly supply it again.
Feed-forward layer #: A transformation applied to each position, often expanding a representation, applying a nonlinearity, and projecting back.
Few-shot prompting #: Providing a small number of examples in the current prompt to demonstrate a task. This can guide behavior without updating model parameters.
Fine-tuning #: Further training of an existing model on a selected objective or dataset.
FlashAttention #: An exact attention implementation that reorganizes computation to reduce memory traffic and avoid storing the full attention matrix in the usual way. It does not make the whole model linear-time.
Floating-point arithmetic #: Arithmetic using a finite numerical format that can approximate a wide range of magnitudes. Rounding means some results differ slightly from exact real-number arithmetic.
Floating-point operation #: An arithmetic operation on floating-point numbers. FLOPs count work; FLOP/s measures how many operations are performed per second.
Forward pass #: Computing intermediate values and outputs from an input using the model’s current parameters. It does not by itself update those parameters.
Gated linear unit #: A neural-network component that multiplies one transformed branch by another gating branch. Variants are used in some transformer feed-forward layers.
Generalization #: Performance on examples or settings beyond those directly learned during training; it must be tested rather than assumed.
Generation #: Producing output tokens from a model. Ordinary autoregressive generation repeatedly predicts a distribution, selects a token, and adds it to the context.
GPTQ #: A post-training weight-quantization method using approximate second-order information to choose a compressed representation. Its quality and speed depend on the model and runtime.
GPU #: Graphics processing unit: a processor suited to running many numerical operations in parallel. GPUs are widely used for neural-network training and inference.
Gradient #: The collection of derivatives indicating how a scalar objective changes as its inputs or parameters change.
Gradient descent #: Updating parameters by subtracting a learning-rate-scaled gradient to seek lower loss. A finite step is not guaranteed to improve every example.
Greedy decoding #: Choosing a highest-scoring token at each step. It does not optimize every property of the whole resulting sequence.
Grounding #: Connecting an answer to supplied evidence or another verifiable source. A citation is useful only when its source actually supports the associated claim.
Grouped-query attention #: An attention design in which groups of query heads share key-value heads. It offers a sharing pattern between separate per-query key-value heads and a single shared set.
Hallucination #: A generated claim that is unsupported or false in the relevant context, even if phrased fluently.
Head #: One attention computation with its own projections, combined with other heads in multi-head attention.
Head dimension #: The number of coordinates in one head’s query, key, or value representation. Cache estimates must use the dimensions actually stored.
Held-out data #: Examples excluded from the parameter updates being evaluated. They help measure performance beyond those training examples, although leakage can compromise the separation.
Hidden state #: An intermediate vector computed while processing an input. Unlike a saved parameter, its value depends on the current sequence and layer.
Host application #: The software around a model that supplies inputs, handles tool requests, and presents results. It must enforce permissions rather than rely on generated text as authorization.
Identity projection #: A transformation that leaves its input vector unchanged. Its identity matrix has ones on the diagonal and zeros elsewhere; the teaching pipeline uses it for inspectable attention calculations.
In-context learning #: Behavior guided by examples or information in the input sequence, without requiring a parameter update.
Inference #: Computing outputs using a model; ordinary inference does not apply optimizer updates to its parameters.
Instruction tuning #: Fine-tuning on examples framed as instructions and desired responses to encourage the model to follow requests. It does not automatically connect the model to live information.
Inter-token latency #: The elapsed time between successive output tokens during generation. A system can start quickly yet generate the rest of an answer slowly.
Key #: A projected vector used to compute compatibility with a query in attention.
KV cache #: Temporary stored attention key and value tensors that can be reused during autoregressive generation.
KV head #: A distinct set of key and value projections used by attention. Several query heads can share it in grouped-query or multi-query designs.
Language model #: A model that assigns probabilities to language sequences or predicts language units from context. The models discussed here mainly predict the next token.
Large language model #: A language model with many learned parameters, usually trained on substantial amounts of text. There is no single parameter threshold that makes a model “large.”
Latency #: Elapsed time for an operation; time to the first output and time per subsequent token are different measurements.
Layer #: A stage of a neural network that transforms a representation. A transformer block contains several sublayers rather than a single operation.
Layer normalization #: Normalizing a layer’s representation using the mean and variance of its features, usually with learned scale and offset. It is different from making token probabilities sum to one.
Learning rate #: A setting controlling the scale of an optimizer update, alongside the optimizer’s other rules.
Lexical retrieval #: Finding material through matching words or other textual features, rather than learned embedding similarity. The retrieval lab uses a simple keyword score.
Likelihood #: The probability assigned to observed data, viewed as a function of model parameters. Training can seek parameters that make the observed examples more likely.
Linear transformation #: A mapping that preserves addition and scaling; matrix multiplication is a common example. Adding a bias makes the full mapping affine rather than strictly linear.
Log-probability #: The logarithm of a probability. For an autoregressive sequence, adding conditional token log-probabilities gives the sequence’s log-probability.
Log-ratio #: The logarithm of one positive quantity divided by another. DPO compares policy-to-reference probability ratios for preferred and rejected responses.
Logistic model #: A model that applies a sigmoid to a score to produce a probability for a binary outcome. The training lab uses a deliberately tiny example.
Logit #: An unnormalized score, often one per vocabulary item before softmax.
Longest-match tokenizer #: A tokenizer that repeatedly selects the longest matching vocabulary entry at the current text position. The site uses this simple rule for teaching; it is not BPE training.
Loss #: A scalar objective used to measure error for training or evaluation. Lower loss does not guarantee truthfulness.
Machine learning #: Building a model whose behavior is adjusted using data, rather than specifying every response as a hand-written rule.
Matrix #: A rectangular table of numbers. Multiplying a vector by a suitable matrix combines its coordinates to produce another vector.
Memorization #: Retention of training-specific information that may allow an example or passage to be reproduced.
Memory bandwidth #: The rate at which data can move to or from memory. A workload can be bandwidth-limited even when its processor can perform arithmetic quickly.
MLP #: Multilayer perceptron: a network of weighted transformations and nonlinearities; transformer feed-forward blocks are often MLPs.
Model #: A mathematical or computational system that maps inputs to outputs. In a trained language model, its calculations depend on numerical parameters learned from data.
Multi-head attention #: Running several attention computations with distinct projections, then combining their outputs. More heads do not automatically mean more independent facts or reasoning steps.
Multi-query attention #: An attention design in which multiple query heads share one set of key and value heads, reducing the amount of distinct key-value state.
Nat #: A unit of information or loss calculated using natural logarithms. A single-target loss of −ln(0.5) is about 0.693 nats.
Natural language processing #: Methods for computers to analyze, transform, or generate human language. Language modeling is one part of this wider field.
Natural logarithm #: The logarithm with base e. These loss examples use it to convert target probabilities into additive penalties.
Near-duplicate #: An example that differs in wording or formatting but substantially repeats another. Exact string matching can miss this overlap.
Negative log-likelihood #: A loss formed by taking minus the logarithm of the probability assigned to observed data. Lower values mean the model assigned greater probability to that data.
Neural network #: A model made of connected numerical transformations with adjustable parameters. Layers transform their inputs into intermediate representations and outputs.
Next-token prediction #: Estimating which token follows a given prefix. During training, the actual following token supplies the target; it need not be the only plausible continuation.
Nonlinearity #: An operation that is not a linear mapping. In a neural network, nonlinearities help the combined layers express more varied input-output relationships.
Normalization #: Putting values into a specified standard form. Probability normalization, text normalization, and layer normalization perform different operations; the surrounding context identifies which one is meant.
Numerical overflow #: A calculation exceeding the largest number representable in its numeric format. Directly exponentiating a very large logit can cause overflow.
Numerical precision #: How finely a numeric format represents values. Lower precision can save storage or computation but introduces different approximation errors.
Numerical stability #: Organizing a calculation to reduce errors from finite precision, overflow, or very small numbers. Subtracting the maximum before softmax is one example.
Optimizer #: An algorithm that updates parameters using gradients and, depending on the algorithm, additional state.
Optimizer state #: Extra values maintained by an optimizer, such as accumulated gradient statistics. They can require memory in addition to parameters and current gradients.
Output projection #: A transformation into the required output space. The final vocabulary projection produces logits; an attention block’s output projection instead combines head outputs.
Output reserve #: A portion of a request’s token budget set aside for the generated response. The shared-budget examples here are illustrative rather than a provider limit.
Overfitting #: Learning patterns specific to the training examples that do not transfer well to new examples or conditions. Low training loss alone cannot rule it out.
Parameter #: A persistent numerical value of a model, usually learned during training.
Parameter update #: An explicit change to model parameters, usually calculated by an optimizer from training signals. A different prompt can change outputs without such an update.
Partial derivative #: A derivative with respect to one input or parameter while holding the others fixed. A gradient collects these derivatives for several variables.
Perplexity #: The exponential of average token negative log-likelihood when natural logarithms are used. Compare it only under compatible data, tokenization, and averaging rules.
Policy #: The distribution a model uses to choose outputs or actions in a learning setup. This differs from a written safety rule or a context-retention policy.
Position embedding #: A vector used to represent a position in a sequence, often combined with a token embedding. It is one family of position mechanisms, not a universal design.
Position encoding #: Information that lets a model distinguish positions or relative positions in a sequence.
Post-training #: Further training after pretraining to shape behavior for selected tasks or preferences. It can include supervised fine-tuning and preference-based methods.
Post-training quantization #: Compressing an already-trained model into a lower-precision representation without running its full original training again. Here “post-training” describes when quantization happens, not assistant fine-tuning.
PPO #: Proximal Policy Optimization: a family of reinforcement-learning updates. Its clipped probability-ratio objective discourages overly large policy changes without hard-bounding every change.
Preference data #: Comparisons or judgments about which outputs are preferred under a specified task and rubric.
Preference learning #: Learning from judgments about which outputs are preferred under a task and rubric. Preference is contextual and does not automatically establish factual correctness.
Preference pair #: Two responses to a prompt with a judgment favoring one over the other. A chosen response can still contain errors.
Prefill #: Processing the supplied prompt before ordinary token-by-token decoding begins, including constructing the attention state needed for generation.
Prefix #: The sequence already available before the next token is predicted. It can contain supplied text and earlier generated tokens.
Pretraining #: An initial large-scale training stage, commonly using prediction objectives over broad data.
Probability calibration #: Agreement between predicted probabilities and observed frequencies on a specified task. A valid probability distribution is not automatically well calibrated.
Probability distribution #: Probabilities assigned to possible outcomes. For the discrete token distributions shown here, the probabilities are nonnegative and add to one.
Probability mass #: The sum of probabilities assigned to a group of discrete outcomes. Top-p sampling retains candidates containing a chosen amount of this mass.
Probability normalization #: Rescaling nonnegative values with a positive total so they add to one. After candidate filtering, renormalization makes the surviving probabilities form a distribution again.
Projection #: A transformation from one vector representation to another, often through a learned matrix. In transformer terminology it need not mean an orthogonal geometric projection.
Prompt #: The input supplied to a model for a request. It can include instructions, conversation text, examples, retrieved passages, and formatting the user does not directly see.
Prompt injection #: Instructions embedded in untrusted material that try to redirect a system away from its authorized task. Clear boundaries help but are not a complete defense by themselves.
Provenance #: Information about where a claim or piece of data came from, such as its document, date, and revision. It helps a reader check whether evidence is suitable.
Proxy #: A measurable stand-in for a broader goal. Optimizing a proxy can improve its score while missing something users actually care about.
Pseudorandom generator #: A deterministic algorithm that produces a random-looking sequence from an initial state. Reusing the seed repeats that sequence only when the algorithm and other conditions match.
Quantization #: Representing values with a smaller set of numerical levels, often to reduce storage and computation costs.
Quantization calibration #: Using representative examples to estimate which values or channels need care during quantization. This differs from checking whether predicted probabilities match observed frequencies.
Quantization error #: The difference between the reconstructed quantized value and its original value. The lesson’s examples state this sign convention explicitly.
Quantization step #: The spacing between adjacent reconstructed values in a uniform quantizer. Rounding to these levels introduces an approximation error.
Query #: A projected vector used to score which keys are compatible with the current attention position.
Query head #: One attention head’s query projection and resulting queries. Its count can differ from the number of distinct key-value heads.
RAG #: Retrieval-augmented generation: supplying retrieved material to a generation system as external evidence.
Reference policy #: A baseline model distribution used to compare or constrain the policy being trained. DPO uses its response probabilities as part of the preference objective.
Regularization #: Adding a constraint or penalty to influence which solutions training prefers. In preference training, a reference-based penalty can discourage large departures from a baseline policy.
Reinforcement learning #: Training a policy using rewards associated with its actions or outcomes. In language-model applications, tokens can be actions and completed responses can receive rewards.
Relative position #: The offset between two sequence positions, such as three tokens earlier. Position mechanisms can make this relationship available to attention.
ReLU #: Rectified linear unit: the activation function max(0, x). It leaves positive inputs unchanged and turns negative inputs into zero.
Residual connection #: An addition that lets a layer contribute an update to an existing representation.
Residual stream #: The running representation to which transformer sublayers add updates through residual connections. It contains input-dependent values rather than a separate set of saved model weights.
Residual update #: The change added to an existing representation through a residual connection. It can reinforce or cancel coordinates rather than only increase them.
Retrieval #: Finding relevant material in an external collection and supplying it to the system. Retrieving a document does not itself update model weights or guarantee a supported answer.
Retriever #: The component that selects candidate passages or records for a request. Its success must be checked separately from the model’s use of the retrieved material.
Reward #: A numerical training signal scoring an action or response under a specified objective. A higher reward does not necessarily mean greater truth or usefulness.
Reward hacking #: Increasing a measured reward by exploiting weaknesses in the scoring rule rather than achieving the intended goal.
Reward model #: A model trained to assign a score to outputs, often based on preference comparisons in an RLHF pipeline.
RLHF #: Reinforcement learning from human feedback; here, the reward-model-based family of post-training recipes.
Rotary position embedding #: RoPE: position-dependent rotations of query and key coordinates that make their attention interaction depend on relative position. It does not simply add a position vector to the input.
Sampling #: Drawing an output token from a distribution, potentially after temperature scaling or truncation.
Scalar #: A single number, rather than a vector or a table of numbers. A loss is commonly a scalar computed from many model outputs.
Scaled dot-product attention #: Compute query–key dot products, divide by the square root of key dimension, apply any mask and softmax, then form a weighted sum of value vectors.
Scaling law #: An empirical relationship between quantities such as model size, training data, compute, and measured loss. Its fitted range and assumptions limit what it predicts.
Score #: A number used to rank or measure something under a specified rule. A next-token logit, a retrieval score and an evaluation score measure different things and are not automatically probabilities.
Search query #: Text or another representation used to request matching material from a search system. This is different from an attention query vector.
Seed #: An initial state for a pseudorandom generator. It enables repeatability only when the rest of the computation is also held fixed.
Self-attention #: Attention whose queries, keys, and values come from representations of the same sequence. A mask can restrict which positions each query may use.
Self-supervised learning #: Training in which targets are derived from the supplied data itself, such as using the next token in a text sequence as the answer to predict.
Sigmoid #: The logistic function 1 / (1 + exp(−x)), which maps real numbers to values between zero and one. This transformation alone does not make a value a reliable probability.
Sinusoidal positional encoding #: A position representation built from sine and cosine functions at different frequencies. It was used in the original Transformer architecture.
Softmax #: A function that exponentiates scores and divides by their sum to obtain a probability distribution.
Special token #: A designated vocabulary entry used for a structural purpose, such as marking a boundary or the end of a sequence. Its meaning depends on the model’s format.
Stochastic gradient descent #: Gradient descent using an estimate from sampled training examples or a minibatch, rather than calculating the gradient over the entire dataset at every step.
Stopping rule #: A condition that ends generation, such as an end token or an output-length limit. Completing a fluent sentence does not itself stop every decoding loop.
Structured output #: Output organized according to a specified format or schema so software can interpret it. Correct formatting alone does not prove correctness or successful execution.
Subword #: A token unit smaller than a complete word, such as a common prefix or word fragment. A tokenizer can mix whole words and smaller pieces.
Summarization #: Replacing longer material with a shorter account of its relevant contents. Summaries can omit or distort details, so they are not lossless memory.
Supervised fine-tuning #: Further training on input–target examples, such as prompts paired with desired responses.
Synthetic data #: Examples generated by a program or another model instead of being directly collected from the original task setting. Such examples still need quality checks.
System message #: An application-supplied message used to express high-level instructions or context in a chat format. Role labels must be interpreted according to the system’s actual design.
Temperature #: A positive divisor applied to logits before softmax. Lower values concentrate probability on higher scores.
Tensor #: A numerical array that can have multiple axes. In model software, vectors, matrices, and batches of such arrays are represented as tensors.
Test set #: Examples reserved for evaluating a developed system. A credible test separates them from both training and the choices tuned to the test answers.
Text normalization #: Converting text to a specified consistent form before processing. Different normalization rules can change which character sequences a tokenizer receives.
Throughput #: The amount of work completed per unit of time, such as output tokens per second. High throughput across many requests does not guarantee low waiting time for one request.
tiktoken #: A byte-pair-encoding tokenizer library. Its encoding must match the model being studied; this site’s toy tokenizer does not reproduce its outputs.
Time to first token #: The delay before the first output token becomes available. It can include queuing and prompt processing, so it is not the same as total completion time.
Token #: One unit in a tokenizer’s vocabulary. It can be a word, part of a word, punctuation, bytes, or another symbol.
Token budget #: The allocation of token capacity to input material and any reserved output. Actual accounting depends on the model and system.
Token ID #: An integer label identifying a vocabulary entry. A larger ID does not mean greater importance or a more complex meaning.
Tokenization #: Converting text into the units and identifiers accepted by a model. The result depends on the tokenizer’s vocabulary and encoding rules.
Tokenizer #: The procedure that converts text into vocabulary entries and token IDs. Training a tokenizer and training a language model are distinct processes.
Tool call #: A structured request that a surrounding application may validate and execute; generating the request is not execution.
Tool result #: The data or error returned after a tool operation. A model’s claim that an operation ran is not a substitute for an actual result.
Tool use #: A workflow in which a model requests an operation and an application validates and executes it, then returns the result to the model.
Top-k #: A decoding restriction that retains a chosen number of highest-scoring candidate tokens before renormalizing.
Top-p #: Nucleus sampling: retain a smallest high-probability candidate set whose cumulative mass reaches a threshold, renormalize it, then sample. The number retained can vary.
Training #: Adjusting a model’s parameters using examples and an objective. Reading a prompt during ordinary inference does not itself perform this update.
Training data #: The examples used to update a model’s parameters. Their contents and sampling mixture influence what the model is exposed to.
Training objective #: The numerical goal used to guide parameter updates, such as reducing prediction loss. It is a measurable proxy for desired behavior, not every property a user may want.
Training target #: The observed or specified output against which a prediction is scored. In next-token training, it is the actual following token in the example.
Training-data extraction #: Attempting to elicit material from a model that was present in its training data. A failed attempt does not establish that every extraction method would fail.
Transformer #: A neural-network architecture using attention and other transformations to process sequences.
Transformer block #: A repeated part of a transformer containing attention, feed-forward transformations, and connecting operations such as residual additions and normalization. Exact arrangements vary.
Transition probability #: The probability of moving to a particular next token or state from the current one. The generator lab supplies these probabilities in a fixed table.
Truncation #: Shortening input by removing selected content. It can discard an important instruction or fact, so which content is removed matters.
Unicode #: A standard for representing characters and text across writing systems. Its characters, code points, and encoded bytes are different units.
Unicode code point #: A numbered position in Unicode. One visible character or emoji can be represented by several code points, so code points and visible characters are not interchangeable.
Uniform distribution #: A distribution giving the same probability to every candidate in a finite set. A uniform random draw over an interval is its continuous counterpart.
User message #: A message representing the user’s input in a conversation format. The complete model input may contain additional messages and context.
Validation set #: Held-out examples used to make development choices, such as selecting settings or checkpoints. Repeatedly tuning against them can make them less independent as a final test.
Value #: A projected vector mixed using attention weights.
Vector #: An ordered list of numbers used to represent an item or an intermediate result. A two-dimensional drawing is only a small example of this representation.
Vector norm #: A measure of vector length. The cosine-similarity examples use the Euclidean norm: the square root of the sum of squared coordinates.
Vocabulary #: The finite set of token IDs recognized by a tokenizer and its corresponding model.
Weight #: A number that scales a contribution in a computation. Model weights are learned parameters; attention weights are input-dependent coefficients computed during a forward pass.
Weight tying #: Reusing the same learned parameter array for more than one role, such as input embeddings and the final vocabulary projection.
Weight-only quantization #: Quantization applied to model weights while other quantities may use higher precision. It reduces weight storage without necessarily shrinking every part of the workload.
Weighted sum #: Multiply each number or vector by its assigned weight, then add the results. Attention uses this operation to mix value vectors.
Workflow #: An ordered set of steps performed by an application, such as receiving a question, retrieving information, calling a tool and presenting a response. The workflow includes more than the underlying model.
Zero vector #: A vector whose coordinates are all zero. It has no direction, so cosine similarity with it is undefined.
Zero-shot prompting #: Asking for a task without supplying worked demonstrations of that task in the current prompt. The model still brings its prior training.

Words behind the words.

No matching terms.