AI & ML Glossary

Specialized computing hardware designed to perform specific types of computations more efficiently than general-purpose processors, particularly for AI and machine learning workloads.

A fundamental evaluation metric measuring the proportion of correct predictions made by a machine learning model out of all predictions, providing a basic measure of model performance.

A mathematical function applied to neural network outputs that introduces non-linearity, enabling networks to learn complex patterns and relationships.

An Activation Function is a mathematical function applied to neural network nodes to determine their output, introducing non-linearity and enabling networks to learn complex patterns.

Agentic AI refers to artificial intelligence systems designed to act autonomously, make decisions, and pursue goals with minimal human supervision, representing the next evolution in AI capabilities.

AI Agents are autonomous systems that perceive their environment, make decisions, and take actions to achieve specific goals using artificial intelligence capabilities.

Intelligent software agents that help users complete tasks through natural language interaction and automated reasoning capabilities.

AI Automation is the use of artificial intelligence technologies to automatically perform tasks, make decisions, and execute processes that traditionally required human intervention.

Artificial General Intelligence refers to AI systems with human-level cognitive abilities across all domains, capable of understanding, learning, and applying intelligence as flexibly as humans.

Artificial Intelligence (AI) is the simulation of human intelligence processes by machines, especially computer systems, enabling them to perform tasks that typically require human cognitive abilities.

Hypothetical AI that surpasses human intelligence in all domains, representing the ultimate goal and potential risk of AI development.

Attention Mechanism is a neural network technique that allows models to focus on relevant parts of input data, improving performance on sequence-to-sequence tasks.

An Autoencoder is an unsupervised neural network architecture that learns efficient data representations by compressing input data and then reconstructing it.

Backpropagation is a supervised learning algorithm used to train neural networks by calculating gradients of the loss function with respect to network weights through backward pass computation.

The maximum rate of data transfer across a communication channel or system component, typically measured in bits per second, determining the throughput capacity of networks, memory, and storage systems.

Batch Normalization is a technique that normalizes layer inputs by adjusting and scaling activations, improving training stability and enabling faster convergence in deep neural networks.

BERT (Bidirectional Encoder Representations from Transformers) is a breakthrough language model that revolutionized NLP through bidirectional context understanding.

High-speed storage that temporarily holds frequently accessed data closer to processing units, reducing latency and improving system performance by minimizing access to slower storage systems.

A Chatbot is an AI-powered conversational agent that simulates human-like dialogue through text or voice interactions, providing automated responses to user queries.

CNN (Convolutional Neural Network) is a deep learning architecture specialized for processing grid-like data such as images, using convolutional layers to detect spatial patterns and features.

Software that translates high-level machine learning model descriptions into optimized, executable code for specific hardware platforms, enabling efficient AI model deployment.

Computer Vision is a field of AI that trains computers to interpret and understand visual information from the world, enabling machines to identify objects, faces, and scenes in images and videos.

The surrounding information that provides meaning and relevance to a particular element, fundamental to how AI models understand and process information.

Advanced AI technology that enables natural, human-like dialogue through sophisticated language understanding and generation capabilities.

Convolutional Neural Networks are deep learning architectures designed for processing grid-like data such as images, using convolutional layers to detect spatial patterns and features.

An independent processing unit within a CPU or GPU that can execute instructions concurrently with other cores, enabling parallel computation and improved performance in multi-threaded applications.

Learn about Cosine and its applications in artificial intelligence and machine learning.

A reasoning technique that encourages language models to break down complex problems into intermediate steps, showing their work step-by-step to arrive at more accurate and explainable solutions.

Central Processing Unit, the primary general-purpose processor in computing systems that executes instructions and coordinates system operations, including AI and ML tasks.

A measure of the difference between two probability distributions, widely used as a loss function in machine learning classification tasks.

Directed Acyclic Graph, a mathematical structure used in computer science and data processing to represent workflows, dependencies, and computational graphs where nodes represent tasks or operations and directed edges represent dependencies without cycles.

A collection of structured data used to train, validate, and test machine learning models, containing examples, labels, and features relevant to specific AI tasks.

A neural network component that generates output sequences from encoded representations, essential in language models, machine translation, and generative AI systems.

Deep Learning is a subset of machine learning that uses multilayered neural networks to model and understand complex patterns in data, mimicking the human brain's information processing.

Deepfakes are AI-generated synthetic media—images, video, or audio—in which a person's likeness or voice is convincingly replaced with someone else's using deep learning techniques.

Generative AI models that create high-quality images, audio, and other content by learning to reverse a gradual noise addition process.

In artificial intelligence, particularly in machine learning and natural language processing (NLP), dimensions refer to the number of numerical values (features) in a vector embedding.

A mathematical measure of how far apart two objects, points, or vectors are in a given space, fundamental to many machine learning algorithms.

Learn about Dot Product and its applications in artificial intelligence and machine learning.

Dropout is a regularization technique that randomly sets a fraction of input units to zero during training to prevent overfitting and improve generalization in neural networks.

Edge AI is the deployment of artificial intelligence algorithms and models directly on local devices at the network edge, enabling real-time processing without relying on cloud connectivity.

A scalar value that represents the factor by which an eigenvector is scaled when a linear transformation is applied, fundamental to understanding matrix behavior, system stability, and dimensionality reduction.

A non-zero vector that, when a linear transformation is applied to it, changes only by a scalar factor, fundamental to understanding linear transformations, dimensionality reduction, and matrix analysis.

Embeddings are dense vector representations that capture semantic meaning and relationships between words, sentences, or other data types in a continuous mathematical space.

A neural network component that transforms input data into meaningful representations, typically used in sequence-to-sequence models and transformers.

Ensemble Learning is a machine learning technique that combines multiple models to create a stronger predictor than any individual model, improving accuracy and robustness through model diversity.

A measure of uncertainty, randomness, or information content in a probability distribution, fundamental to information theory and machine learning.

Learn about Euclidean and its applications in artificial intelligence and machine learning.

A classification metric that combines precision and recall into a single score using their harmonic mean, providing a balanced measure of model performance.

An individual measurable property or characteristic of observed data that serves as input to machine learning models for training and prediction.

Federated Learning is a distributed machine learning approach that trains models across decentralized devices or servers holding local data samples, without centralizing the data.

Neural network architectures and layers where information flows in one direction from input to output, without cycles or feedback loops.

Fine-tuning is the process of adapting a pre-trained AI model to a specific task or domain by training it on additional task-specific data.

A Float32Array is a typed array used to store 32-bit floating point numbers. In AI, it’s often used to store vectors (e.g., embeddings) in memory-efficient formats.

Float64Array stores 64-bit floating point numbers, allowing for double precision. It’s used when high numerical accuracy is more important than saving memory.

Floating-Point Operations Per Second, a measure of computational performance indicating how many floating-point arithmetic operations a processor can execute per second.

Foundation models are large-scale AI models trained on broad datasets that serve as the foundation for multiple downstream applications through adaptation and fine-tuning.

Foundation Models are large AI models trained on broad data that serve as the base for adapting to various downstream tasks across multiple domains and applications.

GAN (Generative Adversarial Network) is a machine learning architecture where two neural networks compete to generate realistic synthetic data through adversarial training.

Gaussian Error Linear Unit, a smooth activation function that weights inputs by their percentile in a Gaussian distribution, widely used in transformers.

Generative AI is a category of artificial intelligence that can create new, original content including text, images, audio, code, and other media by learning patterns from existing data.

GPT (Generative Pre-trained Transformer) is a family of large language models that uses transformer architecture to generate human-like text through autoregressive prediction.

Graphics Processing Unit, a specialized parallel computing processor essential for training and inference of deep learning models and AI applications.

Gradient Descent is a fundamental optimization algorithm used in machine learning to minimize cost functions by iteratively moving in the direction of steepest descent of the gradient.

Safety mechanisms and constraints implemented in AI systems to prevent harmful, inappropriate, or undesired behaviors, ensuring responsible and ethical AI operation within defined boundaries.

Hallucination in AI refers to when language models generate plausible-sounding but factually incorrect, nonsensical, or fabricated information not supported by training data or reality.

Learn about HNSW (Hierarchical Navigable Small World) and its applications in artificial intelligence and machine learning.

Open-source platform and community providing pre-trained AI models, datasets, and tools for natural language processing and machine learning.

Learn about Index and its applications in artificial intelligence and machine learning.

An index is a searchable collection of vectors in a vector database like Pinecone, Weaviate, or Cloudflare Vectorize. It's used to efficiently retrieve similar items using similarity search or nearest neighbor search.

Inference in AI refers to the process of using a trained model to make predictions, generate outputs, or draw conclusions from new input data without further training.

Learn about IVF (Inverted File Index) and its applications in artificial intelligence and machine learning.

Techniques used to circumvent AI safety measures and guardrails, attempting to manipulate AI systems into producing prohibited, harmful, or unintended outputs by exploiting vulnerabilities in their design.

A fundamental computational function or algorithm that performs specific operations in machine learning and computing, particularly in neural networks and parallel processing contexts.

A Large Language Model (LLM) is an advanced AI system trained on vast amounts of text data to understand, generate, and manipulate human language with remarkable sophistication.

A fundamental building block of neural networks where groups of neurons process input data through learned transformations before passing results to the next layer.

Large Language Model Meta AI, a family of foundation language models developed by Meta AI that ranges from 7B to 65B parameters, designed to be efficient, performant, and more accessible for research and development.

A Large Language Model (LLM) is a type of artificial intelligence trained on vast amounts of text data to understand and generate human language.

Large Multimodal Models are AI systems capable of understanding and generating content across multiple modalities like text, images, audio, and video.

Raw, unnormalized prediction scores output by neural networks before applying activation functions, representing the model's confidence in different possible outputs.

Parameter-efficient fine-tuning technique that adapts large language models by training only small rank decomposition matrices.

A Loss Function is a mathematical function that measures the difference between predicted and actual values, guiding neural network training by quantifying prediction errors.

Language Processing Unit, a specialized chip architecture designed specifically for efficient inference of large language models and transformer architectures.

Learn about LSH (Locality-Sensitive Hashing) and its applications in artificial intelligence and machine learning.

LSTM (Long Short-Term Memory) is an advanced recurrent neural network architecture designed to learn long-term dependencies in sequential data by solving the vanishing gradient problem.

Machine Learning (ML) is a subset of artificial intelligence that enables computers to learn and improve from experience without being explicitly programmed for every task.

A rectangular array of numbers, symbols, or expressions arranged in rows and columns, fundamental to linear algebra and essential for representing transformations, data, and computations in machine learning and scientific computing.

Memory Bandwidth Utilization, a performance metric measuring how effectively a computing system uses its available memory bandwidth when executing machine learning workloads.

Open standard that enables secure, controlled interactions between AI applications and external data sources and tools.

Physical storage components that hold data and instructions for immediate access by processors, including various types of volatile and non-volatile memory technologies used in computing systems.

Model FLOPs Utilization, a metric measuring how efficiently a computing system utilizes its theoretical peak floating-point performance when running machine learning models.

Mixture of Experts is a machine learning architecture that uses multiple specialized models (experts) with a gating mechanism to dynamically route inputs to the most relevant experts for processing.

Learn about ML (Machine Learning) and its applications in artificial intelligence and machine learning.

A mechanism that runs multiple attention functions in parallel, allowing models to capture different types of relationships and dependencies simultaneously.

Multimodal AI refers to artificial intelligence systems that can process, understand, and generate content across multiple types of data including text, images, audio, and video.

Natural Language Processing (NLP) is a branch of AI that focuses on enabling computers to understand, interpret, and generate human language in meaningful ways.

Neural Networks are computing systems inspired by biological neural networks that learn to perform tasks by analyzing examples and identifying patterns in data.

Neuromorphic computing mimics the structure and function of biological neural networks in hardware, enabling energy-efficient AI processing inspired by brain architecture.

The basic computational unit in neural networks that receives inputs, applies weights and transformations, and produces an output through an activation function.

A computational unit in neural networks or graphs that processes information, synonymous with neuron in neural networks or vertex in graph structures.

Neural Processing Unit, a specialized processor designed to accelerate artificial intelligence and machine learning computations, optimized for neural network operations.

AI technology that converts images of text into machine-readable digital text format through computer vision and pattern recognition.

An Optimizer is an algorithm that adjusts neural network parameters to minimize the loss function during training, determining how the model learns from data.

Overfitting occurs when a machine learning model learns the training data too well, including noise and irrelevant patterns, resulting in poor performance on new, unseen data.

Learnable variables in machine learning models that are adjusted during training to minimize loss and enable the model to perform its intended task, representing the knowledge acquired by the model.

A metric for evaluating language models that measures how well a model predicts text, with lower perplexity indicating better predictive performance.

A sequence of connected data processing stages where the output of one stage becomes the input of the next, enabling efficient and organized workflows in machine learning and data processing systems.

Pooling layers downsample feature maps in neural networks, reducing computational requirements while preserving important spatial information and providing translation invariance.

A classification metric measuring the proportion of true positive predictions among all positive predictions, indicating the quality and reliability of positive identifications.

Learn about Product Quantization and its applications in artificial intelligence and machine learning.

Input text or instructions given to an AI model to guide its response generation, serving as the primary interface for communicating with language models.

Prompt Engineering is the practice of crafting and optimizing input prompts to effectively communicate with AI language models and achieve desired outputs.

A neural network optimization technique that removes unnecessary weights, neurons, or connections to reduce model size and computational requirements while maintaining performance.

A model optimization technique that reduces the numerical precision of neural network weights and activations, decreasing memory usage and computational requirements while maintaining model performance.

A component or process that converts continuous or high-precision values to discrete, lower-precision representations, essential for model compression, hardware optimization, and efficient deployment of machine learning systems.

A linear data structure that follows the First-In-First-Out (FIFO) principle, widely used in computing for task scheduling, resource management, and asynchronous processing systems.

RAG (Retrieval-Augmented Generation) is an AI technique that combines large language models with external knowledge retrieval to provide more accurate and contextually relevant responses.

AI systems that perform logical thinking, problem-solving, and multi-step inference to reach conclusions and make decisions.

A classification metric measuring the proportion of actual positive cases correctly identified by the model, indicating the model's ability to find all relevant instances.

Regularization is a set of techniques used in machine learning to prevent overfitting by adding constraints or penalties to models, improving their ability to generalize to new data.

Reinforcement Learning is a machine learning approach where agents learn optimal behavior through trial and error by receiving rewards or penalties for their actions.

RLHF is a machine learning approach that uses human preferences and feedback to train AI models, enabling alignment with human values and improving model behavior through reward learning.

Rectified Linear Unit, a simple and effective activation function that outputs the input for positive values and zero for negative values.

Skip connections that add the input of a layer directly to its output, enabling the training of very deep neural networks by facilitating gradient flow.

Retrieval-Augmented Generation combines language models with external knowledge retrieval to generate more accurate, up-to-date, and factually grounded text responses.

RNN (Recurrent Neural Network) is a type of neural network designed for processing sequential data by maintaining memory of previous inputs through recurrent connections.

Robustly Optimized BERT Pretraining Approach, an improved version of BERT that uses optimized training procedures, larger datasets, and refined hyperparameters to achieve better performance on natural language understanding tasks.

A mechanism that allows each position in a sequence to attend to all positions in the same sequence, enabling models to capture dependencies regardless of distance.

Sentiment Analysis is a natural language processing technique that identifies and extracts emotional tone, opinions, and attitudes from text data to understand public sentiment.

A smooth S-shaped activation function that maps inputs to outputs between 0 and 1, commonly used for binary classification and gate mechanisms.

A measure of how alike or related two objects, vectors, or data points are, fundamental to many machine learning and AI applications.

An activation function that converts a vector of raw scores into a probability distribution, commonly used in multi-class classification tasks.

Hyperbolic tangent activation function that maps inputs to outputs between -1 and 1, offering zero-centered outputs and smooth gradients.

A mathematical object that generalizes scalars, vectors, and matrices to higher dimensions, fundamental to deep learning, physics, and multidimensional data representation and computation.

Teraflops (TFLOPS) measure floating-point operations per second at the scale of one trillion, serving as a key metric for comparing the raw computational throughput of AI accelerators and GPUs.

A lightweight execution unit within a process that can run concurrently with other threads, sharing memory space while maintaining independent execution paths for parallel processing.

The basic unit of text processing in natural language models, representing words, subwords, or characters that AI systems use to understand and generate language.

The process of breaking down text into smaller units called tokens for processing by natural language processing and AI models.

The process of breaking down text or other sequential data into smaller units called tokens, which serve as the fundamental input elements for natural language processing and machine learning models.

A system component that converts raw text into tokens (discrete units) that machine learning models can process, serving as the bridge between human language and AI understanding.

The mechanism by which AI models, particularly language models, invoke external functions, APIs, or systems to extend their capabilities beyond text generation, enabling interaction with external tools and services.

Tera Operations Per Second, a performance metric measuring the computational throughput of processors, particularly for AI and machine learning workloads including both integer and floating-point operations.

An advanced reasoning framework that enables language models to explore multiple reasoning paths simultaneously, maintaining a tree-like structure of thoughts to solve complex problems through deliberate search and evaluation.

Tensor Processing Unit, Google's custom ASIC designed specifically for accelerating machine learning workloads, particularly tensor operations and neural networks.

The process of teaching a machine learning model to recognize patterns and make predictions by exposing it to data and adjusting its parameters through iterative optimization.

Transfer Learning is a machine learning technique where a model trained on one task is adapted for use on a related task, leveraging pre-existing knowledge to improve learning efficiency and performance.

Transformer is a neural network architecture that uses attention mechanisms to process sequential data in parallel, revolutionizing natural language processing and AI.

TurboQuant is a quantization technique developed by Together AI that achieves near-lossless INT4 weight compression for large language models through structured, hardware-aligned quantization with minimal calibration overhead.

A metric measuring how effectively computing resources are being used, typically expressed as a percentage of maximum theoretical performance or capacity.

Learn about Vector and its applications in artificial intelligence and machine learning.

A Vector Database is a specialized database system designed to store, index, and query high-dimensional vector data, enabling fast similarity search and retrieval for AI applications.

A vector is a list of numbers that represents data in a format machines can understand — like the meaning of a word, sentence, image, or sound.

The complete set of unique tokens or words that a machine learning model can recognize and use, serving as the foundation for language understanding and generation.

Video Random Access Memory, specialized high-speed memory used by graphics processing units to store frame buffers, textures, and computational data for rendering and parallel processing tasks.

A sequence of connected steps, tasks, or processes designed to accomplish a specific goal or complete a business process, often automated and orchestrated through software systems to ensure efficient and consistent execution.