Loading repository data…
holasoymalva / repository
Glossary of key concepts in Large Language Models (LLMs), Artificial Intelligence (AI), Natural Language Processing (NLP), and Machine Learning (ML). Clear definitions and resources for developers, researchers, and AI enthusiasts exploring generative language technologies.
Your essential reference for Large Language Models, AI, NLP, and Machine Learning terminology
Explore Terms • Contributing • Resources
The LLM Glossary is a comprehensive, community-driven reference guide for understanding the rapidly evolving landscape of Large Language Models and artificial intelligence. Whether you're a developer building AI applications, a researcher exploring cutting-edge techniques, or an enthusiast learning about generative AI, this glossary provides clear, concise definitions for the concepts that matter.
Browse the glossary by category:
A neural network trained on massive text datasets to understand and generate human-like text. LLMs use deep learning architectures (typically Transformers) with billions to trillions of parameters to capture patterns in language.
Key characteristics:
Examples: GPT-4, Claude, Gemini, LLaMA
Resources:
The fundamental unit of text processed by language models. Tokens are typically subword pieces that represent common character sequences.
Common tokenization methods:
Example:
Input: "Tokenization is important"
Tokens: ["Token", "ization", " is", " important"]
Related: Context Window, Vocabulary
The practice of designing inputs (prompts) to effectively communicate with and guide LLMs toward desired outputs.
Key techniques:
Example:
# Zero-shot
"Translate this to French: Hello, world!"
# Few-shot
"Translate to French:
English: Hello → French: Bonjour
English: Thank you → French: Merci
English: Good morning → French: ?"
Resources:
The maximum number of tokens an LLM can process in a single interaction, including both input and output.
Considerations:
Examples:
The foundational neural network architecture for modern LLMs, introduced in 2017. Uses self-attention mechanisms to process sequences in parallel.
Key components:
Variants:
A technique that allows models to focus on relevant parts of the input when processing each token.
Types:
Formula:
Attention(Q, K, V) = softmax(QK^T / √d_k)V
Where Q=Query, K=Key, V=Value, d_k=dimension
An architecture that uses multiple specialized "expert" networks, activating only relevant experts for each input.
Benefits:
Examples: GPT-4, Mixtral, Switch Transformers
The initial phase where models learn general language understanding from large unlabeled datasets.
Objectives:
Scale: Trillions of tokens, thousands of GPU-hours
Adapting a pre-trained model to specific tasks or domains using smaller, task-specific datasets.
Approaches:
An efficient fine-tuning technique that adds small, trainable rank decomposition matrices to model layers while keeping original weights frozen.
Benefits:
Formula:
W' = W + BA
Where W is frozen, B and A are low-rank trainable matrices
Resources:
Training approach that uses human preferences to align model outputs with desired behaviors.
Process:
Impact: Critical for models like ChatGPT, Claude, Gemini
Reducing model precision (e.g., from 32-bit to 8-bit or 4-bit) to decrease memory usage and increase inference speed.
Methods:
Trade-offs: Lower precision can reduce quality but enables deployment on consumer hardware
A hyperparameter controlling randomness in text generation.
Scale:
Implementation: Divides logits before softmax
Techniques to constrain token selection during generation.
Top-k: Sample from k most probable tokens Top-p (Nucleus): Sample from tokens comprising top p probability mass
Best practice: Often use together with temperature for quality control
Infrastructure and techniques for deploying LLMs in production.
Frameworks:
Optimizations: Batching, KV-cache, speculative decoding
A measurement of how well a probability model predicts a sample. Lower perplexity indicates better model performance.
Formula:
PPL = exp(-1/N Σ log P(token_i))
Note: Best for comparing models, not absolute quality assessment
Benchmark measuring knowledge across 57 subjects including STEM, humanities, and social sciences.
Evaluation: Multiple-choice questions, reports accuracy percentage
Coding benchmark measuring ability to generate functionally correct Python code from docstrings.
Metric: pass@k - percentage of problems solved with k attempts
Architecture combining LLMs with external knowledge retrieval to ground responses in factual information.
Architecture:
Benefits: Reduces hallucinations, enables up-to-date information, domain-specific knowledge
Tools: LangChain, LlamaIndex, Haystack
Capability allowing LLMs to invoke external functions or APIs with structured parameters.
Use cases:
Example:
{
"name": "get_weather",
"parameters": {
"location": "San Francisco",
"unit": "celsius"
}
}
Autonomous systems that use LLMs for reasoning and decision-making to accomplish complex tasks.
Components:
Frameworks: AutoGPT, BabyAGI, LangGraph, CrewAI
Dense vector representations of text that capture semantic meaning.
Applications:
Models: OpenAI text-embedding-ada-002, Cohere Embed, Sentence-BERT
We welcome contributions from the community! Here's how you can help:
git checkout -b add-new-term)#### Term Name
Clear, concise definition (1-2 sentences)
**Key points**:
- Important detail 1
- Important detail 2
**Examples/Resources**: Links to papers or implementations
For Beginners:
Selected from shared topics, language and repository description—not editorial ratings.
danigunawan /
A "Glossary of Artificial Intelligence" is a concise reference resource defining key terms, concepts, and terminology related to AI. It provides explanations and definitions to help individuals understand and navigate the field of artificial intelligence, making it a valuable tool for both beginners and experts in the AI domain.