Fine-Tuning AI. Customizing Large Language Models Laurence Moroney

Dostępność produktu

Produkt został zweryfikowany pod kątem ułatwień dostępu więcej przeczytasz w deklaracji dostępności.

Spis treści książki

• Preface
  • Intended Audience
  • Prerequisite Knowledge
  • Hardware and Software Dependencies
  • Book Structure
  • Conventions Used in This Book
  • Using Code Examples
  • OReilly Online Learning
  • How to Contact Us
  • Acknowledgments
• 1. From Generalist to Specialist
  • The Cost of Renting Intelligence
    • The Token Tax
    • The Agentic Multiplier
    • The Instruction Tax
  • The Latency Killer
  • The Data Firewall
    • The Metadata Leak
    • Compliance Issues
    • Model Drift
  • The Specialist Revolution
  • Hardware Is No Longer the Blocker
  • The Spectrum of Customization
• 2. Model Architectures: The DNA of AI
  • The Flow of Information: How Transformers Work
    • Attention Mechanisms
    • Multihead Attention
      • Reducing dimensions in MHA
      • Self-attention versus cross-attention
    • The Feed-Forward Network
    • Layer Normalization and Residual Connections
  • Decoder-Only Dominance
  • Grouped Query Attention
  • Mistral and Mixtral
    • Sliding Window Attention
    • Mixture of Experts and Sparseness
  • Gemma: Googles Open Approach
  • Tokenizers and Embeddings
    • Tokenization
    • Understanding Language and Tokenization
    • Byte Pair Encoding
    • Embeddings
    • Understanding Positional Embeddings with RoPE
  • Other Considerations
    • Instruction Following
    • Reasoning Depth
    • Context Windows and the Lost in the Middle Problem
  • Considerations for Choosing a Model
  • Summary
• 3. The Fine-Tuners Workshop
  • Why Setup Matters
  • The GPU: Your First Decision
    • VRAM: One Number to Rule Them All
    • Understanding GPUs
      • Consumer GPUs
      • Data center GPUs
      • Distributed training with multiple GPUs
  • The Software Stack
    • Choosing Your Operating System
      • Using Linux
      • Using Windows
      • Using macOS
    • Your Programming Environment
    • The Core Library Stack
      • PyTorch
      • Hugging Face Transformers
      • Hugging Face PEFT
      • Bitsandbytes
      • TRL
      • Hugging Face Accelerate
      • Hugging Face Datasets
      • Weights & Biases
    • Experiment Tracking
      • Weights & Biases (wandb)
      • TensorBoard
  • Cloud and Pay-As-You-Go
  • Local Versus Cloud
  • Summary
• 4. Excavating Data
  • Sourcing Raw Data
    • Public Datasets with Hugging Face
    • Kaggle Datasets and Competitions
    • Scraping Domain-Specific Data
      • Web scraping
      • Structured systems
      • PDFs
    • Cleaning Your Data
      • Text normalization
      • Content filtering
      • Deduplication
  • The Risk of Leaking PII
    • Detecting PII
    • PII Redaction Strategies
  • Understanding Data Quality
    • Statistical Profiling
    • Spot Checking
    • Contamination Checks
  • Building Your Data Pipeline
    • Your Output Format
    • Dataset Cards
  • Summary
• 5. Synthetic Data
  • The Teacher-Student Paradigm
  • Generating Instruction-Response Pairs
    • The Naive Approach
    • Seed-Based Generation
    • Prompting the Teacher
    • Batching for Cost Efficiency
    • Understanding Evol-Instruct
    • Generating Chain-of-Thought Reasoning
  • Quality Control for Synthetic Data
    • Hallucination Detection
    • Data Diversity Metrics
    • Filtering the Final Dataset
  • Mixing Real and Synthetic Data
  • Summary
• 6. Preparing Your Dataset for Fine-Tuning
  • The Gap Between Data and Training
  • The Two Paradigms: Completion and Chat
  • Chat Templates: From Messages to Tokens
    • ChatML
    • Llama
    • Mistral/Mixtral
    • Gemma
    • Using Templates with Hugging Face
    • Getting Your Data into the Messages Format
    • Formatting for TRL
    • Using Alpaca with SFTTrainer
  • Loss Masking
  • Other Frameworks for Fine-Tuning
    • Axolotl
    • Unsloth
  • Splitting and Validating Your Dataset
  • Format Validation
  • Inspecting Tokenized Examples
  • Summary
• 7. The Mechanics of Adaptation
  • The Problems with Full Fine-Tuning
    • The Memory Problem
    • Catastrophic Forgetting
    • The Storage Problem
  • Parameter Efficient Fine-Tuning
  • LoRA
    • The Mathematics of LoRA
    • Where LoRA Is Applied
    • The Merge Step
    • Why It Works
  • Understanding LoRA Hyperparameters
    • Rank
    • Alpha and the Scaling Factor
    • Target Modules
    • Dropout
    • Bias
    • Task Type
    • Putting It All Together
  • QLoRA: An Even Smaller Footprint
  • DoRA: Redefining the Adaptation
  • Choosing Your Approach
  • Under the Hood
  • Summary
• 8. The Training Loop
  • Assembling the Pieces
  • The Hugging Face Training Stack
    • Training Configuration with SFTConfig
    • Trainer and SFTTrainer
  • Data Collators
  • Training Hyperparameters
    • Learning Rate
    • Learning Rate Schedule
    • Batch Size and Gradient Accumulation
    • Number of Epochs
    • Weight Decay
    • Maximum Gradient Norm
    • Optimizer
  • Memory Optimizations
    • Gradient Checkpointing
    • Mixed-Precision Training
    • Your Memory Budget: Putting It All Together
  • Preparing the Training Data
    • The Loss Function and Label Masking
    • Training and Validation Split
  • The Complete Training Script
  • Performing the Training Run
  • A Practical Workflow for Hyperparameter Tuning
    • Start with the Defaults
    • Tune the Learning Rate
    • Adjust the Epoch Count
    • Experiment with Rank
    • Apply the Good Enough Principle
  • Logging and Experiment Tracking
    • TensorBoard
    • Weights & Biases
    • Custom Callbacks
  • Reproducibility
  • Other Training Techniques
  • Summary
• 9. Using the Adapter
  • The Anatomy of an Adapter
    • The Adapter for the Model
    • The Adapter Config JSON File
    • The Tokenizer Files
    • The Chat Template
    • The Training Args
  • Downloading Your Adapter
    • Download as a ZIP
    • Mount Google Drive
    • Push to Hugging Face Hub
  • Using the Adapter
    • Merging the Adapter
    • Testing Inference with an Adapter
  • Going Beyond Eyeball Testing
    • Side-by-Side Comparison
    • Measuring Perplexity
  • LLM-as-a-Judge
  • Domain-Specific Benchmarks
  • Common Pitfalls
  • Impact of Using QLoRA
  • Summary
• 10. Alignment and Preference Optimization
  • The Gap: From Knowledge to Behavior
  • RLHF: The Original Pipeline
  • Direct Preference Optimization
    • Building a Preference Dataset
    • Generating the Data
  • The DPO Loss Function
    • The Objective
    • The Role of the Reference Model and KL Divergence
  • Running the DPO Training
    • Preparing the Dataset
    • Loading the Model
    • Configuring and Launching the Trainer
    • Things to Watch for During Training
      • Training Loss (train/loss)
      • Chosen Rewards (train/rewards/chosen)
      • Rejected Rewards (train/rewards/rejected)
      • The Margin (train/rewards/margins)
  • Evaluating the Aligned Model
  • When to Use DPO (and When Not To!)
  • Beyond DPO
  • Summary
• 11. Vision Language Models
  • Why Vision Matters
  • The Two-Tower Architecture
    • The Vision Encoder
    • The Projection Layer
    • What Is LLaVA?
    • How Gemma 4 Puts It All Together
    • The Forward Pass
  • What to Fine-Tune, and What to Freeze!
    • Fine-Tuning Strategies
    • The VRAM Budget
    • How to Choose Your Strategy
  • Building a Vision Dataset
    • The Data Schema
    • Getting the Data from Hugging Face
    • How the Dataset Was Built
      • Image generation
      • Teacher-model annotation
  • The Vision Fine-Tuning Script
    • Loading the Model for Vision
    • Configuring LoRA for Vision
    • Preparing the Dataset
    • The Training Configuration
  • Inference with the Fine-Tuned VLM
  • Evaluating the Vision Model
    • The Hallucination Test
    • LLM-as-a-Judge with Vision
    • Base Models Versus Fine-Tuned Models
  • Domain Examples: Opportunities for Vision Fine-Tuning
    • Medical Imaging
    • Manufacturing and Quality Inspection
    • Document Understanding
  • Practical Considerations
    • Image Resolution and Token Budget
    • Multi-image Conversations
    • When to Fine-Tune the Vision Tower
  • Troubleshooting Common Issues
  • The Complete Workflow
  • Summary
• 12. Diffusion Models
  • Beyond Transformers
  • The Big Idea: Reverse a Destruction Process
    • Gaussian Noise
    • The Forward Process: Controlled Destruction
    • The Reverse Process: Denoising
  • The UNet Architecture
  • Stable Diffusion 1.5
  • The Latent Trick
  • The Text Encoder
  • The Scheduler: Managing Denoising
  • The Full Pipeline
  • Classifier-Free Guidance
  • Fine-Tuning
  • Summary
• 13. Fine-Tuning Diffusion Models
  • Two Recipes, One Architecture
  • DreamBooth: Teaching a Subject
    • The Core Ideas of DreamBooth
    • The General Recipe
      • Step 1: Getting images of your subject
      • Step 2: Generating the class-prior set
      • Step 3: The DreamBooth training step
    • Using Your DreamBooth Adapter
    • The Training Hyperparameters That Matter
      • Rank
      • Steps
      • Class-prior weight
  • Style LoRAs: Teaching a Visual Vocabulary
    • Building a Style Dataset
    • Training
    • Using the Style Adapter
  • Understanding and Sweeping for Strength
  • Compositing Multiple LoRAs
  • Going Beyond Stable Diffusion 1.5
  • Preparing Adapters for Deployment
  • Understanding Common Failure Modes
  • Summary
• 14. Optimization and Quantization
  • The Deployment Gap
  • Merging the Adapter into the Base
    • What Merging Actually Computes
    • The 4-bit Merge Trap
    • Verifying Parity
    • When Not to Merge
  • How Quantization Actually Works
    • A Quick Refresher on Floating Point
    • From Floats to Integers: Scale and Zero-Point
      • Why group size matters
      • Symmetry and activations
  • The Format Zoo
  • Using GGUF with llama.cpp
    • Convert, Then Quantize
    • Understanding the K-Quant Names
    • The Importance Matrix
  • A Better Way Forward: QAT
    • How to Get QAT Gemma
    • Using QAT for Fine-Tuning
  • Measuring Quantization Impact
    • Two Metrics to Track
    • The Bit-Width Sweep
  • KV-Cache Optimization
    • What Is the KV Cache?
    • Grouped Query Attention
    • KV-Cache Quantization
    • PagedAttention
    • Prefix Caching
    • Offloading
  • Putting It All Together: The Quantization Pipeline
  • Avoiding Some Common Pitfalls
  • Summary
• 15. Serving the Expert
  • From Artifact to Service
  • Using Ollama
  • The OpenAI Endpoint Format
  • Using vLLM
    • Gemma 4 Considerations
    • Serving the Unmerged Adapter
  • Serving Many Experts from One Base
  • Production Concerns
    • Structured Output
    • A Gateway in Front
    • Observability and Capacity
  • Conclusion: The Future Is Specialized
• Index

pokaż cały spis treści

ukryj recenzje