MLOps with Databricks. Machine Learning and GenAI Applications End to End Maria Vechtomova

Dostępność produktu

Produkt został zweryfikowany pod kątem ułatwień dostępu więcej przeczytasz w deklaracji dostępności.

Spis treści książki

• Foreword by Matei Zaharia
• Foreword by Alejandro Saucedo
• Preface
  • Why Databricks?
  • Who and What This Book Is For
  • How This Book Is Organized
  • Conventions Used in This Book
  • Using Code Examples
  • OReilly Online Learning
  • How to Contact Us
  • Acknowledgments
• 1. MLOps Principles and Components
  • The Rise of MLOps
  • MLOps Principles
    • Traceability
    • Reproducibility
    • Reliability
    • Observability
  • Team Practices
    • Documentation and Knowledge Preservation
    • Code Hygiene
  • MLOps Components
    • Core Components
      • Code version control
      • Continuous integration and continuous deployment (CI/CD)
      • Workflow orchestration
      • Model and container registry
      • Model training
      • Model Serving
      • Monitoring
      • Data version control
      • Feature store
    • LLM Application Components
      • Vector databases
      • Guardrails
      • AI gateways
      • Memory stores
      • Observability tools
      • Evaluation tools
      • Human review and feedback
    • Choosing Your Tools
  • Databricks MLOps Components
    • Lakeflow Jobs
    • Databricks Compute
      • Classic compute versus serverless compute
      • All-purpose versus job compute
    • MLflow
    • Feature Engineering
    • Databricks Model Serving
    • Data Quality Monitoring
    • Databricks AI Search
    • Unity AI Gateway
    • Databricks AI Guardrails
    • Cross-Cutting Components
      • Unity Catalog
      • Declarative Automation Bundles
      • Lakebase
      • Databricks Apps
  • Conclusion
• 2. Developing on Databricks
  • Cluster Configurations on Databricks
    • All-Purpose and Jobs Compute Configuration
      • Policies and access modes
      • Runtime versions
      • Photon acceleration
      • Worker and driver nodes
      • Advanced settings
      • Cost tracking
    • Serverless Compute
      • Serverless environments
      • Cost tracking
  • Project Setup
    • The Dataset
    • Python Tooling
    • The pyproject.toml file
      • Project section
      • Build-system section
      • Linting and formatting
      • Pytest configurations
    • Project Code: Data Preprocessing
  • Databricks Developer Tools
    • Databricks CLI
    • Declarative Automation Bundles
      • Syntax of the databricks.yml file
      • The databricks.yml file
      • Using DABs
    • Databricks Connect
    • Databricks Extension for Visual Studio Code
    • Running Code in a Databricks Notebook
      • Using serverless compute
      • Using all-purpose compute
  • Conclusion
• 3. MLflow for Traditional ML
  • Experiment Tracking
    • Tracking URI
    • MLflow Experiment
    • MLflow Run
  • Training Data
  • Hyperparameter Tuning and Nested Runs
  • Logging and Registering Models with MLflow
    • Model Logging with the sklearn Flavor
    • Viewing the Logged Model
    • Registering the Model
    • Wrapping the Model Using pyfunc
  • MLflow Evaluation
  • Resource and API Limits
  • Conclusion
• 4. Model Serving: Architectures and Implementation
  • Model Serving
    • Endpoint Deployment
    • A/B Testing
      • Choosing a routing strategy
        • Randomization by customer
        • Session-level assignment
        • Request-level assignment
      • Moving assignment into the model logic
    • Debugging
  • Feature Serving
    • Creating a Feature Table
    • Creating a FeatureSpec
    • Publishing a Table to a Databricks Online Feature Store
    • Endpoint Deployment
  • Model Serving with Feature Lookup
    • Creating Feature Functions and a Feature Table
    • Defining the Training Set
    • Registering the Model and Publishing an Online Table
    • Payload Structure
  • OAuth Authentication
  • Serving Limitations
  • Conclusion
• 5. Machine Learning Model Deployment
  • Machine Learning Pipeline
    • Data Preprocessing
    • Training, Evaluating, and Registering a Model
    • Deploying a Model Serving Endpoint
    • Serverless versus Classic Compute for Lakeflow Jobs
  • Databricks Platform Design
    • How Many Environments Do You Need?
    • Platform Best Practices
    • Access Control Strategy
  • Promoting Code and Models to Higher Environments
    • Git Branching Model
    • Continuous Integration Pipeline
    • Continuous Deployment Pipeline
      • OpenID Connect
      • Branch and deployment protection rules
  • Promoting Models to Higher Environments
  • Testing
    • ML Model Testing
    • Software Testing
      • Unit testing
      • Databricks considerations
      • Integration testing
      • Practical considerations: Integration tests in CI
      • End-to-end testing
  • Conclusion
• 6. Monitoring ML Applications
  • What to Monitor
    • Recommender Systems
    • Regression and Demand Forecasting
    • Large Language Models
    • Concept Drift versus Data Drift
    • Fairness, Bias, and Regulatory Compliance
    • System and Infrastructure Health
  • Introduction to Databricks Data Quality Monitoring
    • Profile Metrics Table
    • Drift Metrics Table
  • Metrics for Monitoring Data Drift
    • Chi-squared Test
    • Kolmogorov-Smirnov Test
    • Total Variation Distance
    • L-Infinity Distance
    • Jensen-Shannon Distance
    • Wasserstein Distance
    • Population Stability Index
  • From Inference Tables to Data Quality Monitoring
    • Inference Table Structure
    • Processing an Inference Table
    • Creating a Data Quality Monitor
    • Using the Dashboard
    • Refreshing the Monitor
  • Alerting
  • Conclusion
• 7. Foundation Models and Context Engineering
  • Introduction to LLMOps
    • What Is an Agent?
    • Structure of the LLMOps Chapters
  • Foundation Models on Databricks
    • Hosting Options
      • Databricks-hosted models (pay-per-token)
      • Provisioned throughput endpoints
      • External models
    • Types of Foundation Models
  • Context Engineering
    • Reference Architecture: The arXiv Curator
    • Getting from Raw Data to Chunks
      • Project configuration
      • Data ingestion
      • PDF processing
      • Chunking
        • Chunking strategies
        • Embedding window considerations
        • Processing chunks
    • Databricks AI Search Index
      • Creating the endpoint and index
      • Querying and retrieval quality
      • Optimizing search quality
    • Genie Agent (Formerly, Genie Space)
      • Databricks AI Search versus Genie
      • Provisioning the SQL warehouse
      • Creating the Genie agent
      • Interacting with Genie
    • MCP Servers on Databricks and Tool Calling
      • Managed MCP ecosystems on Databricks
      • Implementing tool calling
    • Agent Memory with Lakebase
      • Lakebase Autoscaling for agent workloads
      • Lakebase Autoscaling project structure
      • Creating a Lakebase Autoscaling project
      • Interacting with Lakebase
      • Utilizing a connection pool
  • Conclusion
• 8. MLflow for GenAI
  • MLflow Flavors
  • MLflow Tracing
    • Anatomy of a Trace
    • MLflow Tracing APIs
  • Defining a Custom Agent
    • Initialization and Async Handling
    • Core Agent Logic
    • Prediction and Streaming
    • Testing the Agent
  • LLM System Evaluation
    • How to Evaluate LLM Systems
    • MLflow GenAI Evaluation Framework
  • Evaluating, Logging, and Registering the Agent
    • Evaluation in Practice
    • Agent Logging
    • Registering the Agent and Preparing for Serving
  • Conclusion
• 9. Deploying in Monitoring LLM-Based Systems
  • Deploying a Databricks Model Serving Endpoint
    • Unity AI Gateway and Guardrails
      • Guardrails on the LLM level
      • Guardrails as part of the agent logic
      • Implementing deployment
      • Setting up service principal authentication
      • Authentication considerations
      • Calling the Model Serving endpoint
    • Deployment Jobs
      • Data preprocessing job
      • Agent deployment job
    • Deployment Considerations
  • LLM System Monitoring
    • What to Monitor in LLM Systems
    • Monitoring Implementation
      • Deployment notebook
      • Lakeflow job
      • Monitoring dashboard
      • Monitoring considerations
    • Cost Monitoring
      • How Databricks billing works
      • Databricks services and associated costs
      • Tag-based cost attribution
      • Cost-monitoring dashboard
  • Conclusion
• 10. AI Governance
  • The Scope of AI Governance
    • Definition of AI Under the EU AI Act
      • The challenge of definition: A three-factor approach
    • Risk Categories and Whom They Apply To
  • Requirements for High-Risk AI
    • AI Project Lifecycle
    • Article 9: Risk Management System
      • Project scoping (risk identification)
      • Development and testing (mitigation)
      • Post-market monitoring (via a feedback loop)
      • Types of risks and mitigation strategies
        • Data-related and model behavior risks
        • Operational risks
        • Security and cybersecurity risks
    • Article 10: Data and Data Governance
      • Project scoping
      • Dataset/context engineering
      • Model development/adaptation
      • Deployment
      • Monitoring and continual learning
      • Business analysis
    • Article 11: Technical Documentation
    • Article 12: Record-Keeping
    • Article 13: Transparency and Provision of Information to Deployers
    • Article 14: Human Oversight
    • Article 15: Accuracy, Robustness, and Cybersecurity
  • MLOps Checklist
  • Conclusion
• Epilogue
• Index

pokaż cały spis treści

ukryj recenzje