Data Management at Scale

Spis treści książki

• Foreword
• Preface
  • Who Is This Book For?
  • What Will I Learn?
  • Navigating Through This Book
  • Conventions Used in This Book
  • OReilly Online Learning
  • How to Contact Us
  • Acknowledgments
• 1. The Disruption of Data Management
  • Data Management
  • Analytics Is Fragmenting the Data Landscape
  • Speed of Software Delivery Is Changing
  • Networks Are Getting Faster
  • Privacy and Security Concerns Are a Top Priority
  • Operational and Transactional Systems Need to Be Integrated
  • Data Monetization Requires an Ecosystem-to-Ecosystem Architecture
  • Enterprises Are Saddled with Outdated Data Architectures
    • Enterprise Data Warehouse and Business Intelligence
    • Data Lake
    • Centralized View
  • Summary
• 2. Introducing the Scaled Architecture: Organizing Data at Scale
  • Universally Acknowledged Starting Points
    • Each Application Has an Application Database
    • Applications Are Specific and Have Unique Context
    • Golden Source
    • Theres No Escape from the Data Integration Dilemma
    • Applications Play the Roles of Data Providers and Data Consumers
  • Key Theoretical Considerations
    • Object-Oriented Programming Principles
    • Domain-Driven Design
      • Bounded context
      • Ubiquitous language
    • Business Architecture
      • Business capabilities
      • Linking business capabilities with applications
      • Business architecture in practice: A case study
      • Business architecture patterns
  • Communication and Integration Patterns
    • Point-to-Point
    • Silos
    • Hub-Spoke Model
  • Scaled Architecture
    • Golden Sources and Domain Data Stores
    • Data Delivery Contracts and Data Sharing Agreements
    • Eliminating the Siloed Approach
    • Domain-Driven Design on an Enterprise Scale
      • Single-step transformation
    • Read-Optimized Data
    • Data Layer as a Holistic Picture
      • Read-Only Data Store Architecture
      • API Architecture
      • Streaming Architecture
    • Metadata and the Target Operating Model
  • Summary
• 3. Managing Vast Amounts of Data: The Read-Only Data Stores Architecture
  • Introducing the RDS Architecture
  • Command and Query Responsibility Segregation
    • What Is CQRS?
    • CQRS at Scale
  • Read-Only Data Store Components and Services
    • Metadata
    • Data Quality
    • RDS Tiers
    • Data Ingestion
      • Schema metadata
    • Integrating Commercial Off-the-Shelf Solutions
    • Extracting Data from External APIs and SaaSs
    • Historical Data Service
      • Partitioning Full Dimensional Snapshots
      • Maintaining historical data
      • Append-only dimensions
      • Redeliveries and late arriving data
    • Design Variations
    • Data Replication
    • Access Layer
    • File Manipulation Service
    • Delivery Notification Service
    • De-Identification Service
    • Distributed Orchestration
  • Intelligent Consumption Services
  • Populating RDSs on Demand
  • RDS Direct Usage Considerations
  • Summary
• 4. Services and API Management: The API Architecture
  • Introducing the API Architecture
  • What Is Service-Oriented Architecture?
    • Enterprise Application Integration
    • Service Orchestration
    • Service Choreography
    • Public Services and Private Services
    • Service Models and Canonical Data Models
    • Similarities Between SOA and Enterprise Data Warehousing Architecture
      • Canonical model sizing
      • ESB as wrapper for legacy middleware
      • ESB managing application state
  • Modern View on SOA
    • API Gateway
    • Responsibility Model
    • The New Role of the ESB
    • Service Contracts
    • Service Discovery
  • Microservices
    • The Role of the API Gateway Within Microservices
    • Functions
    • Service Mesh
    • Microservices Boundaries
    • Microservices Within the API Reference Architecture
  • Ecosystem Communication
  • API-Based Communication Channels
    • GraphQL
    • Backend for Frontend
  • Metadata
  • Using RDSs for Real-Time and Intensive Reads
  • Summary
• 5. Event and Response Management: The Streaming Architecture
  • Introducing the Streaming Architecture
  • The Asynchronous Event Model Makes the Difference
  • What Do Event-Driven Architectures Look Like?
    • Mediator Topology
    • Broker Topology
    • Event Processing Styles
  • A Gentle Introduction to Apache Kafka
    • Distributed Event Data
    • Apache Kafka Features
  • The Streaming Architecture
    • Event Producers
    • Event Consumers
    • Event Platform
    • Event Sourcing and Command Sourcing
    • Governance Model
    • Business Streams
      • Within the application boundaries
      • Using third-party tooling
      • Using post-ingestion approaches
    • Streaming Consumption Patterns
      • Using the platforms capabilities
      • Microservices and event frameworks
      • Streaming analytics services
      • Event routing on public cloud
    • Event-Carried State Transfer
    • Playing the Role of an RDS
    • Using Streaming to Populate RDSs
    • Controls and Policies for Guiding the Domains
  • Streaming as the Operational Backbone
  • Guarantees and Consistency
    • Consistency Level
    • At Least Once, Exactly Once, and at Most Once Processing
    • Message Order
    • Dead Letter Queue
    • Streaming Interoperability
  • Metadata for Governance and Self-Service Models
  • Summary
• 6. Connecting the Dots
  • Recap of the Architectures
    • RDS Architecture
    • API Architecture
    • Streaming Architecture
    • Strengthening Patterns
  • Enterprise Interoperability Standards
    • Stable Data Endpoints
    • Data Delivery Contracts
    • Accessible and Addressable Data
    • Crossing Network Principles
      • Replicate data toward the place of consumption
      • Think in the spirit of asynchronicity
      • Decoupling domains
      • Exceptions
      • Hub-spoke network topology
  • Enterprise Data Standards
    • Consumption-Optimization Principles
    • Discoverability of Metadata
    • Semantic Consistency
    • Supplying the Corresponding Metadata
    • Data Origination and Movements
  • Reference Architecture
  • Summary
• 7. Sustainable Data Governance and Data Security
  • Data Governance
    • Organization: Data Governance Roles
    • Processes: Data Governance Activities
    • People: Trust and Ethical, Social, and Economic Considerations
    • Technology: Golden Source, Ownership, and Application Administration
    • Data: Golden Sources, Golden Datasets, and Classifications
      • Data governance layer
      • Data labels and classifications
      • Purpose classifications
      • Schema identification
  • Data Security
    • Current Siloed Approach
    • Unified Data Security for Architectures
      • Role-based access control and attribute-based access control
    • Identity Providers
    • Security Reference Architecture and Data Context Approach
    • Security Process Flow
      • Data ingestion
      • Data at rest
      • Access to data
      • Data consumption
      • Data processing at consumer side
      • Onboarding again
  • Practical Guidance
    • RDS Architecture
    • API Architecture
    • Streaming Architecture
    • Intelligent Learning Engine
  • Summary
• 8. Turning Data into Value
  • Consumption Patterns
    • Using Read-Only Data Stores Directly
    • Domain Data Stores
  • Target Operating Model
  • Data Professionals as a Target User Group
  • Business Requirements
  • Nonfunctional Requirements
  • Building the Data Pipeline and Data Model
  • Distributing Integrated Data
  • Business Intelligence Capabilities
  • Self-Service Capabilities
  • Analytical Capabilities
    • Standard Infrastructure for Automated Deployments
    • Stateless Models
    • Prescripted and Configured Workbenches
    • Standardize on Model Integration Patterns
    • Automation
    • Model Metadata
  • Advanced Analytics Reference Architecture
  • Summary
• 9. Mastering Enterprise Data Assets
  • Demystifying Master Data Management
  • Master Data Management Styles
  • MDM Reference Architecture
    • Designing a Master Data Management Solution
    • MDM Distribution
    • Master Identification Numbers
    • Reference Data Versus Master Data
  • Determining the Scope of Your Enterprise Data
  • MDM and Data Quality as a Service
  • Curated Data
    • Metadata Exchange
    • Integrated Views
    • Reusable Components and Integration Logic
    • Data Republishing
  • Relation to Data Governance
  • Summary
• 10. Democratizing Data with Metadata
  • Metadata Management
  • Enterprise Metadata Model
    • Datasets and data objects
    • Data models
    • Interface models
    • Lineage and its integration
    • Data governance and security
  • Enterprise Knowledge Graph
  • Architectural Approaches for Metadata Management
    • Metadata Interoperability
    • Metadata Repositories
  • Marketplace to Provide Rapid Access to Authorized Data
  • Summary
• 11. Conclusion
  • Delivery Model
    • Fully Decentralized Approach
    • Partially Decentralized Approach
    • Structuring Teams
    • InnerSource Strategy
  • Culture
  • Technology Choices
  • The Decline of Traditional Enterprise Architecture
    • Blueprints and Diagrams
    • Modern Skills
    • Control and Governance
  • Last Words
• Glossary
• Index

pokaż cały spis treści

ukryj recenzje