ADR-005: Component-Specific Configuration Handling

Status

✅ Accepted

Date

2024-01-15

Context

Following the decision to implement centralized Configuration Management (ADR-004), a critical detailed design question emerged:

"each component will have a different configuration, how the centralized configuration will handle the different configuration for each module.. how it will hand over the loaded configuration to the respective module"

The Challenge

With 8+ different components (Identity, Memory, Context, Awareness, Observability, Tools, Agent Interface, Security & Privacy), each having:

• Different configuration schemas (cache sizes vs. vector dimensions vs. timeout values)
• Different validation rules (ranges, dependencies, data types)
• Different update sensitivities (hot-reload vs. restart-required)
• Different security requirements (public vs. secret configurations)

The centralized Configuration Management service needed a sophisticated mechanism to:

1. Isolate component configurations while providing unified management
2. Deliver type-safe configurations to each component
3. Handle component-specific validation and update procedures
4. Manage cross-component dependencies when they exist

Decision

Implement a component-registry-based configuration handling system where each component registers its configuration schema with the centralized service, and the service provides type-safe, validated, component-specific configuration delivery.

Core Architecture

Rationale

Why Schema Registration is the Right Approach

1. Type Safety: Each component gets strongly-typed configuration objects
2. Validation: Component-specific validation rules enforced at the service level
3. Isolation: Components only see their own configuration data
4. Flexibility: Easy to add new components without changing the service
5. Security: Component-level access control and field restrictions
6. Performance: Component-specific caching and optimization

Alternatives Considered

Alternative 1: Generic Key-Value Configuration

// Generic approach - component parses its own config
let cache_size: usize = config_service.get("identity.cache_size")?;
let timeout_ms: u64 = config_service.get("identity.timeout_ms")?;

• ❌ No type safety or validation
• ❌ Components must handle parsing and validation
• ❌ No schema enforcement
• ❌ Difficult to manage cross-component dependencies

Alternative 2: Shared Configuration Schema

// Single schema for all components
struct PlatformConfig {
    identity: IdentityConfig,
    memory: MemoryConfig,
    context: ContextConfig,
    // ... all components
}

• ❌ Tight coupling between all components
• ❌ Components receive configurations for other components
• ❌ Versioning becomes complex
• ❌ Security boundaries unclear

Alternative 3: File-Per-Component with Manual Loading

// Each component loads its own file
let config: IdentityConfig = toml::from_str(&fs::read_to_string("identity.toml")?)?;

• ❌ No centralized management
• ❌ No hot-reload capabilities
• ❌ Inconsistent error handling
• ❌ No unified monitoring

Consequences

Positive Outcomes

✅ Type-Safe Configuration: Each component gets strongly-typed config objects ✅ Schema Validation: Invalid configurations caught at service level ✅ Component Isolation: Clean boundaries between component configurations ✅ Flexible Registration: Easy to add new components and config fields ✅ Security Boundaries: Component-level access control ✅ Hot-Reload Support: Sophisticated update handling per component ✅ Cross-Component Dependencies: Managed at the service level ✅ Performance Optimization: Component-specific caching strategies

Implementation Details

Component Schema Registration:

#[derive(Debug, Clone, Serialize, Deserialize, ConfigSchema)]
pub struct IdentityComponentConfig {
    pub cache_size: usize,
    pub session_timeout_seconds: u64,
    pub external_idm: IdmConfig,
}
 
impl ComponentConfig for IdentityComponentConfig {
    fn component_name() -> &'static str { "identity" }
    fn schema_version() -> &'static str { "1.0.0" }
    
    fn schema() -> ComponentSchema {
        ComponentSchema {
            fields: vec![
                SchemaField {
                    name: "cache_size",
                    field_type: ConfigType::UInteger,
                    validation_rules: vec![Range { min: 100, max: 1000000 }],
                    hot_reload: true,
                },
                // ... more fields
            ],
        }
    }
}

Configuration Loading:

// Component requests its specific configuration
let config: IdentityComponentConfig = config_service
    .get_config("identity", &instance_id)
    .await?;
 
// Type-safe access to configuration
let cache_size = config.cache_size; // usize, validated
let timeout = config.session_timeout_seconds; // u64, validated

Update Handling:

// Component subscribes to its own configuration updates
config_service.subscribe_to_config(
    "identity",
    &instance_id,
    |update: ConfigUpdate<IdentityComponentConfig>| {
        // Strongly-typed update with change information
        handle_identity_config_update(update);
    }
).await?;

Configuration Organization

File Structure:

config/
├── components/
│   ├── identity.toml     # Identity-specific config
│   ├── memory.toml       # Memory-specific config
│   └── context.toml      # Context-specific config
├── environments/
│   ├── production/
│   │   ├── identity.toml # Production overrides for identity
│   │   └── memory.toml   # Production overrides for memory
└── schemas/
    └── v1/
        ├── identity.schema.json
        └── memory.schema.json

Configuration Merging:

# Priority: base < environment < instance
Merge Strategy:
  1. Load base component configuration
  2. Apply environment-specific overrides  
  3. Apply instance-specific overrides
  4. Validate against component schema
  5. Deliver type-safe configuration object

Access Control and Security

Component Isolation:

impl ConfigurationService {
    async fn get_config<T: ComponentConfig>(
        &self,
        component_name: &str,
        requester: &ComponentIdentity,
    ) -> Result<T> {
        // Verify component can only access its own config
        if component_name != requester.component_name {
            return Err(AccessDenied("Cross-component access not allowed"));
        }
        
        // Load and validate component-specific configuration
        let config = self.load_component_config(component_name).await?;
        self.validate_against_schema(&config, component_name)?;
        
        Ok(config)
    }
}

Field-Level Security:

pub struct ComponentAccessPolicy {
    pub component_name: String,
    pub field_restrictions: HashMap<String, FieldAccess>,
}
 
pub enum FieldAccess {
    ReadWrite,    // Normal configuration field
    ReadOnly,     // Cannot be modified via API
    Restricted,   // Requires special permissions
    Secret,       // Contains sensitive data
}

Cross-Component Dependencies

Dependency Resolution:

// Example: Identity component needs Security component's encryption setting
impl ConfigurationService {
    async fn resolve_dependencies(
        &self,
        component_configs: HashMap<String, ComponentConfig>,
    ) -> Result<HashMap<String, ComponentConfig>> {
        
        // Resolve dependencies in topological order
        let dependency_order = self.calculate_dependency_order()?;
        
        for component in dependency_order {
            self.resolve_component_dependencies(component, &configs).await?;
        }
        
        Ok(configs)
    }
}

Performance Optimization

Component-Specific Caching:

pub struct ComponentConfigCache {
    // Separate cache per component
    component_caches: HashMap<String, LruCache<String, CachedConfig>>,
    
    // Component-specific TTL
    ttl_config: HashMap<String, Duration>,
}
 
impl ComponentConfigCache {
    async fn get_config<T: ComponentConfig>(
        &self,
        component_name: &str,
        instance_id: &str,
    ) -> Option<T> {
        let component_cache = self.component_caches.get(component_name)?;
        component_cache.get(instance_id)?.try_into().ok()
    }
}

Follow-up Actions

Completed

• ✅ Designed component registration system
• ✅ Created type-safe configuration delivery mechanism
• ✅ Implemented component isolation and security
• ✅ Designed cross-component dependency resolution
• ✅ Created component-specific caching strategy
• ✅ Documented integration patterns for all components

Implementation Requirements

• 📋 Create ComponentConfig trait and derive macros
• 📋 Implement schema registration and validation
• 📋 Build component-specific configuration storage
• 📋 Create type-safe configuration delivery APIs
• 📋 Implement component-level access controls

Component Integration

• 📋 Create configuration schemas for all 8 core components
• 📋 Implement component registration in each component
• 📋 Create migration paths for existing configurations
• 📋 Establish testing patterns for configuration updates

Lessons Learned

1. Type Safety Prevents Errors: Strongly-typed configurations catch errors at compile time
2. Schema Registration Enables Flexibility: Easy to add new components without service changes
3. Component Isolation is Critical: Security and maintainability require clear boundaries
4. Dependencies Need Management: Cross-component dependencies require careful orchestration

Impact on Architecture

Component Development

• Standard Pattern: All components follow the same configuration integration pattern
• Reduced Complexity: Components don't implement configuration management logic
• Type Safety: Compile-time verification of configuration usage

Operational Benefits

• Unified Management: All component configurations managed through single service
• Consistent Validation: Same validation framework across all components
• Centralized Monitoring: Single view of configuration state across platform

Security Improvements

• Access Control: Component-level and field-level security
• Audit Trail: Complete visibility into configuration access and changes
• Secret Management: Consistent secret handling across all components

Related Decisions

• ADR-004: Centralized Configuration Management (provides the foundation)
• ADR-002: Agent Identity Scope Limitation (similar component isolation principle)
• ADR-003: IDM and Agent Identity Separation (similar separation of concerns)

This decision established the detailed mechanisms for how the centralized Configuration Management service delivers component-specific, type-safe, validated configurations while maintaining security boundaries and operational excellence.