Environment Management

Set up and manage development, staging, and production environments for BroxiAI applications

Learn how to properly set up, configure, and manage multiple environments for your BroxiAI applications, from development to production.

Environment Overview

Environment Types

Development Environment

  • Individual developer workspaces

  • Rapid iteration and testing

  • Debug-friendly configurations

  • Local or shared development resources

Staging Environment

  • Production-like testing environment

  • Integration testing

  • User acceptance testing

  • Performance validation

Production Environment

  • Live user-facing systems

  • High availability and performance

  • Security-hardened configuration

  • Comprehensive monitoring

Additional Environments

  • QA/Testing: Dedicated testing environment

  • Demo: Sales and demonstration environment

  • Disaster Recovery: Backup production environment

  • Performance Testing: Load testing environment

Development Environment Setup

Local Development

BroxiAI Workspace Configuration

{
  "environment": "development",
  "debug_mode": true,
  "log_level": "debug",
  "api_settings": {
    "timeout": 30000,
    "retries": 1,
    "rate_limit": false
  },
  "mock_external_apis": true,
  "cache_disabled": true
}

Development-Specific Features

  • Enhanced debugging capabilities

  • Detailed error messages

  • Extended timeout values

  • Disabled caching for testing

  • Mock external services

  • Local file storage

Shared Development Environment

Team Development Setup

Shared Development:
  Infrastructure:
    - Shared vector database instance
    - Development API keys
    - Shared file storage
    - Team collaboration features

  Configuration:
    - Branch-based deployments
    - Feature flag management
    - Shared secrets management
    - Team workflow synchronization

  Best Practices:
    - Isolated namespaces per developer
    - Shared test data sets
    - Consistent environment variables
    - Version control integration

Staging Environment

Purpose and Goals

Staging Environment Objectives

  • Mirror production configuration

  • Integration testing platform

  • User acceptance testing

  • Performance validation

  • Security testing

  • Deployment process validation

Staging Configuration

Infrastructure Setup

Staging Environment:
  Compute:
    - Production-equivalent instance types
    - Scaled-down capacity (30-50% of production)
    - Auto-scaling disabled for predictability
    - Manual scaling for load testing

  Storage:
    - Production-equivalent storage types
    - Subset of production data
    - Backup and recovery testing
    - Data anonymization for privacy

  Network:
    - Production-like network topology
    - SSL certificates for HTTPS
    - Load balancer configuration
    - Security group rules

BroxiAI Configuration

{
  "environment": "staging",
  "debug_mode": false,
  "log_level": "info",
  "monitoring": {
    "detailed_metrics": true,
    "performance_profiling": true,
    "error_tracking": "comprehensive"
  },
  "external_services": {
    "use_production_endpoints": false,
    "sandbox_mode": true,
    "test_credentials": true
  }
}

Data Management in Staging

Test Data Strategy

Test Data Management:
  Data Sources:
    - Anonymized production data
    - Synthetic test data
    - Curated test scenarios
    - Performance testing datasets

  Data Refresh:
    - Weekly data refresh from production
    - Automated anonymization process
    - Test data versioning
    - Data cleanup procedures

  Privacy Compliance:
    - PII removal/masking
    - GDPR compliance
    - Data retention policies
    - Access control implementation

Production Environment

Production Requirements

High Availability Setup

Production Configuration

{
  "environment": "production",
  "debug_mode": false,
  "log_level": "warn",
  "security": {
    "encryption_at_rest": true,
    "encryption_in_transit": true,
    "api_rate_limiting": true,
    "ip_whitelisting": true
  },
  "performance": {
    "caching_enabled": true,
    "compression_enabled": true,
    "connection_pooling": true,
    "query_optimization": true
  },
  "monitoring": {
    "health_checks": true,
    "performance_monitoring": true,
    "error_tracking": true,
    "security_monitoring": true
  }
}

Production Security

Security Hardening

Production Security:
  Network Security:
    - VPC isolation
    - Private subnets for databases
    - Security groups and NACLs
    - DDoS protection
    - Web Application Firewall

  Access Control:
    - IAM roles and policies
    - Multi-factor authentication
    - Principle of least privilege
    - Regular access reviews
    - Audit logging

  Data Protection:
    - Encryption at rest (AES-256)
    - Encryption in transit (TLS 1.3)
    - Key management service
    - Backup encryption
    - Secure data deletion

  Compliance:
    - GDPR compliance measures
    - SOC 2 compliance
    - Regular security audits
    - Penetration testing
    - Vulnerability assessments

Environment Configuration Management

Configuration Strategy

Environment-Specific Variables

Development:
  BROXI_ENVIRONMENT: "development"
  DEBUG_MODE: "true"
  LOG_LEVEL: "debug"
  OPENAI_API_KEY: "${DEV_OPENAI_KEY}"
  PINECONE_ENVIRONMENT: "development"
  CACHE_TTL: "60"

Staging:
  BROXI_ENVIRONMENT: "staging"
  DEBUG_MODE: "false"
  LOG_LEVEL: "info"
  OPENAI_API_KEY: "${STAGING_OPENAI_KEY}"
  PINECONE_ENVIRONMENT: "staging"
  CACHE_TTL: "3600"

Production:
  BROXI_ENVIRONMENT: "production"
  DEBUG_MODE: "false"
  LOG_LEVEL: "warn"
  OPENAI_API_KEY: "${PROD_OPENAI_KEY}"
  PINECONE_ENVIRONMENT: "production"
  CACHE_TTL: "7200"

Secrets Management

Secret Storage Strategy

Secrets Management:
  Development:
    - Local environment files
    - Team shared secrets
    - Version control excluded
    - Simple rotation process

  Staging:
    - Cloud secrets manager
    - Automated rotation
    - Access logging
    - Team access control

  Production:
    - Enterprise secrets manager
    - Automatic rotation
    - Audit logging
    - Strict access control
    - Encryption at rest

Secret Rotation Process

# Example: Automated secret rotation
class SecretRotationManager:
    def __init__(self, environment):
        self.environment = environment
        self.secrets_client = get_secrets_client(environment)
    
    def rotate_api_keys(self):
        """Rotate API keys across all services"""
        for service in self.get_services():
            try:
                # Generate new key
                new_key = service.generate_new_key()
                
                # Test new key
                if self.test_key_validity(service, new_key):
                    # Update secret store
                    self.secrets_client.update_secret(
                        f"{service.name}_api_key", 
                        new_key
                    )
                    
                    # Trigger application restart
                    self.restart_applications()
                    
                    # Revoke old key after grace period
                    self.schedule_key_revocation(service, delay="1h")
                    
            except Exception as e:
                self.log_rotation_error(service, e)
                self.send_alert(f"Secret rotation failed for {service.name}")

Environment Promotion Pipeline

Deployment Pipeline

CI/CD Pipeline Stages

Pipeline Configuration

Pipeline Stages:
  Development:
    triggers: [code_push, pull_request]
    tests: [unit_tests, lint_checks]
    deployment: automatic
    rollback: automatic

  Staging:
    triggers: [dev_success, manual_trigger]
    tests: [integration_tests, security_tests]
    deployment: automatic
    rollback: automatic
    approval: optional

  Production:
    triggers: [staging_success, manual_trigger]
    tests: [smoke_tests, health_checks]
    deployment: manual_approval_required
    rollback: manual_or_automatic
    approval: required

Environment Promotion Process

Promotion Criteria

Promotion Requirements:
  Development to Staging:
    - All unit tests passing
    - Code review completed
    - Security scan passed
    - No critical vulnerabilities

  Staging to Production:
    - Integration tests passing
    - Performance benchmarks met
    - Security validation complete
    - Stakeholder approval obtained
    - Rollback plan validated

Quality Gates:
  Code Quality:
    - Code coverage > 80%
    - No critical code smells
    - Security scan clean
    - Dependency check passed

  Performance:
    - Response time < 3s (p95)
    - Error rate < 1%
    - Resource utilization < 80%
    - Load test passed

Environment Monitoring

Environment-Specific Monitoring

Development Monitoring

Development Metrics:
  Focus Areas:
    - Development velocity
    - Build success rates
    - Test execution times
    - Feature completion rates

  Alerts:
    - Build failures
    - Test failures
    - Development environment downtime
    - Resource exhaustion

Staging Monitoring

Staging Metrics:
  Focus Areas:
    - Integration test results
    - Performance benchmarks
    - Security test results
    - User acceptance feedback

  Alerts:
    - Test failures
    - Performance degradation
    - Security vulnerabilities
    - Data integrity issues

Production Monitoring

Production Metrics:
  Focus Areas:
    - User experience metrics
    - Business KPIs
    - System reliability
    - Security incidents

  Alerts:
    - Service downtime
    - Performance degradation
    - Security breaches
    - Business metric anomalies

Cross-Environment Monitoring

Environment Comparison Dashboard

{
  "environment_comparison": {
    "metrics": [
      "response_time",
      "error_rate",
      "throughput",
      "resource_utilization"
    ],
    "environments": ["development", "staging", "production"],
    "alert_on_drift": true,
    "drift_threshold": "20%"
  }
}

Data Management Across Environments

Data Strategy

Data Flow Management

Data Synchronization

Data Sync Strategy:
  Production to Staging:
    frequency: "weekly"
    process: "anonymized_backup_restore"
    validation: "data_integrity_checks"
    rollback: "previous_week_backup"

  Staging to Development:
    frequency: "on_demand"
    process: "subset_extraction"
    validation: "schema_validation"
    anonymization: "additional_scrubbing"

  Test Data Generation:
    synthetic_data: "faker_library"
    scenario_data: "curated_datasets"
    performance_data: "generated_load_data"

Database Environment Management

Database Configuration per Environment

Database Settings:
  Development:
    instance_type: "small"
    backup_retention: "1_day"
    monitoring: "basic"
    multi_az: false
    encryption: false

  Staging:
    instance_type: "medium"
    backup_retention: "7_days"
    monitoring: "enhanced"
    multi_az: false
    encryption: true

  Production:
    instance_type: "large"
    backup_retention: "30_days"
    monitoring: "comprehensive"
    multi_az: true
    encryption: true
    read_replicas: 2

Environment Troubleshooting

Common Environment Issues

Development Issues

Common Problems:
  Configuration Drift:
    cause: "Inconsistent local setups"
    solution: "Standardized development containers"
    prevention: "Environment validation scripts"

  Resource Conflicts:
    cause: "Multiple developers sharing resources"
    solution: "Namespace isolation"
    prevention: "Resource allocation planning"

  Version Mismatches:
    cause: "Different component versions"
    solution: "Version pinning and documentation"
    prevention: "Automated version checking"

Staging Issues

Common Problems:
  Data Staleness:
    cause: "Infrequent data refreshes"
    solution: "Automated data refresh pipeline"
    prevention: "Scheduled refresh monitoring"

  Performance Differences:
    cause: "Resource configuration mismatch"
    solution: "Environment parity validation"
    prevention: "Regular configuration audits"

  Integration Failures:
    cause: "External service differences"
    solution: "Service virtualization"
    prevention: "Contract testing"

Production Issues

Common Problems:
  Environment Drift:
    cause: "Manual production changes"
    solution: "Infrastructure as code"
    prevention: "Change management process"

  Resource Exhaustion:
    cause: "Unexpected load increases"
    solution: "Auto-scaling and monitoring"
    prevention: "Capacity planning"

  Configuration Errors:
    cause: "Environment-specific misconfigurations"
    solution: "Configuration validation"
    prevention: "Automated configuration testing"

Troubleshooting Tools

Environment Validation Scripts

# Environment validation script
class EnvironmentValidator:
    def __init__(self, environment):
        self.environment = environment
        self.checks = []
    
    def validate_configuration(self):
        """Validate environment configuration"""
        results = {
            "environment": self.environment,
            "timestamp": datetime.now(),
            "checks": []
        }
        
        # Check required environment variables
        results["checks"].append(
            self.check_environment_variables()
        )
        
        # Check service connectivity
        results["checks"].append(
            self.check_service_connectivity()
        )
        
        # Check resource availability
        results["checks"].append(
            self.check_resource_availability()
        )
        
        # Check security configuration
        results["checks"].append(
            self.check_security_configuration()
        )
        
        return results
    
    def check_environment_variables(self):
        required_vars = [
            "BROXI_ENVIRONMENT",
            "OPENAI_API_KEY",
            "PINECONE_API_KEY",
            "DATABASE_URL"
        ]
        
        missing_vars = []
        for var in required_vars:
            if not os.getenv(var):
                missing_vars.append(var)
        
        return {
            "check": "environment_variables",
            "status": "pass" if not missing_vars else "fail",
            "missing_variables": missing_vars
        }

Environment Best Practices

Development Best Practices

Local Development

  • Use consistent development containers

  • Implement environment validation scripts

  • Maintain development documentation

  • Use feature flags for work-in-progress features

  • Implement local testing procedures

Team Development

  • Establish environment provisioning procedures

  • Implement shared resource management

  • Use consistent naming conventions

  • Maintain environment documentation

  • Regular environment cleanup

Staging Best Practices

Environment Parity

  • Match production configuration closely

  • Use production-equivalent data volumes

  • Implement comprehensive testing procedures

  • Validate deployment processes

  • Test disaster recovery procedures

Data Management

  • Regular data refresh procedures

  • Proper data anonymization

  • Test data scenario management

  • Performance testing data sets

  • Data cleanup procedures

Production Best Practices

Security and Compliance

  • Implement comprehensive security measures

  • Regular security audits and updates

  • Compliance validation procedures

  • Incident response procedures

  • Access control and audit logging

Operations and Monitoring

  • Comprehensive monitoring and alerting

  • Automated backup and recovery

  • Performance optimization procedures

  • Capacity planning and scaling

  • Regular disaster recovery testing

Environment Automation

Infrastructure as Code

Environment Provisioning

# Terraform/CloudFormation example
Environment_Template:
  Parameters:
    Environment: [development, staging, production]
    InstanceType: [t3.micro, t3.medium, t3.large]
    DatabaseSize: [20GB, 100GB, 500GB]

  Resources:
    BroxiAI_Application:
      Type: "Container_Service"
      Properties:
        Image: "broxi/app:${var.version}"
        Environment: "${var.environment}"
        Resources: "${var.instance_type}"

    Vector_Database:
      Type: "Managed_Vector_DB"
      Properties:
        Size: "${var.database_size}"
        Encryption: "${var.environment != 'development'}"
        Backups: "${var.environment == 'production'}"

Environment Orchestration

Automated Environment Management

class EnvironmentOrchestrator:
    def __init__(self):
        self.environments = {
            "development": DevelopmentEnvironment(),
            "staging": StagingEnvironment(),
            "production": ProductionEnvironment()
        }
    
    def provision_environment(self, env_name, config):
        """Provision a new environment"""
        environment = self.environments[env_name]
        
        # Provision infrastructure
        environment.provision_infrastructure(config)
        
        # Deploy application
        environment.deploy_application(config.app_version)
        
        # Configure monitoring
        environment.setup_monitoring()
        
        # Validate environment
        validation_results = environment.validate()
        
        return {
            "status": "success" if validation_results.all_passed else "failure",
            "environment": env_name,
            "validation_results": validation_results
        }

Next Steps

After setting up your environments:

  1. Implement Monitoring: Set up environment-specific monitoring

  2. Automate Processes: Create automation for common tasks

  3. Document Procedures: Maintain comprehensive documentation

  4. Train Team: Ensure team understands environment procedures

  5. Regular Reviews: Schedule regular environment health checks

Related Guides

Proper environment management is crucial for reliable software delivery. Invest time in setting up robust environment processes to ensure smooth deployments and reduced production issues.

PreviousProduction ChecklistNextOverview

Last updated