Background: The customer is experiencing significant throttling and delays during frequent and complex schema updates, particularly when executing DDL statements in batches. This issue is compounded by the limitations in memory allocation for schema metadata, which is crucial for handling the customer’s large-scale database with thousands of tables and intricate schema changes.
Core Request: Develop an enhanced memory management system within Cloud Spanner to accommodate extensive schema changes without triggering throttling or excessive delays. This should include the ability for customers to temporarily increase memory allocation during peak schema update activities.
Transparency and Monitoring: Implement features for customers to monitor the memory usage of schema metadata and receive clear indications of when and why throttling occurs. This transparency will assist in better planning and management of schema updates.
Optimization of Schema Versioning: Introduce more efficient mechanisms for handling schema versioning, particularly during batch operations of DDL statements. This might include intelligent compaction of schema versions or optimization strategies that reduce the overall memory footprint of schema changes.
Long-Term Scalability: Ensure that the solution scales effectively for large databases (e.g., 6000 tables, 40 terabytes of data) and supports the customer’s growth and frequent update requirements.
Description
Core Request: Develop an enhanced memory management system within Cloud Spanner to accommodate extensive schema changes without triggering throttling or excessive delays. This should include the ability for customers to temporarily increase memory allocation during peak schema update activities.
Transparency and Monitoring: Implement features for customers to monitor the memory usage of schema metadata and receive clear indications of when and why throttling occurs. This transparency will assist in better planning and management of schema updates.
Optimization of Schema Versioning: Introduce more efficient mechanisms for handling schema versioning, particularly during batch operations of DDL statements. This might include intelligent compaction of schema versions or optimization strategies that reduce the overall memory footprint of schema changes.
Long-Term Scalability: Ensure that the solution scales effectively for large databases (e.g., 6000 tables, 40 terabytes of data) and supports the customer’s growth and frequent update requirements.