An error occurred validating the cluster security descriptor dating with chat room over 10000
Avoiding these temptations and managing configurations in a unified way has made the configuration system a leveraged way to make the site more reliable.• Statically validate configuration changes.
Many configuration systems allow loosely typed configuration, such as JSON structures.
This mechanism provides a basic “sanity check” on all changes to ensure that they do not cause widespread failure.• Hold on to good configurations.
Our configuration system is backed by version control, making it easy to revert changes.connections leading to a spike in error rate— a wrong config change such as a missing secret in an environment variable taking down the service (as an aside, environment variables are best avoided, but that’s a different discussion for a different blog post)Ideally, if testing in the of a service to production.
A subset of production now consists of the canaries which are then sent a small percentage of actual production traffic after they pass a basic health check.
Using a common configuration system ensures that procedures and tools apply to all types of configuration.
At Facebook we have found that teams are sometimes tempted to handle configurations in a one-off way.
Sometimes, despite all best efforts, a bad configuration is deployed.