light client : failsafe crate (circuit breaker) (#9790)

* refactor(light) : N/W calls with `circuit breaker`

* fix(nits) : forgot to commit new files

* Add tests and change CLI args

* Address grumbles

* fix(failsafe-rs) : Santize input to prevent panics

* chore(failsafe) : bump failsafe, (parking_lot)

Bump failsafe to v0.3.0 to enable `parking_lot::Mutex` instead
`spin::Mutex`

* Remove `success_rate`

* feat(circuit_breaker logger)

* feat(CLI): separate CLI args request and response

* Fix tests

* Error response provide request kind

When a reponse `times-out` provide the actual request or requests that failed

* Update ethcore/light/src/on_demand/mod.rs

Co-Authored-By: niklasad1 <niklasadolfsson1@gmail.com>

* Update ethcore/light/src/on_demand/mod.rs

Co-Authored-By: niklasad1 <niklasadolfsson1@gmail.com>

* fix(grumbles): formatting nit

* fix(grumbles)

* Use second resolution on CLI args
* Use `consecutive failure policy` instead of `timeOverWindow`
* Add a couple of tests for `request_guard`

* fix(request_guard): off-by-one error, update tests

parity/cli/tests/config.full.toml

@@ -157,8 +157,11 @@ scale_verifiers = true
 num_verifiers = 6
 
 [light]
-on_demand_retry_count = 15
-on_demand_inactive_time_limit = 15000
+on_demand_response_time_window = 2000
+on_demand_request_backoff_start = 9000
+on_demand_request_backoff_max = 15000
+on_demand_request_backoff_rounds_max = 100
+on_demand_request_consecutive_failures = 1
 
 [snapshots]
 disable_periodic = false

parity/cli/tests/config.toml

@@ -71,8 +71,11 @@ fat_db = "off"
 scale_verifiers = false
 
 [light]
-on_demand_retry_count = 12
-on_demand_inactive_time_limit = 20000
+on_demand_response_time_window = 2000
+on_demand_request_backoff_start = 9000
+on_demand_request_backoff_max = 15000
+on_demand_request_backoff_rounds_max = 10
+on_demand_request_consecutive_failures = 1
 
 [snapshots]
 disable_periodic = true