* Use fewer threads for snapshotting
When taking a snapshot the current default number of threads is equal to half the number of **logical** CPUs in the system. On HT enabled CPUs this value seems a bit high, e.g. 6 snapshotting threads on a 6/12 core/hyperthread CPU. Maybe a better default value is half the number of physical cores?
* fix test
* Replace error chain for network error
* Fix usages and add manual From impls
* OnDemand Error and remove remaining dependencies
* Die error_chain, die.
* DIE
* Hasta la vista, baby
* [whisper] Move needed aes_gcm crypto in-crate
In the latest `parity-crypto` release (upcoming 0.4), the aes GCM features were removed (done to remove the dependency on `ring`).
This PR adds the bare minimum crypto needed for Whisper directly to the crate itself and as those were the only features needed from `parity-crypto`, removes the dependency on that crate altogether.
* Upgrade to parity-crypto 0.4
Reverts using NonZeroU32 (introduced [here](b347599cf7)).
* Check for 0 in `args.arg_keys_iteration`
* Use beta.4
* parity-crypto 0.4.0 is released
* Journal for private txs added
* Tests after adding logging to private tx fixed
* Logs getter and tests added
* Time and amount limit for logs added
* RPC method for log retrieving added
* Correct path name and time validation implemented
* References for parameters added, redundant cloning reworked
* References for parameters added, redundant cloning reworked
* Work with json moved to the separate struct
* Serialization test added
* Fixed build after the merge with head
* Documentation for methods fixed, redundant field removed
* Fixed error usages
* Timestamp trait implemented for std struct
* Commented code removed
* Remove timestamp source, rework serialization test
* u64 replaced with SystemTime
* Path made mandatory for logging
* Source of monotonic time added
* into_system_time method renamed
* Initialize time source by max from current system time and max creation time from already saved logs
* Redundant conversions removed, code a little bit reworked according to review comments
* One more redundant conversion removed, rpc call simplified
* allow CORS requests for Secret Store API (#10582)
* secretstore CORS: fix error with unit tests
* secretstore CORS: removed debug log
* secretstore CORS: add missing response's header
* secretstore CORS: switched to jsonrpc-server-utils for CORS validation
* Fix nasty typo in NodeTable::update (add ;)
* Add limiting for NodeTable
* Add cache for NodeFilter
* Use expect instead of unwrap
* Move node in ordered_ids if it exists there in note_failure and note_success + fix expect msg
* Add comment
* Improve code style
* DRY in note_failure and note_success
* Fix nodes ordering
* Simplify match expression
* Add tests for get_index_to_insert
* Remove get_mut method from NodeTable, Add get method to NodeTable
* Fix table_last_contact_order for macos failing because of lost nanosecond precision
* ethcore: add clique engine (#9981)
* fix broken sync
* correct seal fields
* ethcore: fix comment
* parity: remove duplicate params
* clique: fix whitespaces
* ethcore: fix goerli chain spec
* refactor signer_snapshot into pending/finalized state
* move close_block_extra_data after seal is applied
* refactor most of the logic into the signer_snapshot
* clique: refactor locking logic out of the consensus engine interface
* Fix jsonspec and add an unittest
* Replace space with tabs
* Unbroke sync
* Fix broken sync
* 1/2 state tracking without votes
* 2/2 implement vote tracking
* ci: use travis for goerli
* ci: setup a clique network
* ci: sync a görli node
* add clique deploy script
* ci: fix paths in clique deploy script
* ci: use docker compose
* ci: fix travis job names
* ci: fix build deps
* ci: massively reduce tests
* Revert "ci: massively reduce tests"
This reverts commit 6369f0b069ed2607a7e9f2e1d85489bacdc43384.
* ci: run cargo test directly
* ci: separate build and test stages
* ci: cache rust installation
* ci: simplify ci stages
* ci: make clique deploy script executable
* ci: shutdown goerli sync after 20min
* ci: remove slow sync stage
* ci: use timeout to finish jobs
* ci: fix build path
* ci: use absolute paths to end this confusion
* ci: add geth and parity to path
* ci: be more verbose
* ci: allow for more relaxed caching timeout
* ci: update repositories for custom ppa
* ci: fix typo in file name
* ci: fix docker compose file
* ci: add ethkey to docker
* ci: make sure deploy script is up to date with upstream
* ci: stop docker container after certain time
* ci: force superuser to update permissions on docker files
* ci: reduce run time of script to ~30 min
* ci: remove duplicate caching in travis
* remove trace statements
* clique: add more validation involving the recent signer list
* ethcore: enable constantinople for rinkeby
* ethcore: fix whitespaces in rinkeby spec
* ethcore: reformat goerli.json
* Revert "ci: remove duplicate caching in travis"
This reverts commit a562838d3d194d37f9871dcbe00b783637978f89.
* tmp commit
* another tmp commit
* it builds!
* add sealing capabilities
* add seal_header hook to allow separation of block seal/importing code paths
* clique: remove populate_from_parent.
* add panic
* make turn delay random
* initialize OpenBlock properly in 'enact'
* misc: remove duplicate lines
* misc: fix license headers
* misc: convert spaces to tabs
* misc: fix tabs
* Update Cargo.toml
* Update Cargo.toml
* Update Cargo.toml
* clique: ensure validator restores state before trying to seal
* clique: make 'state' return an Error. Make some error messages more clear
* Fix compile error after rebase & toolchain upgrade
* fix a bunch of import warnings
* Refactor code
* Fix permissions
* Refactoring syncing
* Implement full validator checks
* Refactor util functions to seperate file
* mining 1
* ethcore: add chainspec for kotti
* ethcore: rename pre-goerli configs
* ethcore: load kotti chain spec
* cli: add kotti to params
* Implement working local sealing
* making sealing & syncing work together
* Relax timestamp checking
* ethcore: prepare for the real goerli to launch
* Implement NOTURN wiggle properly & cleanupnup warnings
* Implement vote casting
* Update docs & skip signing if no signer
* Optimize step-service interval
* Record state on local sealed block
* Fix script filemode
* Cleaning up codebase
* restore enact trace logging
* Delete clique.sh and move sync.sh
* remove travis.yml
* Remove dead code
* Cleanup compile warning
* address review comments
* adding more comments and removing unwrap()
* ci: remove sync script
* Address review comments
* fix compile error
* adding better debugging for timing
* Implement an dedicated thread for sealing timing
* fix(add helper for timestamp overflows) (#10330)
* fix(add helper timestamp overflows)
* fix(simplify code)
* fix(make helper private)
* snap: official image / test (#10168)
* official image / test
* fix / test
* bit more necromancy
* fix paths
* add source bin/df /test
* add source bin/df /test2
* something w paths /test
* something w paths /test
* add source-type /test
* show paths /test
* copy plugin /test
* plugin -> nil
* install rhash
* no questions while installing rhash
* publish snap only for release
* fix(docker): fix not receives SIGINT (#10059)
* fix(docker): fix not receives SIGINT
* fix: update with reviews
* update with review
* update
* update
* Don't add discovery initiators to the node table (#10305)
* Don't add discovery initiators to the node table
* Use enums for tracking state of the nodes in discovery
* Dont try to ping ourselves
* Fix minor nits
* Update timeouts when observing an outdated node
* Extracted update_bucket_record from update_node
* Fixed typo
* Fix two final nits from @todr
* change docker image based on debian instead of ubuntu due to the chan… (#10336)
* change docker image based on debian instead of ubuntu due to the changes of the build container
* role back docker build image and docker deploy image to ubuntu:xenial based (#10338)
* Bundle protocol and packet_id together in chain sync (#10315)
Define a new `enum` where devp2p subprotocol packet ids (currently eth and par) are defined. Additionally provide functionality to query id value and protocol of a given id object.
* snap: prefix version and populate candidate channel (#10343)
* snap: populate candidate releases with beta snaps to avoid stale channel
* snap: prefix version with v*
* addressing review comments
* engine: fix copyright header
* scripts: restore permissions on sign command
* ethcore: enforce tabs
* ethcore: enforce tabs
* ethcore: enforce tabs
* addressing comments
* addressing comments
* addressing more comments
* addressing more comments
* addressing more comments
* addressing more comments
* addressing more comments
* json-spec: fix clique epoch to non-zero u64
* ci: enable travis for parity goerli
* ci: don't separate build and test step
* ci: don't run c++ tests on travis
* ci: simplify cargo test to squeeze into travis timeout
* ci: don't run tests on travis at all
* style(fixes)
* fix(add tests)
* fix(recent_signer bug)
* fix(complete all tests)
* fix(nits)
* fix(simplify asserts)
* fix(cliqueState): simplify code
* fix(nits)
* docs(comments what's need to fixed)
* fix(revert unintended changes)
* fix(tests)
* fix(logs): voting logs
* fix(readability + more logs)
* fix(sync)
* docs(add missing licens header)
* fix(log): info! -> trace!
* docs(fix nits) + fix(remove assert)
* perf(use counter instead of vec)
* fix(remove needless block in match)
* fix(faulty comment)
* grumbles(docs for tests)
* fix(nits)
* fix(revert_vote): only remove vote when votes == 0
* fix(vote counter): checked arithmetics
* fix(simplify tests)
* fix(nits)
* fix(clique): err types
* fix(clique utils): make use of errors
* fix(cleanup nits)
* fix(clique sealing): don't read state no signer
* fix(replace Vec<Signers> with BTreeSet<Signers>)
* fix(tests): BTreeSet and more generic helpers
* fix(nits)
* fix(ethcore_block_seal): remove needless `Box`
* fix(faulty log): info -> trace
* fix(checked SystemTime): prevent SystemTime panics
* style(chain cfg): space after `:`
* style(fn enact): fix whitespace
* docs(clique): StepService
* docs(nit): fix faulty comment
* docs(fix typo)
* style(fix bad indentation)
* fix(bad regex match)
* grumble(on_seal_block): make `&mut` to avoid clone
* docs(on_seal_block): fix faulty documentation
* Delete .travis.yml
* docs: remove eth hf references in spec
* Update client.rs
* fix(nits)
* fix(clique step): `RwLock` -> `AtomicBool`
* fix(clique): use `Duration::as_millis`
* Clean up some Clique documentation
* Add trace information to eth_estimateGas (#10519)
* Add trace information to eth_estimateGas
* replace unwrap better version
* change vm::Error formatter to more user-friendly
* remove extra error format
* use map_or instead sequence of map/unwrap_or
* fix(light/on_demand): extract as a trait
* fix(grumble): OnDemand remove needless trait bound
* fix(rpc): fix a bunch of clippy lints
* fix(rpc clippy): remove unused ignored lints
* fix(clippy): fix all redundant_field_names
This commit fixes all uses of `redundant_field_names` and removes the ignored lint `redundant_field_names`
* fix(brain unwrap): replace with expect
* fix(light-rpc): Make `light_sync` generic
The motivation behind this change is to easily mock `light-sync` to make it possible to enable `rpc-integration` tests
for the light-client.
Currently the `rpc's` requires the concrete type `sync::LightSync` which makes it very hard to do so
* fix(bad merge)
* Extract accounts from ethcore.
* Fix ethcore.
* Get rid of AccountProvider in test_helpers
* Fix rest of the code.
* Re-use EngineSigner, fix tests.
* Simplify EngineSigner to always have an Address.
* Fix RPC tests.
* Add deprecation notice to RPCs.
* Feature to disable accounts.
* extract accounts in RPC
* Run with accounts in tests.
* Fix RPC compilation and tests.
* Fix compilation of the binary.
* Fix compilation of the binary.
* Fix compilation with accounts enabled.
* Fix tests.
* Update submodule.
* Remove android.
* Use derive for Default
* Don't build secretstore by default.
* Add link to issue.
* Refresh Cargo.lock.
* Fix miner tests.
* Update rpc/Cargo.toml
Co-Authored-By: tomusdrw <tomusdrw@users.noreply.github.com>
* Fix private tests.
* Patch available private contracts during private transaction
* Key acl ABI added
* Work with secret store keys moved to the separate struct
* Private tx test refactored
* Revert "Private tx test refactored"
This reverts commit 476c132d692c7a886bc7b7cd7fe47b3d7692bd63.
* Test for calling private contract from another one added
* Test fixed
* Redundant tab removed
* ACL contract processing fixed, test added
* Merge with head
* Expect replaced with closure
* prevent silent errors in daemon mode
* change author in Cargo.toml, add preamble to pipe.rs
* set the uid and gid on daemon process, fix permission errors when writing to pid file
* call setup_logger before daemonize to prevent crashing when attempting to create logfile
* map_err for calls to splice and ioctl, fix spaces in Cargo.toml
* split out daemonize to own repo
* removed util/daemonize
* renamed dep to parity-daemonize
* fix(parity-clib): enable `logger`
* bump parity-daemonize
* remove obsolete comment
Co-Authored-By: seunlanlege <seunlanlege@gmail.com>
* fix(grumbles): docs and log in ParityParams
* Add FIXME comment regarding @tomaka grumbles
* Unify logger with the C-API in ParityParams (less type-safety with more from_raw() conversions)
* Add better documentation in the `parity.h`
* Apply suggestions from code review
Co-Authored-By: seunlanlege <seunlanlege@gmail.com>
* docs(parity lib): add link to logging issue
* fix(parity-clib): JNI enable `logger`
* fix(parity-clib): update `Java example`
* Update example to the API changes
* Remove needless printouts which can be controlled via logger instead
* Allow specifying local accounts via CLI
* Add `tx-queue-locals` CLI option
* ethcore: modify miner to check options vec before importing transaction
* modify tests (ethcore/parity)
Resolves#9634
* fix formatting
* fixes: Make prefer HashSet over Vec<>, add test, comment formatting
* Update ethcore/src/miner/miner.rs
Co-Authored-By: insipx <aplaza@liquidthink.net>
* Fix comments and add helper for set->vec conversion
* remove blank line from use statement
* fix helper test
* formatting
* fix test to pass on nightly
* revert test fix for nightly
* fix(ManageNetwork): replace Range -> RangeIncls
Fixes `TODO: Range should be changed to RangeInclusive once stable (https://github.com/rust-lang/rust/pull/50758)`
* fix(tests)
* fix(grumbles): off-by-one error in debug_asserts
* RangeInclusive::end() is inclusive which means that if start and end is equal the `debug_assert(range.end() >
range.start()` will fail which is shouldn't
* Remove re-export of `CallContract` and `RegistryInfo` from `ethcore/client`
* Remove CallContract and RegistryInfo re-exports again
This was missed while fixing merge conflicts
* added BlockChainReset trait, client impl, and cli interface
* show block hashes to be deleted and new best block, update best block in db, better cli interface
* delete BlockNumber from COL_EXTRA
* add TODO comment
* add BlockReciepts to imports
* refactor block_headers_from_best_block, better cli documentation
* exit gracefully if reset arg isn't supplied
* fix cli usage macro
* removed stray int literals
* use Vec::with_capacity
Co-Authored-By: seunlanlege <seunlanlege@gmail.com>
* cast n to usize
* correct imports
* make db reset arg required
* Replace ethcore-logger with env-logger.
* Fix logger initialization in WASM tests.
* uncomment logger initialization in secret store
* Don't use ethcore-logger in whisper.
* Move ethcore-logger within parity dir.
* Uncomment rest from secret-store.
* Use `let _ =` in private_contract for consistency
* `ok()` to `let _ =` fix in service
* Use `let _ = ` for state_db
* Add private tx enabled flag into status packet
* Error log added for the case with no peers available
* Add flag only for supported protocol versions
* Work with private handler refactored
* Log target changed
* Cargo.lock updated
* feat(parity-clib asynchronous rpc queries)
* feat(seperate bindings for ws and rpc)
* Subscribing to websockets for the full-client works
* feat(c binding unsubscribe_from_websocket)
* fix(tests): tweak CMake build config
* Enforce C+11
* refactor(parity-cpp-example) : `cpp:ify`
* fix(typedefs) : revert typedefs parity-clib
* docs(nits)
* fix(simplify websocket_unsubscribe)
* refactor(cpp example) : more subscriptions
* fix(callback type) : address grumbles on callback
* Use it the example to avoid using global variables
* docs(nits) - don't mention `arc`
* fix(jni bindings): fix compile errors
* feat(java example and updated java bindings)
* fix(java example) : run both full and light client
* fix(Java shutdown) : unsubscribe to sessions
Forgot to pass the JNIEnv environment since it is an instance method
* feat(return valid JString)
* Remove Java dependency by constructing a valid Java String in the callback
* fix(logger) : remove `rpc` trace log
* fix(format)
* fix(parity-clib): remove needless callback `type`
* fix(parity-clib-examples) : update examples
* `cpp` example pass in a struct instead to determines `callback kind`
* `java` add a instance variable the class `Callback` to determine `callback kind`
* fix(review comments): docs and format
* Update parity-clib/src/java.rs
Co-Authored-By: niklasad1 <niklasadolfsson1@gmail.com>
* fix(bad merge + spelling)
* fix(move examples to parity-clib/examples)
Fix: new blocks notifications sometimes missing in pubsub RPC
Implement new struct to pass to `new_blocks()` with extra parameter - `has_more_blocks_to_import`, which was previously used to determine whether the notification should be sent. Now it's up to each implementation to decide what to do.
Updated all implementations to behave as before, except `eth_pubsub`, which will send notification even when the queue is not empty.
Update tests.
* Add `is_idle` to LightSync to check importing status
* Use SyncStateWrapper to make sure is_idle gets updates
* Update is_major_import to use verified queue size as well
* Add comment for `is_idle`
* Add Debug to `SyncStateWrapper`
* `fn get` -> `fn into_inner`
* Remove the independent runtimes from `KeyServerHttpListener` and
`KeyServerCore` and instead require a `parity_runtime::Executor`
to be passed upon creation of each.
* Remove the `threads` parameter from both `ClusterConfiguration` structs.
* Implement the `future::Executor` trait for `parity_runtime::Executor`.
* Update tests.
- Update the `loop_until` function to instead use a oneshot to signal
completion.
- Modify the `make_key_servers` function to create and return a runtime.
* Fixing and disabling some tests for windows 10 compatibility.
* Few adjustment for windows in tests (eg bigger timeout for keyserver tests)
* Spaces and temporary single thread ci (to be able to spot the error).
* Clean up serde rename and use rename_all = camelCase when possible
* snake_case for pricing
* Use camelcase for engine
* Use camel case for seal
* Use camel case for validator set
* Use camel case for confirmation payload
* Use camel case for consensus status
* Use camel case for nodekind
* Use kebab case for provenance
* Use camel case for pubsub
* Use lowercase and camelcase for trace
* Use camel case for whisper
* rename Ethash as irregular name
* Replace `tokio_core` with `tokio`.
* Remove `tokio-core` and replace with `tokio` in
- `ethcore/stratum`
- `secret_store`
- `util/fetch`
- `util/reactor`
* Bump hyper to 0.12 in
- `miner`
- `util/fake-fetch`
- `util/fetch`
- `secret_store`
* Bump `jsonrpc-***` to 0.9 in
- `parity`
- `ethcore/stratum`
- `ipfs`
- `rpc`
- `rpc_client`
- `whisper`
* Bump `ring` to 0.13
* Use a more graceful shutdown process in `secret_store` tests.
* Convert some mutexes to rwlocks in `secret_store`.
* Consolidate Tokio Runtime use, remove `CpuPool`.
* Rename and move the `tokio_reactor` crate (`util/reactor`) to
`tokio_runtime` (`util/runtime`).
* Rename `EventLoop` to `Runtime`.
- Rename `EventLoop::spawn` to `Runtime::with_default_thread_count`.
- Add the `Runtime::with_thread_count` method.
- Rename `Remote` to `Executor`.
* Remove uses of `CpuPool` and spawn all tasks via the `Runtime` executor
instead.
* Other changes related to `CpuPool` removal:
- Remove `Reservations::with_pool`. `::new` now takes an `Executor` as an argument.
- Remove `SenderReservations::with_pool`. `::new` now takes an `Executor` as an argument.
* OnDemand no longer loop until there is a query.
All peer known at the time will be queried, and the query fail if all
return no reply.
Returning the failure is done through an empty Vec of reply (the type
of the oneshot channel remains unchanged).
Before this commit the query were send randomly to any peer until there
is a reply (for a query that got no result it was an issue, for other
queries it was quering multiple times the same peers).
After this commit the first query is random but next queries
follows hashmap iterator order.
Test no_capability was broken by this commit (the pending query was
removed).
* OnDemand no longer loop until there is a query.
All peer known at the time will be queried, and the query fail if all
return no reply.
Returning the failure is done through an empty Vec of reply (the type
of the oneshot channel remains unchanged).
Before this commit the query were send randomly to any peer until there
is a reply (for a query that got no result it was an issue, for other
queries it was quering multiple times the same peers).
After this commit the first query is random but next queries
follows hashmap iterator order.
Test no_capability was broken by this commit (the pending query was
removed). If adding some kind of timeout mechanism it could be restored.
* Comment plus better field names.
* No panick on dropped oneshot channel.
* Use Set to avoid counter heuristic
* Cli option `on_demand_nb_retry` for maximum number of retry when doing
on demand query in light client.
* Missing test update for previous commit
* Add a timeout (only when there is no peer to query), that way we do not
set number of query to minimum current number peer or configured number
of query : that way capability test was restored.
* Adding an error type for on_demand, it helps having variant of error
reported at rpc level : choice of rpc error code error might not be
right.
* Duration as constant is nice
* Switch to duration in main too
* Fix indentation (sorry for that).
* Fix error management (bad merge in previous commit)
* Lots of english corrections, major change on the new command parameters :
- use standard '-' instead of '_'
- renaming nb_retry params to 'on-demand-retry-count'
This PR is fixing deadlock for #8918
It avoids some recursive calls on light_sync by making state check optional for Informant.
The current behavior is to display the information when informant checks if block is major version.
This change a bit the informant behavior, but not on most cases.
To remember where and how this kind of deadlock are likely to happen (not seen with Parkinglot deadlock detection because it uses std condvar), I am adding a description of the deadlock.
Also, for the reviewers there may be better solution than modifying the informant.
### Thread1
- ethcore/sync/light_sync/mod.rs
A call to the light handler through any Io (having a loop of rpc query running on like client makes the dead lock way more likely).
At the end of those calls we systematically call `maintain_sync` method.
Here maintain_sync locks `state` (it is the deadlock cause), with a write purpose
`maintain_sync` -> `begin_search` with the state locked open
`begin_search` -> lightcliennt `flush_queue` method
- ethcore/light/src/client/mod.rs
`flush_queue` -> `flush` on queue (HeaderQueue aka VerificationQueue of headers)
- ethcore/src/verification/queue/mod.rs
Condition there is some unverified or verifying content
`flush` wait on a condvar until the queue is empty. The only way to unlock condvar is that worker is empty and unlock it (so thread 2 is Verification worker).
### Thread2
A verification worker at the end of a verify loop (new block).
- ethcore/src/verification/queue/mod.rs
thread loops on `verify` method.
End of loop condition is_ready -> Import the block immediately
calls `set_sync` on QueueSignal which send a BlockVerified ClientIoMessage in inner channel (IoChannel of ClientIoMessage) using `send_sync`
- util/io/src/service_mio.rs
IoChannel `send_sync` method calls all handlers with `message` method; one of the handlers is ImportBlocks IoHandler (with a single inner Client service field)
- ethcore/light/src/client/service.rs
`message` trigger inner method `import_verified`
- core/light/src/client/mod.rs
`import_verified` at the very end notify the listeners of a new_headers, one of the listeners is Informant `listener` method
- parity/informant.rs
`newHeaders` run up to call to `is_major_importing` on its target (again clinet)
- ethcore/sync/src/light_sync/mod.rs
Here `is_major_importing` tries to get state lock (read purpose only) but cannot because of previous state lock, thus deadlock
* Import the `home` crate in `util/dir`.
* Replace uses of `env::home_dir()` with `home::home_dir()`.
* `home` uses a 'correct' impl. on windows and the stdlib impl.
of `::home_dir` otherwise.
* Reexport `home::home_dir` from `util/dir`.
* Bump `util/dir` to 0.1.2.
* Added --tx-queue-no-early-reject flag to disable early tx queue rejects because of low gas price
* Fixed failing tests, clarified comments and simplified no_early_reject field name.
* Added test case for the --tx-queue-no-early-reject flag
* parity: fix UserDefaults json parser
* parity: use serde_derive for UserDefaults
* parity: support deserialization of old UserDefault json format
* parity: make UserDefaults serde backwards compatible
* parity: tabify indentation in UserDefaults
