Define a new `enum` where devp2p subprotocol packet ids (currently eth and par) are defined. Additionally provide functionality to query id value and protocol of a given id object.
* fix(light-rpc): Make `light_sync` generic
The motivation behind this change is to easily mock `light-sync` to make it possible to enable `rpc-integration` tests
for the light-client.
Currently the `rpc's` requires the concrete type `sync::LightSync` which makes it very hard to do so
* fix(bad merge)
* Make the specification of the protocol to which a packet_id belongs to
explicit when calling "SyncRequester::send_packet".
* Remove "SyncIO::send" and leave only "SyncIO::send_protocol"
* Adapt tests to new code.
* Strengthen tests to check if packet_id and protocol match when sending
a devp2p packet.
* Increase the number of block bodies requested during Sync.
* Increase the number of block bodies requested during Sync.
* Check if our peer is an older parity client with the bug
of not handling large requests properly
* Add a ClientVersion struct and a ClientCapabilites trait
* Make ClientVersion its own module
* Refactor and extend use of ClientVersion
* Replace strings with ClientVersion in PeerInfo
* Group further functionality in ClientCapabilities
* Move parity client version data from tuple to its own struct.
* Implement accessor methods for ParityClientData and remove them
from ClientVersion.
* Minor fixes
* Make functions specific to parity return types specific to parity.
* Test for shorter ID strings
* Fix formatting and remove unneeded dependencies.
* Roll back Cargo.lock
* Commit last Cargo.lock
* Convert from string to ClientVersion
* * When checking if peer accepts service transactions just check
if it's parity, remove version check.
* Remove dependency on semver in ethcore-sync
* Remove unnecessary String instantiation
* Rename peer_info to peer_version
* Update RPC test helpers
* Simplify From<String>
* Parse static version string only once
* Update RPC tests to new ClientVersion struct
* Document public members
* More robust parsing of ID string
* Minor changes.
* Update version in which large block bodies requests appear.
* Update ethcore/sync/src/block_sync.rs
Co-Authored-By: elferdo <elferdo@gmail.com>
* Update util/network/src/client_version.rs
Co-Authored-By: elferdo <elferdo@gmail.com>
* Update util/network/src/client_version.rs
Co-Authored-By: elferdo <elferdo@gmail.com>
* Update tests.
* Minor fixes.
* Extract accounts from ethcore.
* Fix ethcore.
* Get rid of AccountProvider in test_helpers
* Fix rest of the code.
* Re-use EngineSigner, fix tests.
* Simplify EngineSigner to always have an Address.
* Fix RPC tests.
* Add deprecation notice to RPCs.
* Feature to disable accounts.
* extract accounts in RPC
* Run with accounts in tests.
* Fix RPC compilation and tests.
* Fix compilation of the binary.
* Fix compilation of the binary.
* Fix compilation with accounts enabled.
* Fix tests.
* Update submodule.
* Remove android.
* Use derive for Default
* Don't build secretstore by default.
* Add link to issue.
* Refresh Cargo.lock.
* Fix miner tests.
* Update rpc/Cargo.toml
Co-Authored-By: tomusdrw <tomusdrw@users.noreply.github.com>
* Fix private tests.
* Patch available private contracts during private transaction
* Key acl ABI added
* Work with secret store keys moved to the separate struct
* Private tx test refactored
* Revert "Private tx test refactored"
This reverts commit 476c132d692c7a886bc7b7cd7fe47b3d7692bd63.
* Test for calling private contract from another one added
* Test fixed
* Redundant tab removed
* ACL contract processing fixed, test added
* Merge with head
* Expect replaced with closure
* fix(ManageNetwork): replace Range -> RangeIncls
Fixes `TODO: Range should be changed to RangeInclusive once stable (https://github.com/rust-lang/rust/pull/50758)`
* fix(tests)
* fix(grumbles): off-by-one error in debug_asserts
* RangeInclusive::end() is inclusive which means that if start and end is equal the `debug_assert(range.end() >
range.start()` will fail which is shouldn't
* Replace ethcore-logger with env-logger.
* Fix logger initialization in WASM tests.
* uncomment logger initialization in secret store
* Don't use ethcore-logger in whisper.
* Move ethcore-logger within parity dir.
* Uncomment rest from secret-store.
* Use `let _ =` in private_contract for consistency
* `ok()` to `let _ =` fix in service
* Use `let _ = ` for state_db
* Add private tx enabled flag into status packet
* Error log added for the case with no peers available
* Add flag only for supported protocol versions
* Work with private handler refactored
* Log target changed
* Cargo.lock updated
* fix#10125
fix service transaction version detection if --identity is enabled, change test to match how --identity actually works
* fix wrong var
* get the index of v, not /
* idx, not idx.len()
* Update ethcore/sync/src/chain/propagator.rs
Co-Authored-By: joshua-mir <43032097+joshua-mir@users.noreply.github.com>
* Update ethcore/sync/src/chain/propagator.rs
Co-Authored-By: joshua-mir <43032097+joshua-mir@users.noreply.github.com>
* change version prefix to a const
* space
Co-Authored-By: joshua-mir <43032097+joshua-mir@users.noreply.github.com>
Fix: new blocks notifications sometimes missing in pubsub RPC
Implement new struct to pass to `new_blocks()` with extra parameter - `has_more_blocks_to_import`, which was previously used to determine whether the notification should be sent. Now it's up to each implementation to decide what to do.
Updated all implementations to behave as before, except `eth_pubsub`, which will send notification even when the queue is not empty.
Update tests.
* Add `is_idle` to LightSync to check importing status
* Use SyncStateWrapper to make sure is_idle gets updates
* Update is_major_import to use verified queue size as well
* Add comment for `is_idle`
* Add Debug to `SyncStateWrapper`
* `fn get` -> `fn into_inner`
* PIP Table Cost relative to average peers instead of max peers
* Add tracing in PIP new_cost_table
* Update stat peer_count
* Use number of leeching peers for Light serve costs
* Fix test::light_params_load_share_depends_on_max_peers (wrong type)
* Remove (now) useless test
* Remove `load_share` from LightParams.Config
Prevent div. by 0
* Add LEECHER_COUNT_FACTOR
* PR Grumble: u64 to u32 for f64 casting
* Prevent u32 overflow for avg_peer_count
* Add tests for LightSync::Statistics
* Rename db_restore => client
* First step: make it compile!
* Second step: working implementation!
* Refactoring
* Fix tests
* PR Grumbles
* PR Grumbles WIP
* Migrate ancient blocks interating backward
* Early return in block migration if snapshot is aborted
* Remove RwLock getter (PR Grumble I)
* Remove dependency on `Client`: only used Traits
* Add test for recovering aborted snapshot recovery
* Add test for migrating old blocks
* Fix build
* PR Grumble I
* PR Grumble II
* PR Grumble III
* PR Grumble IV
* PR Grumble V
* PR Grumble VI
* Fix one test
* Fix test
* PR Grumble
* PR Grumbles
* PR Grumbles II
* Fix tests
* Release RwLock earlier
* Revert Cargo.lock
* Update _update ancient block_ logic: set local in `commit`
* Update typo in ethcore/src/snapshot/service.rs
Co-Authored-By: ngotchac <ngotchac@gmail.com>
* If no subchain heads then try a different peer
* Add log when useless chain head
* Restrict ChainHead useless peer to ancient blocks
* sync: replace `limit_reorg` with `block_set` condition
* Log block set in block_sync for easier debugging
* logging macros
* Match no args in sync logging macros
* Add QueueFull error
* Only allow importing headers if the first matches requested
* WIP
* Test for chain head gaps and log
* Calc distance even with 2 heads
* Revert previous commits, preparing simple fix
This reverts commit 5f38aa885b22ebb0e3a1d60120cea69f9f322628.
* Reject headers with no gaps when ChainHead
* Reset block sync download when queue full
* Simplify check for subchain heads
* Add comment to explain subchain heads filter
* Fix is_subchain_heads check and comment
* Prevent premature round completion after restart
This is a problem on mainnet where multiple stale peer requests will
force many rounds to complete quickly, forcing the retraction.
* Reset stale old blocks request after queue full
* Revert "Reject headers with no gaps when ChainHead"
This reverts commit 0eb865539e5dee37ab34f168f5fb643300de5ace.
* Add BlockSet to BlockDownloader logging
Currently it is difficult to debug this because there are two instances,
one for OldBlocks and one for NewBlocks. This adds the BlockSet to all
log messages for easy log filtering.
* Reset OldBlocks download from last enqueued
Previously when the ancient block queue was full it would restart the
download from the last imported block, so the ones still in the queue would be
redownloaded. Keeping the existing downloader instance and just
resetting it will start again from the last enqueued block.:wq
* Ignore expired Body and Receipt requests
* Log when ancient block download being restarted
* Only request old blocks from peers with >= difficulty
https://github.com/paritytech/parity-ethereum/pull/9226 might be too
permissive and causing the behaviour of the retraction soon after the
fork block. With this change the peer difficulty has to be greater than
or euqal to our syncing difficulty, so should still fix
https://github.com/paritytech/parity-ethereum/issues/9225
* Some logging and clear stalled blocks head
* Revert "Some logging and clear stalled blocks head"
This reverts commit 757641d9b817ae8b63fec684759b0815af9c4d0e.
* Reset stalled header if useless more than once
* Store useless headers in HashSet
* Add sync target to logging macro
* Don't disable useless peer and fix log macro
* Clear useless headers on reset and comments
* Use custom error for collecting blocks
Previously we resued BlockImportError, however only the Invalid case and
this made little sense with the QueueFull error.
* Remove blank line
* Test for reset sync after consecutive useless headers
* Don't reset after consecutive headers when chain head
* Delete commented out imports
* Return DownloadAction from collect_blocks instead of error
* Don't reset after round complete, was causing test hangs
* Add comment explaining reset after useless
* Replace HashSet with counter for useless headers
* Refactor sync reset on bad block/queue full
* Add missing target for log message
* Fix compiler errors and test after merge
* ethcore: revert ethereum tests submodule update
* sync: Validate received BlockHeaders packets against stored request.
* sync: Validate received BlockBodies and BlockReceipts.
* sync: Fix broken tests.
* sync: Unit tests for BlockDownloader::import_headers.
* sync: Unit tests for import_{bodies,receipts}.
* tests: Add missing method doc.
This PR is fixing deadlock for #8918
It avoids some recursive calls on light_sync by making state check optional for Informant.
The current behavior is to display the information when informant checks if block is major version.
This change a bit the informant behavior, but not on most cases.
To remember where and how this kind of deadlock are likely to happen (not seen with Parkinglot deadlock detection because it uses std condvar), I am adding a description of the deadlock.
Also, for the reviewers there may be better solution than modifying the informant.
### Thread1
- ethcore/sync/light_sync/mod.rs
A call to the light handler through any Io (having a loop of rpc query running on like client makes the dead lock way more likely).
At the end of those calls we systematically call `maintain_sync` method.
Here maintain_sync locks `state` (it is the deadlock cause), with a write purpose
`maintain_sync` -> `begin_search` with the state locked open
`begin_search` -> lightcliennt `flush_queue` method
- ethcore/light/src/client/mod.rs
`flush_queue` -> `flush` on queue (HeaderQueue aka VerificationQueue of headers)
- ethcore/src/verification/queue/mod.rs
Condition there is some unverified or verifying content
`flush` wait on a condvar until the queue is empty. The only way to unlock condvar is that worker is empty and unlock it (so thread 2 is Verification worker).
### Thread2
A verification worker at the end of a verify loop (new block).
- ethcore/src/verification/queue/mod.rs
thread loops on `verify` method.
End of loop condition is_ready -> Import the block immediately
calls `set_sync` on QueueSignal which send a BlockVerified ClientIoMessage in inner channel (IoChannel of ClientIoMessage) using `send_sync`
- util/io/src/service_mio.rs
IoChannel `send_sync` method calls all handlers with `message` method; one of the handlers is ImportBlocks IoHandler (with a single inner Client service field)
- ethcore/light/src/client/service.rs
`message` trigger inner method `import_verified`
- core/light/src/client/mod.rs
`import_verified` at the very end notify the listeners of a new_headers, one of the listeners is Informant `listener` method
- parity/informant.rs
`newHeaders` run up to call to `is_major_importing` on its target (again clinet)
- ethcore/sync/src/light_sync/mod.rs
Here `is_major_importing` tries to get state lock (read purpose only) but cannot because of previous state lock, thus deadlock
* Verify private transaction before propagating
* Private transactions queue reworked with tx pool queue direct usage
* Styling fixed
* Prevent resending private packets to the sender
* Process signed private transaction packets via io queue
* Test fixed
* Build and test fixed after merge
* Comments after review fixed
* Signed transaction taken from verified
* Fix after merge
* Pool scoring generalized in order to use externally
* Lib refactored according to the review comments
* Ready state refactored
* Redundant bound and copying removed
* Fixed build after the merge
* Forgotten case reworked
* Review comments fixed
* Logging reworked, target added
* Fix after merge
* Add a `fastmap` crate that provides the H256FastMap specialized HashMap
* Use `fastmap` instead of `plain_hasher`
* Update submodules for Reasons™
* Submodule update
