* Comments and todos
Use `snapshot_sync` as logging target
* fix compilation
* More todos, more logs
* Fix picking snapshot peer: prefer the one with the highest block number
More docs, comments, todos
* Adjust WAIT_PEERS_TIMEOUT to be a multiple of MAINTAIN_SYNC_TIMER to try to fix snapshot startup problems
Docs, todos, comments
* Tabs
* Formatting
* Don't build new rlp::EMPTY_LIST_RLP instances
* Dial down debug logging
* Don't warn about missing hashes in the manifest: it's normal
Log client version on peer connect
* Cleanup
* Do not skip snapshots further away than 30k block from the highest block seen
Currently we look for peers that seed snapshots that are close to the highest block seen on the network (where "close" means withing 30k blocks). When a node starts up we wait for some time (5sec, increased here to 10sec) to let peers connect and if we have found a suitable peer to sync a snapshot from at the end of that delay, we start the download; if none is found and --warp-barrier is used we stall, otherwise we start a slow-sync.
When looking for a suitable snapshot, we use the highest block seen on the network to check if a peer has a snapshot that is within 30k blocks of that highest block number. This means that in a situation where all available snapshots are older than that, we will often fail to start a snapshot at all. What's worse is that the longer we delay starting a snapshot sync (to let more peers connect, in the hope of finding a good snapshot), the more likely we are to have seen a high block and thus the more likely we become to accept a snapshot.
This commit removes this comparison with the highest blocknumber criteria entirely and picks the best snapshot we find in 10sec.
* lockfile
* Add a `ChunkType::Dupe` variant so that we do not disconnect a peer if they happen to send us a duplicate chunk (just ignore the chunk and keep going)
Resolve some documentation todos, add more
* tweak log message
* Don't warp sync twice
Check if our own block is beyond the given warp barrier (can happen after we've completed a warp sync but are not quite yet synced up to the tip) and if so, don't sync.
More docs, resolve todos.
Dial down some `sync` debug level logging to trace
* Avoid iterating over all snapshot block/state hashes to find the next work item
Use a HashSet instead of a Vec and remove items from the set as chunks are processed. Calculate and store the total number of chunks in the `Snapshot` struct instead of counting pending chunks each time.
* Address review grumbles
* Log correct number of bytes written to disk
* Revert ChunkType::Dup change
* whitespace grumble
* Cleanup debugging code
* Fix docs
* Fix import and a typo
* Fix test impl
* Use `indexmap::IndexSet` to ensure chunk hashes are accessed in order
* Revert increased SNAPSHOT_MANIFEST_TIMEOUT: 5sec should be enough
* Replace error chain for network error
* Fix usages and add manual From impls
* OnDemand Error and remove remaining dependencies
* Die error_chain, die.
* DIE
* Hasta la vista, baby
* Fix nasty typo in NodeTable::update (add ;)
* Add limiting for NodeTable
* Add cache for NodeFilter
* Use expect instead of unwrap
* Move node in ordered_ids if it exists there in note_failure and note_success + fix expect msg
* Add comment
* Improve code style
* DRY in note_failure and note_success
* Fix nodes ordering
* Simplify match expression
* Add tests for get_index_to_insert
* Remove get_mut method from NodeTable, Add get method to NodeTable
* Fix table_last_contact_order for macos failing because of lost nanosecond precision
* Increase the number of block bodies requested during Sync.
* Increase the number of block bodies requested during Sync.
* Check if our peer is an older parity client with the bug
of not handling large requests properly
* Add a ClientVersion struct and a ClientCapabilites trait
* Make ClientVersion its own module
* Refactor and extend use of ClientVersion
* Replace strings with ClientVersion in PeerInfo
* Group further functionality in ClientCapabilities
* Move parity client version data from tuple to its own struct.
* Implement accessor methods for ParityClientData and remove them
from ClientVersion.
* Minor fixes
* Make functions specific to parity return types specific to parity.
* Test for shorter ID strings
* Fix formatting and remove unneeded dependencies.
* Roll back Cargo.lock
* Commit last Cargo.lock
* Convert from string to ClientVersion
* * When checking if peer accepts service transactions just check
if it's parity, remove version check.
* Remove dependency on semver in ethcore-sync
* Remove unnecessary String instantiation
* Rename peer_info to peer_version
* Update RPC test helpers
* Simplify From<String>
* Parse static version string only once
* Update RPC tests to new ClientVersion struct
* Document public members
* More robust parsing of ID string
* Minor changes.
* Update version in which large block bodies requests appear.
* Update ethcore/sync/src/block_sync.rs
Co-Authored-By: elferdo <elferdo@gmail.com>
* Update util/network/src/client_version.rs
Co-Authored-By: elferdo <elferdo@gmail.com>
* Update util/network/src/client_version.rs
Co-Authored-By: elferdo <elferdo@gmail.com>
* Update tests.
* Minor fixes.
* discovery: Only add nodes to routing table after receiving pong.
Previously the discovery algorithm would add nodes to the routing table
before confirming that the endpoint is participating in the protocol. This
now tracks in-flight pings and adds to the routing table only after receiving
a response.
* discovery: Refactor packet creation into its own function.
This function is useful inside unit tests.
* discovery: Additional testing for new add_node behavior.
* discovery: Track expiration of pings to non-yet-in-bucket nodes.
Now that we may ping nodes before adding to a k-bucket, the timeout tracking
must be separate from BucketEntry.
* discovery: Verify echo hash on pong packets.
Stores packet hash with in-flight requests and matches with pong response.
* discovery: Track timeouts on FIND_NODE requests.
* discovery: Retry failed pings with exponential backoff.
UDP packets may get dropped, so instead of immediately booting nodes that fail
to respond to a ping, retry 4 times with exponential backoff.
* !fixup Use slice instead of Vec for request_backoff.
* devp2p: Move UDP socket handling from Discovery to Host.
* devp2p: Fix bug with potentially incorrect UDP registration.
This works right now because the Host handler happens to be the first one
registered on the IoService.
* devp2p: Use 0-initialized memory buffer instead of unsafe.
* Remove send_queue field from public interface of Discovery.
* Rename Datagramm to Datagram.
sed -i 's/Datagramm/Datagram/g' util/network-devp2p/src/discovery.rs util/network-devp2p/src/host.rs
sed -i 's/datagramm/datagram/g' util/network-devp2p/src/discovery.rs util/network-devp2p/src/host.rs
* Include target in log statements.
* Update `add_license` script
* run script
* add `remove duplicate lines script` and run it
* Revert changes `English spaces`
* strip whitespaces
* Revert `GPL` in files with `apache/mit license`
* don't append `gpl license` in files with other lic
* Don't append `gpl header` in files with other lic.
* re-ran script
* include c and cpp files too
* remove duplicate header
* rebase nit
* Replace Rlp with UntrustedRlp and unsafely unwrap
All Rlp methods return Result<_,DecoderError> now, so for this first
pass each will be marked with `expect("TODO")`. In the next pass we can
categorise figure out how to handle each case.
* Handle DecoderError for tendermint message
* Unwrap rlp results in TestBlockcChainClient
Rlp should be valid since created manually in tests
* Replace `use rlp::*` with explicit imports
* Remove rlp decode unwraps from light cli request
* Structured rlp encoding for curr best and latest in header chain
* Propogate decoder errors from send_packet
* Fix body uncles rlp index
* Use BodyView in sync and `expect` rlp errors
* Revert bbf28f removing original Rlp for this phase
This can be done again in the next phase, in order that we can leave the ethcore views unchanged
* Restore legacy Rlp and UntrustedRlp
Use legacy Rlp for ethcore views. Will redo replacing Rlp with UntrustedRlp in a subsequent PR
* Fix tests
* Replace boilerplate Encodable/Decodable with derive
* Use BlockView instead of Rlp, remove unwrap
* Remove rlp test_cli unwraps by using BlockView instead of Rlp directly
* Remove unneccesary change to use borrowed hash
* Construct sync block using new_from_header_and_body
