* Add `is_idle` to LightSync to check importing status
* Use SyncStateWrapper to make sure is_idle gets updates
* Update is_major_import to use verified queue size as well
* Add comment for `is_idle`
* Add Debug to `SyncStateWrapper`
* `fn get` -> `fn into_inner`
* Add a dump of the state at the end of transaction for --json-test.
Also fixes json-test output on finish, and allow to put both on err or
out (--out-only and --err-only).
* Dump state resolution from trie, function behind evm-bin feature to avoid misuse.
* Rename 'slow' method to 'to_pod_full'.
Use cache first in 'to_pod_full', for in between commits case.
Change dump activation to use a function pointer instead.
* Fix tests.
* Query and add storage values to dump.
* Switch to use `require` method, even if less efficient it is better in
this case to reuse existing code.
Reuse of `storage_at` was not easy in this case (could not iterate and
use the method at the same time (refcell mutable borrow panics) so
keeping code as is.
* Switch to returning error.
Use 'base_storage_root' instead of 'storage_root'.
Added a test, it will only execute with json-test in ci, or when launch
with the feature.
* Renaming of command line parameters.
Comments fixes.
Minor code changes.
* Fix evmbin cmd parsing test.
* README update.
* Fix extra space and avoid clone call on copiable address.
* Revert test submodule.
* Revert wasm-test submodule.
* Use map_or instead of map + unwrap_or
* restore tests submodule
* PIP Table Cost relative to average peers instead of max peers
* Add tracing in PIP new_cost_table
* Update stat peer_count
* Use number of leeching peers for Light serve costs
* Fix test::light_params_load_share_depends_on_max_peers (wrong type)
* Remove (now) useless test
* Remove `load_share` from LightParams.Config
Prevent div. by 0
* Add LEECHER_COUNT_FACTOR
* PR Grumble: u64 to u32 for f64 casting
* Prevent u32 overflow for avg_peer_count
* Add tests for LightSync::Statistics
* Rename db_restore => client
* First step: make it compile!
* Second step: working implementation!
* Refactoring
* Fix tests
* PR Grumbles
* PR Grumbles WIP
* Migrate ancient blocks interating backward
* Early return in block migration if snapshot is aborted
* Remove RwLock getter (PR Grumble I)
* Remove dependency on `Client`: only used Traits
* Add test for recovering aborted snapshot recovery
* Add test for migrating old blocks
* Fix build
* PR Grumble I
* PR Grumble II
* PR Grumble III
* PR Grumble IV
* PR Grumble V
* PR Grumble VI
* Fix one test
* Fix test
* PR Grumble
* PR Grumbles
* PR Grumbles II
* Fix tests
* Release RwLock earlier
* Revert Cargo.lock
* Update _update ancient block_ logic: set local in `commit`
* Update typo in ethcore/src/snapshot/service.rs
Co-Authored-By: ngotchac <ngotchac@gmail.com>
* ethcore: use Machine::verify_transaction on parent block
also fixes off-by-one activation of transaction permission contract
* ethcore: clarify call to verify_transaction
* Add FIXME link for a state mod statement
* Remove unused benches feature in ethcore
* Fix evm benches compile
* Fix ethash benches compile
* Switch to criterion
* Update test reference.
Block test are really not working so I disabled a few by commenting
directly in source.
* Move ethtest commit cursor.
* Implements 'NoProof' engine from https://github.com/ethereum/tests/issues/464 .
Since tests has been regenerated those one were failing on block
difficulty check.
Update ethereum/tests, waiting for cost fix (block test are still
commented).
* Update tests submodule reference to latest (all test passing except an
identified case).
Fix block reward of constantinople json.
* Restore broken test by using old json tests files.
* Use CanonNoSeal instead of a custom engine, still have to include some
additional tests code.
* Gas upper limit check in json_chain test was bad, moving the test to
verification, the test is running in `verify_header_param`.
Note that test was previously only for ethash, and now for any engine.
* Restore old behavior (gas uper limit only for ethash engine), at the
cost of an additional trait method.
* Proper rpc test fix.
* Update tests submodule, add SStore bug tests.
* Fix json issue tabulation.
Update tests submodule to latest master (lot of new sstore tests
passing)
* Switch ethereum/tests to tag 6.0.0-beta.1 (no tests changes from latest
synch).
* Display hex with separator, use indirection instead of clone for copy
types.
* Fixing and disabling some tests for windows 10 compatibility.
* Few adjustment for windows in tests (eg bigger timeout for keyserver tests)
* Spaces and temporary single thread ci (to be able to spot the error).
* Clean up serde rename and use rename_all = camelCase when possible
* snake_case for pricing
* Use camelcase for engine
* Use camel case for seal
* Use camel case for validator set
* Use camel case for confirmation payload
* Use camel case for consensus status
* Use camel case for nodekind
* Use kebab case for provenance
* Use camel case for pubsub
* Use lowercase and camelcase for trace
* Use camel case for whisper
* rename Ethash as irregular name
* aura: emit ancestry actions for finalizing blocks
* aura: refactor is_epoch_end to get finalized blocks as argument
* ethcore: add is_epoch_end_light method to Engine
The full client now tracks finality by querying the engine on each block import,
and it also persists the finalization state to the DB. For the light client
current it doesn't persist finality information and only keeps track of finality
for epoch signals, by calling `is_epoch_end_light`. This method implements the
previously existing logic of building finality for all the blocks in the current
epoch and then checking the finalized blocks against the transition store.
* ethcore: allow finalizing current block
* aura: fix construction of finality proof
* aura: fix warnings
- missing docs for is_epoch_end_light
- unused method unfinalized_hashes in RollingFinality
* aura: fix clone on copy types
* If no subchain heads then try a different peer
* Add log when useless chain head
* Restrict ChainHead useless peer to ancient blocks
* sync: replace `limit_reorg` with `block_set` condition
* Replace `tokio_core` with `tokio`.
* Remove `tokio-core` and replace with `tokio` in
- `ethcore/stratum`
- `secret_store`
- `util/fetch`
- `util/reactor`
* Bump hyper to 0.12 in
- `miner`
- `util/fake-fetch`
- `util/fetch`
- `secret_store`
* Bump `jsonrpc-***` to 0.9 in
- `parity`
- `ethcore/stratum`
- `ipfs`
- `rpc`
- `rpc_client`
- `whisper`
* Bump `ring` to 0.13
* Use a more graceful shutdown process in `secret_store` tests.
* Convert some mutexes to rwlocks in `secret_store`.
* Consolidate Tokio Runtime use, remove `CpuPool`.
* Rename and move the `tokio_reactor` crate (`util/reactor`) to
`tokio_runtime` (`util/runtime`).
* Rename `EventLoop` to `Runtime`.
- Rename `EventLoop::spawn` to `Runtime::with_default_thread_count`.
- Add the `Runtime::with_thread_count` method.
- Rename `Remote` to `Executor`.
* Remove uses of `CpuPool` and spawn all tasks via the `Runtime` executor
instead.
* Other changes related to `CpuPool` removal:
- Remove `Reservations::with_pool`. `::new` now takes an `Executor` as an argument.
- Remove `SenderReservations::with_pool`. `::new` now takes an `Executor` as an argument.
* prevent zero networkID, closes#8345
Signed-off-by: Seun LanLege <seunlanlege@gmail.com>
* updated networkID of olymic chain spec
Signed-off-by: Seun LanLege <seunlanlege@gmail.com>
* Add signed refund
* Use signed 256-bit integer for sstore gas refund substate
* Fix tests
* Remove signed mod and use i128 directly
* Fix evm test case casting
* Fix jsontests ext signature
* Log block set in block_sync for easier debugging
* logging macros
* Match no args in sync logging macros
* Add QueueFull error
* Only allow importing headers if the first matches requested
* WIP
* Test for chain head gaps and log
* Calc distance even with 2 heads
* Revert previous commits, preparing simple fix
This reverts commit 5f38aa885b22ebb0e3a1d60120cea69f9f322628.
* Reject headers with no gaps when ChainHead
* Reset block sync download when queue full
* Simplify check for subchain heads
* Add comment to explain subchain heads filter
* Fix is_subchain_heads check and comment
* Prevent premature round completion after restart
This is a problem on mainnet where multiple stale peer requests will
force many rounds to complete quickly, forcing the retraction.
* Reset stale old blocks request after queue full
* Revert "Reject headers with no gaps when ChainHead"
This reverts commit 0eb865539e5dee37ab34f168f5fb643300de5ace.
* Add BlockSet to BlockDownloader logging
Currently it is difficult to debug this because there are two instances,
one for OldBlocks and one for NewBlocks. This adds the BlockSet to all
log messages for easy log filtering.
* Reset OldBlocks download from last enqueued
Previously when the ancient block queue was full it would restart the
download from the last imported block, so the ones still in the queue would be
redownloaded. Keeping the existing downloader instance and just
resetting it will start again from the last enqueued block.:wq
* Ignore expired Body and Receipt requests
* Log when ancient block download being restarted
* Only request old blocks from peers with >= difficulty
https://github.com/paritytech/parity-ethereum/pull/9226 might be too
permissive and causing the behaviour of the retraction soon after the
fork block. With this change the peer difficulty has to be greater than
or euqal to our syncing difficulty, so should still fix
https://github.com/paritytech/parity-ethereum/issues/9225
* Some logging and clear stalled blocks head
* Revert "Some logging and clear stalled blocks head"
This reverts commit 757641d9b817ae8b63fec684759b0815af9c4d0e.
* Reset stalled header if useless more than once
* Store useless headers in HashSet
* Add sync target to logging macro
* Don't disable useless peer and fix log macro
* Clear useless headers on reset and comments
* Use custom error for collecting blocks
Previously we resued BlockImportError, however only the Invalid case and
this made little sense with the QueueFull error.
* Remove blank line
* Test for reset sync after consecutive useless headers
* Don't reset after consecutive headers when chain head
* Delete commented out imports
* Return DownloadAction from collect_blocks instead of error
* Don't reset after round complete, was causing test hangs
* Add comment explaining reset after useless
* Replace HashSet with counter for useless headers
* Refactor sync reset on bad block/queue full
* Add missing target for log message
* Fix compiler errors and test after merge
* ethcore: revert ethereum tests submodule update
* `ExecutionsRequest` from light-clients as read-only
This changes so all `ExecutionRequests` from light-clients are executed
as read-only which the `virtual``flag == true ensures.
This boost up the current transaction to always succeed
Note, this only affects `eth_estimateGas` and `eth_call` AFAIK.
* grumbles(revert renaming) : TransactionProof
* grumbles(trace) : remove incorrect trace
* grumbles(state/prove_tx) : explicit `virt`
Remove the boolean flag to determine that a `state::prove_transaction`
whether it should be executed in a virtual context or not.
Because of that also rename the function to
`state::prove_transction_virtual` to make more clear
* Implement CREATE2 gas changes and fix some potential overflowing
* Ignore create2 state tests
* Split CREATE and CREATE2 in gasometer
* Generalize rounding (x + 31) / 32 to to_word_size
* sync: Validate received BlockHeaders packets against stored request.
* sync: Validate received BlockBodies and BlockReceipts.
* sync: Fix broken tests.
* sync: Unit tests for BlockDownloader::import_headers.
* sync: Unit tests for import_{bodies,receipts}.
* tests: Add missing method doc.
* Add new Vm trappable interface
* Exec/Resume interface
* Basic implementation of CallCreateExecutive
* Implement resume_call and resume_create for executive
* Move convertion to call/create result to separate function
* Implement consume that converts resumable to non-resumable
* Use consume for Executive::call/create
* Resumable EVM
* Implement tracing mode without needing subtracers
* Implement vmtracer so it doesn't require extra structs for subtracing
* Use the new tracing mode in executive
* Fix most of the linting errors for cargo build
* Add the concept of stack_depth
* Add back crossbeam
* Fix some test compile
* Fix prefix address test
* Fix evm crate tests
* Fix wasm crate test compile
* Fix wasm runner compile
* Fix jsontests compile
* Fix evmbin compile
* Fix an issue with create nonce and better vm tracing interface
* Fix linting
* Fix evmbin compile
* Fix unconfirmed_substate and static_flag
* Fix an issue in create address logic
* Fix top-level tracing
* Handle builtin tracing
* Fix suicide and reward tracing index stack
* Fix an issue where trap conflicts with tracing
* Fix an issue in parent step vm tracing
* Fix revert tracing
* Fix evmbin tests
* Remove params clone
* Fix TODO proofs
* Fix jsontests compile
* Fix evmbin merge issue
* Fix wasm merge issue
* Fix wasm test
* Fix ethcore merge warnings
* Fix evmbin compile
* Better expect messages and add some trace::skip_one asserts
* ethcore: propose hardfork block number 4230000 for ropsten
* ethcore: propose hardfork block number 9000000 for kovan
* ethcore: enable kip-4 and kip-6 on kovan
* etcore: bump kovan hardfork to block 9.2M
* ethcore: fix ropsten constantinople block number to 4.2M
* ethcore: disable difficulty_test_ropsten until ethereum/tests are updated upstream
* VerificationQueue don't spawn up extra threads
In the verification queue we spawn up worker threads to do the work.
However, if `num-verifiers` is specified we still spawn the maximum
number of threads which consume extra memory.
There is one catch though when `--scale-verifiers` is specified then
we can't do it because all threads are created upon initilization AFAIK.
In my opinion, is doesn't to use both `num-verifiers` and
`scale-verifiers` they are kind of contradictory!
* Fix nits in logic and add tests for verification
* refactor(verification queue) - rm hardcoded const
* Address grumbles in new tests
* Remove hardcoded `MAX_VERIFIERS` constant and replace it by relying
entirely on `num_cpu` crate instead inorder to support CPUs that have
more cores/logical cores
* Add constantinople conf to EvmTestClient.
* Skip some test to update submodule etheureum/tests submodule to latest.
* Put skipping 'under issue' test behind a feature.
* Change blockReward for const-test to pass ethereum/tests
* Update tests to new constantinple definition (change of reward at block
5).
Switch 'reference' to string, that way we can include issues from others
repo (more flexible)Update tests to new constantinple definition (change
of reward at block 5).
Switch 'reference' to string, that way we can include issues from others
repo (more flexible).
* Fix modexp and bn128_mul gas prices in chain config
* Changes `run_test_path` method to append its directory results (without
that it stop testing at the first file failure).
Add some missing tests.
Add skip for those (block create2 is one hundred percent false but on
hive we can see that geth and aleth got similar issue for this item).
* retab current.json
* Update reference to parity issue for failing tests.
* new ethabi migration in progress
* parity migrated to new ethabi
* migrated secred-store to new ethabi
* bump ethabi to 6.0
* fixed review suggestions
* OnDemand no longer loop until there is a query.
All peer known at the time will be queried, and the query fail if all
return no reply.
Returning the failure is done through an empty Vec of reply (the type
of the oneshot channel remains unchanged).
Before this commit the query were send randomly to any peer until there
is a reply (for a query that got no result it was an issue, for other
queries it was quering multiple times the same peers).
After this commit the first query is random but next queries
follows hashmap iterator order.
Test no_capability was broken by this commit (the pending query was
removed).
* OnDemand no longer loop until there is a query.
All peer known at the time will be queried, and the query fail if all
return no reply.
Returning the failure is done through an empty Vec of reply (the type
of the oneshot channel remains unchanged).
Before this commit the query were send randomly to any peer until there
is a reply (for a query that got no result it was an issue, for other
queries it was quering multiple times the same peers).
After this commit the first query is random but next queries
follows hashmap iterator order.
Test no_capability was broken by this commit (the pending query was
removed). If adding some kind of timeout mechanism it could be restored.
* Comment plus better field names.
* No panick on dropped oneshot channel.
* Use Set to avoid counter heuristic
* Cli option `on_demand_nb_retry` for maximum number of retry when doing
on demand query in light client.
* Missing test update for previous commit
* Add a timeout (only when there is no peer to query), that way we do not
set number of query to minimum current number peer or configured number
of query : that way capability test was restored.
* Adding an error type for on_demand, it helps having variant of error
reported at rpc level : choice of rpc error code error might not be
right.
* Duration as constant is nice
* Switch to duration in main too
* Fix indentation (sorry for that).
* Fix error management (bad merge in previous commit)
* Lots of english corrections, major change on the new command parameters :
- use standard '-' instead of '_'
- renaming nb_retry params to 'on-demand-retry-count'
* Enable all Constantinople hard fork changes in constantinople_test.json
* Address grumbles
* Remove EIP-210 activation
* 8m -> 5m
* Temporarily add back eip210 transition so we can get test passed
* Add eip210_test and remove eip210 transition from const_test
* Update & fix JSON state tests.
* Update tests to be able to run ethtest at
021fe3d410773024cd5f0387e62db6e6ec800f32.
- Touch user in state
- Adjust transaction tests to new json format
* Switch to same commit for submodule ethereum/test as geth (next includes constantinople changes).
Added test `json_tests::trie::generic::TrieTests_trieanyorder` and a few
difficulty tests.
* Remove trietestnextprev as it would require to parse differently and
implement it.
* Support new (shitty) format of transaction tests.
* Ignore junk in ethereum/tests repo.
* Ignore incorrect test.
* Update to a later commit
* Move block number to a constant.
* Fix ZK2 test - touched account should also be cleared.
* Fix conflict resolution
* Implement last_checkpoint_storage_at
* Add reverted_storage_at for externalities
* sstore_clears_count -> sstore_clears_refund
* Implement eip1283 for evm
* Add eip1283Transition params
* evm: fix tests
* jsontests: fix test
* Return checkpoint index when creating
* Comply with spec Version II
* Fix docs
* Fix jsontests feature compile
* Address grumbles
* Fix no-checkpoint-entry case
* Remove unnecessary expect
* Add test for State::checkpoint_storage_at
* Add executive level test for eip1283
* Hard-code transaction_checkpoint_index to 0
* Fix jsontests
* Add tests for checkpoint discard/revert
* Require checkpoint to be empty for kill_account and commit
* Get code coverage
* Use saturating_add/saturating_sub
* Fix issues in insert_cache
* Clear the state again
* Fix original_storage_at
* Early return for empty RLP trie storage
* Update comments
* Fix borrow_mut issue
* Simplify checkpoint_storage_at if branches
* Better commenting for gas handling code
* Address naming grumbles
* More tests
* Fix an issue in overwrite_with
* Add another test
* Fix comment
* Remove unnecessary bracket
* Move orig to inner if
* Remove test coverage for this PR
* Add tests for executive original value
* Add warn! for an unreachable cause
* ethcore: don't validate difficulty when ignoring seal check
* ethcore: fix block verification test
* ethcore: document skipped verifications when check_seal is disabled
This PR is fixing deadlock for #8918
It avoids some recursive calls on light_sync by making state check optional for Informant.
The current behavior is to display the information when informant checks if block is major version.
This change a bit the informant behavior, but not on most cases.
To remember where and how this kind of deadlock are likely to happen (not seen with Parkinglot deadlock detection because it uses std condvar), I am adding a description of the deadlock.
Also, for the reviewers there may be better solution than modifying the informant.
### Thread1
- ethcore/sync/light_sync/mod.rs
A call to the light handler through any Io (having a loop of rpc query running on like client makes the dead lock way more likely).
At the end of those calls we systematically call `maintain_sync` method.
Here maintain_sync locks `state` (it is the deadlock cause), with a write purpose
`maintain_sync` -> `begin_search` with the state locked open
`begin_search` -> lightcliennt `flush_queue` method
- ethcore/light/src/client/mod.rs
`flush_queue` -> `flush` on queue (HeaderQueue aka VerificationQueue of headers)
- ethcore/src/verification/queue/mod.rs
Condition there is some unverified or verifying content
`flush` wait on a condvar until the queue is empty. The only way to unlock condvar is that worker is empty and unlock it (so thread 2 is Verification worker).
### Thread2
A verification worker at the end of a verify loop (new block).
- ethcore/src/verification/queue/mod.rs
thread loops on `verify` method.
End of loop condition is_ready -> Import the block immediately
calls `set_sync` on QueueSignal which send a BlockVerified ClientIoMessage in inner channel (IoChannel of ClientIoMessage) using `send_sync`
- util/io/src/service_mio.rs
IoChannel `send_sync` method calls all handlers with `message` method; one of the handlers is ImportBlocks IoHandler (with a single inner Client service field)
- ethcore/light/src/client/service.rs
`message` trigger inner method `import_verified`
- core/light/src/client/mod.rs
`import_verified` at the very end notify the listeners of a new_headers, one of the listeners is Informant `listener` method
- parity/informant.rs
`newHeaders` run up to call to `is_major_importing` on its target (again clinet)
- ethcore/sync/src/light_sync/mod.rs
Here `is_major_importing` tries to get state lock (read purpose only) but cannot because of previous state lock, thus deadlock
* Add EIP-1014 transition config flag
* Remove EIP-86 configs
* Change CREATE2 opcode index to 0xf5
* Move salt to the last item in the stack
* Change sendersaltandaddress scheme to comply with current EIP-1014
* Fix json configs
* Fix create2 test
* Fix deprecated comments
- Update foundation hardcoded header to block 6219777
- Update ropsten hardcoded header to block 3917825
- Update kovan hardcoded header to block 8511489
