Commit Graph

11592 Commits

Author SHA1 Message Date
Nicolas Gotchac
4ddd69cc55 Multithreaded snapshot creation (#9239)
* Add Progress to Snapshot Secondary chunks creation

* Use half of CPUs to multithread snapshot creation

* Use env var to define number of threads

* info to debug logs

* Add Snapshot threads as CLI option

* Randomize chunks per thread

* Remove randomness, add debugging

* Add warning

* Add tracing

* Use parity-common fix seek branch

* Fix log

* Fix tests

* Fix tests

* PR Grumbles

* PR Grumble II

* Update Cargo.lock

* PR Grumbles

* Default snapshot threads to half number of CPUs

* Fix default snapshot threads // min 1
2018-09-13 12:58:49 +02:00
Marek Kotewicz
ef4a61c769
new ethabi (#9511)
* new ethabi migration in progress

* parity migrated to new ethabi

* migrated secred-store to new ethabi

* bump ethabi to 6.0

* fixed review suggestions
2018-09-13 11:04:39 +02:00
Tomasz Drwięga
7dfb5ff5bd Remove initial token for WS. (#9545) 2018-09-13 10:58:52 +02:00
Marek Kotewicz
6b391312ab net_version caches network_id to avoid redundant aquire of sync read lock (#9544)
* net_version caches network_id to avoid redundant aquire of sync read lock, #8746

* use lower_hex display formatting for net_peerCount rpc method
2018-09-13 10:16:24 +02:00
gabriel klawitter
6e62d77e4d correct before_script for nightly build versions (#9543)
- fix gitlab array of strings syntax error
- get proper commit id
- avoid colon in stings
2018-09-12 17:46:56 +03:00
Afri Schoedon
0281cca9af deps: bump kvdb-rocksdb to 0.1.4 (#9539) 2018-09-12 13:23:09 +02:00
Wei Tang
018e2403b1 state: test when contract creation fails, old storage values should re-appear (#9532)
Because more tests won't hurt. :)

Add a test case for https://github.com/ethereum/py-evm/pull/1224#issuecomment-418775512 where if contract creation fails, old storage values (if ever existed) should re-appear.
2018-09-12 11:42:09 +01:00
cheme
61f4534e2a Allow dropping light client RPC query with no results (#9318)
* OnDemand no longer loop until there is a query.
All peer known at the time will be queried, and the query fail if all
return no reply.
Returning the failure is done through an empty Vec of reply (the type
of the oneshot channel remains unchanged).
Before this commit the query were send randomly to any peer until there
is a reply (for a query that got no result it was an issue, for other
queries it was quering multiple times the same peers).
After this commit the first query is random but next queries
follows hashmap iterator order.

Test no_capability was broken by this commit (the pending query was
removed).

* OnDemand no longer loop until there is a query.
All peer known at the time will be queried, and the query fail if all
return no reply.
Returning the failure is done through an empty Vec of reply (the type
of the oneshot channel remains unchanged).
Before this commit the query were send randomly to any peer until there
is a reply (for a query that got no result it was an issue, for other
queries it was quering multiple times the same peers).
After this commit the first query is random but next queries
follows hashmap iterator order.

Test no_capability was broken by this commit (the pending query was
removed). If adding some kind of timeout mechanism it could be restored.

* Comment plus better field names.

* No panick on dropped oneshot channel.

* Use Set to avoid counter heuristic

* Cli option `on_demand_nb_retry` for maximum number of retry when doing
on demand query in light client.

* Missing test update for previous commit

* Add a timeout (only when there is no peer to query), that way we do not
set number of query to minimum current number peer or configured number
of query : that way capability test was restored.

* Adding an error type for on_demand, it helps having variant of error
reported at rpc level : choice of rpc error code error might not be
right.

* Duration as constant is nice

* Switch to duration in main too

* Fix indentation (sorry for that).

* Fix error management (bad merge in previous commit)

* Lots of english corrections, major change on the new command parameters :
 - use standard '-' instead of '_'
 - renaming nb_retry params to 'on-demand-retry-count'
2018-09-12 11:47:01 +02:00
Afri Schoedon
69667317c1
Bump master to 2.2.0 (#9517)
* parity-version: bump master to 2.2.0

* ci: update branch version references

* docker: release master to latest

* deps: bump fs-swap to 0.2.4
2018-09-12 11:32:05 +02:00
Wei Tang
530aac0682
Enable all Constantinople hard fork changes in constantinople_test.json (#9505)
* Enable all Constantinople hard fork changes in constantinople_test.json

* Address grumbles

* Remove EIP-210 activation

* 8m -> 5m

* Temporarily add back eip210 transition so we can get test passed

* Add eip210_test and remove eip210 transition from const_test
2018-09-12 02:08:23 +08:00
Niklas Adolfsson
6e7d8f90b5 [light] Validate account balance before importing transactions (#9417)
* `light::verify_transaction` basic tx validation

* update wasm tests

* Provide `cached_nonce` in `parity_next_nonce` RPC

* nits

* Improve error handeling

Two separate errors for distinguishing between `account not found` and
`insufficient balance`. However, when `next_nonce()` is called and the
account is not found then provide `local_best_next_nonce`!

* Ensure only one n/w request is performed

Refactored to code again:
* Removed `fn cached_next_nonce`
* Removed extra n/w request in `sign` to check balance
* Refactored `fill_optional_field` to request nonce and check account balance

* nits

* grumbles needless clone

* Prevent integer overflow with saturating add & mul

* Call `sign_transaction()` directly from `sign()`

Because the change in `fill_optional_fields` always fill the nonce it is
now possible to call `sign_transaction` directly instead of creating a
`ProspectiveSigner` "object".
2018-09-11 19:20:59 +02:00
cheme
65bf1086a2 In create memory calculation is the same for create2 because the additional parameter was popped before. (#9522) 2018-09-11 23:47:26 +08:00
cheme
98220442b4 Update patricia trie to 0.2.2 (#9525)
* Update patricia trie to 0.2.2 crates. Default dependencies on minor
version only.

* Putting back ethereum tests to the right commit
2018-09-11 16:20:26 +02:00
Hernando Castano
3c3d2ef2b9 Replace hardcoded JSON with serde json! macro (#9489)
* Replace hardcoded JSON with serde json! macro

* Use "{:#x}" formatter instead of "0x{:x}"

* Sort fields of JSON test strings alphabetically

* Stop escaping new lines in evmbin JSON errors

* Remove unnecessary 'to_string()' calls

* Add test with non-empty storage values
2018-09-11 11:36:38 +02:00
gabriel klawitter
67066eb32a
fix typo in version string (#9516) 2018-09-11 11:31:25 +02:00
Wei Tang
631df0fe56 Fix checkpointing when creating contract failed (#9514) 2018-09-11 10:28:31 +02:00
Andronik Ordian
485870296d ci: fix json docs generation (#9515) 2018-09-11 09:38:35 +02:00
Tomasz Drwięga
6e5a1c00dc Update state tests execution model (#9440)
* Update & fix JSON state tests.

* Update tests to be able to run ethtest at
021fe3d410773024cd5f0387e62db6e6ec800f32.

- Touch user in state
- Adjust transaction tests to new json format

* Switch to same commit for submodule ethereum/test as geth (next includes constantinople changes).
Added test `json_tests::trie::generic::TrieTests_trieanyorder` and a few
difficulty tests.

* Remove trietestnextprev as it would require to parse differently and
implement it.

* Support new (shitty) format of transaction tests.

* Ignore junk in ethereum/tests repo.

* Ignore incorrect test.

* Update to a later commit

* Move block number to a constant.

* Fix ZK2 test - touched account should also be cleared.

* Fix conflict resolution
2018-09-10 22:38:30 +02:00
Wei Tang
ba487eaaca
Only check warp syncing for eth_getWorks (#9484)
* Only check warp syncing for eth_getWorks

* Use SyncStatus::is_snapshot_syncing
2018-09-11 01:53:42 +08:00
gabriel klawitter
eb0986c3f9 upload will fail if a file with the same hash is already up (#9479)
* upload will fail if a file with the same hash is already up

* compose version string for nightly releases
2018-09-10 18:22:34 +03:00
Nicolas Gotchac
baf5be09dc Fix Snapshot restoration failure on Windows (#9491)
* Close Blooms DB files before DB restoration

* PR Grumbles I

* PR Grumble

* Grumble
2018-09-10 17:21:57 +02:00
Nicolas Gotchac
f3aed42dd6 Don't error when Snapshot is aborted (#9492)
* Don't error when Snapshot is aborted

* PR Grumble

* PR grumble
2018-09-10 21:26:52 +08:00
Wei Tang
44531e3009
light: give free credits for reserved peers (#9448)
* light: give free credits for reserved peers

* Fix ethcore-light tests

* Test free_flow_params
2018-09-10 19:45:49 +08:00
Afri Schoedon
2177a0179e
parity: print correct keys path on startup (#9501) 2018-09-09 00:43:24 +02:00
Wei Tang
e1f333021f Generalized blockReward and difficultyBombDelays config (#9480)
* Implement multi blockReward

* Implement difficultyBombDelays

* Fix json crate compile

* json keys can only be string
2018-09-09 00:38:07 +02:00
Andronik Ordian
4040d73c60 rpc(debug_getBadBlocks): fix test (#9502) 2018-09-08 17:34:51 +02:00
Tomasz Drwięga
61bd47ccc1 Bad blocks RPC + reporting (#9433)
* Bad blocks RPC.

* Return bad blocks via RPC.

* Fix test.

* More verbose bad block message.

* Expose via CLI.

* Remove stray whitespace.

* Remove stray newline.

* Fix tests.
2018-09-08 04:04:28 +02:00
Wei Tang
915c366056
EIP 1283: Net gas metering for SSTORE without dirty maps (#9319)
* Implement last_checkpoint_storage_at

* Add reverted_storage_at for externalities

* sstore_clears_count -> sstore_clears_refund

* Implement eip1283 for evm

* Add eip1283Transition params

* evm: fix tests

* jsontests: fix test

* Return checkpoint index when creating

* Comply with spec Version II

* Fix docs

* Fix jsontests feature compile

* Address grumbles

* Fix no-checkpoint-entry case

* Remove unnecessary expect

* Add test for State::checkpoint_storage_at

* Add executive level test for eip1283

* Hard-code transaction_checkpoint_index to 0

* Fix jsontests

* Add tests for checkpoint discard/revert

* Require checkpoint to be empty for kill_account and commit

* Get code coverage

* Use saturating_add/saturating_sub

* Fix issues in insert_cache

* Clear the state again

* Fix original_storage_at

* Early return for empty RLP trie storage

* Update comments

* Fix borrow_mut issue

* Simplify checkpoint_storage_at if branches

* Better commenting for gas handling code

* Address naming grumbles

* More tests

* Fix an issue in overwrite_with

* Add another test

* Fix comment

* Remove unnecessary bracket

* Move orig to inner if

* Remove test coverage for this PR

* Add tests for executive original value

* Add warn! for an unreachable cause
2018-09-07 18:51:08 +08:00
Niklas Adolfsson
6888a968f9 Light clippy(fy) (#9473)
* wasm tests

* `clippyfy` light-client

* Revert inefficient change `collect_ready()`
2018-09-06 15:44:40 +02:00
André Silva
4e8e5bbb86 aura: don't report skipped primaries when empty steps are enabled (#9435) 2018-09-06 13:33:46 +02:00
Wei Tang
5752869824 Support millisecond timestamp for instant seal engine (#9469)
* Support millisecond timestamp for instant seal engine

* Forgot to checkin instant_seal mod

* Fix instant seal config

* Fix json crate compile

* Fix private_spec.json

* Option<bool> -> bool
2018-09-06 11:38:00 +02:00
André Silva
39a12622ae ethcore: don't validate difficulty when ignoring seal check (#9470)
* ethcore: don't validate difficulty when ignoring seal check

* ethcore: fix block verification test

* ethcore: document skipped verifications when check_seal is disabled
2018-09-06 04:37:41 +02:00
Niklas Adolfsson
dca88ff85c [light/jsonrpc] Provide the actual account for eth_coinbase RPC and unify error handeling for light and full client (#9383)
* Provide the actual `account` for eth_coinbase

The previous implementation always provided the `zero address` on
`eth_coinbase` RPC. Now, instead the actual address is returned on
success or an error when no account(s) is found!

* full client `eth_coinbase` return err

In the full-client return an error when no account is found instead of
returning the `zero address`

* Remove needless blocks on single import

* Remove needless `static` lifetime on const

* Fix `rpc_eth_author` test
2018-09-05 19:21:08 +02:00
Pierre Krieger
ab9843cb00 Add a Java interface (#9346)
* Add a Java interface

* Use system ABI

* Forgot exception

* Fix param for parity_rpc

* Address concerns
2018-09-05 17:17:12 +02:00
David
72fd1fa58d
Fetch parity-common crates from crates.io (#9410)
* Fetch `parity-common` crates from crates.io

* Add doc tests from `patricia-trie` to `patricia-trie-ethereum`
Fix/update a few deps

* [ethkey] upgrade ethereum-types

* [whisper] update deps

* [network] deps

* [network-devp2p] deps

* [journaldb] deps

* [fastmap] deps

* [miner] deps and test fixes

* [machine] deps

* [json] deps

* [hw] deps

* [ethash] deps

* [registrar] deps

* Update a few more dependencies with new ethabi-*

* [updater] Update deps

* deps

* [ethcore] Update deps

* Use new parity-snappy and parity-rocksdb crates

* Updated submodules

* Use parity-snappy 0.1

* Use kvdb-rocksdb 0.1.2

* Don't use latest ethereum/tests

* Fix merge conflicts errors

* Remove superseeded comment

* Address grumbles: add newlines, add/remove spaces
2018-09-04 20:13:51 +02:00
michaelr524
51eac1926f Fixed typo (#9467) 2018-09-04 15:47:33 +01:00
cheme
c1aed4af45 Fix light client deadlock (#9385)
This PR is fixing deadlock for #8918 

It avoids some recursive calls on light_sync by making state check optional for Informant.

The current behavior is to display the information when informant checks if block is major version.
This change a bit the informant behavior, but not on most cases.

To remember where and how this kind of deadlock are likely to happen (not seen with Parkinglot deadlock detection because it uses std condvar), I am adding a description of the deadlock.
Also, for the reviewers there may be better solution than modifying the informant.

### Thread1 

- ethcore/sync/light_sync/mod.rs

A call to the light handler through any Io (having a loop of rpc query running on like client makes the dead lock way more likely).
At the end of those calls we systematically call `maintain_sync` method.

Here maintain_sync locks `state` (it is the deadlock cause), with a write purpose

`maintain_sync` -> `begin_search` with the state locked open

`begin_search` -> lightcliennt `flush_queue` method

- ethcore/light/src/client/mod.rs

`flush_queue` -> `flush` on queue (HeaderQueue aka VerificationQueue of headers)

- ethcore/src/verification/queue/mod.rs

Condition there is some unverified or verifying content

`flush` wait on a condvar until the queue is empty. The only way to unlock condvar is that worker is empty and unlock it (so thread 2 is Verification worker).

### Thread2

A verification worker at the end of a verify loop (new block).

- ethcore/src/verification/queue/mod.rs

thread loops on `verify` method.

End of loop condition is_ready -> Import the block immediately 

calls `set_sync` on QueueSignal which send a BlockVerified ClientIoMessage in inner channel (IoChannel of ClientIoMessage) using `send_sync`

- util/io/src/service_mio.rs

IoChannel `send_sync` method calls all handlers with `message` method; one of the handlers is ImportBlocks IoHandler (with a single inner Client service field)

- ethcore/light/src/client/service.rs

`message` trigger inner method `import_verified`

- core/light/src/client/mod.rs

`import_verified` at the very end notify the listeners of a new_headers, one of the listeners is Informant `listener` method

- parity/informant.rs

`newHeaders` run up to call to `is_major_importing` on its target (again clinet)

-  ethcore/sync/src/light_sync/mod.rs

Here `is_major_importing` tries to get state lock (read purpose only) but cannot because of previous state lock, thus deadlock
2018-09-04 15:36:34 +01:00
Wei Tang
c12447c7c5 Fix docs of address_hash (#9463) 2018-09-04 11:25:22 +02:00
gabriel klawitter
502bacea82
fix typo in bash script (#9462) 2018-09-04 10:58:13 +02:00
Wei Tang
da5ba4ccc8
Fix a bug in evmbin initial_gas display (#9457) 2018-09-04 16:33:14 +08:00
Andronik Ordian
1f2426226b evmbin: escape newlines in json errors (#9458) 2018-09-04 14:28:49 +08:00
André Silva
c0eb30b604
use kvdb-* and parity-snappy crates from crates.io (#9441)
* use kvdb-* and parity-snappy crates from crates.io

* update rocksdb-sys and snappy-sys
2018-09-03 17:40:11 +01:00
Wei Tang
caca3a8048
Add EIP-1014 transition config flag (#9268)
* Add EIP-1014 transition config flag

* Remove EIP-86 configs

* Change CREATE2 opcode index to 0xf5

* Move salt to the last item in the stack

* Change sendersaltandaddress scheme to comply with current EIP-1014

* Fix json configs

* Fix create2 test

* Fix deprecated comments
2018-08-31 23:43:51 +08:00
gabriel klawitter
f929419676 add tags for runner selection of build-linux jobs (#9451) 2018-08-31 16:41:21 +02:00
Andrew Jones
bc53e78a04
Remove unused BlockStatus::Pending (#9447)
Pending case never instantiated, and only ever matched together with Unknown
2018-08-31 12:13:01 +01:00
André Silva
3dbea1a7a1 ci: only include local paths in coverage script (except target) (#9437) 2018-08-31 01:21:57 +02:00
Afri Schoedon
644d1db5ef Add POA Networks: Core and Sokol (#9413)
* ethcore: add poa network and sokol chainspecs

* rpc: simplify chain spec docs

* cli: rearrange networks by main/test and size/range

* parity: don't blacklist 0x00a328 on sokol testnet

* parity: add sokol and poanet to params and clean up a bit, add tests

* ethcore: add the poa networks and clean up a bit

* ethcore: fix path to poacore chain spec

* parity: rename poa networks to poacore and poasokol

* parity: fix configuration tests

* parity: fix parameter tests

* ethcore: rename POA Core and POA Sokol
2018-08-30 21:32:47 +02:00
André Silva
07d97d5b26 docker: install missing dependencies in arm target dockerfiles (#9436) 2018-08-30 20:03:38 +02:00
Afri Schoedon
c658b46fe1
Random small cleanups (#9423)
* clean up toml files

* update the parity ethereum toolchain docs

* update contribution guide and issue templates

* update desktop and service files

* build clib examples with 8 threads

* update header templates

* replace parity technologies with parity ethereum logo
2018-08-30 19:57:27 +02:00
Wei Tang
8a5c9a8c70 evmbin: Fix gas_used issue in state root mismatch and handle output better (#9418)
* Fix gas used in staterootmismatch, and print full state root hash

* Write trace info for stdjson to stderr

* Fix tests

* Remove struct trait bound
2018-08-29 20:13:45 +03:00