Commit Graph

878 Commits

Author SHA1 Message Date
arkpar
9239a7cd9e v1.3.5 2016-10-08 01:02:41 +02:00
Arkadiy Paronyan
bbaf5ed4f5 Backports into beta (#2512)
* RocksDB version bump

* Preserve cache on reverting the snapshot (#2488)

* Preserve cache on reverting the snapshot

* Renamed merge_with into replace_with

* Renamed and documented snapshotting methods

* Track dirty accounts in the state (#2461)

* State to track dirty accounts

* Removed clone_for_snapshot

* Renaming stuff

* Documentation and other minor fixes

* Replaced MaybeAccount with Option

* Adjustable stack size for EVM (#2483)

* stack size for io workers & evm threshold

* rust way to remember stack size

* right value

* 24kb size

* some stack reduction

* Fixed overflow panic in handshake_panic (#2495)
2016-10-07 10:49:48 +01:00
arkpar
300c0c120b v1.3.4 2016-10-04 22:00:59 +02:00
Arkadiy Paronyan
7f7fe33bde Jumptable cache (#2435)
* v1.3.3

* Jumptable cache
2016-10-02 19:52:53 +02:00
Tomasz Drwięga
63d3a4dd59 Beta Backports (#2396)
* Removing extras data from retracted blocks. (#2375)

* Removing extras data from retracted blocks.

* Adding a test case

Conflicts:
	ethcore/src/blockchain/blockchain.rs

* Fixing transaction queue

Conflicts:
	ethcore/src/miner/transaction_queue.rs

* Prioritizing re-imported transactions (#2372)

* Prioritizing re-imported transactions

* Fixing compilation on beta

Conflicts:
	Cargo.lock
	ethcore/src/client/client.rs
	ethcore/src/miner/transaction_queue.rs

* Post-merge fixes
2016-09-29 13:19:53 +02:00
Nikolay Volf
4d115987cb Accounts bloom (#2357)
* proper bloom

* incremental bloom updates

* crate update

* return of the column

* fix n^2 byteorder write

* add notes to funs

* working bloom commits

* Optimizations

* bloom diag

* migration basic

* migration ongoing

* migration finalizing

* mingration api workarounds

* fix test_client setups

* snapshot bloom update

* review fixes

* just forward keys in the migration

* migration extra tracing

* fix migration path

* remove close pray

* review issues
2016-09-29 11:27:41 +02:00
Arkadiy Paronyan
c9cfcd2728 Canonical state cache (#2308)
* State cache

* Reduced copying data between caches

Whitespace and optional symbols

* Set a limit on storage cache

* Style and docs
2016-09-27 16:44:32 +02:00
Tomasz Drwięga
f5d48cbf2a Handle RLP to string UTF-8 decoding errors (#2217) (#2226) 2016-09-22 14:48:54 +02:00
Arkadiy Paronyan
33084aaa07 v1.3.2 (#2200) 2016-09-20 14:21:17 +08:00
Arkadiy Paronyan
2a82fa0a47 Backports to beta (#2068)
* Fix several RPCs (#1926)

* Fix up pending receipts details.

* Add support for additional params and registry over RPC.

* Fix tests.

* Add test, additional fix.

Fixes #1932.

* Fix up tests.

* Fix test.

* Fix test.

* DB WAL size limit (#1935)

* Limit WAL size

* Check pruning by db modification date (#1924)

* Cache address hash (#1943)

* ECIES without MAC (#1948)

* Use random IV for ECIES AES

* ECIES encrypt/decrypt for a single message

* Derive IV from shared secret

* Apply settings to column families

* fixed #1933 (#1979)

* Fixed neighbours collection (#1996)

* dapps-hosts configuration

* Disabled counter check

* always process trie death row on commit, add more tracing

* fixed transaction addresses mapping, fixes #1971

* simplified iterator

* v1.3.1

* v1.3.1
2016-09-11 14:04:17 +02:00
Arkadiy Paronyan
dfb2ddfdc2 Backports to beta (#1919)
* RPC errors & logs (#1845)

* Refactoring errors in RPC

* Updating jsonrpc-core

* Fixing code_at

* Avoid mentioning obvious segments in proof

[ci:skip]

* fixed cache_manager lock order

* Purging .derefs, fixing clippy warnings. (#1890)

* Fixing clippy warnings

* Purging derefs

* Simplifying engine derefs

* Simplifying more engine derefs

* Adding more details to miner log

* fixed #1889, .DS_Store is no longer treated as key file (#1892)

* fixed #1889, .DS_Store is no longer treated as key file

* ethstore filters directories, hidden files and common system files

* fixed compiling

* fix regression with geth dir

* fix regression with geth dir

* Fix ipc compilation and add ipc feature to test targets (#1902)

* fix compilation and add it to the ci run

* no separator?

* use quotes and spaces

* RocksDB version bump

* Don't return deleted nodes that are not yet flushed (#1908)

* polling & connection timeouts (#1910)

* Peers RPC + UI displaying active/connected/max peers (#1915)

* Peers API

* Bumping Parity-UI

* Fixing tests

* Save nodes removed from backing_overlay until commit (#1917)
2016-08-12 12:16:20 +02:00
Arkadiy Paronyan
52ac5a00f5 Backports for beta (#1888)
* Sync to peers with confirmed fork block only (#1863)

* Fixing gas conversion

* Validating u256->usize conversion

* Update cache usage on commiting block info (#1871)

* Use UntrustedRlp for block verification (#1872)

* take snapshot at specified block and slightly better informants (#1873)

* prettier informant for snapshot creation

* allow taking snapshot at a given block

* minor tweaks

* elaborate on cli

* Send new block hashes to all peers (#1875)

* Reduce max open files (#1876)

* ws-rs update

* Fixing test

* Fixing miner deadlock (#1885)
2016-08-09 14:05:02 +02:00
arkpar
6336bdecf3 Version set to beta 2016-08-06 08:08:27 +02:00
Robert Habermeier
76a7246369 Snapshot creation and restoration (#1679)
* to_rlp takes self by-reference

* clean up some derefs

* out-of-order insertion for blockchain

* implement block rebuilder without verification

* group block chunk header into struct

* block rebuilder does verification

* integrate snapshot service with client service; flesh out implementation more

* initial implementation of snapshot service

* remove snapshottaker trait

* snapshot writer trait with packed and loose implementations

* write chunks using "snapshotwriter" in service

* have snapshot taking use snapshotwriter

* implement snapshot readers

* back up client dbs when replacing

* use snapshot reader in snapshot service

* describe offset format

* use new get_db_path in parity, allow some errors in service

* blockchain formatting

* implement parity snapshot

* implement snapshot restore

* force blocks to be submitted in order

* fix bug loading block hashes in packed reader

* fix seal field loading

* fix uncle hash computation

* fix a few bugs

* store genesis state in db. reverse block chunk order in packed writer

* allow out-of-order import for blocks

* bring restoration types together

* only snapshot the last 30000 blocks

* restore into overlaydb instead of journaldb

* commit version to database

* use memorydbs and commit directly

* fix trie test compilation

* fix failing tests

* sha3_null_rlp, not H256::zero

* move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc

* port archivedb to new overlaydb

* add deletion mode tests for overlaydb

* use new overlaydb, check state root at end

* share chain info between state and block snapshotting

* create blocks snapshot using blockchain directly

* allow snapshot from arbitrary block, remove panickers from snapshot creation

* begin test framework

* blockchain chunking test

* implement stateproducer::tick

* state snapshot test

* create block and state chunks concurrently, better restoration informant

* fix tests

* add deletion mode tests for overlaydb

* address comments

* more tests

* Fix up tests.

* remove a few printlns

* add a little more documentation to `commit`

* fix tests

* fix ref_overlaydb test names

* snapshot command skeleton

* revert ref_overlaydb renaming

* reimplement snapshot commands

* fix many errors

* everything but inject

* get ethcore compiling

* get snapshot tests passing again

* instrument snapshot commands again

* fix fallout from other changes, mark snapshots as experimental

* optimize injection patterns

* do two injections

* fix up tests

* take snapshots from 1000 blocks efore

* address minor comments

* fix a few io crate related errors

* clarify names about total difficulty

[ci skip]
2016-08-05 17:00:46 +02:00
Arkadiy Paronyan
05bfdc508e Split IO and network crates (#1828)
* Abort on panic

* Split IO and network crates

* Restore panic handler

* Fixed doc tests
2016-08-05 10:32:04 +02:00
Robert Habermeier
b20833154e protect unsafety in plainhasher; do more hashing (#1841) 2016-08-05 09:15:55 +02:00
Robert Habermeier
8702a29e6f use mutex in dbtransaction (#1843) 2016-08-04 23:54:26 +02:00
Arkadiy Paronyan
7093651d70 More performance optimizations (#1814)
* Buffered DB

* Use identity hash for MemoryDB

* Various tweaks

* Delayed DB compression

* Reduce last_hashes cloning

* Keep state cache

* Updating tests

* Optimized to_big_int

* Fixing build with stable

* Safer code
2016-08-03 22:03:40 +02:00
Tomasz Drwięga
deceb5fd56 Bumping clippy & fixing warnings (#1823)
* Bumping clippy

* Fixing clippy warnings

* Cargo.lock (latest nightly?)
2016-08-03 19:01:48 +02:00
Marek Kotewicz
531bc79edc removed unused code from util and unnecessary dependency of FixedHash (#1824) 2016-08-03 18:36:45 +02:00
Robert Habermeier
11b65ce53d Remove (almost all) panickers from trie module (#1776)
* memorydb ub patch and other cleanup

* fix denote invocations

* move trie traits into trie module

* replace "denote" with shim

* triedb returns results and no longer panics

* fix warnings

* get ethcore compiling

* warn on trie errors in ethcore

* remove unsafety from node decoder

* restore broken denote behavior for this branch

* fix overlayrecent fallout

* fix triedb tests

* remove unwrap in state

* alter Trie::get to return Result<Option<_>>

* fix refcell error in require

* fix test warnings

* fix json tests

* whitespace

[ci:skip]

* Avoid unneeded match/indentation

* whitespace

* prettify map_or_else

* remove test warning
2016-08-03 18:35:48 +02:00
Robert Habermeier
8c88e2a8cc JournalDB inject (#1806)
* add inject to journaldb

* adjust docs

* add test; fix refcounteddb impl

* fewer panics, fail on invalid insertions or deletions
2016-08-03 16:34:32 +02:00
Gav Wood
9de579366a Miner tweaks (#1797)
* Mining fixes.

- Use queue to determine whether we're mining
- Kick stale hash rates

Fixes #1794
Fixes #1641

* Fix tests.

* Address grumbles.
2016-08-02 17:53:32 +01:00
Nikolay Volf
1b507e0147 Util & ipc clenup (#1807)
* removed frombytes stuff

* removed jsonrpc handler from nano
2016-08-02 15:02:47 +01:00
Marek Kotewicz
dbfc5cd31c fixed trace_transaction crash when block contained suicide (#1781) 2016-07-31 13:47:29 +02:00
arkpar
8c47240248 Min and Max peers setting 2016-07-29 17:30:02 +02:00
Arkadiy Paronyan
57faa37623 Disable WAL (#1765)
* Disable WAL

* Make WAL optional

* Fix tests.

* Update cli.rs
2016-07-29 15:36:00 +02:00
Tomasz Drwięga
e4f0c0b215 Single DB (#1741)
* Consolidation migration

* Started db amalgamation

* Using client constants for columns

* Adding with_columns constructor

* Migrating to single db

* Fixing tests.

* test.sh without verbose

* Fixing warnings

* add migration tests that catch the bug

* make multiple migrations more robust

* add moved v9

* Merge branch 'noop-migrations' into single-db

* spurious line

* clean up migrations ordering

* update comment [ci skip]

* Bumping default number of max_open_files & re-ordering columns.

* fix merge

* fix ignored analysis tests

* Caching best block content

* Faster best_block_header

* Adding progress to v8 migration

* clean up warnings

* Separate hashes and bodies in the DB

* Separate hashes and bodies in the DB

* Fixed tests
2016-07-28 23:46:24 +02:00
Marek Kotewicz
f33cd60dc2 compaction profile used during migration, fixes #1750 (#1751)
* compaction profile used during migration, fixes #1750

* whitespace

[ci:skip]
2016-07-28 20:29:58 +02:00
keorn
30ba10e10c Better handling of multiple migrations (#1747)
* add migration tests that catch the bug

* make multiple migrations more robust

* clean up migrations ordering

* update comment [ci skip]

* remove explicit iter
2016-07-28 12:05:41 +02:00
keorn
02cf48681d Blocks and snapshot compression (#1687)
* new Compressible rlp trait

* new Compressible rlp trait

* make compressed rlp iterable

* make compressed rlp iterable

* invalid rlp slice swapper

* switch compress to swapper, add reverse swapper test case

* add basic account compression test

* add new rlp trait

* new Compressible rlp trait

* make compressed rlp iterable

* invalid rlp slice swapper

* invalid rlp slice swapper

* switch compress to swapper, add reverse swapper test case

* switch compress to swapper, add reverse swapper test case

* add account compress/ decompress test

* make compressor cleaner, use hashmaps for swapper

* improve compression tests

* add a DecompressingDecoder, change Decoder to take refernce

* separate rlp compression related stuff

* new Compressible rlp trait

* new Compressible rlp trait

* new Compressible rlp trait

* make compressed rlp iterable

* make compressed rlp iterable

* make compressed rlp iterable

* invalid rlp slice swapper

* invalid rlp slice swapper

* invalid rlp slice swapper

* switch compress to swapper, add reverse swapper test case

* switch compress to swapper, add reverse swapper test case

* switch compress to swapper, add reverse swapper test case

* add basic account compression test

* add new rlp trait

* add account compress/ decompress test

* make compressor cleaner, use hashmaps for swapper

* improve compression tests

* add a DecompressingDecoder, change Decoder to take refernce

* separate rlp compression related stuff

* DecompressingDecoder test

* initial compressing HashDB wrapper

* remove unused test

* change CompressedDB to struct wrapper with overlay

* simplify compressor

* failed RefCell attempt

* use denote to return reference

* compiled compresseddb

* compressdb test, add overlay emplace

* fix overlay reference count handling

* add immutable compresseddb, make account use hashdb

* simplify using trait objects

* enable hashdb for account

* initial state compression attempt

* wrap state db

* add tests for analyzing db

* add account predicate

* try to compress data fields as rlp too

* remove compression for storage trie

* add a compressing migration

* more compression stats tests

* fix migration import

* nested encoding compression test

* fix decompression, move db stats tests to rlpcompression

* added malformed rlp tests, cover a few edge cases

* new CompressingEncoder struct

* extend migrations to state

* first version working on the whole db

* clean up Compressible impl

* tests cleanup

* add a testing migration

* refactor deep compression using option, add simple compression

* put tests in a module

* fix compressed overlay loading

* simple compression for snapshots

* remove unused DecompressingDecoder

* add a general compressing migration

* add more common rlps to compress

* use static slices for swapper

* add precomputed hashes and invalid rlps

* make decoder private again

* cover more cases with tests

* style

* fix weird indentation

* remove possible panic in payload_info

* make prefix checking safe

* fix db existence check

* remove db dir from test

* pass usize by value [ci skip]

* Improve comment on panic removal.

* add common blocks db rlps

* add compression to blockchain db

* add blocks db migration

* fix the migrations

* remove state compression

* add a separate snapshot swapper

* ability to use different swappers and traversal

* update tests to new interface

* clean up code ordering

* update usage

* fix compilation

* remove unnecessary changes

* move methods to functions to reduce interface

* move test to module

* update common rlps to blocks db

* move tests to tests modules

* remove redundant &
2016-07-27 17:11:41 +02:00
Arkadiy Paronyan
c65ee93542 Use std::sync::Condvar (#1732)
test
2016-07-27 11:39:24 +02:00
Tomasz Drwięga
3f41186b2e Fixing some clippy warnings (#1728)
* Fixing warnings

* Fixing unnecessary ref

* Removing unnecessary operation
2016-07-26 20:31:25 +02:00
Arkadiy Paronyan
53a975d1dc Fixed reading chunked EIP8 handshake (#1712)
* Fixed reading chunked EIP8 handshake

* Added missing break
2016-07-26 16:04:14 +02:00
Gav Wood
b220e07feb Fix broken internal names. (#1711) 2016-07-25 17:45:55 +02:00
Marek Kotewicz
226fe8e0bb cli overhaul (#1600)
* cli commands

* cleanup parity/signer

* cleanup parity/signer

* remove redundant import of signer crate from main.rs

* cli cleanup in progress

* cli cleanup in progress

* moved few commonly used functions to separate methods with tests

* cleaning up blockchain import in progress

* cleaning up blockchain import in progress2

* cleaning up blockchain import in progress3

* tests for database compaction profile parsing

* cleaning up blockchain import in progress4

* cleaning up blockchain import in progress5

* blockchain import

* export blockchain in progress

* cleanup execute_export

* Configuration::to_duration cleaned up

* removed unused code, tests for to_duration

* cleanup Configuration::mode function

* parsing some of the cli params in params.rs

* rpc and signer are no longer optional

* move importing extern crates to main.rs file

* swipe dies from rpc module

* swipe dies from dapps

* finding deprecated

* several tests and fixes for parity

* parity cleanup in progress

* cleanup price parsing

* parity cleanup in progress

* swiped all dies

* parity cleanup in progress

* replace usages of from_str with parse() in parity/params.rs

* removed few more from_str

* split parity/params.rs into params and helpers

* removed wildcard import from configuration.rs

* cleanup directories/path creation

* cleaning up run cmd

* moved LoggerConfig

* defaults for cli params

* fixed indention in raise_fd_limit

* tests for rpc_apis

* tests for default ipc and rpc settings

* ipc socket

* cleanup in progress

* account service

* cleanup miner config

* BlockChain commands use Directiores structure now

* client_config

* network settings and dapps configuration

* removing warnings

* default logger config

* fixed client_path

* overhaul

* fixing export && import

* default export DataFormat

* import and export also upgrade db

* fixed export && import

* polishing pr

* polishing pr

* fixed custom bootnodes

* fixed daemonize on windows

* fixed setting up enable network

* finished pr

* fixed compiling on windows

* Fixed warning; windows build

* Better cache management

* Fixed tests on windows

* Fixed test

* Restored pruning method names

* --cache alias

* Fixed more tests

* Ensure default options actually listed as valid

[ci:skip]
2016-07-25 16:09:47 +02:00
Arkadiy Paronyan
9a8fdeead9 Enable WAL and disable DB repair (#1696)
* Enable WAL

* Disable rewind
2016-07-23 17:05:34 +02:00
Arkadiy Paronyan
7dd29825c4 Ping discovery nodes gradually (#1671) 2016-07-20 12:41:31 +02:00
Arkadiy Paronyan
8a8cfb133f Deadlock on incoming connection (#1672) 2016-07-20 12:40:26 +02:00
gregg dourgarian
d67369a01c fix typos (#1644) 2016-07-19 20:42:23 +02:00
Tomasz Drwięga
038862fa9d Fixing clippy warnings (#1660) 2016-07-19 09:25:51 +02:00
Arkadiy Paronyan
340f0b6f58 Don't ping all nodes on start (#1656)
* Don't ping all nodes on start

* Fixed test
2016-07-19 09:24:19 +02:00
Arkadiy Paronyan
4e447ccc68 More performance optimizations (#1649)
* Use tree index for DB

* Set uncles_hash, tx_root, receipts_root from verified block

* Use Filth instead of a bool

* Fix empty root check

* Flush block queue properly

* Expunge deref
2016-07-19 09:23:53 +02:00
Gav Wood
34b7cf2e0a Pro paths (#1650)
* Util funtions to get OS-standard config path.

* Build fix.
2016-07-17 23:00:47 +02:00
Arkadiy Paronyan
5ab18d1313 Performance optimizations (#1642)
* Optimize ethash verification

* disable WAL for puts

* Clear account cache after commit

* Commit only modified accounts

* Optimize existing block check

* Cache last hashes
2016-07-17 09:18:15 +02:00
Arkadiy Paronyan
64417594c7 Delete values immediately from DB overlay (#1631)
* Delete values immediately from DB overlay

* Match remove behaviour; test

* Warning fixed
2016-07-16 14:48:54 +02:00
Nikolay Volf
7b5d39e0a1 fix network_start regression (#1629)
* fix network_start regression

* network io handler

* move registration to the network start
2016-07-16 11:31:59 +02:00
Gav Wood
d14b6871a5 Cleanup of colour code. Use is_a_tty. (#1621)
* Cleanup of colour code. Use is_a_tty.

* Fix test build.

* Another fix.
2016-07-15 10:11:14 +02:00
Robert Habermeier
517c705ab5 In-memory trie operations (#1408)
* small cleanups in trie

* Memory trie skeleton

* decode nodes from RLP

* memorytrie -> memorytriedb

* implement Trie for MemoryTrie

* partially implement insert

* implement trie insertion

* don't load whole trie into memory, lookup across memory and db

* re-implement insertion and lazily load necessary nodes from DB

* node removal w/o fixing

* reduce churn in node storage

* finish remove implementation

* committing the in-memory trie

* reload root node after commit

* generate elastic arrays when converting nodes to rlp

* port triedbmut tests over to memorytriedb, fix a few bugs.

* hash count and dirty flag

* initial attempt for node inspection

* back to skeleton

* inspection framework

* implement insertion

* no panic paths in insert

* implement deletion without fixing

* node fixing

* commit nodes to db

* tracing targets and bugfix

* get tests to pass with a lot of tracing

* set playpen iterations to 10

* remove some tracing statements

* make TrieMut::root take &mut self

* replace triedbmut with memorytriedb

* treat empty insert value as removal

* add test for null insert

* fix some style concerns

* trie: use nodehandle for root_node, minor cleanup
2016-07-14 18:06:46 +02:00
Nikolay Volf
06e8ae19e3 Merge pull request #1614 from ethcore/cache-pref
Couple of rocksdb optimizations
2016-07-14 12:37:08 +02:00