Commit Graph

22 Commits

Author SHA1 Message Date
Robert Habermeier 76a7246369 Snapshot creation and restoration (#1679)
* to_rlp takes self by-reference

* clean up some derefs

* out-of-order insertion for blockchain

* implement block rebuilder without verification

* group block chunk header into struct

* block rebuilder does verification

* integrate snapshot service with client service; flesh out implementation more

* initial implementation of snapshot service

* remove snapshottaker trait

* snapshot writer trait with packed and loose implementations

* write chunks using "snapshotwriter" in service

* have snapshot taking use snapshotwriter

* implement snapshot readers

* back up client dbs when replacing

* use snapshot reader in snapshot service

* describe offset format

* use new get_db_path in parity, allow some errors in service

* blockchain formatting

* implement parity snapshot

* implement snapshot restore

* force blocks to be submitted in order

* fix bug loading block hashes in packed reader

* fix seal field loading

* fix uncle hash computation

* fix a few bugs

* store genesis state in db. reverse block chunk order in packed writer

* allow out-of-order import for blocks

* bring restoration types together

* only snapshot the last 30000 blocks

* restore into overlaydb instead of journaldb

* commit version to database

* use memorydbs and commit directly

* fix trie test compilation

* fix failing tests

* sha3_null_rlp, not H256::zero

* move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc

* port archivedb to new overlaydb

* add deletion mode tests for overlaydb

* use new overlaydb, check state root at end

* share chain info between state and block snapshotting

* create blocks snapshot using blockchain directly

* allow snapshot from arbitrary block, remove panickers from snapshot creation

* begin test framework

* blockchain chunking test

* implement stateproducer::tick

* state snapshot test

* create block and state chunks concurrently, better restoration informant

* fix tests

* add deletion mode tests for overlaydb

* address comments

* more tests

* Fix up tests.

* remove a few printlns

* add a little more documentation to `commit`

* fix tests

* fix ref_overlaydb test names

* snapshot command skeleton

* revert ref_overlaydb renaming

* reimplement snapshot commands

* fix many errors

* everything but inject

* get ethcore compiling

* get snapshot tests passing again

* instrument snapshot commands again

* fix fallout from other changes, mark snapshots as experimental

* optimize injection patterns

* do two injections

* fix up tests

* take snapshots from 1000 blocks efore

* address minor comments

* fix a few io crate related errors

* clarify names about total difficulty

[ci skip]
2016-08-05 17:00:46 +02:00
Robert Habermeier 8c88e2a8cc JournalDB inject (#1806)
* add inject to journaldb

* adjust docs

* add test; fix refcounteddb impl

* fewer panics, fail on invalid insertions or deletions
2016-08-03 16:34:32 +02:00
Tomasz Drwięga e4f0c0b215 Single DB (#1741)
* Consolidation migration

* Started db amalgamation

* Using client constants for columns

* Adding with_columns constructor

* Migrating to single db

* Fixing tests.

* test.sh without verbose

* Fixing warnings

* add migration tests that catch the bug

* make multiple migrations more robust

* add moved v9

* Merge branch 'noop-migrations' into single-db

* spurious line

* clean up migrations ordering

* update comment [ci skip]

* Bumping default number of max_open_files & re-ordering columns.

* fix merge

* fix ignored analysis tests

* Caching best block content

* Faster best_block_header

* Adding progress to v8 migration

* clean up warnings

* Separate hashes and bodies in the DB

* Separate hashes and bodies in the DB

* Fixed tests
2016-07-28 23:46:24 +02:00
Arkadiy Paronyan 4e447ccc68 More performance optimizations (#1649)
* Use tree index for DB

* Set uncles_hash, tx_root, receipts_root from verified block

* Use Filth instead of a bool

* Fix empty root check

* Flush block queue properly

* Expunge deref
2016-07-19 09:23:53 +02:00
Arkadiy Paronyan da4b1c36cb Enable state queries for overlayrecent db (#1575) 2016-07-11 12:34:29 +02:00
Nikolay Volf 9f4bfd9e7a fix tests 2016-06-27 18:47:50 +02:00
Nikolay Volf 627b67db0a jdb to new settings config 2016-06-27 13:23:50 +02:00
keorn be8f922b3f Replace deprecated hashdb trait names (#1394)
* replace deprecated hashdb method names

* spaces -> tabs
2016-06-23 11:16:11 +02:00
Nikolay Volf c6616431b0 fix tests 2016-06-20 14:20:13 +03:00
Nikolay Volf 7e452ab2e0 configurable cache size 2016-06-20 13:42:04 +03:00
Edward Wang 2b65011706 Use constants for DatabaseConfig (#1318)
Closes #1157
2016-06-18 17:58:28 +02:00
Marek Kotewicz ca008fb541 migration fixes (#1155)
* dont use system temp for migration, do backup

* fixed issue with number of open files on osx

* fixed compilation of util tests
2016-05-27 08:23:29 +02:00
Gav Wood fa95419c27 --pruning=auto option. 2016-04-11 18:42:50 -07:00
Gav Wood 6e97496b27 Support for --pruning=auto. 2016-04-11 15:51:14 -07:00
Tomasz Drwięga 85da55a537 Fixing warnings 2016-04-06 10:55:40 +02:00
Gav Wood ad86feb667 Rename spawn -> boxed_clone 2016-03-28 09:42:50 +02:00
Tomasz Drwięga e58ec31e20 Fixing warnings 2016-03-18 10:14:19 +01:00
Gav Wood 26f41b711c Bring back batching. 2016-03-13 21:54:06 +01:00
Gav Wood 1be92ea8ef Fixes and traces for refcountdb. 2016-03-13 21:21:30 +01:00
Gav Wood 81291622eb Avoid batches for now. 2016-03-13 19:22:42 +01:00
Gav Wood 706c56f56a Usage of LATEST_ERA fixes for archive and ref-counted DBs. 2016-03-13 18:19:52 +01:00
Gav Wood 9e912c7c0d Add new file. 2016-03-13 18:07:10 +01:00