Commit Graph

19 Commits

Author SHA1 Message Date
Robert Habermeier 76a7246369 Snapshot creation and restoration (#1679)
* to_rlp takes self by-reference

* clean up some derefs

* out-of-order insertion for blockchain

* implement block rebuilder without verification

* group block chunk header into struct

* block rebuilder does verification

* integrate snapshot service with client service; flesh out implementation more

* initial implementation of snapshot service

* remove snapshottaker trait

* snapshot writer trait with packed and loose implementations

* write chunks using "snapshotwriter" in service

* have snapshot taking use snapshotwriter

* implement snapshot readers

* back up client dbs when replacing

* use snapshot reader in snapshot service

* describe offset format

* use new get_db_path in parity, allow some errors in service

* blockchain formatting

* implement parity snapshot

* implement snapshot restore

* force blocks to be submitted in order

* fix bug loading block hashes in packed reader

* fix seal field loading

* fix uncle hash computation

* fix a few bugs

* store genesis state in db. reverse block chunk order in packed writer

* allow out-of-order import for blocks

* bring restoration types together

* only snapshot the last 30000 blocks

* restore into overlaydb instead of journaldb

* commit version to database

* use memorydbs and commit directly

* fix trie test compilation

* fix failing tests

* sha3_null_rlp, not H256::zero

* move overlaydb to ref_overlaydb, add new overlaydb without on-disk rc

* port archivedb to new overlaydb

* add deletion mode tests for overlaydb

* use new overlaydb, check state root at end

* share chain info between state and block snapshotting

* create blocks snapshot using blockchain directly

* allow snapshot from arbitrary block, remove panickers from snapshot creation

* begin test framework

* blockchain chunking test

* implement stateproducer::tick

* state snapshot test

* create block and state chunks concurrently, better restoration informant

* fix tests

* add deletion mode tests for overlaydb

* address comments

* more tests

* Fix up tests.

* remove a few printlns

* add a little more documentation to `commit`

* fix tests

* fix ref_overlaydb test names

* snapshot command skeleton

* revert ref_overlaydb renaming

* reimplement snapshot commands

* fix many errors

* everything but inject

* get ethcore compiling

* get snapshot tests passing again

* instrument snapshot commands again

* fix fallout from other changes, mark snapshots as experimental

* optimize injection patterns

* do two injections

* fix up tests

* take snapshots from 1000 blocks efore

* address minor comments

* fix a few io crate related errors

* clarify names about total difficulty

[ci skip]
2016-08-05 17:00:46 +02:00
Arkadiy Paronyan 7093651d70 More performance optimizations (#1814)
* Buffered DB

* Use identity hash for MemoryDB

* Various tweaks

* Delayed DB compression

* Reduce last_hashes cloning

* Keep state cache

* Updating tests

* Optimized to_big_int

* Fixing build with stable

* Safer code
2016-08-03 22:03:40 +02:00
Robert Habermeier 11b65ce53d Remove (almost all) panickers from trie module (#1776)
* memorydb ub patch and other cleanup

* fix denote invocations

* move trie traits into trie module

* replace "denote" with shim

* triedb returns results and no longer panics

* fix warnings

* get ethcore compiling

* warn on trie errors in ethcore

* remove unsafety from node decoder

* restore broken denote behavior for this branch

* fix overlayrecent fallout

* fix triedb tests

* remove unwrap in state

* alter Trie::get to return Result<Option<_>>

* fix refcell error in require

* fix test warnings

* fix json tests

* whitespace

[ci:skip]

* Avoid unneeded match/indentation

* whitespace

* prettify map_or_else

* remove test warning
2016-08-03 18:35:48 +02:00
Arkadiy Paronyan 64417594c7 Delete values immediately from DB overlay (#1631)
* Delete values immediately from DB overlay

* Match remove behaviour; test

* Warning fixed
2016-07-16 14:48:54 +02:00
Marek Kotewicz bcb63bce12 serde is no longer util dependency (#1534)
* removed old json-tests

* simplify folds in triehash.rs

* removed unused json_aid

* removed unused squeeze.rs

* json branching tests for trie

* removing todos from util

* separated UsingQueue and Table

* further cleanup, removing unused code

* serde serialization of hash moved to rpc module

* uint wrapper for rpc in progress

* serialization of uint moved to rpc module

* updated eth-secp256k1

* updated igd, serde is no longer dependency of util

* loading trie consensus tests

* renamed aliases in rpc imports
2016-07-06 11:23:29 +02:00
debris 7904464d24 use mem::replace instead of mem::swap in ArchiveDB, add aux_remove() 2016-06-27 14:01:06 +02:00
debris 1b4f67151f fatdb and fatdb iterator module 2016-06-27 09:16:34 +02:00
keorn be8f922b3f Replace deprecated hashdb trait names (#1394)
* replace deprecated hashdb method names

* spaces -> tabs
2016-06-23 11:16:11 +02:00
Tomasz Drwięga 85da55a537 Fixing warnings 2016-04-06 10:55:40 +02:00
Robert Habermeier 40d3301c68 Update deprecated HashDB methods in docs. 2016-03-30 01:36:35 -04:00
Gav Wood 88e23fcb9a Merge pull request #634 from ethcore/jdb_option2
JournalDB with in-memory overlay (option2)
2016-03-12 10:01:17 +01:00
arkpar 8f54c24e47 Merged changes from jdb_option1, keep LATEST_ERA from decreasing 2016-03-11 11:52:11 +01:00
Tomasz Drwięga 8709dd28f8 Fixing clippy warnings 2016-03-11 10:58:51 +01:00
Gav Wood bc2fb14b5d Add memory usage reports.
Update to be similar to master.
2016-03-06 22:39:04 +01:00
arkpar d77d9ad9d8 JournalDB with history overlay 2016-03-06 19:57:00 +01:00
debris c90d64662a added license in every *.rs file 2016-02-05 13:40:41 +01:00
Gav Wood fad2f3a23d More doc updates. All Gav Wood docs done. 2016-02-03 14:51:45 +01:00
Tomusdrw bad2a244df Adding documentation request comments. 2016-01-19 17:02:01 +01:00
Gav Wood 6ea8eaa3b5 Move ethcore files back into root. 2016-01-17 13:11:25 +01:00