265 lines
8.5 KiB
Markdown
265 lines
8.5 KiB
Markdown
# chaind-eth development tester recipe
|
|
|
|
chaind-eth is a socket server that acts as a automated transaction handler for an EVM network.
|
|
|
|
It capabilities are (unchecked box means feature not yet completed):
|
|
|
|
- [x] unix socket server to accept raw, signed RLP evm transactions
|
|
- [x] stateful queueing system following full local and remote lifecycle of the transaction
|
|
- [x] transaction dispatcher unit
|
|
- [x] transaction retry unit (for errored or suspended transactions)
|
|
- [x] blockchain listener that updates state of transactions in queue
|
|
- [x] CLI transaction listing tool, filterable by:
|
|
* [x] transaction range with lower and/or upper bound
|
|
* [x] only show transaction with errors
|
|
* [x] only show transaction that have not yet completed
|
|
- [x] systemd unit / socket service
|
|
- [x] sql storage backend
|
|
- [ ] filesystem storage backend
|
|
|
|
|
|
## prerequisites
|
|
|
|
For these examples you need:
|
|
|
|
- linux (tested on 5.12.x, perhaps wsl/macos will work too, no guarantees, though)
|
|
- python 3.9.x
|
|
- pip
|
|
- virtualenv
|
|
- socat
|
|
- sqlite
|
|
- an EVM RPC endpoint
|
|
|
|
For any python command / executable used below:
|
|
|
|
* add `-v` or `-vv` to get more information about what is going on
|
|
* use with `--help` for information on how to use and parameters that can be passed
|
|
|
|
|
|
## setting up the database backend
|
|
|
|
Currently there is no more practical way of setting up the database backend than to pull the repository and run a database migration script :/
|
|
|
|
```
|
|
git clone https://gitlab.com/chaintool/chaind
|
|
cd chaind
|
|
python -m venv .venv
|
|
. .venv/bin/activate
|
|
pip install --extra-index-url https://pip.grassrootseconomics.net:8433 -r requirements.txt
|
|
# the following will set up your database in ~/.local/share/chaind/eth/chaind.sqlite
|
|
PYTHONPATH=. CHAIND_DOMAIN=eth DATABASE_ENGINE=sqlite python scripts/migrate.py
|
|
```
|
|
|
|
|
|
## usage example
|
|
|
|
### create an empty working directory
|
|
|
|
In terminal window A
|
|
|
|
```
|
|
d=$(mktemp -d) && cd $d
|
|
```
|
|
|
|
### create a chaind-eth sandbox
|
|
|
|
```
|
|
python -m venv .venv
|
|
. .venv/bin/activate
|
|
pip install --extra-index-url https://pip.grassrootseconomics.net:8433 "chaind-eth>=0.0.3a5"
|
|
```
|
|
|
|
### start the services
|
|
|
|
In terminal window B
|
|
|
|
```
|
|
cd <working directory>
|
|
. .venv/bin/activate
|
|
export DATABASE_ENGINE=sqlite
|
|
export RPC_PROVIDER=<your_provider>
|
|
export CHAIN_SPEC=<chain_spec_of_provider>
|
|
chaind-eth-server --session-id testsession
|
|
```
|
|
|
|
In terminal window C
|
|
|
|
```
|
|
cd <working directory>
|
|
. .venv/bin/activate
|
|
export DATABASE_ENGINE=sqlite
|
|
export RPC_PROVIDER=<your_provider>
|
|
export CHAIN_SPEC=<chain_spec_of_provider>
|
|
chaind-eth-syncer
|
|
```
|
|
|
|
### prepare test transactions
|
|
|
|
Create two transactions from sender in keyfile (which needs to have gas balance) to a newly created account
|
|
|
|
```
|
|
export WALLET_KEY_FILE=<path_to_keyfile>
|
|
export WALLET_PASSWORD=<keyfile_password_if_needed>
|
|
export RPC_PROVIDER=<your_provider>
|
|
export CHAIN_SPEC=<chain_spec_of_provider>
|
|
|
|
# create new account and store address in variable
|
|
eth-keyfile -z > testkey.json
|
|
recipient=$(eth-keyfile -z -d testkey.json)
|
|
|
|
# create transactions
|
|
eth-gas --raw -a $recipient 1024 > tx1.txt
|
|
eth-gas --raw -a $recipient 2048 > tx2.txt
|
|
eth-gas --raw -a $recipient 4096 > tx3.txt
|
|
```
|
|
|
|
### send test transactions to queue
|
|
|
|
```
|
|
cat tx1.txt | socat UNIX-CLIENT:/run/user/$UID/chaind/eth/testsession/chaind.sock -
|
|
cat tx2.txt | socat UNIX-CLIENT:/run/user/$UID/chaind/eth/testsession/chaind.sock -
|
|
cat tx3.txt | socat UNIX-CLIENT:/run/user/$UID/chaind/eth/testsession/chaind.sock -
|
|
```
|
|
|
|
### check status of transactions
|
|
|
|
|
|
`chainqueue-list` outputs details about transactions in the queue.
|
|
|
|
Provided the initial database migration was executed as described above, the execution would look as follows:
|
|
|
|
```
|
|
export DATABASE_ENGINE=sqlite
|
|
export DATABASE_NAME=$HOME/.local/share/chaind/eth/chaind.sqlite
|
|
export CHAIN_SPEC=<chain_spec_of_provider>
|
|
sender=$(eth-keyfile -d $WALLET_KEY_FILE)
|
|
chainqueue-list $sender
|
|
```
|
|
|
|
To show a summary only instead all transactions:
|
|
|
|
```
|
|
chainqueue-list --summary $sender
|
|
```
|
|
|
|
The `chaind-list` tool can be used to list by session id. Following the above examples:
|
|
|
|
```
|
|
export DATABASE_ENGINE=sqlite
|
|
export CHAIN_SPEC=<chain_spec_of_provider>
|
|
chaind-list testsession
|
|
```
|
|
|
|
The `chainqueue-list` and `chaind-list` tools both provides the same basic filtering. Use `--help` to see the details.
|
|
|
|
|
|
### Retrieve transaction by hash
|
|
|
|
The socket server returns the transaction hash when a transaction is submitted.
|
|
|
|
If a socket server is given a transaction hash, it will return the transaction data for that hash (if it exists).
|
|
|
|
Extending the previous examples, this will output the original signed transaction:
|
|
|
|
```
|
|
eth-gas --raw -a $recipient 1024 > tx1.txt
|
|
cat tx1.txt | socat UNIX-CLIENT:/run/user/$UID/chaind/eth/testsession/chaind.sock - | cut -b 4- > hash1.txt
|
|
cat hash1.tx | socat UNIX-CLIENT:/run/user/$UID/chaind/eth/testsession/chaind.sock - | cut -b 4- > tx1_recovered.txt
|
|
diff tx1_recovered.txt tx1.txt
|
|
# should output 0
|
|
echo $?
|
|
```
|
|
|
|
The first 4 bytes of the data returned from the socket is a 32-bit big-endian result code. The data payload follows from the 5th byte.
|
|
|
|
|
|
## Batch processing
|
|
|
|
The `chaind-eth-send` executable generates signed transactions with data from a csv file.
|
|
|
|
The data columns must be in the following order:
|
|
|
|
1. receipient address
|
|
2. transaction value
|
|
3. token specifier (optional, network fee token if not given)
|
|
4. network fee token value (optional)
|
|
|
|
|
|
If the gas token value (4) is not given for a gas token transaction, the transaction value (2) will be used.
|
|
|
|
By default the signed transactions are output as hex to stdout, each on a separate line.
|
|
|
|
If a valid `--socket` is given (i.e. the socket of the `chaind-eth-server`) the transactions will be send to the socket instead. The hash of the transaction will be output to standard output.
|
|
|
|
|
|
### Using token symbols
|
|
|
|
If token symols are to be used in some or all values of column 3, then a valid `--token-index` executable address is required (in this case, a smart contract implementing the [`registry`](https://gitlab.com/grassrootseconomics/cic-contracts/-/blob/master/solidity/Registry.sol) contract interface).
|
|
|
|
|
|
### Input validity checks
|
|
|
|
The validity of the input data is verified _before_ actual execution takes place.
|
|
|
|
These checks include:
|
|
|
|
- The token can be made sense of.
|
|
- The values can be parsed to integer amounts.
|
|
- The recipient address is a valid checksum address.
|
|
|
|
The checks do however _not_ include whether the token balances of the signer are sufficient to successfully execute the transactions on the network.
|
|
|
|
|
|
### CSV input example
|
|
|
|
```
|
|
0x72B70906fD07c72f2d96aAA250C2D31662D0d809,10,0xb708175e3f6Cd850643aAF7B32212AFad50e2549
|
|
0xD536CB6d1d9B8d33875E0ba0Aa3515eD7478f889,0x2a,GFT,100
|
|
0xeE08b59a95E822AE346489038D25750C8EdfcC25,0x029a
|
|
```
|
|
|
|
This will result in the following transactions:
|
|
|
|
1. send 10 tokens from token contract `0xb708175e3f6Cd850643aAF7B32212AFad50e2549` to recipient `0x72B70906fD07c72f2d96aAA250C2D31662D0d809`.
|
|
2. send 42 `GFT` tokens along with 100 network gas tokens to recipient `0xD536CB6d1d9B8d33875E0ba0Aa3515eD7478f889`
|
|
3. send 666 network gas tokens to recipient `0xeE08b59a95E822AE346489038D25750C8EdfcC25`
|
|
|
|
|
|
### Resending transactions
|
|
|
|
Since the `chaind-eth-server` does not have access to signing keys, resending stalled transactions is also a separate external action.
|
|
|
|
The `chaind-eth-resend` executable takes a list of signed transactions (e.g. as output from `chaind-eth-send` using the socket) and automatically increases the fee price of the transaction to create a replacement.
|
|
|
|
As with `chaind-eth-send`, the resend executable optionally takes a socket argument that sends the transaction directly to a socket. Otherwise, the signed transactions are send to standard output.
|
|
|
|
For example, the following will output details of the transaction generated by `chaind-eth-resend`, in which the fee price has been slightly incremented:
|
|
|
|
```
|
|
eth-gas --raw --fee-price 100000000 -a $recipient 1024 > tx1.txt
|
|
chaind-eth-resend tx1.txt > tx1_bump.txt
|
|
cat tx1_bump.txt | eth-decode
|
|
```
|
|
|
|
|
|
### Retrieving transactions for resend
|
|
|
|
The `chaind-list` tool can be used to retrieve transactions with the same filters as `chainqueue-list`, but also allowing results limited a specific session id.
|
|
|
|
As with `chainqueue-list`, which column to output can be customized. This enables creation of signed transaction lists in the format accepted by `chaind-eth-resent`.
|
|
|
|
One examples of criteria for transactions due to be resent may be:
|
|
|
|
```
|
|
# get any pending transaction in session "testsession"
|
|
export DATABASE_ENGINE=sqlite
|
|
chaind-list -o signedtx --pending testsession
|
|
```
|
|
|
|
Note that the `chaind-list` tool requires a connection to the queueing backend.
|
|
|
|
|
|
## systemd
|
|
|
|
TBC
|