mirror of
https://github.com/grassrootseconomics/cic-chain-events.git
synced 2024-11-25 16:46:45 +01:00
39 lines
1.8 KiB
Markdown
39 lines
1.8 KiB
Markdown
## Functionality
|
|
|
|
## Filters
|
|
|
|
Filters are initialized in `cmd/filters.go` and implemented in `internal/filters/*.go` folder. You will need to modify these files to suite your indexing needs.
|
|
|
|
The existing implementation demo's tracking Celo stables transfer events and gives a rough idea on how to write filters. The final filter should always emit an event to NATS JetStream.
|
|
|
|
## Syncers
|
|
|
|
### Head syncer
|
|
|
|
The head syncer processes newely produced blocks independently by connecting to the geth websocket endpoint.
|
|
|
|
### Janitor
|
|
|
|
The janitor syncer checks for missing (blocks) gaps in the commited block sequence and queues them for processing. It can also function as a historical syncer to process older blocks.
|
|
|
|
With the default `config.toml`, The janitor can process around 950-1000 blocks/min.
|
|
|
|
**Ordering**
|
|
|
|
Missed/historical blocks are not guaranteed to be processed in order, however a low concurrency setting would somewhat give an "in-order" behaviour (not to be relied upon in any case).
|
|
|
|
## Block fetchers
|
|
|
|
The default GraphQL block fetcher is the recommended fetcher. An experimental RPC fetcher implementation is also provided as an example.
|
|
|
|
## Pipeline
|
|
|
|
The pipeline fetches a whole block with its full transaction and receipt objects, executes all loaded filters serially and finally commits the block number to the db. Blocks are processed atomically by the pipeline; a failure in one of the filters will trigger the janitor to re-queue the block and process the block again.
|
|
|
|
## Store
|
|
|
|
The postgres store keeps track of commited blocks and syncer curosors. Schema:
|
|
|
|
- The `blocks` table keeps track of processed blocks.
|
|
- The `syncer_meta` table keeps track of the lower_bound cursor. Below the lower_bound cursor, all blocks are guarnteed to have been processsed hence it is safe to trim the `blocks` table below that pointer.
|