You can support us by donating Eth or any token to our ENS address farghaly.eth or directly to the following Ethereum address:
0x5dfA5dB515748f3E2BCc8677A9332f4aFFad5A5E
Thank you!
Welcome
Kotal is an open source alternative to centralized API gateway, Node as a service, and Staking service providers. Like Infura in Ethereum ecosystem, Onfinality in Polkadot ecosystem, and Hiro in Stacks ecosystem.
Kotal is multi-client, protocol-agnostic, and cloud-agnostic Blockchain Kubernetes Operator that makes it easy to deploy highly-available self-managing self-healing blockchain infrastructure on any cloud.
Kotal has been tested on Amazon Web Service, Google Cloud Platform, Microsoft Azure, DigitalOcean, Alibab Cloud, Tencent cloud 🤯.
For all the fields associated with the Node API resource:
Full Bitcoin node reference is documented .
Installation
Requirements
Kubernetes cluster
Cert Manager
Kubernetes cluster
For development and testing purposes we recommend using (Kubernetes in Docker), it's an easy way to create and tear down kubernetes clusters in seconds. For staging and production purposes you can use any public cloud kubernetes as a service like , or .
kind installation guide can be found .
Cert Manager
Cert Manager is a native Kubernetes certificate management controller. It can help with issuing certificates from a variety of sources, such as , , a simple signing key pair, or self-signed. It will ensure certificates are valid and up to date, and attempt to renew certificates at a configured time before expiry.
Kotal webhook component uses cert-manager for issuing certificates to default and validate your Blockchain resources such as nodes, beacon nodes, ipfs peers ... etc.
cert-manager installation guide can be found .
Install kotal
The latest version of Kotal operator v0.3.0 can be installed into your Kubernetes cluster simply by:
Bitcoin nodes that enable JSON-RPC server are required to use basic authentication while calling JSON-RPC methods.
Apply rpc-user-password.yaml to create the password secret:
Deploy Bitcoin RPC Node
The following manifest describes a Bitcoin node that syncs Bitcoin mainnet network: mainnet, enables HTTP JSON-RPC server rpc: true, defines JSON-RPC user credentials rpcUsers: ... and uses client:
Apply bitcoin.yaml manifest:
Kotal operator will notice your bitcoin-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets neccessary.
You can fetch the deployed Bitcoin Node using:
It will return an output similar to the following:
Fetch Node Logs
Get the pods that has been created by Kotal for the node:
It will return an output similar to the following:
Get the logs of the running node:
It will return node logs similar to the following:
Call JSON-RPC Method
By default, Bitcoin node JSON-RPC server port is 8332, which can be changed using rpcPort.
Forward localhost:8332 calls to the node pod:
In another terminal window call getblockhash JSON-RPC method to get the genesis block hash:
You will get JSON result similar to the following:
Finally you can delete the node by:
Kubernetes garbage collector will delete all the resources that has been created by Bitcoin Node controller.
Quick Start
Create rinkeby.yaml manifest that describes an Ethereum node which uses go-ethereum client, connecting to rinkeby network, and enabling JSON-RPC server:
Within a couple of seconds, the node will be up and running. You can get the node using:
kubectl get nodes.ethereum
It will return an output similar to the following:
Kotal will create all the necessary pods, volumes, services for the node. Get the pods by:
It will return an output similar to the following:
Finally delete the node by:
Congratulations!
rpc-user-password.yaml
# WARNING: DON'T use the following secret in production
apiVersion: v1
kind: Secret
metadata:
name: kotal-rpc-user-password
stringData:
password: s3cr3t
kubectl apply -f rpc-user-password.yaml
NAME CLIENT Network Consensus
rinkeby-geth-node geth rinkeby poa
For all the fields associated with the Node API resource:
Full NEAR node reference is documented .
Ethereum
We've extended Kubernetes with Node custom resource which can be used to deploy Ethereum nodes from the given spec.
Using Ethereum Node custom resource, you can describe any public or private network node you want, and Kotal will create all the necessary Kubernetes resources like pods, persistent volumes, services, configmaps, and secrets for you.
If you want to get all the deployted ethereum Node(s):
Which will report node name, client, network, and consensus.
IPFS
IPFS is a distributed system for storing and accessing files, websites, applications, and data.
We've extended Kubernetes with Peer and ClusterPeer custom resources which can be used to deploy IPFS peers, swarms, and clusters from the given spec.
Peer
Polkadot
Polkadot is a sharded protocol that enables blockchain networks to operate together seamlessly. Polkadot is a 100% open-source project created to enable a decentralized web and better society.
We've extended Kubernetes with Node custom resource which can be used to Polkadot, Kusama, Rococo, and substrate-based chains from the given spec.
Node
Node
Filecoin
We've extended Kubernetes with Node custom resource which can be used to create Filecoin nodes across different filecoin networks like Mainnet or calibration using lotus client from a given spec.
Here's an example of a Filecoin node that syncs calibration chain:
For all the fields associated with the Node API resource:
If you want to get all the fields associated with ethereum
Node
:
For a comprehensive reference on Ethereum Node, check our ethereum reference
Multi-client Support
Support for OpenEthereum (parity) client has been deprecated. OpenEthereum team is working with Erigon on a smooth transition path for users. More information can be found here
Peer is an ipfs peer running go-ipfs client and connecting to the public ipfs swarm or a private swarm secured by swarm key. Here's an example of a very basic ipfs peer connecting to the public ipfs swarm.
For all the fields associated with the Peer API resource:
ClusterPeer is an ipfs cluster peer running ipfs-cluster-service client, connecting to ipfs peer and optional bootstrap cluster peer(s). ClusterPeers provide data orchestration across a swarm of IPFS daemons by allocating, replicating and tracking a global pinset distributed among multiple peers. Here's an example of cluster peer connecting to ipfs peer.
For all the fields associated with the ClusterPeer API resource:
Full ipfs cluster peer reference is documented here.
We've extended Kubernetes with Node custom resource which can be used to deploy Chainlink nodes from the given spec.
apiVersion: chainlink.kotal.io/v1alpha1
kind: Node
metadata:
name: my-node
spec:
# your node spec goes here
Using Chainlink Node custom resource, you can describe any chainlink node, connect to EVM chain node, add jobs and satisfy requests, and Kotal will create all the necessary Kubernetes resources like pods, persistent volumes, services, configmaps, and secrets for you.
If you want to get all the deployted chainlink Node(s):
kubectl get nodes.chainlink
Which will report node name, client, Ethereum chain ID, and Link contract address.
If you want to get all the fields associated with chainlink Node:
For a comprehensive reference on Chainlink Node, check our
Deploy Filecoin Node
Deploy Node
This is a simple filecoin Node that joins calibration test network.
Let's deploy the node:
kubectl apply -f calibration.yaml
Kotal operator will notice your calibration-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets.
It will return an output similar to the following:
Fetch Node Logs
Get the pods created for the node:
It will return an output similar to the following:
If pod STATUS is Pending, most probably it's due to not enough cpu and memory, because Kotal allocates lots of cpu cores and memory for public nodes. The required node resources can be changed using spec.resources.cpu, spec.resources.memory, and spec.resources.storage.
Check the logs of the running node:
Finally you can delete the filecoin node and all its resources by
and kubernetes garbage collector will delete all resources created by the node controller.
Stacks
We've extended Kubernetes with Node custom resource which can be used to deploy Stacks rpc and miner nodes from the given spec.
Node
Node is Stacks node using Stacks Blockchain Node client, connecting to and syncing a specific chain, optionally mining or exposing JSON-RPC server to be used by clients.
For all the fields associated with the Node API resource:
Full Stacks node reference is documented .
Deploy Validator Client
Staking is the act of depositing 32 ETH to activate validator software. As a validator you’ll be responsible for storing data, processing transactions, and adding new blocks to the blockchain. This will keep Ethereum secure for everyone and earn you new ETH in the process. This process, known as proof-of-stake, is being introduced by the Beacon Chain.
Deploy Validator Client
In this validator client, we're using ConsenSys Teku Ethereum 2.0 client client: teku
NAME CLIENT NETWORK AGE
teku-validator teku pyrmont 1m
kubectl get pods
NAME READY STATUS RESTARTS AGE
teku-validator-0 1/1 Running 0 1m
kubectl logs -f teku-validator-0
...
21:21:30.668 INFO - Storing validator data in: /opt/teku/kotal-data/validator
21:21:30.751 INFO - Loading 1 validator keys...
21:21:31.759 INFO - Loaded 1 Validators: 83dbb18
...
kubectl delete validator teku-validator
Deploy Stacks RPC Node
Stacks node requires a Bitcoin node to connect to on the back-end for Proof of Transfer. You can deploy a Bitcoin node using Kotal operator like we did in this guide. In this tutorial, we will be using Bitcoin node hosted by Stacks Foundation.
Make sure Bitcoin node is fully synced before deploying your Stacks node.
Bitcoin node JSON-RPC user password
Let's store Stacks Foundation Bitcoin node JSON-RPC password in a Kubernetes secret to be used by our node:
Apply bitcoin-node-rpc-password.yaml to create the password secret:
Deploy Stacks RPC Node
The following manifest describes a Stacks node that syncs Stacks mainnet network: mainnet, and connects to the bitcoin node using configurations in bitcoinNode: ... for Proof of Transfer:
Apply stacks.yaml manifest:
Kotal operator will notice your stacks-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets neccessary.
You can fetch the deployed Stacks Node using:
It will return an output similar to the following:
Fetch Node Logs
Get the pods that has been created by Kotal for the node:
It will return an output similar to the following:
Get the logs of the running node:
It will return node logs similar to the following:
Call JSON-RPC Method
By default, Stacks node JSON-RPC server port is 20443, which can be changed using rpcPort.
Forward localhost:20443 calls to the node pod:
HTTP server will start after Bitcoin headers sync, till then you will get the following error if you tried to send JSON-RPC calls:
In another terminal window call info JSON-RPC method to retrieve information about the Core API including the server version:
You will get JSON result similar to the following:
Finally you can delete the node by:
Kubernetes garbage collector will delete all the resources that has been created by Stacks Node controller.
Deploy Stacks Miner Node
Stacks node requires a Bitcoin node to connect to on the back-end for Proof of Transfer. You can deploy a Bitcoin node using Kotal operator like we did in this guide. In this tutorial, we will be using Bitcoin node hosted by Stacks Foundation.
Make sure Bitcoin node is fully synced before deploying your Stacks node.
Generating Miner Private Key
Don't use the following Miner private key in production!
Generate Key chain using tool. Stacks CLI can be installed using:
Generate key chain using:
It will return an output similar to the following
Store privateKey from the output above into a Kubernetes secret in data field called key:
Bitcoin node JSON-RPC user password
Let's store Stacks Foundation Bitcoin node JSON-RPC password in a Kubernetes secret to be used by our node:
Apply bitcoin-node-rpc-password.yaml to create the password secret:
Deploy Stacks Miner Node
The following manifest describes a Stacks node that syncs Stacks mainnet network: mainnet, and connects to the bitcoin node using configurations in bitcoinNode: ... for Proof of Transfer, and loading miner private key using seedPrivateKeySecretName:
Apply stacks.yaml manifest:
Kotal operator will notice your stacks-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets neccessary.
You can fetch the deployed Stacks Node using:
It will return an output similar to the following:
Note Miner is true in the previous output 🔥
Fetch Node Logs
Get the pods that has been created by Kotal for the node:
It will return an output similar to the following:
Get the logs of the running node:
It will return node logs similar to the following:
Finally you can delete the node by:
Kubernetes garbage collector will delete all the resources that has been created by Stacks Node controller.
Deploy Beacon Node
The Beacon Chain is the coordination mechanism of the new network, responsible for creating new blocks, making sure those new blocks are valid, and rewarding validators with ETH for keeping the network secure. Source
Deploy Beacon Node
In this beacon node, we're using ConsenSys Teku Ethereum 2.0 client client: teku, and syncing the pyrmont network beacon chain network: pyrmont. We also enable REST API server rest: true at port 8888 restPort: 8888. We're connecting to Ethereum 1 endpoint eth1Endpoints: .... Check Ethereum on how to deploy an Ethereum node.
Let's deploy the beacon node:
Kotal operator will notice your teku-beacon-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets.
You can fetch the deployed Ethereum 2.0 BeaconNode using:
It will return an output similar to the following:
Fetch Beacon Node Logs
Get the pods created for the beacon node:
It will return an output similar to the following:
Get the logs of the running beacon node:
Call /eth/v1/beacon/genesis REST API
Get the service created for the beacon node:
It will return an output similar to the following:
Forward the localhost:8888 calls to the beacon node
In another terminal window, send REST API call to get chain genesis details:
You'll get a result similar to the following:
Finally delete the beacon node:
Kubernetes garbage collector will delete all the resources that has been created by Kotal Ethereum 2.0 BeaconNode controller.
Ethereum 2.0
We've extended Kubernetes with BeaconNode and Validator custom resources which can be used to deploy Ethereum 2.0 beacon nodes and validator clients from the given spec.
Kotal seperates between beacon node and validator client by using different API resources BeaconNode and Validator. So you can't run a beacon node and validator client in the same process/container.
Beacon Node
BeaconNode is Ethereum 2.0 beacon node -optionally- connecting to Ethereum 1 endpoint and syncing the beacon chain.
Here's an example of ConsenSys Teku beacon node BeaconNode syncing mainnet beacon chain, connecting to Ethereum 1 JSON RPC endpoint, enabling REST API server which is listening at port 8888:
For all the fields associated with the BeaconNode API resource:
Full beacon node reference is documented .
Validator Client
Validator is Ethereum 2.0 validator client connecting to beacon node endpoint and validating blocks using the loaded keystores.
Here's an example of ConsenSys Teku validator client connecting to beacon node endpoint, validating pyrmont blocks using 1 loaded validator keystore from its secret:
For all the fields associated with the Validator API resource:
Full validator client reference is documented .
Multi-client Support
Kotal supports the following Ethereum 2.0 beacon node and validator clients:
by ConsenSys.
by Sigma Prime.
by Prysmatic Labs.
Setting validator or beacon node client is as simple as changing the spec.client to the desired client.
Deploy Kusama Validator
Deploy Kusama Node
The following manifest describes Kusama node that syncs Kusama relay chain network: kusama, in archive mode pruning: false, and uses client:
Deploy IPFS peer
Deploy Peer
This is a simple ipfs peer that joins the public ipfs swarm, and will starts with the all default settings that's similar to running ipfs init then ipfs daemon on your machine.
Let's deploy the peer:
Deploy Public Rinkeby Node
Deploy Rinkeby Node
Rinkeby is a Proof of Authority public Ethereum test network, used by developers to test their dApps.
The following manifest describes an Ethereum node that joins rinkeby network network: rinkeby, and uses Hyperledger Besu client client: besu:
Deploy NEAR Validator Node
In this tutorial, we will learn how to deploy a NEAR validator node, and load a validator key into it.
Generate Validator Key
Generate validator account using tool. Which can be installed using:
Deploy NEAR RPC Node
Deploy RPC Node
The following manifest describes a NEAR node that syncs NEAR mainnet network: mainnet, enables HTTP JSON-RPC server rpc: true and uses client:
Kotal operator will notice your simple-peer and will create all the necessary pods, persistent volumes, services, configmaps, and secrets.
It will return an output similar to the following:
Fetch Peer Logs
Get the pods created for the peer:
It will return an output similar to the following:
Check the logs of the running peer:
You'll get output similar to the following:
Your peer is up and running and ready to receive api calls.
Read file Using IPFS HTTP API
Let's forward localhost:5001 calls to simple-peer:5001
In another terminal window, send http api call using cURL:
You'll get the following output:
You can also view ipfs webui by visiting http://0.0.0.0:5001/webui
Finally, delete the ipfs peer:
Kubernetes garbage collector will delete all the resources that has been created by Kotal IPFS Peer controller.
Kotal operator will notice your near-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets neccessary.
You can fetch the deployed NEAR Node using:
It will return an output similar to the following:
Fetch Node Logs
Get the pods that has been created by Kotal for the node:
It will return an output similar to the following:
Get the logs of the running node:
It will return node logs similar to the following:
Call JSON-RPC Method
By default, NEAR node JSON-RPC server port is 3030, which can be changed using rpcPort.
Forward localhost:3030 calls to the node pod:
In another terminal window call status JSON-RPC method to get the general status of our node (sync status, nearcore node version, protocol version) and the current set of validators:
You will get JSON result similar to the following:
Finally you can delete the node by:
Kubernetes garbage collector will delete all the resources that has been created by Kotal NEAR Node controller.
NAME NETWORK CLIENT MINER
stacks-node mainnet stacks false
kubectl get pods
NAME READY STATUS RESTARTS AGE
stacks-node-0 1/1 Running 0 1m
kubectl logs -f stacks-node-0
INFO [1649597250.010714] [testnet/stacks-node/src/main.rs:113] [main] Loading config at path /home/stacks/kotal-config/config.toml
INFO [1649597250.599067] [testnet/stacks-node/src/run_loop/neon.rs:341] [main] Start syncing Bitcoin headers, feel free to grab a cup of coffee, this can take a while
INFO [1649597251.949664] [src/burnchains/bitcoin/spv.rs:923] [main] Syncing Bitcoin headers: 0.3% (2000 out of 731268)
INFO [1649597253.010039] [src/burnchains/bitcoin/spv.rs:923] [main] Syncing Bitcoin headers: 0.5% (4000 out of 731268)
INFO [1649597254.019798] [src/burnchains/bitcoin/spv.rs:923] [main] Syncing Bitcoin headers: 0.8% (6000 out of 731268)
INFO [1649597254.992070] [src/burnchains/bitcoin/spv.rs:923] [main] Syncing Bitcoin headers: 1.1% (8000 out of 731268)
INFO [1649597256.214756] [src/burnchains/bitcoin/spv.rs:923] [main] Syncing Bitcoin headers: 1.4% (10000 out of 731268)
INFO [1649597257.276950] [src/burnchains/bitcoin/spv.rs:923] [main] Syncing Bitcoin headers: 1.6% (12000 out of 731268)
INFO [1649597258.383248] [src/burnchains/bitcoin/spv.rs:923] [main] Syncing Bitcoin headers: 1.9% (14000 out of 731268)
...
kubectl port-forward bitcoin-node-0 20443
curl: (52) Empty reply from server
curl localhost:20443/v2/info
{
"peer_version": 402653189,
"pox_consensus": "9bed78c9f02d2ba5a5950fe88c4746436b47c730",
"burn_block_height": 666051,
"stable_pox_consensus": "0000000000000000000000000000000000000000",
"stable_burn_block_height": 666050,
"server_version": "stacks-node No Version Info (No Branch Info:No Commit Info, release build, linux [x86_64])",
"network_id": 1,
"parent_network_id": 3652501241,
"stacks_tip_height": 0,
"stacks_tip": "0000000000000000000000000000000000000000000000000000000000000000",
"stacks_tip_consensus_hash": "0000000000000000000000000000000000000000",
"genesis_chainstate_hash": "74237aa39aa50a83de11a4f53e9d3bb7d43461d1de9873f402e5453ae60bc59b",
"unanchored_tip": null,
"unanchored_seq": null,
"exit_at_block_height": null
}
NAME NETWORK CLIENT MINER
stacks-node mainnet stacks true
kubectl get pods
NAME READY STATUS RESTARTS AGE
stacks-node-0 1/1 Running 0 1m
kubectl logs -f stacks-node-0
INFO [1649610683.233993] [testnet/stacks-node/src/main.rs:113] [main] Loading config at path /home/stacks/kotal-config/config.toml
INFO [1649610683.795606] [testnet/stacks-node/src/run_loop/neon.rs:341] [main] Start syncing Bitcoin headers, feel free to grab a cup of coffee, this can take a while
INFO [1649610686.606279] [src/burnchains/bitcoin/spv.rs:923] [main] Syncing Bitcoin headers: 0.3% (2000 out of 731294)
INFO [1649610687.625014] [src/burnchains/bitcoin/spv.rs:923] [main] Syncing Bitcoin headers: 0.5% (4000 out of 731294)
INFO [1649610688.634560] [src/burnchains/bitcoin/spv.rs:923] [main] Syncing Bitcoin headers: 0.8% (6000 out of 731294)
INFO [1649610689.677533] [src/burnchains/bitcoin/spv.rs:923] [main] Syncing Bitcoin headers: 1.1% (8000 out of 731294)
INFO [1649610690.737928] [src/burnchains/bitcoin/spv.rs:923] [main] Syncing Bitcoin headers: 1.4% (10000 out of 731294)
INFO [1649610691.745002] [src/burnchains/bitcoin/spv.rs:923] [main] Syncing Bitcoin headers: 1.6% (12000 out of 731294)
INFO [1649610692.789689] [src/burnchains/bitcoin/spv.rs:923] [main] Syncing Bitcoin headers: 1.9% (14000 out of 731294)
...
kubectl delete -f stacks.yaml
kubectl apply -f beacon.yaml
kubectl get beaconnodes
NAME CLIENT Network AGE
teku-beacon-node teku pyrmont 1m
kubectl get pods
NAME READY STATUS RESTARTS AGE
teku-beacon-node-0 1/1 Running 0 1m
kubectl logs -f teku-beacon-node-0
kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
teku-beacon-node ClusterIP 10.96.197.218 <none> 9000/UDP,9000/TCP,8888/TCP 7s
NAME READY STATUS RESTARTS AGE
kusama-node-0 1/1 Running 0 1m
kubectl logs -f kusama-node-0
2021-10-18 15:32:38 ----------------------------
2021-10-18 15:32:38 This chain is not in any way
2021-10-18 15:32:38 endorsed by the
2021-10-18 15:32:38 KUSAMA FOUNDATION
2021-10-18 15:32:38 ----------------------------
2021-10-18 15:32:38 Parity Polkadot
2021-10-18 15:32:38 ✌️ version 0.9.11-bfd38ed62-x86_64-linux-gnu
2021-10-18 15:32:38 ❤️ by Parity Technologies <[email protected]>, 2017-2021
2021-10-18 15:32:38 📋 Chain specification: Kusama
2021-10-18 15:32:38 🏷 Node name: validator-node-sample
2021-10-18 15:32:38 👤 Role: AUTHORITY
2021-10-18 15:32:38 💾 Database: RocksDb at /polkadot/kotal-data/chains/ksmcc3/db/full
2021-10-18 15:32:38 ⛓ Native runtime: kusama-9110 (parity-kusama-0.tx7.au2)
2021-10-18 15:32:38 🔨 Initializing Genesis block/state (state: 0xb000…ef6b, header-hash: 0xb0a8…dafe)
2021-10-18 15:32:38 👴 Loading GRANDPA authority set from genesis on what appears to be first startup.
2021-10-18 15:32:39 ⏱ Loaded block-time = 6s from block 0xb0a8d493285c2df73290dfb7e61f870f17b41801197a149ca93654499ea3dafe
2021-10-18 15:32:39 👶 Creating empty BABE epoch changes on what appears to be first startup.
2021-10-18 15:32:39 🏷 Local node identity is: 12D3KooWFEHU9FXqGCL6ify4rqcon31WYViCnUfA6aVYTjRXqJ5F
2021-10-18 15:32:39 📦 Highest known block at #0
2021-10-18 15:32:39 〽️ Prometheus exporter started at 0.0.0.0:9615
2021-10-18 15:32:39 Listening for new connections on 127.0.0.1:9944.
2021-10-18 15:32:39 👶 Starting BABE Authorship worker
...
NAME READY STATUS RESTARTS AGE
simple-peer-0 1/1 Running 0 5m
kubectl logs -f simple-peer-0
Initializing daemon...
go-ipfs version: 0.8.0-ce693d7
Repo version: 11
System version: amd64/linux
Golang version: go1.14.4
Swarm listening on /ip4/10.244.0.6/tcp/4001
Swarm listening on /ip4/10.244.0.6/udp/4001/quic
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/127.0.0.1/udp/4001/quic
Swarm listening on /ip6/::1/tcp/4001
Swarm listening on /ip6/::1/udp/4001/quic
Swarm listening on /p2p-circuit
Swarm announcing /ip4/10.244.0.6/tcp/4001
Swarm announcing /ip4/10.244.0.6/udp/4001/quic
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/127.0.0.1/udp/4001/quic
Swarm announcing /ip6/::1/tcp/4001
Swarm announcing /ip6/::1/udp/4001/quic
API server listening on /ip4/0.0.0.0/tcp/5001
WebUI: http://0.0.0.0:5001/webui
Gateway (readonly) server listening on /ip4/0.0.0.0/tcp/8080
Daemon is ready
kubectl port-forward simple-peer-0 5001
curl -X POST "http://127.0.0.1:5001/api/v0/cat?arg=QmQPeNsJPyVWPFDVHb77w8G42Fvo15z4bG2X8D2GhfbSXc/readme"
Hello and Welcome to IPFS!
██╗██████╗ ███████╗███████╗
██║██╔══██╗██╔════╝██╔════╝
██║██████╔╝█████╗ ███████╗
██║██╔═══╝ ██╔══╝ ╚════██║
██║██║ ██║ ███████║
╚═╝╚═╝ ╚═╝ ╚══════╝
If you're seeing this, you have successfully installed
IPFS and are now interfacing with the ipfs merkledag!
-------------------------------------------------------
| Warning: |
| This is alpha software. Use at your own discretion! |
| Much is missing or lacking polish. There are bugs. |
| Not yet secure. Read the security notes for more. |
-------------------------------------------------------
Check out some of the other files in this directory:
./about
./help
./quick-start <-- usage examples
./readme <-- this file
./security-notes
Joining any Ethereum network is as easy as setting node's spec.network value, which accepts values like mainnet, rinkeby, goerli, xdai ... etc. After the node is created, you can't change spec.network.
Apply rinkeby.yaml manifest:
Kotal operator will notice your rinkeby-besu-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets neccessary.
You can fetch the deployed Ethereum Node using:
It will return an output similar to the following:
Fetch Node Logs
Get the pods that has been created by Kotal for the node:
It will return an output similar to the following:
If pod STATUS is Pending, most probably it's due to not enough cpu and memory, because Kotal allocates lots of cpu cores and memory for public nodes. The required node resources can be changed using spec.resources.cpu, spec.resources.memory, and spec.resources.storage.
Get the logs of the running node:
Call JSON-RPC Method
Let's update our node by enabling JSON-RPC HTTP server:
Kotal defaults node missing spec parameters like:
JSON-RPC HTTP server port is defaulted to 8545.
JSON-RPC HTTP server enabled modules are defaulted to: eth, net, and web3.
Blocks Synchronization mode is defauled to fast in public networks.
For a comprehensive reference on Ethereum Node and default spec parameter values, check our
Apply the new version of rinkeby.yaml:
Forward localhost:8545 calls to the node pod:
In another terminal window call eth_syncing JSON-RPC method
You will get JSON result similar to the following:
Finally you can delete the node by:
Kubernetes garbage collector will delete all the resources that has been created by Kotal Ethereum Node controller.
Validator account must have the name validator.
An ed25519 private key will be generated and saved in current user home directory under .near-credentials/testnet
It will return an output similar to the following
Change private_key field to secret_key.
Generate Secret for Validator Private Key
Create a Kubernetes secret from the generated private key in the previous step:
Deploy NEAR Validator Node
The following manifest describes a NEAR node that syncs NEAR testnet network: testnet, enables HTTP JSON-RPC server rpc: true, loads validator key from the Kubernetes secret that we've created in the previous step, and uses NEAR Core client:
Apply near.yaml manifest:
Kotal operator will notice your near-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets neccessary.
You can fetch the deployed NEAR Node using:
It will return an output similar to the following:
Note that validator is true in the prevous output 😇
Fetch Node Logs
Get the pods that has been created by Kotal for the node:
It will return an output similar to the following:
Get the logs of the running node:
It will return node logs similar to the following:
Finally you can delete the node by:
Kubernetes garbage collector will delete all the resources that has been created by Kotal NEAR Node controller.
Chainlink node requires access to an EVM chain node with WebSocket server enabled. Here's how to deploy a simple Ethereum Rinkeby node with WebSocket server enabled:
After deploying this node, it can be accessed in-cluster using ws://besu-rinkeby-node:8546 🔥. Read for more details on how to deploy public and private network nodes.
PostgreSQL Database
Chainlink requires access to database instance.
You can use a hosted database a service to obtain PostgreSQL database:
You can also deploy your own PostgreSQL database instance from official PostgreSQL .
Keystore and API Password
Chainlink node requires a strong password for securing access to keystore (wallet) and API. We will store this password in Kubernetes secret my-password:
TLS Certificate
Chainlink optionally requires TLS certificate to secure access to the node API. Using we can issue a simple self-signed certificate and store it in a Kubernetes secret:
Deploying Chainlink Node
With access to Ethereum Rinkeby node, PostgreSQL database, tls certificate and a strong password. We're ready to deploy our Chainlink node to create jobs and satisfy requests:
In this manifest, we're describing Chainlink node that connects to Ethereum Rinkeby node using ethereumWsEndpoint, setting ethereum chain ID using ethereumChainId, setting link contract address using linkContractAddress, connecting to postgress database instance using databaseURL, setting wallet password using keystorePasswordSecretName which accepts a name of k8s secret, setting tls configuration using certSecretName which accepts k8s secret name that holds tls.key and tls.crt, and finally setting API credentials using apiCredentials.
Apply chainlink.yaml manifest:
Kotal operator will notice your chainlink-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets neccessary.
You can fetch the deployed Chainlink Node using:
It will return an output similar to the following:
If you added -o wide it will return more info like link contract address
Fetch Node Logs
Get the pods that has been created by Kotal for the node:
It will return an output similar to the following:
Get the logs of the running node:
Open Chainlink User Interface
Chainlink node comes with a user interface for managing the node, and it's being served by default on port 6688.
Forward localhost:6688 calls to the node pod:
Open https://localhost:6688 and it will take you to Chainlink node UI. It will ask you for API credentials. Enter the email address in your node's .spec.apiCredentials.email, and in the password field, enter fE2xXKDnR3ns489X.
🔥🔥🔥
Finally you can delete the node by:
Kubernetes garbage collector will delete all the resources that has been created by Kotal Chainlink Node controller.
Bitcoin
Syntax
Type
Description
Default
required
string
Bitcoin network to join and sync
network
network is Bitcoin network to join and sync. Possible values are mainnet and testnet.
network is immutable, can't be changed after node is created.
p2pPort
p2pPort is p2p communications port.
p2pHost
p2pHost is p2p communications host.
rpc
rpc enables JSON-RPC server.
rpcPort
rpcPort is JSON-RPC server port.
rpcHost
rpcHost is JSON-RPC server host.
rpcUsers
rpcUsers is a list of JSON-RPC users credentials:
Syntax
Type
Description
wallet
wallet loads local wallet and enables wallet RPC calls.
txIndex
txIndex maintains a full transaction index.
resources
resources allocates compute and storage resources to the node.
Syntax
Type
Description
Default
memory and storage requests and limits must use the pattern ^[1-9][0-9]*[KMGTPE]i$ for example 1500Mi, 30Gi, and 1Ti.
cpu requests and limits must use the pattern ^[1-9][0-9]*m?$ for example 1000m (which is equal to 1), 1500m, 2, and 4.
cpuLimit can't be less than cpu.
memoryLimit can't be less than or equal to memory.
storageClass field is immutable, it cannot be changed after creation.
Deploy Polkadot Node
Deploy Polkadot Node
The following manifest describes a Polkadot node that syncs Polkadot relay chain network: polkadot, enables HTTP JSON-RPC server rpc: true on custom port rpcPort: 8888, and uses Parity Polkadot client:
Apply polkadot.yaml manifest:
Kotal operator will notice your polkadot-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets neccessary.
You can fetch the deployed Polkadot Node using:
It will return an output similar to the following:
Fetch Node Logs
Get the pods that has been created by Kotal for the node:
It will return an output similar to the following:
Get the logs of the running node:
It will return node logs similar to the following:
Call JSON-RPC Method
Forward localhost:8888 calls to the node pod because rpcPort: 8888:
In another terminal window call eth_getBlock JSON-RPC method to get the latest synced block:
You will get JSON result similar to the following:
Finally you can delete the node by:
Kubernetes garbage collector will delete all the resources that has been created by Kotal Polkadot Node controller.
Deploy Private Clique Network
Deploy Private Network Node
In this tutorial we will use the following private keys and their corresponding addresses.
Let's describe an Ethereum Node that uses a custom genesis block to join a private Proof of Authority network using Clique consensus configuration in the genesis block.
In this node, we're using Hyperledger besu client client: besu, enabling JSON-RPC server rpc: true so we can query number of peers later in this tutorial, and we're loading the node private key from Kubernetes secretd called besu-clique-nodekey.
The node private key will give the node a unique identity and node URL, and will allow the node to generate blocks, because the address 0xbAa5f05af4A67A467cEcA89085f162aFb4206Aaa that's corresponding to the node private key is in the initial block signers.
We're defining a genesis block that uses the value 4444 as network and chain identifier, and we start the chain with 3 signers as defined by spec.genesis.clique.signers.
This node private key secret can be created by:
Private key must not start with 0x, and must be stored in secret data field called key.
Let's deploy the node:
Kotal operator will notice your besu-clique-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets.
You can fetch the deployed Ethereum Node using:
It will return an output similar to the following:
Fetch Node Logs
Get the pods that has been created by Kotal for the node:
It will return an output similar to the following:
Get the logs of the running node:
Deploy a Second Node
Let's deploy another go-ethereum node, and connect it to the previous node in our private proof of authority network.
Genesis block must be the same in both nodes, or they will fork at genesis block, and won't reach consensus.
In this node, we're using go-ethereum client client: geth, starting the PoA consensus engine miner: true, setting the second address in the genesis signers list spec.genesis.clique.signers as the coinbase coinbase: "0xc1381ED43B327e3C7A1ADb21285f1e9cB82Bc00d", and loading the signer account private key and password from kubernetes secrets privateKeySecretName: ... and passwordSecretName: .... We're connecting to the first node using staticNodes option which accepts Node name or enode url.
staticNodes accept Node name or enode URL. Node name has the format of name.namespace, namespace is optional if Node is in the same namespace. If the node doesn't exist, or is not up and running yet, Kotal will not raise an error.
You can create the private key and password secrets using:
Deploy the second node using:
Kotal operator will notice your second geth-clique-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets.
You can fetch the deployed Ethereum Nodes using:
It will return an output similar to the following:
Call JSON-RPC Method net_peerCount
Get the pods that has been created by Kotal for the node:
It will return an output similar to the following:
Forward localhost:8545 calls to the node pod:
In another terminal window call net_peerCount JSON-RPC method
You will get JSON result similar to the following:
Homework
Deploy a third node that uses Nethermind client, and signing blocks using the third key in the signers list spec.genesis.clique.signers. Nethermind client is similar to geth, you will import signer account private key and password from kubernetes secrets, and use the same genesis as the other nodes.
Finally you can delete all the nodes by:
Kubernetes garbage collector will delete all the resources that has been created by Kotal Ethereum Node controller.
Kubernetes secret name holding node Ed25519 private key
string
Kubernetes secret name holding node Ed25519 validator key
number
Minimum number of peers to start syncing/producing blocks
5
boolean
Keeps old blocks in the storage
false
number
p2p protocol tcp port
24567
string
p2p host
0.0.0.0
boolean
Enables JSON-RPC server
false
number
JSON-RPC server listening port
3030
string
JSON-RPC server listening host
0.0.0.0
number
Prometheus exporter port
9615
string
Prometheus exporter host
0.0.0.0
string
Telemetry service URL
array
Boot nodes to bootstrap network from
object
node compute and storage resources to alloacte
network
network is NEAR network to join and sync. Possible values are mainnet, testnet, and betanet.
network is immutable, it can't be changed after node is created.
nodePrivateKeySecretName
nodePrivateKeySecretName is Kubernetes secret name holding node Ed25519 private key in data field key, from which node public key is generated, and p2p messages are signed.
validatorSecretName
validatorSecretName is Kubernetes secret name holding node Ed25519 validator key in data field key.
minPeers
minPeers is the minimum number of peers to start syncing/producing blocks.
archive
archive keeps old blocks in the storage.
p2pPort
p2pPort is p2p protocol tcp port.
p2pHost
p2pHost is p2p host address.
rpc
rpc enables JSON-RPC server.
rpcPort
rpcPort is JSON-RPC server listening port.
rpcHost
rpcHost is JSON-RPC server listening host address.
prometheusPort
prometheusPort is Prometheus exporter port.
prometheusHost
prometheusHost is Prometheus exporter host address.
telemetryURL
telemetryURL is Telemetry service URL.
bootnodes
bootnodes is a list of boot nodes to bootstrap network from
resources
resources allocates compute and storage resources to the node.
Syntax
Type
Description
Default
cpu
string
number of cpu cores this node requires
4
cpuLimit
string
memory and storage requests and limits must use the pattern ^[1-9][0-9]*[KMGTPE]i$ for example 1500Mi, 30Gi, and 1Ti.
cpu requests and limits must use the pattern ^[1-9][0-9]*m?$ for example 1000m (which is equal to 1), 1500m, 2, and 4.
cpuLimit can't be less than cpu.
memoryLimit can't be less than or equal to memory.
storageClass field is immutable, it cannot be changed after creation.
In this tutorial we will use the following private keys and their corresponding addresses.
DON'T use these keys in production.
Address
Private Key
Let's describe an Ethereum Node that uses a custom genesis block to join a private Proof of Authority network using ibft2 consensus configuration in the genesis block.
In this node, we're using Hyperledger besu client client: besu, enabling JSON-RPC server rpc: true so we can query number of peers later in this tutorial, and we're loading the node private key from Kubernetes secretd called besu-ibft2-nodekey.
The node private key will give the node a unique identity and node URL, and will allow the node to generate blocks, because the address 0xbAa5f05af4A67A467cEcA89085f162aFb4206Aaa that's corresponding to the node private key is in the initial block validators.
We're defining a genesis block that uses the value 4444 as network and chain identifier, and we start the chain with 3 validators as defined by spec.genesis.ibft2.validators.
This node private key secret can be created by:
Private key must not start with 0x, and must be stored in secret data field called key.
Let's deploy the node:
Kotal operator will notice your besu-ibft2-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets.
You can fetch the deployed Ethereum Node using:
It will return an output similar to the following:
Fetch Node Logs
Get the pods that has been created by Kotal for the node:
It will return an output similar to the following:
Get the logs of the running node:
Deploy a Second Node
Let's deploy another go-ethereum node, and connect it to the previous node in our private proof of authority network.
Genesis block must be the same in both nodes, or they will fork at genesis block, and won't reach consensus.
In this node, we're using go-ethereum client client: geth, starting the PoA consensus engine miner: true, setting the second address in the genesis validators list spec.genesis.ibft2.validators as the coinbase coinbase: "0xc1381ED43B327e3C7A1ADb21285f1e9cB82Bc00d", and loading the validator account private key and password from kubernetes secrets privateKeySecretName: ... and passwordSecretName: .... We're connecting to the first node using staticNodes option which accepts Node name or enode url.
staticNodes accept Node name or enode URL. Node name has the format of name.namespace, namespace is optional if Node is in the same namespace. If the node doesn't exist, or is not up and running yet, Kotal will not raise an error.
You can create the private key and password secrets using:
Deploy the second node using:
Kotal operator will notice your second geth-ibft2-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets.
You can fetch the deployed Ethereum Nodes using:
It will return an output similar to the following:
Call JSON-RPC Method net_peerCount
Get the pods that has been created by Kotal for the node:
It will return an output similar to the following:
Forward localhost:8545 calls to the node pod:
In another terminal window call net_peerCount JSON-RPC method
You will get JSON result similar to the following:
Homework
Deploy a third node that uses Nethermind client, and signing blocks using the third key in the validators list spec.genesis.ibft2.validators. Nethermind client is similar to geth, you will import validator account private key and password from kubernetes secrets, and use the same genesis as the other nodes.
Finally you can delete all the nodes by:
Kubernetes garbage collector will delete all the resources that has been created by Kotal Ethereum Node controller.
Stacks
Syntax
Type
Description
Default
required
string
Stacks network to join and sync
network
network is Stacks network to join and sync. Possible values are mainnet and testnet.
network is immutable, can't be changed after node is created.
rpcPort
rpcPort is JSON-RPC server port.
rpcHost
rpcHost is JSON-RPC server host.
p2pPort
p2pPort is p2p bind port.
p2pHost
p2pHost is p2p bind host.
bitcoinNode
bitcoinNode is Bitcoin node details for Stacks node to connect and query:
Syntax
Type
Description
miner
miner enables mining.
seedPrivateKeySecretName
seedPrivateKeySecretName is Kubernetes secret name holding seed private key used for mining.
ethereumWsEndpoint is Ethereum node's WebSocket server endpoint.
ethereumHttpEndpoints
ethereumHttpEndpoints is array of Ethereum HTTP endpoints used for heavy requests and as fallback in case of Ethereum node bug or failure.
databaseURL
databaseURL is Postgres database connection URL.
keystorePasswordSecretName
keystorePasswordSecretName is k8s secret name that holds keystore (wallet) password in data field called password.
apiCredentials
apiCredentials is the credetials used to access Chainlink node UI:
Syntax
Type
Description
corsDomains
corsDomains is a list of domains from which to accept cross origin requests.
certSecretName
certSecretName is k8s secret name that holds TLS private key in tls.key data field and TLS certificate in tls.crt data field.
apiPort
apiPort is the port used for node API and GUI.
tlsPort
tlsPort is the port used for HTTPS connections.
p2pPort
p2pPort is the port used for p2p communcations.
secureCookies
secureCookies enables secure cookies for authentication.
logging
logging is logging verbosity level. Possible values are debug, info, warn, error, and panic.
resources
resources allocates compute and storage resources to the node.
Syntax
Type
Description
Default
memory and storage requests and limits must use the pattern ^[1-9][0-9]*[KMGTPE]i$ for example 1500Mi, 30Gi, and 1Ti.
cpu requests and limits must use the pattern ^[1-9][0-9]*m?$ for example 1000m (which is equal to 1), 1500m, 2, and 4.
cpuLimit can't be less than cpu.
memoryLimit can't be less than or equal to memory.
storageClass field is immutable, it cannot be changed after creation.
Deploy IPFS cluster peer
IPFS Cluster provides data orchestration across a swarm of IPFS daemons by allocating, replicating and tracking a global pinset distributed among multiple peers. Source
Prerequisites
Before we start, you need to:
Step 1 - Deploy IPFS Peer
Deploy an ipfs peer Peer because it's required by ClusterPeer. Check our tutorial.
Step 2 - Generate Cluster Secret
Create a cluster secret which secures access to the cluster:
Cluster secret must be 32-byte hex-encoded (64 characters) without the leading 0x. It must be hold in data field called secret in the Kubernetes secret.
Step 3 - Generate ID and Private Key
Create cluster peer ID and private key using
It will return an output similar to the following:
Step 4 - Deploy Private Key Secret
create Kubernetes secret from the generated private key in step(3) above.
Deploying First Cluster Peer
Now we're ready to deploy the cluster peer and connect it to the peer we've deployed in step(1)
Note that id and privateKeySecretName are optional, they're useful to get predictable cluster peer id to be used in future steps as bootstrap peer.
Let's deploy cluster peer manifest file:
Kotal operator will notice your cluster-peer and will create all the necessary pods, persistent volumes, services, configmaps, and secrets.
You can fetch the deployed IPFS ClusterPeer using:
It will return an output similar to the following:
Fetch Cluster Peer Logs
Get the pods created for our cluster peer:
It will return an output similar to the following:
Check the logs of the running cluster peer:
It will return an output similar to the following:
As you can see from the logs that our cluster peer is up and running as indicated by the healthy logs IPFS Cluster is READY.
Update Cluster Pinset
Let's add some content to the pinset by executing commands inside cluster peer container:
Let's fetch the pinset of the ipfs peer by executing commands inside peer -created in step(1)- container:
As you can see, content pinned by the cluster peer, has been added to the pinset of the ipfs peer and available in its local storage.
Deploying Second Cluster Peer
The only prerequisite to the second cluster peer is to:
Step 1 - Deploy IPFS Peer
Deploy an ipfs peer Peer, and name itpeer-sample-two. Check our tutorial.
We will reuse cluster secret that was generated in step(2) during deploying first cluster peer above.
The first cluster peer multiaddress is predictable because we've used a pre-generated id and private key in step(3). /ip4/{cluster peer k8s service name}/tcp/9096/p2p/{cluster peer ID}
Deploying this cluster manifest will deploy a cluster peer that connects to the first cluster peer and will connect to its own ipfs peer peer-sample-two instance.
You can fetch the deployed cluster peers by:
It will return an output similar to the following:
Fetch Second Cluster Peer Logs
Get the pods created by our cluster peer:
It will return an output similar to the following:
Check the logs of cluster peer two:
It will return an output similar to the following:
As you can see cluster peer two is up and running as indicated by the healthy logs and IPFS Cluster is READY.
Cluster peer two has joined the first cluster peer as indicated by the highlighted log line joined 12D...uiq's cluster
Finally, delete all peers and cluster peers:
Kubernetes garbage collector will delete all the resources that has been created by Kotal IPFS Peer and ClusterPeer controllers.
Generating a 2048 bit ed25519 key...
Success!
ID for generated key: 12D3KooWBcEtY8GH4mNkri9kM3haeWhEXtQV7mi81ErWrqLYGuiq
CAESQOH/DvUJmeJ9z6m3wAStpkrlBwJQxIyNSK0YGf0EI5ZRGpwsWxl4wmgReqmHl8LQjTC2iPM0QbYAjeY3Z63AFnI=
apiVersion: ipfs.kotal.io/v1alpha1
kind: ClusterPeer
metadata:
Name: cluster-peer
spec:
consensus: crdt
# from step(3)
id: "12D3KooWBcEtY8GH4mNkri9kM3haeWhEXtQV7mi81ErWrqLYGuiq"
# from step(4)
privateKeySecretName: cluster-peer-privatekey
# from step(1)
peerEndpoint: /dns4/peer-sample/tcp/5001
# from step(2)
clusterSecretName: cluster-secret
kubectl apply -f cluster-peer.yaml
kubectl get clusterpeers
NAME CLIENT CONSENSUS
cluster-peer ipfs-cluster-service crdt
kubectl get pods
NAME READY STATUS RESTARTS AGE
cluster-peer-0 1/1 Running 0 2m
kubectl logs -f cluster-peer-0
2021-04-30T00:31:24.109Z INFO service ipfs-cluster-service/daemon.go:46 Initializing. For verbose output run with "-l debug". Please wait...
2021-04-30T00:31:24.133Z INFO cluster ipfs-cluster/cluster.go:136 IPFS Cluster v0.13.2+git812c8e3631d4d6771a6e04112e12cd7726e23f7a listening on:
/ip4/10.244.0.8/tcp/9096/p2p/12D3KooWBcEtY8GH4mNkri9kM3haeWhEXtQV7mi81ErWrqLYGuiq
/ip4/127.0.0.1/tcp/9096/p2p/12D3KooWBcEtY8GH4mNkri9kM3haeWhEXtQV7mi81ErWrqLYGuiq
2021-04-30T00:31:24.134Z INFO ipfsproxy ipfsproxy/ipfsproxy.go:320 IPFS Proxy: /ip4/127.0.0.1/tcp/9095 -> /ip4/127.0.0.1/tcp/5001
2021-04-30T00:31:24.134Z INFO restapi rest/restapi.go:521 REST API (HTTP): /ip4/127.0.0.1/tcp/9094
2021-04-30T00:31:24.134Z INFO crdt [email protected]/crdt.go:278 crdt Datastore created. Number of heads: 0. Current max-height: 0
2021-04-30T00:31:24.136Z INFO cluster ipfs-cluster/cluster.go:651 Cluster Peers (without including ourselves):
2021-04-30T00:31:24.136Z INFO cluster ipfs-cluster/cluster.go:653 - No other peers
2021-04-30T00:31:24.136Z INFO cluster ipfs-cluster/cluster.go:666 ** IPFS Cluster is READY **
$ kubectl exec -it cluster-peer-0 -- sh
# now we're inside cluster-peer container
$ echo "kotal simplifies Blockchain DevOps" > kotal.txt
# add kotal.txt to the pinset
$ ipfs-cluster-ctl add kotal.txt
added QmUiwYtz1VxeryVCpTLWqrbeUi9HyGH4AnpSG6BrmdMhzL kotal.txt
# leave cluster-peer container
$ exit
$ kubectl exec -it peer-sample-0 -- sh
# now we're inside peer-sample container
# List objects pinned to local storage
$ ipfs pin ls
QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn recursive
QmUiwYtz1VxeryVCpTLWqrbeUi9HyGH4AnpSG6BrmdMhzL recursive
$ ipfs cat QmUiwYtz1VxeryVCpTLWqrbeUi9HyGH4AnpSG6BrmdMhzL
kotal simplifies Blockchain DevOps
# leave ipfs peer container
$ exit
cluster-peer-two.yaml
apiVersion: ipfs.kotal.io/v1alpha1
kind: ClusterPeer
metadata:
Name: cluster-peer-two
spec:
# from step(1)
peerEndpoint: /dns4/peer-sample-twp/tcp/5001
# generated during deploying first cluster peer in step(2)
clusterSecretName: cluster-secret
# first cluster peer listening address
bootstrapPeers:
- /ip4/cluster-peer/tcp/9096/p2p/12D3KooWBcEtY8GH4mNkri9kM3haeWhEXtQV7mi81ErWrqLYGuiq
kubectl apply -f cluster-peer-two.yaml
kubectl get clusterpeers
NAME CLIENT CONSENSUS
cluster-peer ipfs-cluster-service crdt
cluster-peer-two ipfs-cluster-service crdt
kubectl get pods
NAME READY STATUS RESTARTS AGE
cluster-peer-0 1/1 Running 0 10m
cluster-peer-two-0 1/1 Running 0 2m
kubectl logs -f cluster-peer-two-0
2021-05-01T19:17:12.043Z INFO service ipfs-cluster-service/daemon.go:46 Initializing. For verbose output run with "-l debug". Please wait...
2021-05-01T19:17:12.352Z INFO cluster ipfs-cluster/cluster.go:136 IPFS Cluster v0.13.2+git812c8e3631d4d6771a6e04112e12cd7726e23f7a listening on:
/ip4/10.244.0.35/tcp/9096/p2p/12D3KooWGVZek7gdarvj7gePvnV9x8ekSeSdruYojgj5dmU466Wv
/ip4/127.0.0.1/tcp/9096/p2p/12D3KooWGVZek7gdarvj7gePvnV9x8ekSeSdruYojgj5dmU466Wv
2021-05-01T19:17:12.441Z INFO restapi rest/restapi.go:521 REST API (HTTP): /ip4/127.0.0.1/tcp/9094
2021-05-01T19:17:12.441Z INFO ipfsproxy ipfsproxy/ipfsproxy.go:320 IPFS Proxy: /ip4/127.0.0.1/tcp/9095 -> /ip4/127.0.0.1/tcp/5001
2021-05-01T19:17:12.441Z INFO service ipfs-cluster-service/daemon.go:218 Bootstrapping to /dns4/cluster-peer/tcp/9096/p2p/12D3KooWBcEtY8GH4mNkri9kM3haeWhEXtQV7mi81ErWrqLYGuiq
2021-05-01T19:17:12.442Z INFO crdt [email protected]/crdt.go:278 crdt Datastore created. Number of heads: 0. Current max-height: 0
2021-05-01T19:17:12.442Z INFO crdt crdt/consensus.go:277 'trust all' mode enabled. Any peer in the cluster can modify the pinset.
2021-05-01T19:17:12.447Z INFO cluster ipfs-cluster/cluster.go:651 Cluster Peers (without including ourselves):
2021-05-01T19:17:12.447Z INFO cluster ipfs-cluster/cluster.go:653 - No other peers
2021-05-01T19:17:12.448Z INFO cluster ipfs-cluster/cluster.go:666 ** IPFS Cluster is READY **
2021-05-01T19:17:12.551Z INFO cluster ipfs-cluster/cluster.go:1020 12D3KooWGVZek7gdarvj7gePvnV9x8ekSeSdruYojgj5dmU466Wv: joined 12D3KooWBcEtY8GH4mNkri9kM3haeWhEXtQV7mi81ErWrqLYGuiq's cluster
# these extra arguments will be passed as follow to the client
# --arg1 val1 --toggle --arg2 val2
extraArgs:
"--arg1": val1
"--toggle": ""
"--arg2": val2
# generate binary node key and save it in node.key file
subkey generate-node-key node.key
# convert node.key binary file into hexadecimal node.key.hex file
xxd -p -c 32 node.key > node.key.hex
Kotal uses only go-ipfs client for IPFS peers, that's why there's no .spec.client
option.
Syntax
Type
Description
Default
Array
List of initial configuration profiles
default-datastore
initProfiles
initProfiles is al list of initial ipfs configuration profile.
initialProfiles available values are server, randomports, default-datastore, local-discovery, test, default-networking, flatfs, badgerds, and lowpower.
initProfiles can't be updated (immutable).
profiles
profiles is the list of configuration profiles to apply after peer initialization.
profiles available values are server, randomports, default-datastore, local-discovery, test, default-networking, flatfs, badgerds, and lowpower.
apiHost
apiHost is API server host.
If you set apiHost to host other than 0.0.0.0, api calls forwarded to the container won't hit the API server. This is useful if you want to disallow calls to API server.
apiPort
apiPort is API server port.
gatewayHost
gatewayHost is local ipfs gateway host.
If you set gatewayHost to host other than 0.0.0.0, gateway won't work. This is useful if you want to disable to access gateway from outside.
gatewayPort
gatewayPort is API server port.
routing
routing is the content routing mechanism.
routing available values are none, dht, dhtclient, dhtserver.
swarmKeySecretName
swarmKeySecretName is the kubernetes secret name that's holding the swarm key in a key called secret.
resources
resources allocates compute and storage resources to the peer.
resources object has the following fields:
Syntax
Type
Description
Defalt
cpu
string
number of cpu cores this peer requires
1
cpuLimit
Memory and storage requests and limits must use the pattern ^[1-9][0-9]*[KMGTPE]i$ for example 1500Mi, 30Gi, and 1Ti.
cpu requests and limits must use the pattern ^[1-9][0-9]*m?$ for example 1000m (which is equal to 1 core), 1500m which is 1.5 core, 2 cores, and 4 cores.
cpuLimit can't be less than cpu.
memoryLimit can't be less than or equal to memory.
storageClass field is immutable, it cannot be changed after creation.
ClusterPeer
Kotal uses only ipfs-cluster-service for IPFS cluster peers that runs along with go-ipfs, that's why there's no .spec.client option.
Syntax
Type
Description
Default
string
Cluster peer ID
id
id is cluster peer id derived from private key.
id is required if privateKeySecretName is provided.
It will return an output similar to the following:
privateKeySecretName
privateKeySecretName is the name of the k8s secret holding base64 cluster peer privatekey in a key called key.
privateKeySecretName is required if id is provided.
privateKeySecretName can be generated using ipfs-key tool.
It will return an output similar to the following:
trustedPeers
trustedPeers is a list of peer IDs that can manage the cluster pinset in crdt consensus clusters.
trustedPeers is ignored in raft consensus clusters.
trustedPeers default value is * which is trust all peers.
bootstrapPeers
bootstrapPeers is a list of peers to connect to on startup.
bootstrapPeers will be trusted peers in crdt clusters.
consensus
consensus is the cluster consensus algorithm.
peerEndpoint
peerEndpoint id ipfs peer http API endpoint.
peerEndpoint is required for the cluster peer to function correctly.
clusterSecretName
clusterSecretName is the k8s secret name holding 32-bit hex-encoded(without 0x) cluster secret in a key called secret.
clusterSecretName can be generated using openssl tool:
resources
resources allocates compute and storage resources to the peer.
resources object has the following fields:
Syntax
Type
Description
Defalt
cpu
string
number of cpu cores this peer requires
1
cpuLimit
Memory and storage requests and limits must use the pattern ^[1-9][0-9]*[KMGTPE]i$ for example 1500Mi, 30Gi, and 1Ti.
cpu requests and limits must use the pattern ^[1-9][0-9]*m?$ for example 1000m (which is equal to 1 core), 1500m which is 1.5 core, 2 cores, and 4 cores.
cpuLimit can't be less than cpu.
memoryLimit can't be less than or equal to memory.
Generating a 2048 bit ed25519 key...
Success!
ID for generated key: 12D3KooWT2bqgwZPxHthAGBV9Ut8ZLraz1LARtB7vG3mF26Mtof1
CAESQLepaunFGa/PI0oNS3plrbVSInqab/X/U1laEDe2V2cL/7kbF9H6x3xBiwpbVnYT/jDA8EhAznXALlbwzEsuKaw=
ipfs-key -type ed25519 | base64
Generating a 2048 bit ed25519 key...
Success!
ID for generated key: 12D3KooWT2bqgwZPxHthAGBV9Ut8ZLraz1LARtB7vG3mF26Mtof1
CAESQLepaunFGa/PI0oNS3plrbVSInqab/X/U1laEDe2V2cL/7kbF9H6x3xBiwpbVnYT/jDA8EhAznXALlbwzEsuKaw=
In this tutorial we will use the following private keys and their corresponding addresses.
DON'T use these keys in production.
Address
Private Key
Let's describe an Ethereum Node that uses a custom genesis block to join a private Proof of Work network using Ethash consensus configuration in the genesis block.
In this node, we're using Hyperledger besu client client: besu, enabling JSON-RPC server rpc: true so we can query number of peers later in this tutorial, turning on Proof of Work mining miner: true, setting the address that will collect block reward using coinbase, and we're loading the node private key from Kubernetes secretd called besu-ethash-nodekey, this is the private from which the coinbase address is derived.
We're defining a genesis block that uses the value 4444 as network and chain identifier, and used ethash proof of work consensus engine.
ethash is required in proof of work networks, even if it's empty like in our example: ethash: {}.
This node private key secret can be created by:
Private key must not start with 0x, and must be stored in secret data field called key.
Let's deploy the node:
Kotal operator will notice your besu-ethash-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets.
You can fetch the deployed Ethereum Node using:
It will return an output similar to the following:
Fetch Node Logs
Get the pods that has been created by Kotal for the node:
It will return an output similar to the following:
Get the logs of the running node:
Deploy a Second Node
Let's deploy another go-ethereum node, and connect it to the previous node in our private proof of work network.
Genesis block must be the same in both nodes, or they will fork at genesis block, and won't reach consensus.
In this node, we're using go-ethereum client client: geth, turning on Proof of Work mining miner: true, setting the address that will collect block reward using coinbase, and loading the miner account private key and password from kubernetes secrets privateKeySecretName: ... and passwordSecretName: .... We're connecting to the first node using staticNodes option which accepts Node name or enode url.
staticNodes accept Node name or enode URL. Node name has the format of name.namespace, namespace is optional if Node is in the same namespace. If the node doesn't exist, or is not up and running yet, Kotal will not raise an error.
You can create the private key and password secrets using:
Deploy the second node using:
Kotal operator will notice your second geth-ethash-node and will create all the necessary pods, persistent volumes, services, configmaps, and secrets.
You can fetch the deployed Ethereum Nodes using:
It will return an output similar to the following:
Call JSON-RPC Method net_peerCount
Get the pods that has been created by Kotal for the node:
It will return an output similar to the following:
Forward localhost:8545 calls to the node pod:
In another terminal window call net_peerCount JSON-RPC method
You will get JSON result similar to the following:
Homework
Deploy a third node that uses Nethermind client, and use the third private key and address from the table above. Nethermind client is similar to geth, you will import signer account private key and password from kubernetes secrets, and use the same genesis as the other nodes.
Finally you can delete all the nodes by:
Kubernetes garbage collector will delete all the resources that has been created by Kotal Ethereum Node controller.
network is Ethereum 2.0 network to join and sync its beacon chain.
client
client is the Ethereum 2.0 client to use.
client possible values are teku,prysm,lighthouse, and nimbus.
eth1Endpoints
eth1Endpoints is array of Ethereum 1 JSON RPC endpoints.
prysm, teku, and lighthouse clients support multiple endpoints in eth1Endpoints.
nimbus client supports only 1 endpoint in eth1Endpoints.
hosts
hosts is a list of host names to whitelist for RPC access (server enforced).
corsDomains
corsDomains is a list of domains from which to accept cross-origin requests (browser enforced).
rest
rest enables REST API server.
REST API server is only supported by teku and lighthouse.
restHost
restHost is the REST API server host.
restPort
restPort is the REST API server port.
rpc
rpc enables JSON RPC server.
JSON RPC server is only supported by nimbus and prysm.
rpcHost
rpcHost is the JSON RPC server host.
rpcPort
rpcPort is the JSON RPC server port.
grpc
grpc enables GRPC gateway server.
GRPC gateway is only supported by prysm client.
grpcHost
grpcHost is the GRPC gateway server host.
grpcPort
grpcPort is the GRPC gateway server port.
p2pPort
p2pPort is the p2p and discovery port.
certSecretName
certSecretName is k8s secret name that holds TLS private key in data field called tls.key and TLS certificate in data field called tls.crt.
certSecretName is supported only by prysm client.
logging
logging is logging verbosity level.
Different levels are supported by different Ethereum 2.0 beacon node clients.
Logging/Client
Teku
Lighthouse
Prysm
Nimbus
resources
resources allocates compute and storage resources to the node.
resources object has the following fields:
Syntax
Type
Description
Defalt
memory and storage requests and limits must use the pattern ^[1-9][0-9]*[KMGTPE]i$ for example 1500Mi, 30Gi, and 1Ti.
cpu requests and limits must use the pattern ^[1-9][0-9]*m?$ for example 1000m (which is equal to 1 core), 1500m which is 1.5 core, 2 cores, and 4 cores.
cpuLimit can't be less than cpu.
memoryLimit can't be less than or equal to memory.
storageClass field is immutable, it cannot be changed after creation.
Validator
Syntax
Type
Description
Default
network
network is the Network to validate blocks for.
client
client is the Ethereum 2.0 client to use.
beaconEndpoints
beaconEndpoints is a list of beacon node endpoints.
lighthouse is the only client that supports multiple endpoints. All other clients supports only a single endpoint.
certSecretName
certSecretName is k8s secret name that holds TLS certificate in data field called tls.crt.
certSecretName is supported only by prysm client.
graffiti
graffiti is the text to include in proposed blocks.
walletPasswordSecretName
walletPasswordSecretName is prysm wallet password kubernetes secret.
walletPasswordSecretName is supported only by prysm client.
Wallet password secret must be deployed before deploying the validator.
Wallet password secret must contain the password in key named password.
keystores
keystores is array of keystore objects.
Syntax
Type
Description
Validator keystore public key is required in case of client: lighthouse
Keystore secret must be deployed before deploying the validator.
Keystore secret must contain the BLS12-381 keystore JSON file in data field key called keystore key, and password in password key.
logging
logging is logging verbosity level.
Different levels are supported by different Ethereum 2.0 beacon node clients.
Teku validator client doesn't support logging. Verbosity level will be ignored 🙈
Logging/Client
Lighthouse
Prysm
Nimbus
resources
resources allocates compute and storage resources to the node.
resources object has the following fields:
Syntax
Type
Description
Defalt
memory and storage requests and limits must use the pattern ^[1-9][0-9]*[KMGTPE]i$ for example 1500Mi, 30Gi, and 1Ti.
cpu requests and limits must use the pattern ^[1-9][0-9]*m?$ for example 1000m (which is equal to 1 core), 1500m which is 1.5 core, 2 cores, and 4 cores.
cpuLimit can't be less than cpu.
memoryLimit can't be less than or equal to memory.
fatal
✔️
❌
✔️
✔️
none
❌
❌
❌
✔️
notice
❌
❌
❌
✔️
error
✔️
✔️
✔️
✔️
warn
✔️
✔️
✔️
✔️
info
✔️
✔️
✔️
✔️
debug
✔️
✔️
✔️
✔️
trace
✔️
❌
✔️
✔️
all
✔️
❌
❌
❌
critical
❌
✔️
❌
❌
panic
❌
❌
✔️
❌
number of cpu cores this node is limited to
8
memory
string
memory this node requires
8Gi
memoryLimit
string
memory this node is limited to
16Gi
storage
string
disk space this node requires
200Gi
storageClass
string
Node volume storage class
Cluster's default storage class will be used as defined by cluster admin or cloud provider
network is the public network name to join, like rinkeby.
network can't be provided in private networks, where .spec.genesis is not null.
network can't be updated (immutable).
highly available
highlyAvailable controls if Ethereum nodes will be scheduled on different Kubernetes nodes.
topology key
topologyKey is Kubernetes node label key used to distribute ethereum nodes pods on different kubernetes nodes.
bootnodes
bootnodes is ethereum node URLs for p2p network discovery and bootstraping.
bootnodes accepts ethereum node URL enodeURL or a reference to kotal Node in the form of name.namespace where namespace is optional if referenced node is in the same namespace.
Ethereum clients have hardcoded bootnodes for public main and test networks. bootnodes will override these bootnodes.
client
Support for OpenEthereum (parity) client has been deprecated. OpenEthereum team is working with Erigon on a smooth transition path for users. More information can be found
client is the Ethereum client name powering the node.
client possible values are besu, geth, nethermind.
Only besu client can be used if network consensus is ibft2.
Only besu can be used in fixed difficulty proof of work networks, where spec.genesis.ethash.fixedDifficulty is not null.
coinbase
coinbase is ethereum account to which mining rewards are paid.
coinbase is required if the node is mining miner: true.
corsDomains
corsDomains is a list of domains from which to accept cross-origin requests (browser enforced).
Default value * will be used if HTTP RPC server is enabled rpc: true or web socket server is enabled ws: true or graphQL server is enabled graphql: true.
graphql
graphql enables the GraphQL server.
Nethermind client doesn't support GraphQL.
Geth (Go-Ethereum) GraphQL server can be used only if RPC is enabled as of geth
graphqlPort
graphqlPort GraphQL server listening port.
The default value 8547 will be used if the graphQL server is enabled graphql: true.
hosts
hosts is a list of host names to whitelist for RPC access (server enforced).
import
import is the ethereum account to import. Only for nodes running with geth or nethermind.
During account creation, it will be encrypted with the password, and during import it will be unlocked using the same password.
Nodes that import accounts can't enable HTTP RPC server, web socket server, or GraphQL server to prevent funds drainage if exposed to the internet.
Account must be imported if the node is running with geth or nethermind clients wants to be a signer or miner.
Syntax
Type
Description
logging
logging is Ethereum node logging verbosity level.
logging possible values are off, fatal, error, warn, debug, info, trace and all.
Different clients support different logging vrbosity levels as shown in the following table:
Logging/Client
Hyperledger Besu
Go Ethereum
Nethermind
miner
miner enables node mining or signing blocks.
nodePrivateKeySecretName
nodePrivateKeySecretName is the node private key.
nodePrivateKeySecretName is required if the node is a boot node bootnode: true, or if besu node with client: besu is a signer in proof of authority clique network or validator in ibft2 network.
p2pPort
p2pPort is node p2p port for communication (TCP) and discovery (UDP).
resources
resources allocates compute and storage resources to the node.
Syntax
Type
Description
Default
memory and storage requests and limits must use the pattern ^[1-9][0-9]*[KMGTPE]i$ for example 1500Mi, 30Gi, and 1Ti.
cpu requests and limits must use the pattern ^[1-9][0-9]*m?$ for example 1000m (which is equal to 1), 1500m, 2, and 4.
cpuLimit can't be less than cpu.
memoryLimit can't be less than or equal to memory.
storageClass field is immutable, it cannot be changed after creation.
rpc
rpc enables the HTTP RPC server.
rpcPort
rpcPort is the HTTP RPC server listening port.
Default value 8545 will be used if the HTTP RPC server is enabled with rpc: true.
rpcAPI
rpcAPI is a list of RPC services to enable.
Default value ["web3", "eth", "net] will be used if HTTP RPC server is enabeld with rpc: true.
staticNodes
staticNodes is a set of trusted ethereum nodes to maintain connection to.
Static nodes are exempt from maximum peer and remote connection limits. Ethereum clients periodically initiating a connection to any unconnected static node.
staticNodes accepts ethereum node URL enodeURL or a reference to kotal Node in the form of name.namespace where namespace is optional if referenced node is in the same namespace.
syncMode
Nethermind client which was activated by syncMode: light.
syncMode is Blockchain synchronization mode.
syncMode possible values are light, full, snap or fast.
light and snap sync modes are only supported by go-ethereum client.
ws
ws enables the web socket server.
wsPort
wsPort is the web socket server listening port.
The default value 8546 is used if the web socket server is enabled with ws: true.
wsAPI
wsAPI is a list of rpc services to enable.
the default value ["web3", "eth", "net] will be used if the web socket server is enabeld with ws: true.
genesis
genesis is the genesis block configuration.
Genesis block configuration genesis is required in private networks.
Syntax
Type
Description
Default
chainId
chainId is the chain id value used in transaction signature to prevent transactions reply
chainId is required.
chainId can't reuse existing public network chain id to avoid transaction replay.
Network
Chain id
networkId
id is the network id used for p2p communications between network nodes in private networks.
id is required in private networks.
id can't be provided while joining a public network.
id can't be updated (immutable).
coinbase
coinbase is the beneficiary (ethereum address) of mining reward.
coinbase is optional.
difficulty
difficulty is the difficulty of the genesis block.
difficulty is optional.
mixHash
mixHash is combined with the nonce to prove the effort spent to create a block.
mixHash is optional.
gasLimit
gasLimit is the total gas limit for all transactions in a block.
gasLimit is optional.
nonce
nonce is a random number used in block computation.
nonce is optional.
timestamp
timestamp is block creation date.
timestamp is optional.
forks
forks is an object, where the key is fork name and the value is the block number at which to activate this fork.
forks is optional.
if forks is missing, all forks will be activated at block 0 (genesis block) except DAO.
Later forks like muirglacier can't be activated before earlier forks like homestead. They must be ordered as in the following table:
Syntax
Type
Description
accounts
accounts is an array of accounts to fund or store code.
accounts is optional.
accounts is recommended for networks where signers or validators are not rewarded with eth.
a single account has no defaults.
Syntax
Type
Description
ethash
ethash is Proof of Work consensus engine configuration.
ethash is optional.
ethash can be set only in proof of work private networks.
ethash.fixedDifficulty has no default value.
ethash.fixedDifficulty is only supported by Hyperledger Besu Client client: besu
Syntax
Type
Description
Default
clique
clique is Proof of Authority clique consensus engine configuration.
clique is optional.
clique can be set only in proof of authority clique private networks.
At least one signer in clique.signers is required.
Syntax
Type
Description
Default
ibft2
ibft2 is IBFT2 engine configuration.
ibft2 is optional.
ibft2 can be set only in IBFT2 private networks.
At least one validator in ibft2.validators is required.
Syntax
Type
Description
Default
❌
❌
error
✔️
✔️
✔️
warn
✔️
✔️
✔️
debug
✔️
✔️
✔️
info
✔️
✔️
✔️
trace
✔️
❌
✔️
all
✔️
✔️
❌
number of cpu cores this node is limited to
3 in private, 6 in public
memory
string
memory this node requires
4Gi in private, 8Gi in public
memoryLimit
string
memory this node is limited to
8Gi in private, 16Gi in public
storage
string
disk space this node requires
100Gi in private, 6Ti in mainnet with full sync, 750Gi in mainnet wit fast sync, 25Gi in public test networks
storageClass
string
Node volume storage class
Cluster's default storage class will be used as defined by cluster admin or cloud provider
used in network p2p communications
string
benefeciary (ethereum address) of mining reward
address(0)
string
difficulty (hexadecimal number) of the genesis block
0x1
string
hash (hexadecimal) combined with nonce to prove effort spent to create block
0x00..00
string
total gas limit (hexadecimal number) for all transactions in a block
0x47b760
string
random hexadecimal number used in block computation
0x0
string
genesis block creation date (hexadecimal)
0x0
Array
array of accounts to prefund and store code
object
supported forks and corresponding block number
object
Proof of Work consensus configuration
object
Proof of Authority consensus configuration
object
IBFT2 consensus configuration
5
kotti
6
ethereum classic
61
Mordor
63
development
2018
eip150
number
fork activation block number
eip155
number
fork activation block number
eip158
number
fork activation block number
byzantium
number
fork activation block number
constantinople
number
fork activation block number
petersburg
number
fork activation block number
istanbul
number
fork activation block number
muirglacier
number
fork activation block number
berlin
number
fork activation block number
london
number
fork activation block number
arrowGlacier
number
fork activation block number
code
string
bytecode in hexadecimal
storage
map
key is the storage location in hexadecimal, and value in hexadecimal is the storage value
number of blocks after which to reset all votes
1000
signers
Array
array of ethereum addresses
number of blocks after which to reset all votes
1000
validators
array
array of ethereum addresses
requestTimeout
number
timeout for each consensus round in seconds
10
messageQueueLimit
number
message queue limit
1000
duplicateMessageLimit
number
duplicate messages limit
100
futureMessagesLimit
number
future messages buffer limit
1000
futureMessagesMaxDistance
number
maximum height from current chain height for buffering future messages