gton

gton is a Go implementation of a TON full node with a liteserver API. Its validator subsystem is under development and does not yet provide a complete production validation pipeline. The node is designed to be an efficient API access point for services, backends of projects, indexers, wallets, and other infrastructure that needs fast synchronization and stable data serving under heavy load.

The project focuses on:

• Fast block sync and live updates;
• Memory efficiency
• Liteserver optimized for live data availability, with smart cache;
• LSM Pebble-based storage with a sharded cell DB
• Runtime metrics for sync, storage, p2p, and liteserver observability.

The project is under active development. Storage format and configuration may change without backward compatibility.

Telegram group: @gtonnode

• Minimum RAM: 48 GB.
• Recommended RAM: 64 GB.
• Minimum CPU: 8 cores.
• Minimum disk: 512 GB SSD.

Build the node binary:

go build -o gton-node ./cmd/node

Or download from releases.

The first run creates config.json and exits:

./gton-node

Review the generated config, enable the liteserver and metrics if needed, then start the node again:

If global.config.json is missing, the node downloads it automatically. To replace the global config before startup:

./gton-node --global-config https://ton-blockchain.github.io/global.config.json

The generated config defaults to:

• ADNL: 0.0.0.0:30303;
• DHT: 0.0.0.0:30304;
• storage directory: data;
• liteserver: disabled, non-final mode disabled, listen address 0.0.0.0:7445.

Open the external ADNL/DHT (UDP) ports on your firewall and make sure adnl.external_addr points to the node's public address. If you enable the liteserver for external clients, open its port too (TCP).

After startup node will sync with the latest blockchain state. First it will download state and then blocks to apply them. This could take around 1 - 3 hours depending on your hardware and chain.

The main binary is ./cmd/node.

./gton-node [flags]

Supported flags:

Build the standalone genesis utility:

go build -o gton-genesis ./cmd/genesis

The first run without arguments creates genesis.json with standard testnet settings and placeholders for three validators and one bootstrap DHT node. It does not create or modify any node config:

./gton-genesis

Validator signing keys are created later through validator control and are stored in the validator database, not in config.json. For a pre-seeded test genesis, use the keys produced by the genesis workflow; use the node commands for the network identities:

./gton-node --config node-0.json --adnl-id
./gton-node --config node-0.json --dht-descriptor

Before exporting the DHT descriptor, set adnl.external_addr to an address reachable by the other nodes. Once all placeholders have been replaced, the second run creates a complete Pebble node database, the global config, and a resolved lock file:

./gton-genesis

All paths have defaults and can be overridden independently:

./gton-genesis \
  --genesis ./genesis.json \
  --data ./data \
  --global-config ./global.config.json \
  --lock ./genesis.lock.json

Start the seeded node with its own config and the generated artifacts:

./gton-node \
  --config node-0.json \
  --data-dir ./data \
  --global-config-file ./global.config.json

Distribute the same global.config.json to every initial node. The closed generated data directory may also be copied to all three nodes before startup; it contains no node identity or private key. Alternatively, start one seeded node first and let the other nodes download the declared zerostates through the normal protocol. genesis.lock.json records the exact block hashes, validator key IDs, and bounceable testnet addresses for operator verification.

Use this when the node must receive broadcasts from a private/custom overlay and optionally expose pending non-final shard blocks through its liteserver.

Custom overlays are configured directly in config.json. The local node must be listed in nodes; otherwise it will ignore the overlay. Set block_sender=true if this node should send block broadcasts to the overlay, and msg_sender=true if it should send external messages.

Minimal custom_overlays example:

{
  "custom_overlays": [{
    "name": "private-a",
    "nodes": [{
      "adnl_id": "<base64 32-byte ADNL id>",
      "msg_sender": true,
      "msg_sender_priority": 0,
      "block_sender": true
    }],
    "sender_shards": [],
    "skip_public_msg_send": false
  }]
}

sender_shards may be empty to send all shards. To limit sending to a shard, add entries with workchain and shard:

"sender_shards": [{
  "workchain": 0,
  "shard": -9223372036854775808
}]

skip_public_msg_send skips public-overlay external message broadcasting only when the message is actually sent through a matching custom overlay.

Set liteserver.non_final_enabled=true in the node config and restart:

{
  "liteserver": {
    "enabled": true,
    "non_final_enabled": true
  }
}

Non-final data is kept in memory only. It is not applied to persistent state or written to storage, and it is dropped when the corresponding final shard blocks arrive or cache limits are reached. Supported liteserver methods use it transparently when the request references a pending non-final block; regular final-block requests do not touch the non-final cache. getValidatorGroups is not supported for non-final blocks.

After startup, the process reads commands from stdin. This is useful for manual diagnostics and maintenance without a separate RPC control interface.

Example:

status
status db
status validator
serialize 48500000
migrate 48500000

After startup, the node runs a background maintenance routine. It keeps stored data useful for serving requests while controlling disk usage. Only one heavy maintenance job runs at a time, so the node does not run several large cleanup or rebuild tasks at once.

The routine can run these jobs:

Creates snapshot files with the full chain state at selected masterchain key blocks. These files let the node serve state downloads to other nodes and provide a stable point for later storage maintenance.

Builds a fresh cell database from a serialized state snapshot, catches it up to the current chain state, and then switches to it. This keeps the state database compact and lets the node remove old state data after it is no longer needed.

After a successful full serialization, immediately removes snapshot groups beyond persistent_state_keep_recent. Background cleanup also removes expired leftovers. A snapshot still needed by an unfinished migration is kept temporarily even when it exceeds the configured limit.

Removes old archive packages and related block metadata after the configured archive retention period. This keeps archive storage inside the configured history window instead of growing forever.

Use archival liteserver mode when the node should start from the zero state, import archive data forward, keep archive packages forever, and serve liteserver requests over the full stored history.

Enable it by setting ton.sync_before=-1 and keeping both storage retention knobs disabled:

{
  "ton": {
    "sync_before": -1,
    "state_ttl": 0,
    "archive_ttl": 0
  },
  "liteserver": {
    "enabled": true
  }
}

In this mode startup rejects non-zero state_ttl or archive_ttl. state_ttl=0 keeps the current-state cell database generation, and archive_ttl=0 disables archive package pruning. Disk usage grows with the archived history, so size the storage volume for long-term retention before enabling this mode.

The config is a JSON file. If it does not exist, the node generates a new file with ADNL/DHT/liteserver keys and default values, then exits so you can review it.

Simplified example:

{
  "ton": {
    "global_config_path": "global.config.json",
    "sync_before": 14400,
    "sync_until": 0,
    "state_ttl": 172800,
    "archive_ttl": 604800,
    "next_checkpoint_blocks": 200,
    "archive_checkpoint_blocks": 2000,
    "checkpoint_bytes": 268435456,
    "sync_backpressure_windows": 4
  },
  "adnl": {
    "key": "<base64 ed25519 seed>",
    "listen_addr": "0.0.0.0:30303",
    "external_addr": "203.0.113.10:30303"
  },
  "dht": {
    "key": "<base64 ed25519 seed>",
    "listen_addr": "0.0.0.0:30304"
  },
  "liteserver": {
    "enabled": true,
    "non_final_enabled": false,
    "key": "<base64 ed25519 seed>",
    "listen_addr": "0.0.0.0:7445",
    "master_block_cache": 128,
    "shard_block_cache": 1024,
    "allow_duplicate_externals": false,
    "send_message_broadcast_bytes_per_second": 0,
    "send_message_broadcast_max_delay_ms": 100,
    "send_message_broadcast_fanout": 5
  },
  "storage": {
    "dir": "data",
    "cell_total_cache_size": 8589934592,
    "decoded_cell_cache_enabled": true,
    "decoded_cell_cache_shards": 64,
    "decoded_cell_cache_entries": 131072,
    "cell_record_cache_bytes": 4294967296,
    "cell_shard_memtable_size": 268435456,
    "cell_memtable_stop_writes_threshold": 4,
    "large_boc_shard_read_workers": 2,
    "persistent_state_large_boc_batch_size": 524288,
    "persistent_state_keep_recent": 1,
    "state_serialize_one_pass": false,
    "artifact_file_max_open": 512
  },
  "metrics": {
    "enabled": true,
    "listen_addr": "127.0.0.1:9090",
    "namespace": "gton"
  },
  "validator": {
    "enabled": false,
    "control": {
      "listen_addr": "127.0.0.1:3030",
      "key": "<base64 ed25519 seed>",
      "clients": [
        {
          "id": "<base64 client short key id>",
          "permissions": 15
        }
      ]
    }
  },
  "custom_overlays": [],
  "disable_state_serialization": false
}

Dedicated network shared by the validator and standalone collator. Generated configs always include this block with enabled: false, a random independent key, and the detected public IP. With enabled: false or an omitted block, both roles use adnl.key and the ordinary P2P listener. Old configs without the block remain unchanged; an omitted enabled also means false.

"consensus_adnl": {
  "enabled": false,
  "key": "<base64-encoded 32-byte Ed25519 seed>",
  "listen_addr": "0.0.0.0:30305",
  "external_addr": "203.0.113.10:30305"
}

Set enabled: true to use the dedicated network. Its key and both addresses are required and validated only when enabled. The key must differ from the ordinary ADNL and DHT keys. The dedicated network has its own ADNL/RLDP connections, consensus overlays and identity-bound FastSync overlays; QUIC listens on the ADNL port plus 1000 (31305 in this example). Both dedicated ports use UDP and must be reachable at the advertised address. With NAT, forward both ports; the advertised QUIC port is also the advertised ADNL port plus 1000. Collisions with the ordinary ADNL, QUIC and DHT listeners or advertised endpoints are rejected.

Public/custom overlays continue through the ordinary network. FastSync block synchronization, publication and membership certificates use the dedicated identity, matching the validator/collator registration. Both transports share the process DHT, blockchain state and broadcast processing; no second synchronizer is started. Full-node queries on the dedicated listener require FastSync membership. Changes take effect on restart.

Use ./gton-node --config config.json --consensus-adnl-id for validator elections and collator registration. --adnl-id continues to print the ordinary node identity. The validator-control getConfig and addValidatorAdnlAddress use the selected consensus identity.

To move an existing validator to the dedicated port while preserving its identity, move its existing adnl.key seed into consensus_adnl.key and assign a new seed to adnl.key, then set consensus_adnl.enabled: true. Update memberships and certificates tied to the ordinary node identity as needed. This preserves validator key bindings and session storage; the feature changes no database format. A conflicting, unexpired validator key binding causes startup to fail with the affected IDs. Choosing a new consensus identity requires the corresponding election or collator registration; editing JSON does not rewrite existing key bindings.

Sizing the two cell caches. cell_total_cache_size and decoded_cell_cache_entries are independent knobs over different tiers, and raising one does not move the other. cell_total_cache_size is pebble's block cache: compressed blocks in one off-heap arena, so its cost is bytes and nothing else, and below it the OS page cache holds the rest of celldb for free. That is where bulk capacity belongs — raise it freely on a large machine.

decoded_cell_cache_entries bounds fully decoded *cell.Cell trees on the Go heap. Each entry is roughly 10 live objects and ~820 B, and every GC mark cycle scans all of them, so its cost is paid per collection rather than once. Mark cost tracks the object COUNT, which is why the knob is an entry count and not a byte budget, and why the default is small relative to the pebble cache. Raising it by a factor of eight raises steady-state mark work by about the same factor. Earlier releases derived this capacity from cell_total_cache_size; that coupling is gone, so a node that raises the pebble cache no longer silently multiplies its live object count.

For the same reason the knob has a hard ceiling of 1048576 entries, enforced when the config is loaded rather than clamped silently: a value above it fails startup with the reason. That ceiling is about 10 M live objects and ~820 MiB, and it is the same bound the removed derivation was clamped to — when the derivation went away the clamp went with it, which left the knob unbounded above and let a config put back exactly the object count the resize had removed. The deprecated alias service_decoded_cell_cache_entries is checked against the same ceiling when it is the value that wins.

There is one decoded cell cache for the whole process — the lightserver, archive import, sync, collation and validation all share it. It is not splittable per consumer: the collator and the validator must hold the same *cell.Cell for a given parent, because the validator's live-successor carry-back compares tip states by pointer, and two caches cannot both supply one object.

The control server implements the MTC election path: getStats, getConfig, generateKeyPair, addValidatorPermanentKey, addValidatorTempKey, exportPublicKey, addValidatorAdnlAddress, and sign. Generated validator keys and their election metadata are persisted under storage.dir/validator; the trusted-client allowlist and authenticated connections remain in memory.

Unknown JSON fields are ignored throughout the node config, including @type keys in overlay definitions copied from the C++ node.

List of private overlay definitions. Empty by default. Each overlay has:

Each nodes entry has:

When true, automatic persistent state serialization is disabled. Manual serialization through the serialize console command is still a separate service operation.

To enable the Prometheus endpoint, add:

{
  "metrics": {
    "enabled": true,
    "listen_addr": "127.0.0.1:9090",
    "namespace": "gton"
  }
}

Metrics are exposed at:

http://127.0.0.1:9090/metrics

The exported metrics cover liteserver latency, sync lag, block download/apply, checkpoint persistence, p2p queues, rebroadcasting, blocksync, and Pebble/cell DB status. The full metric list and PromQL examples are documented in METRICS.md. A Grafana dashboard is available in metrics.json.

The node can call a statically linked extension from a custom node binary.

The extension is compiled into the custom binary, so node builds do not need cgo.

For a standard node build with an additional extension, use node.Run as the main entry point:

package main

import (
    "github.com/xssnick/gton/cmd/node/node"
    "my/project/extension"
)

func main() {
    node.Run(extension.New)
}

node.Run uses the same CLI and config bootstrap as the standard ./cmd/node binary. It reads config.json (or the path passed through --config), starts the configured liteserver and HTTP API as root-owned servers, and separately composes the supplied extension factories. Multiple factories can be passed to node.Run.

gton.RunNode is the lower-level entry point for a binary that owns its full startup flow. It accepts already resolved gton.NodeOptions; it does not read config.json or configure the built-in API servers automatically. Loading a config with Config.RuntimeOptions and passing only RuntimeOptions.Node therefore does not enable the liteserver or HTTP API. A custom bootstrap must configure NodeOptions.Liteserver and NodeOptions.HTTPAPI explicitly.

For example, a fully custom binary can parse its own CLI and config files, then pass typed startup values to gton.RunNode:

package main

import (
    "context"
    "flag"
    "os"

    "github.com/xssnick/gton"
    nodeconfig "github.com/xssnick/gton/cmd/node/config"
    "github.com/rs/zerolog"
    "github.com/xssnick/tonutils-go/liteclient"
    "my/project/extension"
)

func main() {
    flags := flag.NewFlagSet("my-node", flag.ExitOnError)
    configPath := flags.String("config", nodeconfig.DefaultPath, "path to node config JSON")
    flags.Parse(os.Args[1:])

    cfg, err := nodeconfig.Load(*configPath)
    if err != nil {
        panic(err)
    }

    ctx := context.Background()
    runtimeOpts, err := cfg.RuntimeOptions(gton.DefaultNodeOptions())
    if err != nil {
        panic(err)
    }

    globalConfigPath := runtimeOpts.GlobalConfigPath
    if _, err = nodeconfig.EnsureGlobalConfig(ctx, globalConfigPath, nodeconfig.DefaultGlobalConfigURL, false); err != nil {
        panic(err)
    }
    globalConfig, err := liteclient.GetConfigFromFile(globalConfigPath)
    if err != nil {
        panic(err)
    }

    opts := runtimeOpts.Node
    opts.GlobalConfig = globalConfig
    opts.Logger = zerolog.New(os.Stdout).Level(zerolog.InfoLevel).With().Timestamp().Logger()
    opts.Extension = extension.New

    err = gton.RunNode(ctx, opts)
    if err != nil {
        panic(err)
    }
}

Custom binaries can load the node config themselves and keep extension config in a separate file:

nodeCfg, err := nodeconfig.Load("node.json")
if err != nil {
    panic(err)
}
runtimeOpts, err := nodeCfg.RuntimeOptions(gton.DefaultNodeOptions())
if err != nil {
    panic(err)
}
globalConfig, err := liteclient.GetConfigFromFile(runtimeOpts.GlobalConfigPath)
if err != nil {
    panic(err)
}
extensionCfg, err := extension.LoadConfig("extension.json")
if err != nil {
    panic(err)
}

opts := runtimeOpts.Node
opts.GlobalConfig = globalConfig
opts.Logger = zerolog.New(os.Stdout).Level(zerolog.InfoLevel).With().Timestamp().Logger()
opts.Extension = extension.NewFactory(extensionCfg)

err = gton.RunNode(context.Background(), opts)

See example_extension/ for a complete static apply logger and live transaction replay checker example. The example is a separate Go module with a local replace to the repository root, so it does not become part of the main module's ./... package set.

Optional hooks adapters for the API servers live in extensions/httpapi and extensions/liteserver. They were moved out of api/httpapi and api/liteserver so core API packages never import the extension SDK. Custom binaries using the old api/*/NewExtension entry points must switch those two imports to extensions/*; no forwarding shim is kept in core.

The extension factory must have this signature:

func New(node hooks.Node) (hooks.Extension, error)

hooks.Node gives the extension a small capability surface. Network can send external messages and submit an already decoded local block to the normal node pipeline. PrivateOverlays creates extension-owned fixed-membership overlays; it exposes authenticated message, query, and broadcast transport without depending on validator types. BlockBroadcasts accepts validator-produced candidate and accepted-block artifacts for node-owned publication. Store is a read-only live view backed by the same store/cache layer used by the liteserver, while TVM, Commands, Metrics, and Logger expose their corresponding runtime capabilities. None of these APIs exposes block download control or direct storage mutation.

The returned value receives hook events through:

type Extension interface {
    Start(context.Context) error
    Close(context.Context) error
    OnBlockApplied(context.Context, BlockAppliedEvent) error
    OnExternalMessage(context.Context, ExternalMessageEvent) error
    OnBlockReceived(context.Context, BlockReceivedEvent) error
}

OnBlockApplied is called after a block state update is applied and before that block flow can continue to checkpoint or persist. Block apply hook delivery is at least once: the same block apply call may be repeated if the node crashes or is hard-stopped before the block flow is fully persisted/checkpointed. If the method returns an error, the node retries the same event after a short delay and does not advance that block flow until the extension returns nil. This retry only blocks the affected block dependency branch; shard block apply is parallel, so other independent branches may continue and may call the same extension concurrently.

Start is called after the P2P node is ready and before the sync coordinator and state/maintenance workers start. P2P callbacks can begin as the node becomes ready, so factories must initialize all hook-visible state eagerly; Start should only launch background work. During shutdown the node stops and joins all ordinary hook delivery before calling Close; no new hook can begin and every already admitted hook has returned. P2P remains available during Close so an extension can deterministically retire private overlays, then the node stops the network. Close should block until the extension exits or the context is done. Extensions with no background work should return nil.

OnExternalMessage is called after an external message is accepted by TVM emulation and before it is rebroadcast further. event.IsLocal is true when the message came from this node's local API path and false for overlay broadcasts. If the method returns an error, that external message is dropped without retry.

OnBlockReceived is called when a block is received from broadcasts, downloads, or archive imports. event.IsSigned is true for signed block artifacts and downloaded/imported full blocks, and false for shard block candidates. If the method returns an error, the node logs it and continues the block flow.

Extension implementations must be thread-safe. They must also treat every object received in hook events as borrowed for the duration of the call only. Do not retain event fields, cells, metadata, state pointers, or payload slices after the hook returns. If an extension needs data later, it must copy or serialize the exact data it owns before returning.

BlockAppliedEvent includes block/proof BOC payloads, block root cell, block metadata, applied state roots, and the inclusion masterchain reference/state for shard blocks. The block id is available as event.Meta.ID. InclusionMasterRef and InclusionMasterState are nil for masterchain blocks. The node does not resolve execution masterchain data on the apply hot path. Extensions that need the execution master reference should parse tlb.Block from BlockRoot; state can then be taken from the event roots when it is already present or loaded through the normal read-only store APIs.

BlockReceivedEvent includes the received block/proof BOC payloads, block root cell, block metadata, and IsSigned. The block id is available as event.Meta.ID.

A minimal extension looks like:

package extension

import (
    "context"

    "github.com/xssnick/gton/service/hooks"
)

type extension struct{}

func New(node hooks.Node) (hooks.Extension, error) {
    return extension{}, nil
}

func (extension) Start(ctx context.Context) error {
    return nil
}

func (extension) Close(ctx context.Context) error {
    return nil
}

func (extension) OnBlockApplied(ctx context.Context, event hooks.BlockAppliedEvent) error {
    return nil
}

func (extension) OnExternalMessage(ctx context.Context, event hooks.ExternalMessageEvent) error {
    return nil
}

func (extension) OnBlockReceived(ctx context.Context, event hooks.BlockReceivedEvent) error {
    return nil
}

Build a custom node binary from the wrapper package:

cd example_extension
CGO_ENABLED=0 go build -o gton-node-with-extensions .