Introduction

• About Psibase

Linux Installation

This downloads and extracts both wasi-sdk and psidk. wasi-sdk provides clang and other tools and provides the C and C++ runtime libraries built for WASM. psidk provides libraries and tools for working with psibase. The set of additional packages you need varies with Linux distribution; see the sections below.

For convenience, consider adding the environment variables below to ~/.bashrc or whatever is appropriate for the shell you use.

If you're using docker, use the -p8080:8080 option to expose psibase's http port.

export WASI_SDK_PREFIX=~/work/wasi-sdk-14.0
export PSIDK_PREFIX=~/work/psidk
export PATH=$PSIDK_PREFIX/bin:$PATH

mkdir -p ~/work
cd ~/work
wget https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-14/wasi-sdk-14.0-linux.tar.gz
tar xf wasi-sdk-14.0-linux.tar.gz

cd ~/work
wget https://github.com/gofractally/psibase/releases/download/pre-0.1.1/psidk-linux.tar.gz
tar xf psidk-linux.tar.gz

Ubuntu 20.04 and 22.04

Run these as root:

apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -yq cmake wget binaryen

Ubuntu 18.04

Run these as root:

apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -yq apt-transport-https ca-certificates gnupg software-properties-common wget
wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | apt-key add -
apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'
DEBIAN_FRONTEND=noninteractive apt-get install -yq cmake

cd /usr/local
wget https://github.com/WebAssembly/binaryen/releases/download/version_109/binaryen-version_109-x86_64-linux.tar.gz
tar xf binaryen-version_109-x86_64-linux.tar.gz --strip-components=1

Fedora 36

Run this as root:

dnf -y install cmake wget binaryen

Rust Installation

Not Yet Functional

This documents some of our current thoughts and ongoing development for Rust service support. The installation procedure and examples are currently non-functional.

Installation

To get started with Rust service development, install the following:

• Psidk C++ Support. We're using its psibase, psinode, and psitest executables.
• Rust version 1.65.0 or later: see the commands below. If you don't have rustup installed, then follow the rustup installation instructions.
• Rust's wasi support: see the commands below.
• cargo-psibase: see the commands below.

TODO: switch back to release after 1.65 is stable

rustup update
rustup toolchain install beta
rustup default beta
rustup target add wasm32-wasi
cargo install cargo-psibase

Do not use cargo install to fetch psibase, psinode, or psitest; the executables in those packages are just placeholders. If you did, then run cargo uninstall psibase psinode psitest to fix.

You may have to periodically rerun cargo install cargo-psibase to keep up with ongoing development.

Psinode and Psibase

psidk comes with two executables for working with chains:

• psinode runs a chain. It can optionally be a producer or a non-producer node on a chain. It also optionally hosts an http interface which provides RPC services, GraphQL services, and hosts web UIs. On-chain services define most of the http interface.
• psibase is a command-line client for interacting with the chain. It connects to the http interface on a running node.

psinode has an explicit interface; it won't boot a new chain or connect to an existing chain unless you instruct it to. It also won't open any ports you didn't request or store its database at a location you didn't tell it about.

psinode

psinode has the following command-line interface:

psinode [OPTIONS] <DATABASE>

<DATABASE>, which is required, is a path to the psibase database. psinode creates it if it does not already exist.

If you don't give it any other options, psinode will just sit there with nothing to do. There are three important options for creating and running a local test chain:

• -p or --producer tells psinode to produce blocks. It will not start production on an empty chain until you boot the chain (below). Its argument is a name for the producer. psinode will only produce blocks when it is this producer's turn according to consensus. Multiple distinct nodes must not use the same producer name.
• -o or --host tells psinode to host the http interface. Its argument is a domain name which supports virtual hosting. e.g. if it's running on your local machine, use psibase.127.0.0.1.sslip.io. Right now it always hosts on address 0.0.0.0 (TODO). The port defaults to 8080 but can be configured with --port. The http interface also accepts p2p websocket connections from other nodes (see --peer).
• -s or --sign tells psinode a private key with which to sign blocks. It must match the producer's public key. If the producer has no key set, then it may be omitted.

Three more options are important for connecting multiple nodes together in a network:

• --port tells psinode the TCP port for the http interface. The default port is 8080. This option is only useful with -o.
• --peer tells psinode a peer to sync with. The argument should have the form host:port. This argument can appear any number of times.
• --p2p tells psinode to allow external nodes to peer to it over its http interface at /native/p2p.

There is one more option which is useful for local development. Production deployments shouldn't use this:

• --slow stops it from complaining when it is unable to lock memory for its database. It will still attempt to lock memory, but if it fails it will continue to run, but more slowly. If you don't run with --slow and it fails, psinode will give suggestions on how to configure Linux to allow psinode to lock memory.

psinode does not include https hosting; use a reverse proxy to add that when hosting a public node.

Options can also be specified in a configuration file loaded from <DATABASE>/config. If an option is specified on both the command line and the config file, the command line takes precedence.

slow     = yes
producer = prod
host     = 127.0.0.1.sslip.io
port     = 8080

The configuration file also controls logging.

psibase

psibase provides commands for booting a chain, creating accounts, deploying services, and more. Notable options and commands for service development:

• -a or --api tells it which api endpoint to connect to. This defaults to http://psibase.127.0.0.1.sslip.io:8080/.
• boot boots an empty chain; see below
• deploy deploys a service; see Basic Service

Booting a chain

A chain doesn't exist until it's booted. This procedure boots a chain suitable for local development.

Start psinode

psinode -p prod -o psibase.127.0.0.1.sslip.io my_psinode_db --slow

This will:

• Open a database named my_psinode_db in the current directory; it will create it if it does not already exist.
• Host a web UI and an RPC interface at http://psibase.127.0.0.1.sslip.io:8080/.
• Produce blocks once the chain is booted.

Boot the chain

In a separate terminal, while psinode is running, run the following:

psibase boot -p prod

This will create a new chain which has:

• A set of system services suitable for development
• A set of web-based user interfaces suitable for development
• prod as the sole block producer

psibase boot creates system accounts with no authentication, making it easy to manage them. If you intend to make the chain public, use boot's -k or --key option to set the public key for those accounts.

You may now interact with the chain using:

• The web UI at http://psibase.127.0.0.1.sslip.io:8080/
• Additional psibase commands

Connecting to an existing chain

psinode \
    --peer some_domain_or_ip:8080 \
    -o psibase.127.0.0.1.sslip.io \
    my_psinode_db

This will:

• Open a database named my_psinode_db in the current directory; it will create it if it does not already exist.
• Host a web UI and an RPC interface at http://psibase.127.0.0.1.sslip.io:8080/.
• Connect to a peer at some_domain_or_ip:8080. The peer option may be repeated multiple times to connect to multiple peers.

If the database is currently empty, or if the database is on the same chain as the peers, this will grab blocks from the peers and replay them. Any peers must have their --p2p option enabled.

Be kind; please rewind

If you're starting a new node on an existing chain, it's polite to replay from a block file instead of fetching the entire chain from peer nodes.

psinode
    --replay-blocks filename \
    -o psibase.127.0.0.1.sslip.io \
    my_psinode_db

This will:

• Open a database named my_psinode_db in the current directory; it will create it if it does not already exist.
• Host a web UI and an RPC interface at http://psibase.127.0.0.1.sslip.io:8080/.
• Replay the blocks from filename. The RPC interface is available during replay; this allows you to use the UI to monitor progress.

If you combine both the --replay-blocks and the --peer options, then psinode will connect to peers after it has finished replaying from the block file. If you use both the --replay-blocks and the --p2p option, the node will accept incoming p2p requests after it has finished. TODO: allow p2p during replay to allow outgoing blocks; don't allow incoming ones until finished.

If your database isn't empty and is on the same chain that's stored in the block file, then psinode will replay the blocks that it hasn't already processed. This can be used to quickly catch a node up which has been offline a while.

Creating a block file

The --save-blocks file option will save all blocks to a file. You can use this to bootstrap other nodes.

Cloning a node

psinode's database is portable between machines. Copying the database may be faster than replaying from a block file. Be sure to shut down a node before copying its database to prevent corruption.

SIGANY

psinode uses triedent as its database. triedent locks a large amount of space in memory-mapped files, plus psinode reserves a lot of memory for executing many WASMs simultaneously. This makes psinode a tempting target for Linux's out-of-memory (OOM) killer, which strikes suddenly with SIGKILL. Triedent works to make its database robust against this, but not against kernel crashes, filesystem corruption, or power outages. If the OOM killer strikes, psinode's database should survive. If your machine looses power, or you use a remote filesystem, remote block store, or distributed block store, then psinode's database is vulnerable to undetectable corruption. Be especially cautious about using Kubernetes; it has a nasty habit of yanking volumes before the kernel has finished flushing a stopped container's memory-mapped files, causing corruption.

Since it's near-impossible to do SIGKILL coverage testing, we're going with a more aggressive option for now. During beta, psinode doesn't gracefully shutdown for SIGINT or SIGHUP. Instead, these kill psinode as aggressively as SIGKILL does. There may be a delay. This isn't psinode cleaning after itself; this is the kernel saving psinode's memory-mapped files.

Serving https

These instructions cover using nginx and Let's Encrypt to add https support to psinode.

psinode doesn't support https itself. It would create several complications if it did:

• psinode's binary release supports multiple distributions. Unfortunately different distributions have incompatible versions of the OpenSSL library. About the only way to resolve that is to statically link OpenSSL, which makes it hard to keep up with its security fixes.
• Since psinode hosts a variable set of subdomains, https requires wildcard certificates. Let's Encrypt's wildcard certificates require a periodic dance between the https server, the Let's Encrypt API, and DNS entries to confirm ownership of the domain. certbot knows how to do this dance using nginx.

These instructions cover using GoDaddy. Unfortunately every DNS provider needs a different certbot plugin to support wildcard certificates; see this issue. If you're using a different DNS provider, then check the lists at DNS Plugins and Third-party plugins. We're using the dns-godaddy plugin.

These instructions cover using Ubuntu 22.04.

Domain

The rest of these instructions assume you're hosting on my-psinode-domain.com; adjust them to your domain. You need 2 DNS entries (A records): one for the domain and one for the wildcard (*).

Install packages

This assumes you've already followed the Linux Installation guide, including the Ubuntu 22.04 instructions. We're using pip packages instead of snap packages since, as of this writing, certbot-dns-godaddy doesn't function correctly when using the certbot snap package.

Make sure certbot isn't already installed: which certbot. If it is, uninstall it or it will conflict with these instructions.

Run the following as root:

apt-get update
apt-get -y install nginx python3 python3-venv libaugeas0
python3 -m venv /opt/certbot/
/opt/certbot/bin/pip install --upgrade pip
/opt/certbot/bin/pip install certbot certbot-nginx certbot-dns-godaddy
ln -s /opt/certbot/bin/certbot /usr/bin/certbot

GoDaddy credentials

certbot-dns-godaddy needs a credentials file from developer.godaddy.com/keys. This allows it to respond to DNS challenges from Let's Encrypt. Select "Production"; "ote" (the default) won't work. The file looks like the following:

dns_godaddy_secret = 0123456789abcdef0123456789abcdef01234567
dns_godaddy_key = abcdef0123456789abcdef01234567abcdef0123

Protect this file! It should only be readable by root. certbot runs periodically to renew wildcard certificates and it fetches the credentials from this file each time.

Creating the certificate

Run the following as root. If this process succeeds, certbot will create the certificate in /etc/letsencrypt/live/my-psinode-domain.com/.

Adjust the arguments to point to your credentials file, use your email address, and use your domain.

certbot \
  --authenticator dns-godaddy \
  --installer nginx \
  --email email_goes_here@email_goes_here \
  --agree-tos \
  -d 'my-psinode-domain.com,*.my-psinode-domain.com' \
  --dns-godaddy-credentials /root/.secrets/certbot/godaddy.ini \
  --dns-godaddy-propagation-seconds 900 \
  --keep-until-expiring \
  --non-interactive \
  --expand

Ignore this message:

Missing command line flag or config entry for this setting:
Which server blocks would you like to modify?

Auto renew

First, test renew works:

certbot --dry-run renew

If this operates correctly, schedule auto renewal:

echo "0 0,12 * * * root /opt/certbot/bin/python -c 'import random; import time; time.sleep(random.random() * 3600)' && certbot renew -q" | tee -a /etc/crontab > /dev/null

Twice a day, at 0:00 and 12:00, this will wait randomly up to 1 hour then run auto renewal.

Upgrade certbot

Update certbot every once in a while, e.g. when you update your system packages.

/opt/certbot/bin/pip install --upgrade certbot certbot-nginx certbot-dns-godaddy

Configure nginx

Create /etc/nginx/sites-available/psinode. Replace my-psinode-domain.com with your domain.

server {
    listen 443 ssl;

    # This includes both the domain and subdomains
    server_name             my-psinode-domain.com *.my-psinode-domain.com;

    ssl_certificate         /etc/letsencrypt/live/my-psinode-domain.com/fullchain.pem;
    ssl_certificate_key     /etc/letsencrypt/live/my-psinode-domain.com/privkey.pem;
    include                 /etc/letsencrypt/options-ssl-nginx.conf;

    # Allow larger upload than nginx's default
    client_max_body_size    2m;

    location / {
        proxy_pass       http://127.0.0.1:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }

    # Deny public access to the /native/admin interface
    # Note: this is only effective if port 8080 isn't publicly exposed
    location /native/admin {
        deny all;
        return 403;
    }

    # Optional; deny incoming p2p requests
    # Note: this is only effective if port 8080 isn't publicly exposed
    # location /native/p2p {
    #     deny all;
    #     return 403;
    # }
}

To enable it:

ln -s /etc/nginx/sites-available/psinode /etc/nginx/sites-enabled/psinode

service nginx restart

Running psinode

psinode needs to know what domain it's being hosted at. Use its -o/--host option:

psinode -o my-psinode-domain.com remaining_options...

Logging

• Config File
  • Console Logger
  • File Logger
  • Differences from JSON
• Log Filters
• Log Formatters

Logging in psinode can be configured at startup in the server's configuration file (found in <DATABASE>/config) or through the HTTP API while the server is running.

Config File

Each logger has a section in the config file called [logger.<NAME>]. The name of the logger is only significant to identify the logger.

[logger.stderr]
type   = console
filter = %Severity% >= info
format = [%TimeStamp%] [%Severity%]: %Message%

Each logger should have the following properties

Console logger

The console logger writes to the server's stderr. It does not use any additional configuration. There should not be more than one console logger.

File logger

The file logger writes to a named file and optionally provides log rotation and deletion. Multiple file loggers are supported as long as they do not write to the same files.

filename and target can contain patterns which will be used to generate multiple file names. The pattern should result in a unique name or old log files may be overwritten. The paths are relative to the server's root directory.

Both rotation and log deletion trigger when any condition is reached.

When log files are deleted, the oldest logs will be deleted first. All files that match the target pattern are assumed to be log files and are subject to deletion.

Example:

[logger.file]
type         = file
filter       = %Severity >= info%
format       = [%TimeStamp%]: %Message%
filename     = psibase.log
target       = psibase-%Y%m%d-%N.log
rotationTime = 00:00:00Z
rotationSize = 16 MiB
maxFiles     = 128
maxSize      = 1 GiB

Differences from JSON

The config file format is intended to allow manual editing and is therefore more permissive than the JSON format used by the HTTP API, which is designed as a machine-to-machine interface.

• rotationSize and maxSize can specify units, such as KiB, MiB, GiB, etc. The JSON format requires a Number in bytes only.
• A per-channel format is split into multiple properties instead of fields of a subobject. This is a consequence of the flat structure of the INI format.
• TODO: currently rotationTime is pretty human-unfriendly

Log Filters

Every logger has an associated filter. Filters determine whether to output any particular log record. The filter %Attribute% tests whether an attribute is present. %Attribute% op value tests that an attribute is present and meets a specific condition. Filters can be grouped using parentheses or combined using the boolean operators and, or, and not.

Examples:

• Everything except debug messages: %Severity% >= info
• Everything about a specific peer: %PeerId% = 42
• Warnings, errors, and blocks: %Severity% >= warning or %Channel% = block

Log Formatters

Formatters specify how a log record is formatted. In JSON, a formatter can be either a template string or an object. If the formatter is an object the keys should be channel names or "default".

Examples:

• [%TimeStamp%] [%Severity%] [%RemoteEndpoint%]: %Message%

• {
    "default": "[%TimeStamp%] [%Severity%]: %Message%",
    "p2p": "[%TimeStamp%] [%Severity%] [%RemoteEndpoint%]: %Message%",
    "block": "[%TimeStamp%] [%Severity%]: %Message% %BlockId%"
  }
  

Formatters have several attributes that are not available for filters.

Services

A service is like a smart contract on other chains, but has these differences:

• A service may serve either static or dynamic web pages which interact with the block chain.
• A service may provide RPC queries, GraphQL queries, and even help construct transactions. These may all rely on chain state.
• A service is often paired with an Applet, forming a bidirectional trust relationship.
• Most psibase functionality comes from services instead of from native functions.
• The term "service" is common in computer systems and doesn't imply a legal or business relationship.

Psibase services are WASM. psidk supports writing services in C++, Rust, and AssemblyScript (TODO).

Basic C++ Service

Here is a basic service definition. Place example.cpp and CMakeLists.txt in an empty folder.

example.cpp

#include <psibase/psibase.hpp>

// The service
struct ExampleService
{
   // Add two numbers
   int32_t add(int32_t a, int32_t b) { return a + b; }

   // Multiply two numbers
   int32_t multiply(int32_t a, int32_t b) { return a * b; }
};

// Reflect the service's methods. This enables
// PSIBASE_DISPATCH and other mechanisms to operate.
PSIO_REFLECT(ExampleService,  //
             method(add, a, b),
             method(multiply, a, b))

// Allow users to invoke reflected methods inside transactions.
// Also allows other services to invoke these methods.
PSIBASE_DISPATCH(ExampleService)

CMakeLists.txt

# All cmake projects need these
cmake_minimum_required(VERSION 3.16)
project(example)

# Generate compile_commands.json to aid vscode and other editors
set(CMAKE_EXPORT_COMPILE_COMMANDS on)

# psidk requires C++20
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

# Libraries for building services and tests
find_package(psidk REQUIRED)

# Build example.wasm service
add_executable(example example.cpp)
target_link_libraries(example psibase-service-simple-malloc)

# These symlinks help vscode
execute_process(COMMAND ln -sfT ${psidk_DIR} ${CMAKE_CURRENT_BINARY_DIR}/psidk)
execute_process(COMMAND ln -sfT ${WASI_SDK_PREFIX} ${CMAKE_CURRENT_BINARY_DIR}/wasi-sdk)

Building

This will create example.wasm:

mkdir build
cd build
cmake `psidk-cmake-args` ..
make -j $(nproc)

Deploying the service

This, when run on a local test chain, will:

• Create the example account, if it doesn't already exist. The account won't be secured; anyone can authorize as this account without signing. Caution: this option should not be used on production or public chains. -i is a shortcut for --create-insecure-account.
• Deploy the example.wasm service on that service.

psibase deploy -i example example.wasm

Trying the service

Even though other services may call into our service's add and multiply methods, we haven't provided end users with a way to construct transactions which use them. That's the topic of the next section, Minimal User Interface.

Homework

There's a potentially-exploitable bug in add and multiply. What is it? Why is it more dangerous in C++ than it is in psibase's other service languages? How can you avoid it?

vscode support

The following files configure vscode:

• .vscode/c_cpp_properties.json
• .vscode/settings.json

Code completion and symbol lookup does not work until the project is built (above).

Minimal User Interface

psidk can provide a minimal UI to your services. This UI can help get you started developing your own services, but isn't suitable for end users.

Here is the service definition. Place example.cpp and CMakeLists.txt in an empty folder.

example.cpp

#include <psibase/psibase.hpp>

struct ExampleService
{
   int32_t add(int32_t a, int32_t b) { return a + b; }
   int32_t multiply(int32_t a, int32_t b) { return a * b; }

   // This action serves HTTP requests
   std::optional<psibase::HttpReply> serveSys(psibase::HttpRequest request)
   {
      // serveSimpleUI serves UI files to the browser and
      // provides an RPC interface for preparing transactions.
      return serveSimpleUI<ExampleService, true>(request);
   }
};

PSIO_REFLECT(ExampleService,  //
             method(add, a, b),
             method(multiply, a, b),
             method(serveSys, request))

PSIBASE_DISPATCH(ExampleService)

CMakeLists.txt

CMakeLists.txt is the same as the one in Basic Service.

Building

This will create example.wasm:

mkdir build
cd build
cmake `psidk-cmake-args` ..
make -j $(nproc)

Deploying the service

The --register-proxy option (shortcut -p) registers the service with the proxy-sys service. Registered services may:

• Optionally serve files via HTTP
• Optionally respond to RPC requests
• Optionally respond to GraphQL requests

proxy-sys calls into the service's serveSys action. See the next section, Calling Other Services, to see how services do this.

psibase deploy -ip example example.wasm

Trying the service

If you're running a test chain locally, then it will typically be at http://psibase.127.0.0.1.sslip.io:8080/. If this is the case, then prefix the domain with the service name: http://example.psibase.127.0.0.1.sslip.io:8080/.

Sys suffix

There are 2 common suffixes used by psibase services:

• Trusted system services have account names which end with -sys. Only chain operators may create accounts with this suffix.
• psibase standard action names end with Sys or _Sys (case insensitive); serveSys is one of these actions. You should avoid this suffix when defining your own actions if they're not implementing one of the existing standards documented in this book. If you don't avoid it, your service may misbehave when future standards are adopted. e.g. don't create an action named emphasys.

How it works

• psinode forwards most http requests to the proxy-sys service.
• If the URL begins with /common, proxy-sys forwards the request to the common-sys service. common-sys provides shared resources, such as js library code and an RPC request handler for packing transactions.
• proxy-sys looks at the request domain. If it begins with the name of a registered service, it calls that service's serveSys action to process the request.
• psibase::serveSimpleUI handles the following requests:
  • GET / returns a minimal html file which references the /common/SimpleUI.mjs script. This script generates the UI dynamically.
  • GET /action_templates returns a template json structure (below). This lets the UI know which actions are available and sample values for their arguments. This isn't a schema; it's only suitable for simple cases.
  • POST /pack_action/add accepts the arguments for the add action as a JSON object, converts it to binary, and returns the result.

For more detail, see Web Services.

/action_templates result

{
  "add": {
    "a": 0,
    "b": 0
  },
  "multiply": {
    "a": 0,
    "b": 0
  },
  "serveSys": { ... }
}

Calling Other Services

Services may synchronously call into other services, for example to transfer tokens, fetch database rows, or even to use common library facilities (this example).

Arithmetic Service

Let's start by breaking up our previous example into a header file (.hpp) and an implementation file (.cpp). This makes it easier for other services to call into it.

arithmetic.hpp

#include <psibase/psibase.hpp>

// The header includes the class definition
struct Arithmetic
{
   // The account this service is normally installed on
   static constexpr auto service = psibase::AccountNumber("arithmetic");

   // The header declares but doesn't implement the actions
   int32_t add(int32_t a, int32_t b);
   int32_t multiply(int32_t a, int32_t b);

   std::optional<psibase::HttpReply> serveSys(psibase::HttpRequest request);
};

// The header does the reflection
PSIO_REFLECT(  //
    Arithmetic,
    method(add, a, b),
    method(multiply, a, b),
    method(serveSys, request))

// Do NOT include PSIBASE_DISPATCH in the header

arithmetic.cpp

#include "arithmetic.hpp"

// The implementation file has the action definitions
int32_t Arithmetic::add(int32_t a, int32_t b)
{
   return a + b;
}

int32_t Arithmetic::multiply(int32_t a, int32_t b)
{
   return a * b;
}

std::optional<psibase::HttpReply> Arithmetic::serveSys(  //
    psibase::HttpRequest request)
{
   return serveSimpleUI<Arithmetic, true>(request);
}

// The implementation file has the dispatcher
PSIBASE_DISPATCH(Arithmetic)

Caller Service

This service calls into the arithmetic service.

caller.hpp

#include <psibase/psibase.hpp>

struct Caller
{
   static constexpr auto service = psibase::AccountNumber("caller");

   int32_t mult_add(int32_t a, int32_t b, int32_t c, int32_t d);

   std::optional<psibase::HttpReply> serveSys(psibase::HttpRequest request);
};

PSIO_REFLECT(  //
    Caller,
    method(mult_add, a, b, c, d),
    method(serveSys, request))

caller.cpp

// This service
#include "caller.hpp"

// Other service
#include "arithmetic.hpp"

int32_t Caller::mult_add(int32_t a, int32_t b, int32_t c, int32_t d)
{
   // This allows us to call into the Arithmetic service. It fetches
   // the account number from Arithmetic::service.
   auto otherService = psibase::to<Arithmetic>();

   // Compute the result. Calls into the Arithmetic service 3 times.
   return otherService.add(          //
       otherService.multiply(a, b),  //
       otherService.multiply(c, d));
}

std::optional<psibase::HttpReply> Caller::serveSys(  //
    psibase::HttpRequest request)
{
   return serveSimpleUI<Caller, true>(request);
}

PSIBASE_DISPATCH(Caller)

CMakeLists.txt

CMakeLists.txt is almost the same as previous examples, but instead of building 1 service, it builds 2.

Building

This is the same as before.

mkdir build
cd build
cmake `psidk-cmake-args` ..
make -j $(nproc)

Deploying the service

Let's deploy both services.

psibase deploy -ip arithmetic arithmetic.wasm
psibase deploy -ip caller caller.wasm

Trying the service

If you're running a test chain locally, then the caller service's user interface is at http://caller.psibase.127.0.0.1.sslip.io:8080/.

What's Happening?

When a service calls another, the system pauses its execution and runs that other service. The system returns the result back to the original caller and resumes execution. This behavior is core to most of psibase's functionality. e.g. the SystemService::TransactionSys service receives a transaction then calls a service for each action within the transaction. These services may call more services, creating a tree of actions.

"Action" may refer to:

• One of the actions (requests) within a transaction
• A call from one service to another
• A method on a service

The system keeps each service alive during the entire transaction. This allows some interesting capabilities:

• Services may call each other many times without repeating the WASM startup overhead.
• Services may call into each other recursively. Be careful; you need to either plan for this or disable it. TODO: make it easy for services to opt out of recursion.
• A service method may store intermediate results in global variables then return. Future calls to that service, within the same transaction, have access to those global variables. They're wiped at the end of the transaction.

Who called me?

A service may call psibase::getSender to find out which service or user called it. A service also may use psibase::getReceiver to get the account that the service is running on.

void MyService::doSomething()
{
   psibase::check(
       psibase::getSender() == expectedAccount,
       "you're not who I expected");
}

Services and Events

• Defining a service
• Reserved action names
• Calling other services
• Defining events
• Recursion safety
• psibase::getSender
• psibase::getReceiver
• psibase::Service
  • psibase::Service::emit
  • psibase::Service::events
• psibase::Actor
• psibase::to
• psibase::from
• psibase::EventEmitter
• psibase::EventReader

Defining a service

To define a service:

• Make a struct or class. It may optionally publicly inherit from psibase::Service to gain the psibase::Service::emit and psibase::Service::events convenience methods.
• Define or declare the set of methods
• Reflect the methods using PSIO_REFLECT. Only reflected methods become actions; these are available for transactions and for other services to call using psibase::Actor or psibase::call.

Example without convenience base class:

struct MyService
{
   // The account this service is normally installed on. This definition
   // is optional.
   static constexpr auto service = psibase::AccountNumber("myservice");

   void        doSomething(std::string_view str);
   std::string somethingElse(uint32_t x, psibase::AccountNumber y);
};

PSIO_REFLECT(MyService,
             method(doSomething, str),
             method(somethingElse, x, y))

Example with convenience base class:

struct MyService: psibase::Service<MyService>
{
   // The account this service is normally installed on. This definition
   // is optional.
   static constexpr auto service = psibase::AccountNumber("myservice");

   void        doSomething(std::string_view str);
   std::string somethingElse(uint32_t x, psibase::AccountNumber y);
};

PSIO_REFLECT(MyService,
             method(doSomething, str),
             method(somethingElse, x, y))

Reserved action names

Psibase standard action names end with Sys or _Sys (case insensitive). You should avoid this suffix when defining your own actions if they're not implementing one of the existing standards documented in this book. If you don't avoid it, your service may misbehave when future standards are adopted. e.g. don't create an action named emphasys.

Calling other services

psibase::Actor supports calling other services. psibase::to and psibase::from simplify obtaining an actor.

To call another service:

auto result =
    psibase::to<OtherServiceClass>(otherServiceAccount)
    .someMethod(args...);

If OtherServiceClass defines service within it, you may omit the account name:

auto result =
    psibase::to<OtherServiceClass>()
    .someMethod(args...);

Defining events

See the following for a description of the various types of events:

• psibase::DbId::historyEvent
• psibase::DbId::uiEvent
• psibase::DbId::merkleEvent

To define events for a service, declare the event functions as below, then reflect them using the 4 macros below. Each of the History, Ui, and Merkle structs must be present and reflected, even when they don't have any events declared within.

After you have defined your events, use psibase::Service::emit to emit them and psibase::Service::events to read them.

struct MyService: psibase::Service<MyService> {
   struct Events {
      // Events which live a long time
      struct History {
         // These functions don't need implementations;
         // they only define the interface
         void myEvent(uint32_t a, std::string s);
         void anotherEvent(psibase::AccountNumber account);
      };

      // Events which live a short time
      struct Ui {
         void updateDisplay();
      };

      // Events which live in Merkle trees
      struct Merkle {
         void credit(
            psibase::AccountNumber from,
            psibase::AccountNumber to,
            uint64_t amount);
      };
   };
};

PSIBASE_REFLECT_EVENTS(MyService)

PSIBASE_REFLECT_HISTORY_EVENTS(
   MyService,
   method(myEvent, a, s),
   method(anotherEvent, account))

PSIBASE_REFLECT_UI_EVENTS(
   MyService,
   method(updateDisplay))

PSIBASE_REFLECT_MERKLE_EVENTS(
   MyService,
   method(credit, from, to, amount))

Recursion safety

• By default, services support recursion. TODO: make it opt-in instead.
• When a service is called multiple times within a transaction, including recursively, each action gets a fresh DerivedService instance. However, it runs in the same WASM memory space as the other executing actions for that service. Global variables and static variables are shared.
• Potential hazards to watch out for:
  • If a call modifies member variables within a Service instance, other calls aren't likely to see it.
  • If a call modifies global or static variables, this will effect both the other currently-executing calls, and subsequent calls.
  • If a call modifies the database, other currently-executing calls will see the change only if they read or re-read the database.
  • When you call into any service; assume it can call you back unless you opted out of recursion. TODO: make it possible to opt out of recursion.
  • Calling other services while you are iterating through the database can be dangerous, since they can call back into you, causing you to change the database.

The notes above use the following definition of "call":

• Using actions in a transaction to enter the service
• Using psibase::call or psibase::Actor to enter or reenter the service

Calling service methods directly (e.g. this->doSomething()) don't count in this definition.

psibase::getSender

psibase::AccountNumber psibase::getSender();

The account which authorized the currently-executing action.

psibase::getReceiver

psibase::AccountNumber psibase::getReceiver();

The account which received the currently-executing action.

psibase::Service

template<typename DerivedService>
struct psibase::Service {
    emit(...);   // Emit events
    events(...); // Read events
};

Services may optionally inherit from this to gain the emit and events convenience methods.

Template arguments:

• DerivedService: the most-derived service class that inherits from Service

psibase::Service::emit

EventEmitter<DerivedService> psibase::Service::emit() const;

Emit events.

The following examples use the example definitions in Defining Events. After you have defined your events, you can use emit to emit them. Examples:

auto eventANumber = this->emit().history().myEvent(a, s);
auto eventBNumber = this->emit().ui().updateDisplay();
auto eventCNumber = this->emit().merkle().credit(from, to, amount);

These functions return a psibase::EventNumber, aka uint64_t, which uniquely identifies the event. This number supports lookup; see Service::events.

emit is just a convenience method with the following definition:

EventEmitter<DerivedService> emit() const
{
   return EventEmitter<DerivedService>();
}

Here's how to do the above when the service doesn't inherit from psibase::Service:

EventEmitter<MyService> emitter;
auto eventANumber = emitter.history().myEvent(a, s);
auto eventBNumber = emitter.ui().updateDisplay();
auto eventCNumber = emitter.merkle().credit(from, to, amount);

psibase::Service::events

EventReader<DerivedService> psibase::Service::events() const;

Read events.

The following examples use the example definitions in Defining Events. After you have defined your events, you can use events to read them. Examples:

auto eventAArguments = this->events().history().myEvent(eventANumber).unpack();
auto eventBArguments = this->events().ui().updateDisplay(eventBNumber).unpack();
auto eventCArguments = this->events().merkle().credit(eventCNumber).unpack();

These functions take a psibase::EventNumber, aka uint64_t, which uniquely identifies the event. These numbers were generated by Service::emit.

The functions return psio::shared_view_ptr<std::tuple<event argument types>>. You can get the tuple using unpack(), like above.

There are restrictions on when events can be read; see the following for details:

• DbId::historyEvent
• DbId::uiEvent
• DbId::merkleEvent

events is just a convenience method with the following definition:

EventReader<DerivedService> events() const
{
   return EventReader<DerivedService>();
}

Here's how to do the above when the service doesn't inherit from psibase::Service:

auto EventReader<MyService> reader;
auto eventAArguments = reader.history().myEvent(eventANumber).unpack();
auto eventBArguments = reader.ui().updateDisplay(eventBNumber).unpack();
auto eventCArguments = reader.merkle().credit(eventCNumber).unpack();

psibase::Actor

template<typename T = void>
struct psibase::Actor {
    psibase::AccountNumber sender;   // Use this authority
    psibase::AccountNumber receiver; // Send actions to this account

    Actor(...);      // Constructor
    from(...);       // Use `other` authority
    to(...);         // Select a service to send actions to
    operator->(...); // Return this
    operator*(...);  // Return *this
};

Calls other services.

Template arguments:

• T: the service class for the receiver

Actor methods

Actor uses reflection to get the set of methods on T. It adds methods to itself with the same names, arguments, and return types to simplify calling.

For example, if SomeService has this set of methods:

struct SomeService : psibase::Service<SomeService>
{
   void        doSomething(std::string_view str);
   std::string doAnother(uint32_t x, psibase::AccountNumber y);
};
PSIO_REFLECT(SomeService,
             method(doSomething, str),
             method(doAnother, x, y))

Then Actor<SomeService> will have the same methods. Actor's methods:

• Pack their arguments, along with sender and receiver into Action
• Use call to synchronously call receiver with the action data
• Unpack the return value from the synchonous call
• Return it

psibase::Actor::Actor

psibase::Actor::Actor(
    psibase::AccountNumber sender,
    psibase::AccountNumber receiver
);

Constructor.

This actor will send actions to receiver using sender authority.

You probably don't need this constructor; use psibase::to or psibase::from.

Non-priviledged services may only use their own authority.

psibase::Actor::from

Actor<T> psibase::Actor::from(
    psibase::AccountNumber other
) const;

Use other authority.

This returns a new Actor object instead of modifying this.

Non-priviledged services may only use their own authority.

psibase::Actor::to

template<typename Other>
Actor<Other> psibase::Actor::to(
    uint64_t otherReceiver
) const;

Select a service to send actions to.

Template arguments:

• Other: the service's class

Arguments

• otherReceiver: the account the service runs on

This returns a new Actor object instead of modifying this.

psibase::Actor::operator->

Actor<T> * psibase::Actor::operator->() const;

Return this.

psibase::Actor::operator*

Actor<T> & psibase::Actor::operator*() const;

Return *this.

psibase::to

template<DefinesService Service>
Actor<Service> psibase::to();

Call a service.

Template arguments:

• Service: the receiver's class

Returns an Actor for calling receiver using the current service's authority. This version sets receiver to Service::service; this works if Service defined a const member named service which identifies the account that service is normally deployed on.

See the other overload

Example use:

auto result = to<OtherServiceClass>().someMethod(args...);

psibase::to

template<typename Service>
Actor<Service> psibase::to(
    psibase::AccountNumber receiver
);

Call a service.

Template arguments:

• Service: the receiver's class

Returns an Actor for calling receiver using the current service's authority.

See the other overload

Example use:

auto result = to<OtherServiceClass>(otherServiceAccount).someMethod(args...);

psibase::from

Actor<psibase::EmptyService> psibase::from(
    psibase::AccountNumber u = psibase::AccountNumber()
);

Call a service.

• If u is 0 (the default), then use this service's authority (getReceiver).
• If u is non-0, then use u's authority. Non-priviledged services may only use their own authority.

See psibase::to; it covers the majority use case.

Example use:

auto result =
  from(userAccount)
  .to<OtherServiceClass>(otherServiceAccount)
  .someMethod(args...);

psibase::EventEmitter

template<typename T = void>
struct psibase::EventEmitter {
    EventEmitter(...); // Constructor
    ui(...);           // Emit Ui events
    history(...);      // Emit History events
    merkle(...);       // Emit Merkle events
    from(...);         // Emit events from sender
    operator->(...);   // Return this
    operator*(...);    // Return *this
};

Emits events.

Template arguments:

• T: the service class which defines the events (e.g. MyService), or
• T: the inner-most struct within the service class which defines the events (e.g. MyService::Events::History)

Emit methods

EventEmitter uses reflection to get the set of events on T. It adds methods to itself with the same names and arguments.

For example, assume SomeService has the set of events in Defining Events. EventEmitter<MyService> e will support the following:

• e.history().myEvent(a, s);
• e.ui().updateDisplay();
• e.merkle().credit(from, to, amount);

These functions return a psibase::EventNumber, aka uint64_t, which uniquely identifies the event. This number supports lookup; see Service::events.

psibase::EventEmitter::EventEmitter

psibase::EventEmitter::EventEmitter(
    psibase::DbId elog = psibase::DbId::historyEvent
);

Constructor.

Initialize the emitter with:

• elog: the event log to emit to

Service::emit is a shortcut to constructing this.

psibase::EventEmitter::ui

auto psibase::EventEmitter::ui() const;

Emit Ui events.

See Emit Methods

psibase::EventEmitter::history

auto psibase::EventEmitter::history() const;

Emit History events.

See Emit Methods

psibase::EventEmitter::merkle

auto psibase::EventEmitter::merkle() const;

Emit Merkle events.

See Emit Methods

psibase::EventEmitter::from

auto psibase::EventEmitter::from(
    psibase::AccountNumber sender
) const;

Emit events from sender.

This returns a new EventEmitter object instead of modifying this.

You probably don't need this; use Service::emit instead.

psibase::EventEmitter::operator->

auto * psibase::EventEmitter::operator->() const;

Return this.

psibase::EventEmitter::operator*

auto & psibase::EventEmitter::operator*() const;

Return *this.

psibase::EventReader

template<typename T = void>
struct psibase::EventReader {
    EventReader(...); // Constructor
    ui(...);          // Read Ui events
    history(...);     // Read History events
    merkle(...);      // Read Merkle events
    from(...);        // Read events from sender
    operator->(...);  // Return this
    operator*(...);   // Return *this
};

Reads events.

Template arguments:

• T: the service class which defines the events (e.g. MyService), or
• T: the inner-most struct within the service class which defines the events (e.g. MyService::Events::History)

Reader methods

EventReader uses reflection to get the set of events on T. It adds methods to itself with the same names and arguments.

For example, assume SomeService has the set of events in Defining Events. EventReader<MyService> e will support the following:

• auto eventAArguments = e.history().myEvent(eventANumber).unpack();
• auto eventBArguments = e.ui().updateDisplay(eventBNumber).unpack();
• auto eventCArguments = e.merkle().credit(eventCNumber).unpack();

These functions take a psibase::EventNumber, aka uint64_t, which uniquely identifies the event. These numbers were generated by Service::emit.

The functions return psio::shared_view_ptr<std::tuple<event argument types>>. You can get the tuple using unpack(), like above.

There are restrictions on when events can be read; see the following for details:

• DbId::historyEvent
• DbId::uiEvent
• DbId::merkleEvent

psibase::EventReader::EventReader

psibase::EventReader::EventReader(
    psibase::DbId elog = psibase::DbId::historyEvent
);

Constructor.

Initialize the reader with:

• elog: the event log to read from

Service::events is a shortcut for constructing this.

psibase::EventReader::ui

auto psibase::EventReader::ui() const;

Read Ui events.

See Reader Methods

psibase::EventReader::history

auto psibase::EventReader::history() const;

Read History events.

See Reader Methods

psibase::EventReader::merkle

auto psibase::EventReader::merkle() const;

Read Merkle events.

See Reader Methods

psibase::EventReader::from

auto psibase::EventReader::from(
    psibase::AccountNumber sender
);

Read events from sender.

This returns a new EventReader object instead of modifying this.

You probably don't need this; use Service::events instead.

psibase::EventReader::operator->

auto * psibase::EventReader::operator->() const;

Return this.

psibase::EventReader::operator*

auto & psibase::EventReader::operator*() const;

Return *this.

Table

• psibase::ServiceTables
  • Modifying table set
  • Prefix format
• psibase::Table
  • Schema changes
  • Data format
• psibase::TableIndex
• psibase::KvIterator
• psibase::KeyView

psibase::ServiceTables

template<typename ...Tables>
struct psibase::ServiceTables {
    psibase::AccountNumber account; // the service runs on this account

    ServiceTables(...); // Constructor
    open(...);          // Open by table number
    open(...);          // Open by table type
};

Defines the set of tables in a service.

Template arguments:

• Tables: one or more Table types; one for each table the service supports.

Modifying table set

You may add additional tables at the end.

Don't do any of the following; these will corrupt data:

• Don't remove any tables
• Don't reorder the tables

Prefix format

ServiceTables gives each table the following prefix. See Data format.

psibase::ServiceTables::ServiceTables

explicit psibase::ServiceTables::ServiceTables(
    psibase::AccountNumber account
);

Constructor.

account is the account the service runs on.

psibase::ServiceTables::open

template<std::uint16_t Table>
auto psibase::ServiceTables::open() const;

Open by table number.

This gets a table by number. The first table is 0.

e.g. auto table = MyServiceTables{myServiceAccount}.open<2>();

Returns a Table.

psibase::ServiceTables::open

template<typename T>
auto psibase::ServiceTables::open() const;

Open by table type.

This gets a table by the table's type.

e.g. auto table = MyServiceTables{myServiceAccount}.open<MyTable>();

Returns a Table.

psibase::Table

template<typename T, auto Primary, auto ...Secondary>
struct psibase::Table {
    Table(...);    // Construct table with prefix
    Table(...);    // Construct table with prefix
    put(...);      // Store `value` into the table
    erase(...);    // Remove `key` from table
    remove(...);   // Remove object from table
    getIndex(...); // Get a primary or secondary index
};

Stores objects in the key-value database.

Template arguments:

• T: Type of object stored in table
• Primary: fetches primary key from an object
• Secondary: fetches a secondary key from an object. This is optional; there may be 0 or more secondary keys.

Primary and Secondary may be:

• pointer-to-data-member. e.g. &MyType::key
• pointer-to-member-function which returns a key. e.g. &MyType::keyFunction
• non-member function which takes a const T& as its only argument and returns a key
• a callable object which takes a const T& as its only argument and returns a key

Schema changes

You may modify the schema of an existing table the following ways:

• Add new optional<...> fields to the end of T, unless T is marked definitionWillNotChange(). When you read old records, these values will be std::nullopt.
• If T has a field with type X, then you may add new optional<...> fields to the end of X, unless X is marked definitionWillNotChange(). This rule is recursive, including through vector, optional, and user-defined types.
• Add new secondary indexes after the existing ones. Old records will not appear in the new secondary indexes. Remove then re-add records to fill the new secondary indexes.

Don't do any of the following; these will corrupt data:

• Don't modify any type marked definitionWillNotChange()
• Don't remove definitionWillNotChange() from a type; this changes its serialization format
• Don't add new fields to the middle of T or any types that it contains
• Don't add new non-optional fields to T or any types that it contains
• Don't reorder fields within T or any types that it contains
• Don't remove fields from T or any types that it contains
• Don't change the types of fields within T or any types that it contains
• Don't reorder secondary indexes
• Don't remove secondary indexes

Data format

The key-value pairs have this format:

Each secondary index is numbered 1 <= i <= 255 above. The secondary indexes point to the primary index.

Table serializes keys using psio::to_key. It serializes T using fracpack.

psibase::Table::Table

psibase::Table::Table(
    psibase::DbId    db,
    psibase::KeyView prefix
);

Construct table with prefix.

The prefix separates this table's data from other tables; see Data format.

This version of the constructor copies the data within prefix.

psibase::Table::Table

psibase::Table::Table(
    psibase::DbId        db,
    std::vector<char> && prefix
);

Construct table with prefix.

The prefix separates this table's data from other tables; see Data format.

psibase::Table::put

void psibase::Table::put(
    const T & value
);

Store value into the table.

If an object already exists with the same primary key, then the new object replaces it. If the object has any secondary keys which have the same value as another object, but not the one it's replacing, then put aborts the transaction.

psibase::Table::erase

template<compatible_key<psibase::Table::key_type> Key>
void psibase::Table::erase(
    Key && key
);

Remove key from table.

This is equivalent to looking an object up by the key, then calling remove if found. The key must be the primary key.

psibase::Table::remove

void psibase::Table::remove(
    const T & oldValue
);

Remove object from table.

psibase::Table::getIndex

template<int Idx>
auto psibase::Table::getIndex() const;

Get a primary or secondary index.

If Idx is 0, then this returns the primary index, else it returns a secondary index.

The result is TableIndex.

psibase::TableIndex

template<typename T, typename K>
struct psibase::TableIndex {
    TableIndex(...);  // Construct with prefix
    begin(...);       // Get iterator to first object
    end(...);         // Get iterator past the end
    lower_bound(...); // Get iterator to first object with `key >= k`
    upper_bound(...); // Get iterator to first object with `key > k`
    subindex(...);    // Divide the key space
    get(...);         // Look up object by key
};

A primary or secondary index in a Table.

Use Table::getIndex to get this.

Template arguments:

• T: Type of object stored in table
• K: Type of key this index uses

psibase::TableIndex::TableIndex

psibase::TableIndex::TableIndex(
    psibase::DbId        db,
    std::vector<char> && prefix,
    bool                 is_secondary
);

Construct with prefix.

prefix identifies the range of database keys that the index occupies.

psibase::TableIndex::begin

KvIterator<T> psibase::TableIndex::begin() const;

Get iterator to first object.

psibase::TableIndex::end

KvIterator<T> psibase::TableIndex::end() const;

Get iterator past the end.

psibase::TableIndex::lower_bound

template<CompatibleKeyPrefix<K> K2>
KvIterator<T> psibase::TableIndex::lower_bound(
    K2 && k
) const;

Get iterator to first object with key >= k.

If the index's key is an std::tuple, then k may be the first n fields of the key.

Returns end if none found.

psibase::TableIndex::upper_bound

template<CompatibleKeyPrefix<K> K2>
KvIterator<T> psibase::TableIndex::upper_bound(
    K2 && k
) const;

Get iterator to first object with key > k.

If the index's key is an std::tuple, then k may be the first n fields of the key.

Returns end if none found.

psibase::TableIndex::subindex

template<CompatibleKeyPrefix<K> K2>
TableIndex<T, KeySuffix<K2, K>> psibase::TableIndex::subindex(
    K2 && k
);

Divide the key space.

Assume K is a std::tuple<A, B, C, D>. If you call subindex with a tuple with the first n fields of K, e.g. std::tuple(aValue, bValue), then subindex returns another TableIndex which restricts its view to the subrange. e.g. it will iterate and search for std::tuple(cValue, dValue), holding aValue and bValue constant.

psibase::TableIndex::get

template<compatible_key<K> K2>
std::optional<T> psibase::TableIndex::get(
    K2 && k
) const;

Look up object by key.

If a matching key is found, then it returns a fresh object; it does not cache.

psibase::KvIterator

template<typename T>
struct psibase::KvIterator {
    KvIterator(...);       // constructor
    operator++(...);       // preincrement (++it)
    operator++(...);       // postincrement (it++)
    operator--(...);       // predecrement (--it)
    operator--(...);       // postdecrement (it--)
    moveTo(...);           // Move iterator
    moveTo(...);           // Move iterator
    keyWithoutPrefix(...); // Get serialized key without prefix
    operator*(...);        // get object
    operator<=>(...);      // Comparisons
};

An iterator into a TableIndex.

Use TableIndex::begin, TableIndex::end, TableIndex::lower_bound, or TableIndex::upper_bound to get an iterator.

psibase::KvIterator::KvIterator

psibase::KvIterator::KvIterator(
    psibase::DbId        db,
    std::vector<char> && key,
    std::size_t          prefixSize,
    bool                 isSecondary,
    bool                 isEnd
);

constructor.

• db identifies the database the table lives in.
• See the "Key" column of Data format; the key field contains this.
• prefixSize is the number of bytes within key which covers the index's prefix (includes the index number byte).
• isSecondary is true if this iterator is for a secondary index.
• isEnd is true if this iterator points past the end.

You don't need this constructor in most cases; use TableIndex::begin, TableIndex::end, TableIndex::lower_bound, or TableIndex::upper_bound instead.

psibase::KvIterator::operator++

KvIterator<T> & psibase::KvIterator::operator++();

preincrement (++it).

This moves the iterator forward.

The iterator has circular semantics. If you increment an end iterator, then it moves to the beginning of the index, or back to end again if empty.

psibase::KvIterator::operator++

KvIterator<T> psibase::KvIterator::operator++(
    int 
);

postincrement (it++).

This moves the iterator forward.

The iterator has circular semantics. If you increment an end iterator, then it moves to the beginning of the index, or back to end again if empty.

Note: postincrement (it++) and postdecrement (it--) have higher overhead than preincrement (++it) and predecrement (--it).

psibase::KvIterator::operator--

KvIterator<T> & psibase::KvIterator::operator--();

predecrement (--it).

This moves the iterator backward.

The iterator has circular semantics. If you decrement a begin iterator, then it moves to end.

psibase::KvIterator::operator--

KvIterator<T> psibase::KvIterator::operator--(
    int 
);

postdecrement (it--).

This moves the iterator backward.

The iterator has circular semantics. If you decrement a begin iterator, then it moves to end.

Note: postincrement (it++) and postdecrement (it--) have higher overhead than preincrement (++it) and predecrement (--it).

psibase::KvIterator::moveTo

void psibase::KvIterator::moveTo(
    int result
);

Move iterator.

This moves the iterator to the most-recent location found by raw::kvGreaterEqual, raw::kvLessThan, or raw::kvMax.

result is the return value of the raw call.

You don't need this function in most cases; use TableIndex::begin, TableIndex::end, TableIndex::lower_bound, TableIndex::upper_bound, or the iterator's increment or decrement operators instead.

psibase::KvIterator::moveTo

void psibase::KvIterator::moveTo(
    std::span<const char> k
);

Move iterator.

This moves the iterator to k. k does not include the prefix. May be used for GraphQL cursors; see keyWithoutPrefix.

psibase::KvIterator::keyWithoutPrefix

std::span<const char> psibase::KvIterator::keyWithoutPrefix() const;

Get serialized key without prefix.

The returned value can be passed to moveTo, e.g. for GraphQL cursors.

psibase::KvIterator::operator*

T psibase::KvIterator::operator*() const;

get object.

This reads an object from the database. It does not cache; it returns a fresh object each time it's used.

psibase::KvIterator::operator<=>

std::weak_ordering psibase::KvIterator::operator<=>(
    const KvIterator<T> & rhs
) const;

Comparisons.

psibase::KeyView

struct psibase::KeyView {
    std::span<const char> data; 
};

A serialized key (non-owning).

The serialized data has the same sort order as the non-serialized form

Web Services

• Routing
• Registration
• Interfaces
  • psibase::ServerInterface
  • psibase::HttpRequest
  • psibase::HttpReply
  • psibase::StorageInterface
• Helpers
  • psibase::serveSimpleUI
  • psibase::serveActionTemplates
  • psibase::servePackAction
  • psibase::WebContentRow
  • psibase::storeContent
  • psibase::serveContent
  • psibase::serveGraphQL
  • psibase::makeConnection
    • psibase::PageInfo
    • psibase::Edge
    • psibase::Connection
  • psibase::EventDecoder
  • psibase::EventQuery
  • psibase::makeEventConnection

Routing

psinode passes most HTTP requests to the SystemService::ProxySys service, which then routes requests to the appropriate service's serveSys action (see diagram). The services run in RPC mode; this prevents them from writing to the database, but allows them to read data they normally can't. See psibase::DbId.

SystemService::CommonSys provides services common to all domains under the /common tree. It also serves the chain's main page.

SystemService::PsiSpaceSys provides web hosting to non-service accounts.

psinode directly handles requests which start with /native, e.g. /native/push_transaction. Services don't serve these.

Registration

Services which wish to serve HTTP requests need to register using the SystemService::ProxySys service's SystemService::ProxySys::registerServer action. There are multiple ways to do this:

• psibase deploy has a --register-proxy option (shortcut -p) that can do this while deploying the service.
• psibase register-proxy can also do it. TODO: implement psibase register-proxy.
• A service may call registerServer during its own initialization action.

A service doesn't have to serve HTTP requests itself; it may delegate this to another service during registration.

Interfaces

Services which serve HTTP implement these interfaces:

• psibase::ServerInterface (required)
  • psibase::HttpRequest
  • psibase::HttpReply
• psibase::StorageInterface (optional)

psibase::ServerInterface

struct psibase::ServerInterface {
    serveSys(...); // Handle HTTP requests
};

Interface for services which serve http.

proxy.sys uses this interface to call into services to respond to http requests.

Do not inherit from this. To implement this interface, add a serveSys action to your service and reflect it.

psibase::ServerInterface::serveSys

std::optional<HttpReply> psibase::ServerInterface::serveSys(
    psibase::HttpRequest request
);

Handle HTTP requests.

Define this action in your service to handle HTTP requests. You'll also need to register your service.

serveSys can do any of the following:

• Return std::nullopt to signal not found. psinode produces a 404 response in this case.
• Abort. psinode produces a 500 response with the service's abort message.
• Return a psibase::HttpReply. psinode produces a 200 response with the body and contentType returned.
• Call other services.

A service runs in RPC mode while serving an HTTP request. This mode prevents database writes, but allows database reads, including reading data and events which are normally not available to services; see psibase::DbId.

psibase::HttpRequest

struct psibase::HttpRequest {
    std::string       host;        // Fully-qualified domain name
    std::string       rootHost;    // host, but without service subdomain
    std::string       method;      // "GET" or "POST"
    std::string       target;      // Absolute path, e.g. "/index.js"
    std::string       contentType; // "application/json", "text/html", ...
    std::vector<char> body;        // Request body, e.g. POST data
};

An HTTP Request.

Most services receive this via their serveSys action. SystemService::ProxySys receives it via its serve exported function.

psibase::HttpReply

struct psibase::HttpReply {
    std::string             contentType; // "application/json", "text/html", ...
    std::vector<char>       body;        // Response body
    std::vector<HttpHeader> headers;     // HTTP Headers
};

An HTTP reply.

Services return this from their serveSys action.

psibase::StorageInterface

struct psibase::StorageInterface {
    storeSys(...); // Store a file
};

Interface for services which support storing files.

Some services support storing files which they then serve via HTTP. This is the standard interface for these services.

Do not inherit from this. To implement this interface, add a storeSys action to your service and reflect it.

psibase::StorageInterface::storeSys

void psibase::StorageInterface::storeSys(
    std::string_view  path,
    std::string_view  contentType,
    std::vector<char> content
);

Store a file.

Define this action in your service to handle file storage requests. This action should store the file in the service's tables. The service can then serve these files via HTTP.

• path: absolute path to file. e.g. /index.html for the main page
• contentType: text/html, text/javascript, application/octet-stream, ...
• content: file content

The psibase upload command uses this action.

storeContent simplifies implementing this.

Helpers

These help implement basic functionality:

• psibase::serveSimpleUI
• psibase::serveActionTemplates
• psibase::servePackAction
• psibase::WebContentRow
• psibase::storeContent
• psibase::serveContent
• psibase::serveGraphQL
• psibase::makeConnection
  • psibase::PageInfo
  • psibase::Edge
  • psibase::Connection
• psibase::EventDecoder
• psibase::EventQuery
• psibase::makeEventConnection

Here's a common pattern for using these functions:

std::optional<psibase::HttpReply> serveSys(psibase::HttpRequest request)
{
   if (auto result = psibase::serveActionTemplates<ExampleService>(request))
      return result;

   if (auto result = psibase::servePackAction<ExampleService>(request))
      return result;

   if (request.method == "GET" && request.target == "/")
   {
      static const char helloWorld[] = "Hello World";
      return psibase::HttpReply{
            .contentType = "text/plain",
            .body        = {helloWorld, helloWorld + strlen(helloWorld)},
      };
   }

   return std::nullopt;
}

psibase::serveSimpleUI

template<typename Service, bool IncludeRoot>
std::optional<HttpReply> psibase::serveSimpleUI(
    const psibase::HttpRequest & request
);

Serve a developer UI.

This function serves a simple developer UI to help get you started. The UI it generates is not suitable for end users.

This serves the following:

• GET /action_templates: provided by serveActionTemplates
• POST /pack_action/x: provided by servePackAction
• GET /, but only if IncludeRoot is set. This returns the following HTML body:

<html>
<div id="root" class="ui container"></div>
<script src="/common/SimpleUI.mjs" type="module"></script>
</html>

psibase::serveActionTemplates

template<typename Service>
std::optional<HttpReply> psibase::serveActionTemplates(
    const psibase::HttpRequest & request
);

Handle /action_templates request.

If request is a GET to /action_templates, then this returns a JSON object containing a field for each action in Service. The field names match the action names. The field values are objects with the action arguments, each containing sample data.

If request doesn't match the above, then this returns std::nullopt.

psibase::servePackAction

template<typename Service>
std::optional<HttpReply> psibase::servePackAction(
    const psibase::HttpRequest & request
);

Handle /pack_action/ request.

If request is a POST to /pack_action/x, where x is an action on Service, then this parses a JSON object containing the arguments to x, packs them using frackpac, and returns the result as an application/octet-stream.

If request doesn't match the above, or the action name is not found, then this returns std::nullopt.

psibase::WebContentRow

struct psibase::WebContentRow {
    std::string       path;        // Absolute path to content, e.g. "/index.mjs"
    std::string       contentType; // "text/html", "text/javascript", ...
    std::vector<char> content;     // Content body
};

Content for serving over HTTP.

This the table row format for services which store and serve HTTP files using storeContent and serveContent.

Also includes this definition:

using WebContentTable = Table<WebContentRow, &WebContentRow::path>;

psibase::storeContent

template<typename ...Tables>
void psibase::storeContent(
    std::string &&                   path,
    std::string &&                   contentType,
    std::vector<char> &&             content,
    const ServiceTables<Tables...> & tables
);

Store web content in table.

This stores web content into a service's WebContentTable. serveContent serves this content via HTTP.

Example use:

// Don't forget to include your service's other tables in this!
using Tables = psibase::ServiceTables<psibase::WebContentTable>;

void MyService::storeSys(
   std::string path, std::string contentType, std::vector<char> content)
{
   psibase::check(getSender() == getReceiver(), "wrong sender");
   psibase::storeContent(std::move(path), std::move(contentType), std::move(content),
                         Tables{getReceiver()});
}

psibase::serveContent

template<typename ...Tables>
std::optional<HttpReply> psibase::serveContent(
    const psibase::HttpRequest &     request,
    const ServiceTables<Tables...> & tables
);

Serve files via HTTP.

This serves files stored by storeContent.

Example use:

// Don't forget to include your service's other tables in this!
using Tables = psibase::ServiceTables<psibase::WebContentTable>;

std::optional<psibase::HttpReply> MyService::serveSys(
   psibase::HttpRequest request)
{
   if (auto result = psibase::serveContent(request, Tables{getReceiver()}))
      return result;
   return std::nullopt;
}

psibase::serveGraphQL

template<typename QueryRoot>
std::optional<HttpReply> psibase::serveGraphQL(
    const psibase::HttpRequest & request,
    const QueryRoot &            queryRoot
);

Handle /graphql request.

This handles graphql requests, including fetching the schema.

• GET /graphql: Returns the schema.
• GET /graphql?query=...: Run query in URL and return JSON result.
• POST /graphql?query=...: Run query in URL and return JSON result.
• POST /graphql with Content-Type = application/graphql: Run query that's in body and return JSON result.
• POST /graphql with Content-Type = application/json: Body contains a JSON object of the form {"query"="..."}. Run query and return JSON result.

queryRoot should be a reflected object; this shows up in GraphQL as the root Query type. GraphQL exposes both fields and const methods. Fields may be any reflected struct. Const methods may return any reflected struct. They should return objects by value.

See makeConnection example.

psibase::makeConnection

template<typename Connection, typename T, typename Key>
Connection psibase::makeConnection(
    const TableIndex<T, Key> &         index,
    const std::optional<Key> &         gt,
    const std::optional<Key> &         ge,
    const std::optional<Key> &         lt,
    const std::optional<Key> &         le,
    std::optional<uint32_t>            first,
    std::optional<uint32_t>            last,
    const std::optional<std::string> & before,
    const std::optional<std::string> & after
);

GraphQL Pagination through TableIndex.

You rarely need to call this directly; see the example.

Template arguments:

• Connection: Connection
• T: Type stored in index
• Key: Key in index

Arguments:

• index: TableIndex to paginate through
• gt: Restrict range to keys greater than this
• ge: Restrict range to keys greater than or equal to this
• lt: Restrict range to keys less than to this
• le: Restrict range to keys less than or equal to this
• first: Stop after including this many items at the beginning of the range
• last: Stop after including this many items at the end of the range
• before: Opaque cursor value. Resume paging; include keys before this point
• after: Opaque cursor value. Resume paging; include keys after this point

By default, makeConnection pages through the entire index's range. gt, ge, lt, and le restrict the range. They can be used in any combination (set intersection).

first, last, before, after, PageInfo, Connection, and Edge match the GraphQL Cursor Connections Specification (aka GraphQL Pagination). first and after page through the range defined above in the forward direction. last and before page through the range defined above in the reverse direction. Combining first with last isn't recommended, but matches the behavior in the specification.

makeConnection example

This demonstrates exposing a table's contents via GraphQL. This example doesn't include a way to fill the table; that's left as an exercise to the reader. Hint: service-based RPC and GraphQL only support read-only operations; you must use actions to write to a table.

#include <psibase/Service.hpp>
#include <psibase/dispatch.hpp>
#include <psibase/serveGraphQL.hpp>
#include <psibase/serveSimpleUI.hpp>

struct MyType
{
   uint32_t    primaryKey;
   std::string secondaryKey;
   std::string moreData;

   std::string someFn(std::string arg1, std::string arg2) const
   {
      return arg1 + secondaryKey + arg2 + moreData;
   }
};
PSIO_REFLECT(MyType,
   primaryKey, secondaryKey, moreData,
   method(someFn, arg1, arg2))

using MyTable  = psibase::Table<
   MyType, &MyType::primaryKey, &MyType::secondaryKey>;
using MyTables = psibase::ServiceTables<MyTable>;

struct Query
{
   psibase::AccountNumber service;

   auto rowsByPrimary() const {
      return MyTables{service}.open<MyTable>().getIndex<0>();
   }
   auto rowsBySecondary() const {
      return MyTables{service}.open<MyTable>().getIndex<1>();
   }
};
PSIO_REFLECT(Query, method(rowsByPrimary), method(rowsBySecondary))

struct ExampleService : psibase::Service<ExampleService>
{
   std::optional<psibase::HttpReply> serveSys(psibase::HttpRequest request)
   {
      if (auto result = psibase::serveSimpleUI<ExampleService, true>(request))
         return result;
      if (auto result = psibase::serveGraphQL(request, Query{getReceiver()}))
         return result;
      return std::nullopt;
   }
};
PSIO_REFLECT(ExampleService, method(serveSys, request))

PSIBASE_DISPATCH(ExampleService)

This example doesn't call makeConnection directly; it's automatic. If a member function on a query object:

• is const, and
• is reflected, and
• has no arguments, and
• returns a TableIndex

then the system exposes that function to GraphQL as a function which takes gt, ge, lt, le, first, last, before, and after. The system calls makeConnection automatically.

serveGraphQL generates this GraphQL schema and processes queries which conform to it:

type MyType {
    primaryKey: Float!
    secondaryKey: String!
    moreData: String!
    someFn(arg1: String! arg2: String!): String!
}
type PageInfo {
    hasPreviousPage: Boolean!
    hasNextPage: Boolean!
    startCursor: String!
    endCursor: String!
}
type MyTypeEdge {
    node: MyType!
    cursor: String!
}
type MyTypeConnection {
    edges: [MyTypeEdge!]!
    pageInfo: PageInfo!
}
type Query {
    rowsByPrimary(
         gt: Float ge: Float lt: Float le: Float
         first: Float last: Float
         before: String after: String): MyTypeConnection!
    rowsBySecondary(
         gt: String ge: String lt: String le: String
         first: Float last: Float
         before: String after: String): MyTypeConnection!
}

Things of note:

• rowsByPrimary and rowsBySecondary automatically have makeConnection's arguments.
• MyTypeEdge and MyTypeConnection are automatically generated from MyType.
• Returned rows (MyType) include MyType's fields and the someFn method. Only const methods are exposed.
• serveGraphQL automatically chooses GraphQL types which cover the range of numeric types. When no suitable match is found (e.g. no GraphQL type covers the range of int64_t), it falls back to String.

psibase::PageInfo

struct psibase::PageInfo {
    bool        hasPreviousPage; 
    bool        hasNextPage;     
    std::string startCursor;     
    std::string endCursor;       
};

GraphQL support for paging.

This lets the query clients know when more data is available and what cursor values can be used to fetch that data.

psibase::Edge

template<typename Node, psio::FixedString EdgeName>
struct psibase::Edge {
    Node        node;   
    std::string cursor; 
};

GraphQL support for paging.

node contains the row data. cursor identifies where in the table the node is located.

psibase::Connection

template<typename Node, psio::FixedString ConnectionName, psio::FixedString EdgeName>
struct psibase::Connection {
    std::vector<Edge> edges;    
    psibase::PageInfo pageInfo; 
};

GraphQL support for paging.

edges contain the matching rows. pageInfo gives clients information needed to resume paging.

psibase::EventDecoder

template<typename Events>
struct psibase::EventDecoder {
    psibase::DbId          db;      
    uint64_t               eventId; 
    psibase::AccountNumber service; 
};

GraphQL support for decoding an event.

If a GraphQL query function returns this type, then the system fetches and decodes an event.

The GraphQL result is an object with these fields, plus more:

type MyService_EventsUi {
    event_db: Float!                   # Database ID (uint32_t)
    event_id: String!                  # Event ID (uint64_t)
    event_found: Boolean!              # Was the event found in db?
    event_service: String!             # Service that created the event
    event_supported_service: Boolean!  # Is this service the one
                                       #    that created it?
    event_type: String!                # Event type
    event_unpack_ok: Boolean!          # Did it decode OK?
}

EventDecoder will only attempt to decode an event which meets all of the following:

• It's found in the EventDecoder::db database (event_found will be true)
• Was written by the service which matches the EventDecoder::service field (event_supported_service will be true)
• Has a type which matches one of the definitions in the Events template argument

If decoding is successful, EventDecoder will set the GraphQL event_unpack_ok field to true. It will include any event fields which were in the query request. It will include all event fields if the query request includes the special field event_all_content. EventDecoder silently ignores any requested fields which don't match the fields of the decoded event.

EventDecoder example

This example assumes you're already serving GraphQL and have defined events for your service. It's rare to define a query method like this one; use EventQuery instead, which handles history, ui, and merkle events.

struct Query
{
   psibase::AccountNumber service;

   auto getUiEvent(uint64_t eventId) const
   {
      return EventDecoder<MyService::Events::Ui>{
         DbId::uiEvent, eventId, service};
   }
};
PSIO_REFLECT(Query, method(getUiEvent, eventId))

Example query:

{
  getUiEvent(eventId: "13") {
    event_id
    event_type
    event_all_content
  }
}

Example reply:

{
  "data": {
    "getUiEvent": {
      "event_id": "13",
      "event_type": "credited",
      "tokenId": 1,
      "sender": "symbol-sys",
      "receiver": "alice",
      "amount": {
        "value": "100000000000"
      },
      "memo": {
        "contents": "memo"
      }
    }
  }
}

psibase::EventQuery

template<typename Events>
struct psibase::EventQuery {
    psibase::AccountNumber service; 

    history(...); // Decode history events
    ui(...);      // Decode user interface events
    merkle(...);  // Decode merkle events
};

GraphQL support for decoding multiple events.

If a GraphQL query function returns this type, then the system returns a GraphQL object with the following query methods:

type MyService_Events {
    history(ids: [String!]!): [MyService_EventsHistory!]!
    ui(ids: [String!]!):      [MyService_EventsUi!]!
    merkle(ids: [String!]!):  [MyService_EventsMerkle!]!
}

These methods take an array of event IDs. They return arrays of objects containing the decoded (if possible) events. See EventDecoder for how to interact with the return values; MyService_EventsHistory, MyService_EventsUi, and MyService_EventsMerkle all behave the same.

EventQuery example

This example assumes you're already serving GraphQL and have defined events for your service.

struct Query
{
   psibase::AccountNumber service;

   auto events() const
   {
      return psibase::EventQuery<MyService::Events>{service};
   }
};
PSIO_REFLECT(Query, method(events))

Example query:

{
  events {
    history(ids: ["3", "4"]) {
      event_id
      event_all_content
    }
  }
}

Example reply:

{
  "data": {
    "events": {
      "history": [
        {
          "event_id": "3",
          "tokenId": 1,
          "creator": "token-sys",
          "precision": {
            "value": 8
          },
          "maxSupply": {
            "value": "100000000000000000"
          }
        },
        {
          "event_id": "4",
          "prevEvent": 1,
          "tokenId": "3",
          "setter": "token-sys",
          "flag": "untradeable",
          "enable": true
        }
      ]
    }
  }
}

psibase::EventQuery::history

auto psibase::EventQuery::history(
    const std::vector<uint64_t> & ids
) const;

Decode history events.

psibase::EventQuery::ui

auto psibase::EventQuery::ui(
    const std::vector<uint64_t> & ids
) const;

Decode user interface events.

psibase::EventQuery::merkle

auto psibase::EventQuery::merkle(
    const std::vector<uint64_t> & ids
) const;

Decode merkle events.

psibase::makeEventConnection

template<typename Events>
auto psibase::makeEventConnection(
    psibase::DbId                      db,
    uint64_t                           eventId,
    psibase::AccountNumber             service,
    std::string_view                   fieldName,
    std::optional<uint32_t>            first,
    const std::optional<std::string> & after
);

Magic Numbers

• psibase::AccountNumber
• psibase::MethodNumber

psibase::AccountNumber

struct psibase::AccountNumber {
    uint64_t value; // Number form

    AccountNumber(...); // Construct the empty name
    AccountNumber(...); // Construct from 64-bit value
    AccountNumber(...); // Construct from string (name)
    str(...);           // Get string (name)
    operator<=>(...);   // Comparisons
};

An account number or name.

Psibase account numbers are 64-bit values which are compressed from strings (names). Names may be 0 to 18 characters long and contain the characters a-z, 0-9, and - (hyphen). Non-empty names must begin with a letter.

There are some names which meet the above rules, but fail to compress down to 64 bits. These names are invalid. Likewise, there are some 64-bit integers which aren't the compressed form of valid names; these are also invalid.

The empty name "" is value 0.

psibase::AccountNumber::AccountNumber

psibase::AccountNumber::AccountNumber();

Construct the empty name.

psibase::AccountNumber::AccountNumber

explicit psibase::AccountNumber::AccountNumber(
    uint64_t value
);

Construct from 64-bit value.

This doesn't do any checking; if value isn't valid, then str will return a string which doesn't round-trip back to value.

psibase::AccountNumber::AccountNumber

explicit psibase::AccountNumber::AccountNumber(
    std::string_view s
);

Construct from string (name).

This does minimal checking; if s isn't valid, then str will return a string which doesn't match s. Many, but not all, invalid names produce value 0.

psibase::AccountNumber::str

std::string psibase::AccountNumber::str() const;

Get string (name).

psibase::AccountNumber::operator<=>

std::strong_ordering psibase::AccountNumber::operator<=>(
    const psibase::AccountNumber & 
) const;

Comparisons.

Compares by 64-bit value. This does not sort by the string (name) form.

psibase::MethodNumber

struct psibase::MethodNumber {
    uint64_t value; // Number form

    MethodNumber(...); // Construct the empty name
    MethodNumber(...); // Construct from 64-bit value
    MethodNumber(...); // Construct from string (name)
    str(...);          // Get string (name)
    operator<=>(...);  // Comparisons
};

A method number or name.

Psibase method numbers are 64-bit values which are compressed from strings (names). They contain the characters a-z, and 0-9. Non-empty names must begin with a letter. A-Z round-trips as a-z. _ (underscore) is dropped.

There are some names which meet the above rules, but fail to compress down to 64 bits. These names are invalid. Likewise, there are some 64-bit integers which aren't the compressed form of valid names; these are also invalid. Some invalid names fall back to a hash (below).

There is a special case when bit 48 is set. str() returns a string which begins with # followed by 16 letters. This is an alternative hex representation which represents the hash of a name which failed to compress. Once a name is in this form, it will round trip with no further changes.

The empty name "" is value 0.

psibase::MethodNumber::MethodNumber

psibase::MethodNumber::MethodNumber();

Construct the empty name.

psibase::MethodNumber::MethodNumber

explicit psibase::MethodNumber::MethodNumber(
    uint64_t v
);

Construct from 64-bit value.

This doesn't do any checking; if value isn't valid, then str will return a string which doesn't round-trip back to value.

psibase::MethodNumber::MethodNumber

explicit psibase::MethodNumber::MethodNumber(
    std::string_view s
);

Construct from string (name).

This does minimal checking; if s isn't valid, then str will return a string which doesn't match s. Many, but not all, invalid names produce a hashed value. Some produce 0.

psibase::MethodNumber::str

std::string psibase::MethodNumber::str() const;

Get string (name).

psibase::MethodNumber::operator<=>

std::strong_ordering psibase::MethodNumber::operator<=>(
    const psibase::MethodNumber & 
) const;

Comparisons.

Compares by 64-bit value. This does not sort by the string (name) form.

Native Functions

Native functions give services the ability to print debugging messages, abort transactions on errors, access databases and event logs, and synchronously call other services. There aren't many native functions since services implement most psibase functionality.

• Types For Native Functions
• Wrapped Native Functions
• Raw Native Functions

Types For Native Functions

• psibase::Action
• psibase::DbId

psibase::Action

struct psibase::Action {
    psibase::AccountNumber sender;  // Account sending the action
    psibase::AccountNumber service; // Service to execute the action
    psibase::MethodNumber  method;  // Service method to execute
    std::vector<char>      rawData; // Data for the method
};

A synchronous call.

An Action represents a synchronous call between services. It is the argument to call and can be fetched using getCurrentAction.

Transactions also contains actions requested by the transaction authorizers.

psibase::DbId

enum psibase::DbId: uint32_t {
    service             = 0, // Services should store their tables here
    writeOnly           = 1, // Data for RPC
    subjective          = 2, // The first 64 bits of the key match the service.
    nativeConstrained   = 3, // Tables used by native code
    nativeUnconstrained = 4, // Tables used by native code
    blockLog            = 5, // Block log
    historyEvent        = 6, // Long-term history event storage
    uiEvent             = 7, // Short-term history event storage
    merkleEvent         = 8, // Events which go into the merkle tree
    blockProof          = 9, // block signatures
    numDatabases        = 10, // Number of defined databases
};

Identify database to operate on.

Native functions expose a set of databases which serve various purposes. This enum identifies which database to use when invoking those functions.

psibase::DbId::service

Services should store their tables here.

The first 64 bits of the key match the service.

psibase::DbId::writeOnly

Data for RPC.

Write-only during transactions, and read-only during RPC. Individual nodes may modify this database, expire data from this database, or wipe it entirely at will.

The first 64 bits of the key match the service.

psibase::DbId::subjective

The first 64 bits of the key match the service..

psibase::DbId::nativeConstrained

Tables used by native code.

This database enforces constraints during write. Only writable by privileged services, but readable by all services.

Some writes to this database indicate chain upgrades. If a privileged service writes to a table that an older node version doesn't know about, or writes new fields to an existing table that an older node doesn't know about, then that node will reject the write. If the producers accepted the write into a block, then the node will stop following the chain until it's upgraded to a newer version.

psibase::DbId::nativeUnconstrained

Tables used by native code.

This database doesn't enforce constraints during write. Only writable by privileged services, but readable by all services.

psibase::DbId::blockLog

Block log.

Transactions don't have access to this, but RPC does.

psibase::DbId::historyEvent

Long-term history event storage.

Write-only during transactions, and read-only during RPC. Individual nodes may modify this database, expire data from this database, or wipe it entirely at will.

TODO: this policy may eventually change to allow time-limited read access during transactions.

Key is an auto-incremented, 64-bit unsigned number.

Value must begin with:

• 32 bit: block number
• 64 bit: service

Only usable with these native functions:

• putSequential
• getSequential

TODO: right now the value must begin with the service. Revisit whether beginning with the block number is useful.

psibase::DbId::uiEvent

Short-term history event storage.

These events are erased once the block that produced them becomes final. They notify user interfaces which subscribe to activity.

Write-only during transactions, and read-only during RPC. Individual nodes may modify this database, expire data from this database, or wipe it entirely at will.

Key is an auto-incremented, 64-bit unsigned number.

Value must begin with:

• 32 bit: block number
• 64 bit: service

Only usable with these native functions:

• putSequential
• getSequential

TODO: right now the value must begin with the service. Revisit whether beginning with the block number is useful.

psibase::DbId::merkleEvent

Events which go into the merkle tree.

TODO: read support; right now only RPC mode can read

Services may produce these events during transactions and may read them up to 1 hour (configurable) after they were produced, or they reach finality, which ever is longer.

Key is an auto-incremented, 64-bit unsigned number.

Value must begin with:

• 32 bit: block number
• 64 bit: service

Only usable with these native functions:

• putSequential
• getSequential

TODO: right now the value must begin with the service. Revisit whether beginning with the block number is useful.

psibase::DbId::blockProof

block signatures.

psibase::DbId::numDatabases

Number of defined databases.

This number may grow in the future

Wrapped Native Functions

These functions wrap the Raw Native Functions.

• psibase::abortMessage
• psibase::call
• psibase::check
• psibase::getCurrentAction
• psibase::getCurrentActionView
• psibase::getKey
• psibase::getResult
• psibase::getSequential
• psibase::getSequentialRaw
• psibase::kvGet
• psibase::kvGetOrDefault
• psibase::kvGetRaw
• psibase::kvGetSize
• psibase::kvGetSizeRaw
• psibase::kvGreaterEqual
• psibase::kvGreaterEqualRaw
• psibase::kvLessThan
• psibase::kvLessThanRaw
• psibase::kvMax
• psibase::kvMaxRaw
• psibase::kvPut
• psibase::kvPutRaw
• psibase::kvRemove
• psibase::kvRemoveRaw
• psibase::putSequential
• psibase::putSequentialRaw
• psibase::setRetval
• psibase::setRetvalBytes
• psibase::writeConsole

psibase::abortMessage

void psibase::abortMessage(
    std::string_view message
);

Abort with message.

Message should be UTF8.

psibase::call

std::vector<char> psibase::call(
    const psibase::Action & action
);

Call a service and return its result.

psibase::call

std::vector<char> psibase::call(
    const char * action,
    uint32_t     len
);

Call a service and return its result.

action must contain a fracpacked Action.

psibase::call

std::vector<char> psibase::call(
    psio::input_stream action
);

Call a service and return its result.

action must contain a fracpacked Action.

psibase::check

void psibase::check(
    bool             cond,
    std::string_view message
);

Abort with message if !cond.

Message should be UTF8.

psibase::getCurrentAction

psibase::Action psibase::getCurrentAction();

Get the currently-executing action.

This function unpacks the data into the Action struct. For large data, getCurrentActionView can be more efficient.

If the service, while handling action A, calls itself with action B:

• Before the call to B, getCurrentAction() returns A.
• After the call to B, getCurrentAction() returns B.
• After B returns, getCurrentAction() returns A.

Note: The above only applies if the service uses call. Actor uses call.

psibase::getCurrentActionView

psio::shared_view_ptr<Action> psibase::getCurrentActionView();

Get the currently-executing action.

This function creates a view, which can save time for large data. For small data, getCurrentAction can be more efficient.

If the service, while handling action A, calls itself with action B:

• Before the call to B, getCurrentAction() returns A.
• After the call to B, getCurrentAction() returns B.
• After B returns, getCurrentAction() returns A.

Note: The above only applies if the service uses call. Actor uses call.

psibase::getKey

std::vector<char> psibase::getKey();

Get key.

Other functions set the key.

psibase::getResult

std::vector<char> psibase::getResult();

Get result.

Other functions set result.

psibase::getResult

std::vector<char> psibase::getResult(
    uint32_t size
);

Get result when size is known.

Other functions set result.

Caution: this does not verify size.

psibase::getSequential

template<typename V, typename Type>
std::optional<V> psibase::getSequential(
    psibase::DbId                  db,
    uint64_t                       id,
    const psibase::AccountNumber * matchService = nullptr,
    const Type *                   matchType = nullptr,
    psibase::AccountNumber *       service = nullptr,
    Type *                         type = nullptr
);

Get a sequentially-numbered record, if available.

• If matchService is non-null, and the record wasn't written by matchService, then return nullopt. This prevents a spurious abort from mismatched serialization.
• If matchType is non-null, and the record type doesn't match, then return nullopt. This prevents a spurious abort from mismatched serialization.
• If service is non-null, then it receives the service that wrote the record. It is left untouched if the record is not available.
• If type is non-null, then it receives the record type. It is left untouched if either the record is not available or if matchService is not null but doesn't match.

psibase::getSequentialRaw

std::optional<std::vector<char>> psibase::getSequentialRaw(
    psibase::DbId db,
    uint64_t      id
);

Get a sequentially-numbered record, if available.

psibase::kvGet

template<typename V, typename K>
std::optional<V> psibase::kvGet(
    psibase::DbId db,
    const K &     key
);

Get a key-value pair, if any.

psibase::kvGet

template<typename V, typename K>
std::optional<V> psibase::kvGet(
    const K & key
);

Get a key-value pair, if any.

psibase::kvGetOrDefault

template<typename V, typename K>
V psibase::kvGetOrDefault(
    psibase::DbId db,
    const K &     key
);

Get a value, or the default if not found.

psibase::kvGetOrDefault

template<typename V, typename K>
V psibase::kvGetOrDefault(
    const K & key
);

Get a value, or the default if not found.

psibase::kvGetRaw

std::optional<std::vector<char>> psibase::kvGetRaw(
    psibase::DbId      db,
    psio::input_stream key
);

Get a key-value pair, if any.

psibase::kvGetSize

template<typename K>
std::optional<uint32_t> psibase::kvGetSize(
    psibase::DbId db,
    const K &     key
);

Get size of stored value, if any.

psibase::kvGetSize

template<typename K>
std::optional<uint32_t> psibase::kvGetSize(
    const K & key
);

Get size of stored value, if any.

psibase::kvGetSizeRaw

std::optional<uint32_t> psibase::kvGetSizeRaw(
    psibase::DbId      db,
    psio::input_stream key
);

Get size of stored value, if any.

psibase::kvGreaterEqual

template<typename V, typename K>
std::optional<V> psibase::kvGreaterEqual(
    psibase::DbId db,
    const K &     key,
    uint32_t      matchKeySize
);

Get the first key-value pair which is greater than or equal to key.

If one is found, and the first matchKeySize bytes of the found key matches the provided key, then returns the value. Use getKey to get the found key.

psibase::kvGreaterEqual

template<typename V, typename K>
std::optional<V> psibase::kvGreaterEqual(
    const K & key,
    uint32_t  matchKeySize
);

Get the first key-value pair which is greater than or equal to key.

If one is found, and the first matchKeySize bytes of the found key matches the provided key, then returns the value. Use getKey to get the found key.

psibase::kvGreaterEqualRaw

std::optional<std::vector<char>> psibase::kvGreaterEqualRaw(
    psibase::DbId      db,
    psio::input_stream key,
    uint32_t           matchKeySize
);

Get the first key-value pair which is greater than or equal to key.

If one is found, and the first matchKeySize bytes of the found key matches the provided key, then returns the value. Use getKey to get the found key.

psibase::kvLessThan

template<typename V, typename K>
std::optional<V> psibase::kvLessThan(
    psibase::DbId db,
    const K &     key,
    uint32_t      matchKeySize
);

Get the key-value pair immediately-before provided key.

If one is found, and the first matchKeySize bytes of the found key matches the provided key, then returns the value. Use getKey to get the found key.

psibase::kvLessThan

template<typename V, typename K>
std::optional<V> psibase::kvLessThan(
    const K & key,
    uint32_t  matchKeySize
);

Get the key-value pair immediately-before provided key.

If one is found, and the first matchKeySize bytes of the found key matches the provided key, then returns the value. Use getKey to get the found key.

psibase::kvLessThanRaw

std::optional<std::vector<char>> psibase::kvLessThanRaw(
    psibase::DbId      db,
    psio::input_stream key,
    uint32_t           matchKeySize
);

Get the key-value pair immediately-before provided key.

If one is found, and the first matchKeySize bytes of the found key matches the provided key, then returns the value. Use getKey to get the found key.

psibase::kvMax

template<typename V, typename K>
std::optional<V> psibase::kvMax(
    psibase::DbId db,
    const K &     key
);

Get the maximum key-value pair which has key as a prefix.

If one is found, then returns the value. Use getKey to get the found key.

psibase::kvMax

template<typename V, typename K>
std::optional<V> psibase::kvMax(
    const K & key
);

Get the maximum key-value pair which has key as a prefix.

If one is found, then returns the value. Use getKey to get the found key.

psibase::kvMaxRaw

std::optional<std::vector<char>> psibase::kvMaxRaw(
    psibase::DbId      db,
    psio::input_stream key
);

Get the maximum key-value pair which has key as a prefix.

If one is found, then returns the value. Use getKey to get the found key.

psibase::kvPut

template<typename K, NotOptional V>
void psibase::kvPut(
    psibase::DbId db,
    const K &     key,
    const V &     value
);

Set a key-value pair.

If key already exists, then replace the existing value.

psibase::kvPut

template<typename K, NotOptional V>
void psibase::kvPut(
    const K & key,
    const V & value
);

Set a key-value pair.

If key already exists, then replace the existing value.

psibase::kvPutRaw

void psibase::kvPutRaw(
    psibase::DbId      db,
    psio::input_stream key,
    psio::input_stream value
);

Set a key-value pair.

If key already exists, then replace the existing value.

psibase::kvRemove

template<typename K>
void psibase::kvRemove(
    psibase::DbId db,
    const K &     key
);

Remove a key-value pair if it exists.

psibase::kvRemove

template<typename K>
void psibase::kvRemove(
    const K & key
);

Remove a key-value pair if it exists.

psibase::kvRemoveRaw

void psibase::kvRemoveRaw(
    psibase::DbId      db,
    psio::input_stream key
);

Remove a key-value pair if it exists.

psibase::putSequential

template<typename Type, NotOptional V>
uint64_t psibase::putSequential(
    psibase::DbId          db,
    psibase::AccountNumber service,
    Type                   type,
    const V &              value
);

Add a sequentially-numbered record.

Returns the id.

psibase::putSequentialRaw

uint64_t psibase::putSequentialRaw(
    psibase::DbId      db,
    psio::input_stream value
);

Add a sequentially-numbered record.

Returns the id.

psibase::setRetval

template<typename T>
void psibase::setRetval(
    const T & retval
);

Set the return value of the currently-executing action.

psibase::setRetvalBytes

void psibase::setRetvalBytes(
    psio::input_stream s
);

Set the return value of the currently-executing action.

psibase::writeConsole

void psibase::writeConsole(
    const std::string_view & sv
);

Write message to console.

Message should be UTF8.

Raw Native Functions

This is the set of raw native functions (wasm imports). They are available for services to use directly, but we recommend using the Wrapped Native Functions instead.

• psibase::raw::abortMessage
• psibase::raw::call
• psibase::raw::getCurrentAction
• psibase::raw::getKey
• psibase::raw::getResult
• psibase::raw::getSequential
• psibase::raw::kvGet
• psibase::raw::kvGreaterEqual
• psibase::raw::kvLessThan
• psibase::raw::kvMax
• psibase::raw::kvPut
• psibase::raw::kvRemove
• psibase::raw::putSequential
• psibase::raw::setRetval
• psibase::raw::writeConsole

psibase::raw::abortMessage

void psibase::raw::abortMessage(
    const char * message,
    uint32_t     len
);

Abort with message.

Message should be UTF8.

psibase::raw::call

uint32_t psibase::raw::call(
    const char * action,
    uint32_t     len
);

Call a service, store the return value into result, and return the result size.

action must contain a fracpacked Action.

Use getResult to get result.

psibase::raw::getCurrentAction

uint32_t psibase::raw::getCurrentAction();

Store the currently-executing action into result and return the result size.

The result contains a fracpacked Action; use getResult to get it.

If the service, while handling action A, calls itself with action B:

• Before the call to B, getCurrentAction() returns A.
• After the call to B, getCurrentAction() returns B.
• After B returns, getCurrentAction() returns A.

Note: The above only applies if the service uses call. Actor uses call.

psibase::raw::getKey

uint32_t psibase::raw::getKey(
    const char * dest,
    uint32_t     destSize
);

Copy min(destSize, key_size) bytes of the most-recent key into dest and return key_size.

Other functions set or clear the key. getResult, getKey, and writeConsole are the only raw functions which leave the current result and key intact.

psibase::raw::getResult

uint32_t psibase::raw::getResult(
    const char * dest,
    uint32_t     destSize,
    uint32_t     offset
);

Copy min(destSize, resultSize - offset) bytes from result + offset into dest and return resultSize.

If offset >= resultSize, then skip the copy.

Other functions set or clear result. getResult, getKey, and writeConsole are the only raw functions which leave the current result and key intact.

psibase::raw::getSequential

uint32_t psibase::raw::getSequential(
    psibase::DbId db,
    uint64_t      id
);

Get a sequentially-numbered record.

If id is available, then sets result to value and returns size. If id does not exist, returns -1 and clears result.

psibase::raw::kvGet

uint32_t psibase::raw::kvGet(
    psibase::DbId db,
    const char *  key,
    uint32_t      keyLen
);

Get a key-value pair, if any.

If key exists, then sets result to value and returns size. If key does not exist, returns -1 and clears result. Use getResult to get result.

psibase::raw::kvGreaterEqual

uint32_t psibase::raw::kvGreaterEqual(
    psibase::DbId db,
    const char *  key,
    uint32_t      keyLen,
    uint32_t      matchKeySize
);

Get the first key-value pair which is greater than or equal to the provided key.

If one is found, and the first matchKeySize bytes of the found key matches the provided key, then sets result to value and returns size. Also sets key. Otherwise returns -1 and clears result. Use getResult to get result and getKey to get found key.

psibase::raw::kvLessThan

uint32_t psibase::raw::kvLessThan(
    psibase::DbId db,
    const char *  key,
    uint32_t      keyLen,
    uint32_t      matchKeySize
);

Get the key-value pair immediately-before provided key.

If one is found, and the first matchKeySize bytes of the found key matches the provided key, then sets result to value and returns size. Also sets key. Otherwise returns -1 and clears result. Use getResult to get result and getKey to get found key.

psibase::raw::kvMax

uint32_t psibase::raw::kvMax(
    psibase::DbId db,
    const char *  key,
    uint32_t      keyLen
);

Get the maximum key-value pair which has key as a prefix.

If one is found, then sets result to value and returns size. Also sets key. Otherwise returns -1 and clears result. Use getResult to get result and getKey to get found key.

psibase::raw::kvPut

void psibase::raw::kvPut(
    psibase::DbId db,
    const char *  key,
    uint32_t      keyLen,
    const char *  value,
    uint32_t      valueLen
);

Set a key-value pair.

If key already exists, then replace the existing value.

psibase::raw::kvRemove

void psibase::raw::kvRemove(
    psibase::DbId db,
    const char *  key,
    uint32_t      keyLen
);

Remove a key-value pair if it exists.

psibase::raw::putSequential

uint64_t psibase::raw::putSequential(
    psibase::DbId db,
    const char *  value,
    uint32_t      valueLen
);

Add a sequentially-numbered record.

Returns the id.

psibase::raw::setRetval

void psibase::raw::setRetval(
    const char * retval,
    uint32_t     len
);

Set the return value of the currently-executing action.

psibase::raw::writeConsole

void psibase::raw::writeConsole(
    const char * message,
    uint32_t     len
);

Write message to console.

Message should be UTF8.

Basic Rust Service

Not Yet Functional

This documents some of our current thoughts and ongoing development for Rust service support. The installation procedure and examples are currently non-functional.

Installation

Follow the Rust Installation Guide.

Getting Started

Run the following to create a project:

cargo new --lib example
cd example
cargo add psibase

This creates the following files:

example
├── Cargo.toml      Project configuration
├── Cargo.lock      Versions of dependencies
└── src
    └── lib.rs      Service source file

You don't need to manually make any changes to Cargo.toml to follow the examples in this book. Contrary to other Rust WASM guides, do not add a crate-type entry to Cargo.toml; cargo-psibase doesn't need it and it can cause problems in some situations. You should never edit Cargo.lock by hand.

Replace the content of lib.rs with the following. This is our initial service:

#[psibase::service]
mod service {
    #[action]
    fn add(a: i32, b: i32) -> i32 {
        a + b
    }

    #[action]
    fn multiply(a: i32, b: i32) -> i32 {
        a * b
    }
}

Deploying the Service

This, if you have a local test chain running, will:

• Build the service.
• Create the example account, if it doesn't already exist. The account won't be secured; anyone can authorize as this account without signing. Caution: this option should not be used on production or public chains. -i is a shortcut for --create-insecure-account.
• Deploy the just-built service on that account.

cargo psibase deploy -i example

Where's the pub?

The service module and the actions within it don't need to be public. Instead, the [psibase::service] macro generates public definitions which wrap the actions. You don't need to make the actions public to document them; the macro copies documentation from the action definitions to the generated definitions. It also copies documentation from the service module.

Psibase and Cargo

There are two related commands for interacting with psibase blockchains:

• The psibase utility knows how to interact with blockchains.
• cargo psibase builds, tests, and deploys Rust services on blockchains.

Here's an example of how they differ: psibase deploy has an argument which must point to an existing WASM. cargo psibase deploy builds and deploys the service identified by Cargo.toml.

Testing the Service

The next section, Testing Services covers testing our service.

Homework

There's a bug in both add and multiply. What is it? Even though it's still a bug, and can be exploitable in some situations, how is it less dangerous in Rust than it is in C++? How can you avoid it?

Testing Rust Services

TODO

JSON Format

Both C++ and Rust services support typed JSON serialization. C++ services use psio::to_json and psio::from_json. Rust services use serde_json.

Structs

Both psio and serde_json represent structs-with-fields as JSON objects.

Numbers

64-bit Numbers are incredibly tricky in JSON, thanks in part to JavaScript, and thanks in part to common JSON libraries in type-safe languages.

• JavaScript's number type can handle integers up to 53 bits unsigned, 54 signed. Extra precision is silently truncated. e.g. 10000000000000001 == 10000000000000000.
• JavaScript's BigInt type supports arbitrary precision, but JavaScript's built-in JSON conversions don't support it.

The cleanest workaround seems to be to store 64-bit integers in quoted strings, but several widely-used JSON libraries in type-safe languages decided against that workaround, and reject incoming quoted numbers. serde_json used to support it (input only), but hit some nasty conflicts and had to remove it. serde_json provides customization (serialize_with and deserialize_with), but that gets cumbersome in nested types, e.g. Option<Vec<u64>>.

Instead of trying to get type-safe JSON libraries to work around JavaScript's limitations, it's probably time we ask JavaScript to pull its own weight. JavaScript JSON Libraries exist which handle BigInt.

psio::to_json (C++) does not place 64-bit numbers in quoted strings, but for a time psio::from_json will accept numbers in quoted strings for backwards compatibility. serde_json (Rust) does not place or accept numbers in quoted strings, unless you use customization. JavaScript needs a JSON library or will silently truncate values.

• TODO: update psio::to_json to not quote numbers
• TODO: Rethink GraphQL numeric handling. It currently relies on to_json.
• TODO: update JavaScript code

Strings

psio::to_json and psio::from_json use JSON strings for std::string. serde_json uses JSON strings for rust's various string types.

Optional

Both psio (std::optional) and serde_json (Option) represent the empty case as null and the non-empty case as the inner type.

Vectors and Arrays

Both psio and serde_json represent vectors (std::Vector, Vec) and arrays (std::array, [] (Rust)) as JSON arrays.

Byte vectors and arrays

Byte vectors and arrays create a tricky problem. The JSON array-of-numbers representation is wasteful and annoying for this; hex strings are more compact and readable. Base-64 is even more compact, but it is near impossible to read and there are 7 standard RFC encodings, plus more.

When should a vector or array use a hex representation? It's convenient to automatically switch when the element type is an 8-bit number, but this can be jarring for new service developers who don't expect it. We could have a separate "bytes" type, but that's annoying for experienced developers in both Rust and C++. serdes already made a choice: vectors by default use the array notation in JSON. serialize_with and deserialize_with can opt into other representations, but they are cumbersome in nested types, e.g. Option<Vec<u8>>. That leaves a separate "bytes" type. Both Rust and C++ support move semantics to handle any efficiency issues.

• TODO: C++: A fixed-size bytes type
• TODO: C++: Switch existing byte vectors and arrays within psibase structs to bytes wrappers
• TODO: C++: Remove hex representation from std::vector<*> and std::array<*>
• TODO: C++: fracpack support for encoding psio::bytes as a vector instead of a struct of vector
• TODO: C++: fracpack support for encoding new fixed-size bytes type as an array instead of a struct of array
• TODO: Rust: A bytes type
• TODO: Rust: A fixed-size bytes type
• TODO: Rust: fracpack support for encoding bytes type as a vector instead of a struct of vector
• TODO: Rust: fracpack support for encoding fixed-size bytes type as an array instead of a struct of array

Tuples

Both psio and serde_json represent tuples as JSON arrays. The empty tuple has a problem. psio renders std::tuple<>{} as you'd expect: []. serde_json, however, renders (), the unit, as null.

• TODO: psio json support for tuples

Variants / Enums

There are probably as many ways to represent these in JSON as there are grains of sand on the beach. serde_json supports 4 approaches. Of these, only the externally tagged and adjacently tagged representations cover all situations unambiguously.

• TODO: pick one. It will be easier on rust devs if we choose serde_json's default (externally tagged). It looks like I can take advantage of the syntax of the externally-tagged option to represent nested types in the schema without falling back on a DSL.

Schema Format

Psibase has a schema format which describes the fracpack-format data and JSON-format data it uses for action arguments, event content, and database content.

TODO: implement schema

Type Definitions

Type definitions live in the userType array:

{
    "userType": [
        {definition},
        {definition},
        ...
    ]
}

Each definition has at least these fields:

• "name": name of the type
• Exactly one of the "alias", "structFields", or "unionFields" fields

A definition may also have these optional fields:

• "customJson", a boolean, indicates the type uses custom JSON serialization. We recommend against using this in most cases since it requires special handling in all serializers and deserializers. Psibase uses it for public and private keys, signatures, psibase::AccountNumber, and psibase::MethodNumber. The serialization libraries which communicate with psibase (e.g. the js library) support this set, but not additional ones. Only valid for structs.
• "definitionWillNotChange", a boolean, indicates the definition for this type will not change in the future. It opts into an alternative fracpack encoding which saves 2 bytes. Only valid for structs.
• "methods". Only valid for structs.

Alias Definitions

An alias definition creates an alternative name for a type:

{
    "name": "NameGoesHere",
    "alias": {type reference}
}

The syntax for referencing types appears below.

Struct Definitions

A struct definition has this form:

{
    "name": "NameGoesHere",
    "structFields": [
        {
            "name": "field1",
            "type": {type reference}
        },
        ...
    ]
}

Struct Upgradeability

The following change to a struct maintains backwards binary and JSON compatibility, but only if definitionWillNotChange isn't true:

• Add additional std::optional (C++) or Option (Rust) fields to the end of the struct

The following break backwards compatibility; if you do these, data will end up corrupted:

• Don't reorder or drop fields
• Don't add new fields at the beginning or middle
• Don't add new non-optional fields to the end
• Don't change anything if definitionWillNotChange (defaults to false) is true
• Don't change definitionWillNotChange

Union Definitions

A union definition describes an std::variant in C++ or an enum in Rust.

{
    "name": "NameGoesHere",
    "unionFields": [
        {
            "name": "alternative0",
            "type": {type reference}
        },
        ...
    ]
}

In C++, each type comes from the variant's type parameters. The names come from (TODO: variant name reflection support).

In Rust, both the names and the types come from the Rust definition:

#[derive(psibase::Schema)]
enum MyEnumType {
    Example0,                        // Type: TODO; need to define a C++ equivalent
    Example1(u64),                   // Type: u64
    Example2(u64, u32),              // Type: tuple of u64, u32
    Example3((u64,)),                // Type: tuple of u64 (extra parenthesis required)
    Example4 { foo: u64, bar: u32 }, // Type: a struct
    Example5(),                      // Type: empty tuple
    Example6(()),                    // Not supported
}

serde_json has an interesting gap. Tuples render as [...], but (), the unit, renders as null. There is no true empty tuple in Rust; () comes closest, but fills multiple roles in Rust, which may explain why serde_json chose null for it. Example5 opens up an opportunity since serde_json renders it as {"Example5":[]}. The schema treats it as an empty tuple because it renders like an empty tuple would, if serde_json supported empty tuples. Empty tuples are useful because the fracpack format for empty tuples supports adding new optional fields to them in a compatible way. e.g. the following definition is compatible with the above definition:

#[derive(psibase::Schema)]
enum MyEnumType {
    ...
    Example5((Option<u64>, )),  // No longer empty, but still compatible
    ...
}

The schema format doesn't support Example6 because serde_json renders it as {"Example6":null}.

The schema format doesn't support discriminants in Rust since fracpack numbers alternatives starting at 0 with no gaps. The schema format also doesn't support enum in C++; use std::variant instead.

// Not supported
#[derive(psibase::Schema)]
enum DoesNotWork {
    x = 4,
    y = 7,
}

Union Upgradeability

The following changes to a union maintain backwards binary and JSON compatibility:

• Add additional alternatives at the end of the enum (Rust) or variant (C++)
• Add additional std::optional (C++) or Option (Rust) items to the end of an alternative's tuple
• Add additional std::optional (C++) or Option (Rust) fields to the end of an alternative's struct

In addition, the following changes maintain backwards binary compatibility, but break JSON compatibility:

• Switch an alternative's type from a struct to a tuple, maintaining the order and types of the fields
• Switch an alternative's type from a tuple to struct, maintaining the order and types of the fields

The following break backwards compatibility; if you do these, data will end up corrupted:

• Don't drop alternatives from the enum (Rust) or variant (C++).
• Don't reorder alternatives within the enum (Rust) or variant (C++).
• Don't add additional alternatives to the beginning or middle of the enum (Rust) or variant (C++).
• Don't reorder or drop fields from an alternative's struct or tuple.
• Don't add new fields at the beginning or middle of an alternative's struct or tuple.
• Don't add non-optional fields to the end of an alternative's struct or tuple.
• Don't switch an alternative from a single type, e.g. Example(u64), to a tuple, e.g. Example((u64,)) or Example(u64,u32), or to a struct, e.g. Example{...}
• Don't switch an alternative from a tuple or a struct to a single type.
• Don't switch an alternative from no data to one with data, or vice-versa. You may switch from an empty tuple, e.g. Example(()), to a non-empty one containing optionals, e.g. Example(Option<u64>,Option<u32>).

Method Definitions

A struct definition may have methods on it.

{
    "name": "MyStruct",
    "structFields": [...],
    "methods": [
        {
            "name": "myMethod",
            "returns": {type reference},
            "args": [
                {
                    "name": "arg0",
                    "type": {type reference}
                },
                ...
            ]
        },
        ...
    ]
}

If a method doesn't return a value, returns should be {"builtinType": "void"}.

Method Upgradeability

The following changes maintain backwards binary and JSON compatibility:

• Add a new method
• Add an optional argument at the end of a method's existing arguments

The following break backwards compatibility; if you do these, data will end up corrupted, or history unreadable:

• Don't remove or rename methods; make them abort instead
• Don't change the return type of a method
• Don't add non-optional arguments to an existing method
• Don't add arguments to the beginning or middle of an existing method's argument list

Type References

We used {type reference} to indicate a type reference in the definitions above. This can be one of the following:

• {"builtinType": "u32"} - a built-in type
• {"userType": "Foo"} - a type defined in the userType array
• {"vector": {inner type}} - a vector of inner type
• {"optional": {inner type}} - an optional of inner type
• {"tuple": [{inner type}, ...]} - a tuple of inner types
• {"array": {inner type}, "size": 8} - a fixed-size array of inner type

Built-in types live in a separate namespace from user-defined types to minimize conflicts in the future if more built-in types are added.

Tuple Upgradeability

The following change to a tuple maintains backwards binary and JSON compatibility:

• Add additional std::optional (C++) or Option (Rust) inner types to the end of the tuple

The following break backwards compatibility; if you do these, data will end up corrupted:

• Don't reorder or drop fields from a tuple
• Don't add new fields at the beginning or middle from a tuple
• Don't add new non-optional fields to the end of a tuple

Built-in Types

{"builtinType":"..."} can name one of the following built-in types:

• void: only supported as a method return type
• bool
• u8, u16, u32, u64: unsigned integers
• i8, i16, i32, i64: signed integers
• f32, f64: floating-point types
• string

TODO

• events
• tables

Schema Schema

The schema schema defines both the JSON format and the binary (fracpack) format of schemas.

TODO...

fracpack

Psibase uses a new binary format, fracpack, which has the following goals:

• Quickly pack and unpack data, making it suitable for service-to-service communication, node-to-node communication, blockchain-to-outside communication, and database storage.
• Forwards and backwards compatibility; it supports adding new optional fields to the end of structs and tuples, even when they are embedded in variable-length vectors, fixed-length arrays, optional, and other structs and tuples.
• Option to read without unpacking (almost zero-copy); helps to efficiently handle large data.
• Doesn't require a code generator to support either C++ or Rust; macros and metaprogramming handle it.
• Efficient compression when combined with the compression algorithm from Cap 'n' Proto. Note: psibase doesn't currently use this.

Psibase uses fracpack for all of its message formats and uses it for database storage. Wherever psibase uses binary data, it's in fracpack format. There is one slight deviation: cryptographic types, even though they are normally stored in fracpack format, use a different binary format when encoded as a string (e.g. PUB_K1_898DAWuc...). Encoders and decoders take care of the extra conversion. e.g. when converting action arguments from json to binary, they decode the base-58 string, verify the checksum, then convert the resulting binary into fracpack format.

This document describes fracpack's binary format; it does not describe either C++'s or Rust's reflection, encoding, and decoding facilities.

Fixed-Size vs Variable-Size Objects

fracpack classifies objects into two categories: fixed size and variable size. The format for fixed-size objects is similar to what you'd get if you used memcpy, except subobjects, if any, are packed tightly without padding; fracpack doesn't align data. The format for variable-size objects is more complicated; see Variable-Size Objects.

Fixed-Size Objects

Numeric types

Numeric types are in twos-complement little-endian format. They are unaligned. This is the currently-supported set:

• Boolean: 1 byte; either 0 or 1
• Unsigned integer sizes (bits): 8, 16, 32, 64
• Signed integer sizes (bits): 8, 16, 32, 64
• Floating point: 32 and 64 bits; Intel format
• TODO: void; 0 bytes. This is for tagged-union alternatives which have no payload

Non-Extensible Fixed-Size Structs

A non-extensible struct is one that will never gain new fields in the future. Non-extensibility requires an explicit opt-in; fracpack considers structs extensible by default. fracpack's extensibility mechanism is unrelated to inheritance; fracpack doesn't model inheritance and its C++ implementation doesn't understand it.

A struct which is non-extensible and which contains only fixed-sized subobjects is itself fixed size. Its subobjects are packed in order without padding. A struct is variable size if it is extensible (the default) or if it contains any variable-sized data within it. Variable-size structs have a different encoding.

Fixed-Length Arrays of Fixed-Size Objects

A fixed-length array which contains fixed-size objects is itself fixed size. Its subobjects are packed in order without padding.

Variable-Size Objects

Variable-size objects contain two parts: a fixed-size (but sometimes growable) area and a variable-size area. Child objects live in one (or both) of these areas within the parent.

Variable-Size Structs

A struct is variable size if it is extensible (the default) or if it contains any variable-sized data within it. It has the following layout:

uint16_t            fixed_size;     // The amount of data in fixed_data
uint8_t[fixed_size] fixed_data;     // Fixed-size objects and offsets
                                    //   to variable-size objects
uint8_t[]           variable_data;  // Variable-size inner objects

Fixed-size subobjects live entirely within fixed_data. Variable-size subobjects have an Offset Pointer in fixed_data which points into variable_data. fixed_size, when present, enables decoders to safely skip newly-added optional fields. fixed_size is only present for extensible (the default) structs. Non-extensible structs don't have it since the fixed size is always known in that case.

Tuples

Tuples have the same encoding as structs, except they don't have a non-extensible option; tuples are always extensible, so they're always variable-size.

Offset Pointers

When a subobject is variable size, it has an Offset Pointer which points to the beginning of that subobject. Offset pointers are unsigned 32-bit integers which record the difference between the target address and the pointer's address. Values 0-3 point to within the offset pointer itself; they have special meaning:

• 0: indicates a vector or string is empty
• 1: indicates an optional is empty
• 2,3: reserved for future use

The special values are a space-saving measure. e.g. suppose we have an std::optional<std::string>. The 3 possible states, nullopt, empty, and !empty, are encoded as 1, 0, or an offset.

Fixed-Length Arrays of Variable-Size Objects

These have the following encoding:

uint8_t[fixed_size] fixed_data;     // Offsets to Variable-size objects
uint8_t[]           variable_data;  // Variable-size inner objects

fixed_size has a known size, so isn't recorded.

Vectors and Strings

Vectors and Strings have the following encoding:

uint32_t            fixed_size;     // The amount of data in fixed_data
uint8_t[fixed_size] fixed_data;     // Fixed-size objects or offsets to
                                    //   Variable-size objects
uint8_t[]           variable_data;  // Variable-size objects

fixed_size indirectly encodes the vector's length. If it's a vector of some type T, and that type is fixed size, then then vector length is fixed_size / T's size. If T is variable size, then the vector length is fixed_size / 4. Note that fixed_size here is 32 bits instead of 16, which is used in structs.

Optionals

Optionals always use a variable-size encoding, whether their inner data is fixed-size or variable size. An optional represents an empty state by using 1 for its Offset Pointer. It represents non-empty as an Offset Pointer which points to the contained data. If the contained data is variable-size, then it already uses an offset pointer; Optional reuses this pointer to save space.

There are 2 special rules for optionals embedded within structs and tuples:

• If any field is optional, then all the remaining fields must also be optional.
• If the last n optional fields are empty, then they must be omitted from fixed_data.

TODO: Implement these rules in Rust. Verify them in C++.

TODO: This rule is incompatible with non-extensible structs since fixed_size isn't present.

TODO: Reconsider this rule. Its purpose is to guarantee that decoding and re-encoding a fracpacked binary results in an identical binary. There are situations which prevent the guarantee, e.g. unrecognized but populated fields in an upgraded struct, tuple, or union, accidental NaN normalization, and potentially more. Psibase doesn't rely on this guarantee. It places a burden on both fracpack implementors and on users.

Tagged Unions

Tagged Unions (std::variant in C++ or enum in Rust) have the following encoding:

uint8_t         tag;    // Identifies the alternative; 0 is first
uint32_t        size;   // The amount of data
uint8_t[size]   data;   // Selected inner object

Alternatives are sequentially-numbered starting from 0. The size field allows decoders to safely skip newly-added alternatives that they are not aware of.

Safety Checking

Fracpack places additional requirements on packed data to enable quick safety checks while decoding.

• variable_data must be in the same order as fixed_data
• Offset pointers must leave no gaps, except when unknown fields are skipped (e.g. extensible structs, tuples, and unknown enum entries)
• Additional rules for Optionals
• Tagged Unions' tag field must be less than 128. This allows a potential extension in the future which will be signaled by the MSB.

TODO: finish this list. Make sure both the C++ and Rust implementations' checkers enforce the rules.

HTTP and Javascript

• TLDR: Pushing a transaction
• Routing and Virtual Hosts (http)
  • CORS and authorization (http)
• Native services
  • Push transaction (http)
  • Boot chain (http)
• Common services
  • Common files (http)
  • rootdomain and siblingUrl (js)
  • Pack transaction (http)
  • Simple RPC wrappers (js)
  • Conversions (js)
  • Transactions (js)
  • Signing (js)
  • Key Conversions (js)
  • React GraphQL hooks (js)
• Root services
• Service-provided services
  • Packing actions (http)
• Node administrator services
  • Peer management
  • Logging

TLDR: Pushing a transaction

This example works directly in the browser without bundling, transpiling, etc.

TODO: npm package which supports bundling

<!DOCTYPE html>
<html>
    <body>
        See the console

        <!-- type="module" enables es6 imports -->
        <!-- it also allows using await outside of functions -->
        <script src="script.mjs" type="module"></script>
    </body>
</html>

// Use these if your script is NOT hosted by psinode:
import { getTaposForHeadBlock, signAndPushTransaction }
    from 'http://psibase.127.0.0.1.sslip.io:8080/common/rpc.mjs';
const baseUrl = 'http://psibase.127.0.0.1.sslip.io:8080';

// Use these if your script is hosted by psinode:
//    import {getTaposForHeadBlock, signAndPushTransaction} from '/common/rpc.mjs';
//    const baseUrl = '';

try {
    const transaction = {
        tapos: {
            ...await getTaposForHeadBlock(baseUrl),
            // expire after 10 seconds
            expiration: new Date(Date.now() + 10000),
        },
        actions: [
            {
                sender: "sue",          // account requesting action
                service: "example",     // service executing action
                method: "add",          // method to execute
                data: {                 // arguments to method
                    "a": 0,
                    "b": 0
                }
            }
        ],
    };
    const privateKeys = [
        'PVT_K1_2bfGi9rYsXQSXXTvJbDAPhHLQUojjaNLomdm3cEJ1XTzMqUt3V',
    ];

    // Don't forget the await!
    const trace = await signAndPushTransaction(baseUrl, transaction, privateKeys);

    console.log("Transaction executed");
    console.log("\ntrace:", JSON.stringify(trace, null, 4));
} catch (e) {
    console.log("Caught exception:", e.message);
    if (e.trace)
        console.log(JSON.stringify(e.trace, null, 4));
}

Routing and Virtual Hosts (http)

psinode provides virtual hosting. Domains have 2 categories:

• root domain, e.g. psibase.127.0.0.1.sslip.io. This hosts the main page for the chain and also provides service-independent services.
• service domain, e.g. my-service.psibase.127.0.0.1.sslip.io. This hosts user interfaces and RPC services for individual services. Services must register for this service.

The above table describes how psinode normally routes HTTP requests. Only the highest-priority rule is fixed. The proxy-sys service, which handles the remaining routing rules, is customizable, both by distinct chains and by individual node operators.

CORS and authorization (http)

psinode always accepts CORS requests, since services would break without it. psinode does not currently handle any HTTP authentication or authorization.

Native services

psinode's native code handles any target which begins with /native, regardless of domain. Targets which begin with /native but aren't recognized produce a 404.

Push transaction (http)

POST /native/push_transaction pushes a transaction. The user must pack the transaction using fracpack and pass in the binary as the request body. See Pack transaction (http) for an RPC request which packs transactions. TODO: describe how to pack without using RPC; currently waiting for the transaction format to stabilize, for schema support, and for WASM ABI support.

If the transaction succeeds, or if the transaction fails but a trace is available, then psinode returns a 200 reply with a JSON body (below). If the transaction fails and a trace is not available, then it returns a 500 error with an appropriate message.

{
    "actionTraces": [...],  // Detailed execution information for debugging.
    "error": "..."          // Error message. Field will be empty or missing on success.
    // TODO: events?
}

If a transaction succeeds, the transaction may or may not make it into a block. If it makes it into a block, it may get forked back out. TODO: add lifetime tracking and reporting to psinode.

Future psinode versions may trim the action traces when not in a developer mode.

Boot chain (http)

POST /native/push_boot boots the chain. This is only available when psinode does not have a chain yet. Use the psibase boot command to boot a chain. TODO: document the body content.

Common services

• Common files (http)
• rootdomain and siblingUrl (js)
• Pack transaction (http)
• Simple RPC wrappers (js)
• Conversions (js)
• Transactions (js)
• Signing (js)
• Key Conversions (js)
• React GraphQL hooks (js)

The common-sys service provides services which start with the /common* path across all domains. It handles RPC requests and serves files.

Common files (http)

common-sys serves files stored in its tables. Chain operators may add files using the storeSys action (psibase upload). psibase boot installs this default set of files while booting the chain:

rootdomain and siblingUrl (js)

getRootDomain calls the /common/rootdomain/ endpoint, which returns the root domain for the queried node (e.g. psibase.127.0.0.1.sslip.io). The result is cached so subsequent calls will not make additional queries to the node.

siblingUrl makes it easy for scripts to reference other services' domains. It automatically navigates through reverse proxies, which may change the protocol (e.g. to HTTPS) or port (e.g. to 443) from what psinode provides. baseUrl may point within either the root domain or one of the service domains. It also may be empty, null, or undefined for scripts running on webpages served by psinode.

Example uses:

• siblingUrl(null, '', '/foo/bar'): Gets URL to /foo/bar on the root domain. This form is only usable by scripts running on webpages served by psinode.
• siblingUrl(null, 'other-service', '/foo/bar'): Gets URL to /foo/bar on the other-service domain. This form is only usable by scripts running on webpages served by psinode.
• siblingUrl('http://psibase.127.0.0.1.sslip.io:8080/', '', '/foo/bar'): Like above, but usable by scripts running on webpages served outside of psinode.
• siblingUrl('http://psibase.127.0.0.1.sslip.io:8080/', 'other-service', '/foo/bar'): Like above, but usable by scripts running on webpages served outside of psinode.

Pack transaction (http)

/common/pack/Transaction and /common/pack/SignedTransaction use fracpack to convert unsigned and signed transactions to binary. They accept JSON as input and return the binary data.

Transaction has these fields:

{
  "tapos": {
    "expiration": "..." // When transaction expires (UTC)
                        // Example value: "2022-05-31T21:32:23Z"
                        // Use `new Date(...)` to generate the correct format.
  },
  "actions": [],        // See Action
  "claims": []          // See Claim
}

TODO: document additional tapos fields once they're operational

SignedTransaction has these fields:

{
  "transaction": {},    // This may be the Transaction object (above),
                        // or it may be a hex string containing the packed
                        // transaction.
  "proofs": []          // See Proof
}

Action has these fields. See Packing actions (http).

{
  "sender": "...",      // The account name authorizing the action
  "service": "...",     // The service name to receive the action
  "method": "...",      // The method name of the action
  "rawData": "..."      // Hex string containing action data (arguments)
}

Claim has these fields. See Signing (js) to fill claims and proofs.

{
  "service": "...",     // The service which verifies the proof meets
                        // the claim, e.g. "verifyec-sys"
  "rawData": "..."      // Hex string containing the claim data.
                        // e.g. `verifyec-sys` expects a public key
                        // in fracpack format.
}

Proof is a hex string containing data which proves the claim. e.g. verifyec-sys expects a signature in fracpack format. See Signing (js) to fill claims and proofs.

Simple RPC wrappers (js)

/common/rpc.mjs exports a set of utilities to aid interacting with psinode's RPC interface.

Conversions (js)

/common/rpc.mjs exports these conversion functions.

Transactions (js)

/common/rpc.mjs exports these functions for packing and pushing transactions.

Signing (js)

/common/rpc.mjs exports these functions for signing and pushing transactions

signTransaction signs a transaction; it also packs any actions if needed. baseUrl must point to within the root domain, one of the service domains, or be empty or null; see rootdomain and siblingUrl (js). transaction must have the following form:

{
    tapos: {
        refBlockIndex: ...,   // Identifies block
        refBlockSuffix: ...,  // Identifies block
        expiration: "..."     // When transaction expires (UTC)
                              // Example value: "2022-05-31T21:32:23Z"
                              // Use `new Date(...)` to generate the correct format.
    },
    actions: [],        // See below
}

Action has these fields:

{
  sender: "...",    // The account name authorizing the action
  service: "...",   // The service name to receive the action
  method: "...",    // The method name of the action

  data: {...},      // Method's arguments. Not needed if `rawData` is present.
  rawData: "...",   // Hex string containing packed arguments. Not needed if `data` is present.
}

privateKeys is an array which may contain a mix of:

• Private keys in text form, e.g. "PVT_K1_2bfGi9r..."
• {keyType, keyPair}. See Key Conversions (js).

Key Conversions (js)

/common/keyConversions.mjs exports functions which convert Elliptic KeyPair objects and Elliptic Signature objects to and from psibase's text and fracpack forms. Each function accepts or returns a {keyType, keyPair} or {keyType, signature}, where keyType is one of the following values:

export const KeyType = {
  k1: 0,
  r1: 1,
};

Here are example private and public keys in text form:

PVT_K1_2bfGi9rYsXQSXXTvJbDAPhHLQUojjaNLomdm3cEJ1XTzMqUt3V
PUB_K1_6MRyAjQq8ud7hVNYcfnVPJqcVpscN5So8BhtHuGYqET5BoDq63

PVT_R1_fJ6ASApAc9utAL4zfNE4qwo22p7JpgHHSCVJ9pQfw4vZPXCq3
PUB_R1_7pGpnu7HZVwi8kiLLDK2MJ6aYYS23eRJYmDXSLq5WZFCN6WEqY

TODO: even though the JS library supports both k1 and r1 types, psibase only currently supports k1.

React GraphQL hooks (js)

TODO

Root services

TODO

Service-provided services

TODO

Packing actions (http)

TODO

Node administrator services

Peer management

/native/admin/peers lists the currently connected peers.

Each peer has the following fields:

/native/admin/connect /native/admin/disconnect

Logging

• Console Logger
• File Logger
• Syslog Logger (TODO)
• Websocket Logger

/native/admin/loggers provides GET and PUT access to the server's logging configuration.

The body is a JSON object which has a field for each logger. The name of the logger is only significant to identify the logger. When the log configuration is changed, if the new configuration has a logger with the same name and type as one in the old configuration, the old logger will be updated to the new configuration without dropping or duplicating any log records.

{
    "console": {
        "type": "console",
        "filter": "%Severity% >= info",
        "format": "[%TimeStamp%] [%Severity%]: %Message%"
    }
}

All loggers must have the following fields:

Additional fields are determined by the logger type.

Console logger

The console logger writes to the server's stderr. It does not use any additional configuration. There should not be more than one console logger.

File logger

The file logger writes to a named file and optionally provides log rotation and deletion. Multiple file loggers are supported as long as they do not write to the same files.

filename and target can contain patterns which will be used to generate multiple file names. The pattern should result in a unique name or old log files may be overwritten. The paths are relative to the server's root directory.

Both rotation and log deletion trigger when any condition is reached.

When log files are deleted, the oldest logs will be deleted first. All files that match the target pattern are assumed to be log files and are subject to deletion.

Example:

{
    "file-log": {
        "type": "file",
        "filter": "%Severity >= info%",
        "format": "[%TimeStamp%]: %Message%",
        "filename": "psibase.log",
        "target": "psibase-%Y%m%d-%N.log",
        "rotationTime": "00:00:00Z",
        "rotationSize": 16777216,
        "maxSize": 1073741824
    }
}

Websocket logger

/native/admin/log is a websocket endpoint that provides access to server logs as they are generated. Each message from the server contains one log record. Messages sent to the server should be JSON objects representing the desired logger configuration for the connection.

The server begins sending log messages after it receives the first logger configuration from the client. The client can change the configuration at any time. The configuration change is asynchronous, so the server will continue to send messages using the old configuration for a short period after client sends the update but before the server processes it.

Applets

• What is an applet
• Decentralized front-end
• Security model
  • Security model comparison
  • Security model implications
• Scaling state and computation

What is an applet?

An applet is the name for the client-side code served to the browser by a Psinode and inherently coupled to an account on a psibase blockchain. An applet could be as simple as a static web page, or it could be a dynamic interface to the service running on its corresponding account on the blockchain. Since applets may be linked to services, they are solely responsible for presenting any information that would be relevant to a user trying to use the service. Anyone may write traditional front-end user-interfaces that interact with psibase chains, but they will be unable to take advantage of the robust infrastructure provided to all front-ends that are deployed as psibase applets, such as permission and key management services.

Decentralized front-end

If applets are the front-ends to blockchain services, then any full-node running the blockchain also contains all front-ends to those services. That means that the front-ends are fully decentralized, and there is no single server that could shut down access to that service. Even if certain full-nodes are attempt to censor the applet for some services, users could simply find another full-node from which to request the applet, or they could run their own full node to gain full access to all blockchain applets and services.

This also means that for each applet deployed by a developer, there will be numerous web addresses & domain names that all point to the same applet, just as there are many API endpoints to which a user can submit transactions on all previous-generation blockchains.

Security model

An applet is served by its own service, and it is therefore capable of executing actions on its own service on behalf of the user without requiring the user to manually grant permission. It's functionally an extension of the service itself.

But, for an applet to execute actions and operations on another applet would require that the user gives permission to the originating applet to act on their behalf within the context of another applet.

The core infrastructure provided for applet development handles these permission requests automatically so that applets can simply focus on writing the logic that performs the necessary operations. All permissions management and authentication will be automatically facilitated for all applets that are served from a psibase chain.

Security model comparison

Below shows the comparison between a sample vote interaction in a previous generation blockchain application vs a psibase applet.

sequenceDiagram
title Previous-gen vote action

actor Alice
participant Frontend
participant Authenticator
participant Blockchain

Alice->>Frontend: Click "vote"
Frontend->>Authenticator: Alice wants to Vote
Authenticator->>Alice: Do you authorize this vote?
Alice-->>Authenticator: Yes
Authenticator->>Blockchain: Vote
Blockchain-->>Authenticator: return
Frontend-->>Alice: Voted!

sequenceDiagram
title Psibase vote action

actor Alice
participant Applet
participant Blockchain

Alice->>Applet: Click "vote"
Applet->>Blockchain: Vote
Blockchain-->>Applet: return
Applet-->>Alice: Voted!

Hopefully this makes it clear that the steps needed to use psibase applets have the potential to be minimized to produce a seamless and immersive UX that meets the expectations of users of traditional (non-blockchain) web applications. Note that the above example implies that "vote" is an action on the blockchain Service that is coupled to the Applet. If the action is part of a third-party service unrelated to the applet, then the user is still required to give manual permission for the applet to submit the relevant action to the blockchain.

Security model implications

The positive implications of this security model are significant. The majority of the breakdown in user-experience in other "Web 3" applications comes from the need for a user to use external Authenticators and authenticators to grant front-ends permission to sign transactions on their behalf. In psibase blockchains, all such signature requests are eliminated for any transactions that only involve the applet updating the blockchain state for its own service. psibase applications therefore have the potential to offer users the most seamless possible user-experience out of all decentralized applications.

Scaling state and computation

Applets have the ability to not only store data in blockchain state through services, but they are able to store data in local browser-storage on the client-side. Imagine, for example, a contact-list applet that stored the "friends" of a particular user. It would be wasteful and a potential breach of privacy to store such a list of friends on-chain. In psibase, a contact-list applet could simply store the account names of a user's friends in browser-local storage. That list of friends could then be used, with user permission, by other third-party applets where it would be helpful to have a list of "friends" for a given account (such as games, or instant-messaging, etc.). Therefore, psibase applets not only have the ability to leverage infrastructure to to share public storage (the psibase blockchain), but they can request that the user grant them access to additional data stored client-side. This allows for much more data-rich applications to be built for a much lower-cost, as the data need not all be stored on the blockchain side.

Similarly, computation may be performed client-side that drastically minimizes the computation needed to be done on-chain. The psibase infrastructure includes libraries that facilitate inter-applet communication to allow applets to pass messages between each other, perform computation, store data, and interact with external services purely on the client-side. This means it is also easy to reuse operations defined in one applet in another applet, allowing for massively composable applications to be built that need not "reinvent the wheel" each time they need to perform an operation that is already performed by another applet.

For example, if a blockchain service requires that actions are always called on it in a particular order, the corresponding applet for that service may define a single operation that takes the parameters necessary to call both actions in the correct order, and expose that operation to other client-side applets. Third-party applets, in this case, would not ever need to know the requirement that actions are called in a particular order, and would instead rely on the discrete operation defined in the applet. For more information on this inter-applet communication, see Inter-applet communication.

Applet interfaces

• What are they
  • Operation example
• Queries
  • Query example
• Security
• Error handling
• Event sourcing
• Message routing
• Composability
• Action scripts

What are they?

Applets must define, in a structured way, the operations they know how to perform. If done correctly, these operations can not only be used internally to the applet who owns the operations, but these operations could be called by any other applet loaded client-side on a user's browser. This allows one applet to create an interface defining interactions with its own local state, its service, other applet local state, or other services. This drastically increases the level of abstraction it is possible for applets on a psibase blockchain to provide each other.

Operation Example

To create a new symbol, it is necessary to first transfer an amount of tokens to the symbol service that is sufficient to cover the cost of the new tokens. But rather than forcing every other applet for whom the process of creating a new symbol is part of their own operational requirements to independently learn the process needed to create a symbol, the symbol service can simply define an operation that performs the correct sequence to create a symbol. If it does so, then it effectively abstracts the complexity of symbol creation from the outside world, who now only need to know that they call the "Create" operation on the symbol applet.

// applet/symbol-sys/index.js

initializeApplet(async () => {
    setOperations([
        {
            id: "create",
            exec: async (sender, {symbol}) => {
                let maxDebit = getCurrentSymbolPrice() * 1.1;
                let tokenApplet = new AppletId("token-sys", "");
                operation(tokenApplet, "credit", {
                    symbol: "PSI",
                    receiver: "symbol-sys",
                    amount: maxDebit,
                    memo: "To cover cost of symbol creation",
                });
                action("symbol-sys", "create", {symbol, maxDebit});
            },
        }
    ]);
});

In the above sample implementation of the symbol-sys:create operation, there is not only a call to the symbol-sys:create action on the symbol service, but there is also a call to the token-sys:credit operation, which is an operation defined by another applet. In Psibase, tokens and symbols are separate. In fact, tokens know nothing about symbols, but there is a standard symbol service which knows about tokens. So if one was to attempt to credit tokens to another by submitting an action directly to the token service, it would first be necessary to look up the tokenId associated with a particular symbol in the symbol service. But in this case, the token-sys applet defined an operation that accepts a symbol in order to hide that complexity from all other applets.

Hopefully this starts to elucidate how the concept of client-side operations can be used to improve abstraction and composability in psibase services & applets beyond what is possible in other blockchain development environments.

Queries

Queries allow applets to query each other for information on the client side. In order to return the result that the caller is looking for, it may be necessary for an applet to reference chain state or local (client-side) storage. The applet being queried always knows which applet was the sender of the request, and can therefore may want to prompt the user to explicitly allow the sharing of data if the data is potentially sensitive. Both operations and queries can return objects. The only difference between operations and queries is that queries are not allowed to call action() or operation(), and may only call another query().

Query example

The account-sys applet defines a mechanism to allow other applets to ask for the name of the currently logged-in user, using something similar to the following definition:

// applet/account-sys/index.js

initializeApplet(async () => {
    setQueries([
        {
            id: "getLoggedInUser",
            exec: ({sender}) => {
                return users.find(u=>u.loggedIn === true);
            },
        }
    ]);
});

In the above example implementation, the sender is not checked, and therefore any applet would be permitted to know the name of the currently logged-in user. This is just an example, however, the actual account-sys applet will prompt the user to explicitly confirm or deny the right of another applet to determine the name of the currently logged-in user.

Security

From the perspective of each applet, the origin of the window on whom postMessage is called during all inter-applet communication is restricted to be the common-sys applet. That means it is not possible for an attacker applet to embed your applet within their own and mimic common-sys.

In common-sys, each time an applet is opened in an iFrame, all incoming messages from it are restricted to the domain listed in the initial src property of the iFrame. Therefore, if the iFrame navigates away to a different origin, common-sys will reject all messages from it. Furthermore, all messages sent from core to a particular iFrame are explicitly restricted such that they will only be processed if the origin of the iFrame matches the origin when the iFrame was first opened.

Error handling

Both operation() and query() return a javascript Promise object to the caller. If there are any errors thrown during the execution of an operation or query, the errors will be routed back to the reject path of the corresponding Promise. That means callers can simply wrap their calls to operation() and query() in a standard try/catch block to listen for errors.

// applet/my-applet/index.js

try
{
    operation(tokenApplet, "credit", {
        symbol: "PSI",
        receiver: "dlarimer",
        amount: 5,
        memo: "Happy birthday!",
    });
} 
catch (e) 
{
    console.error(e);
}

Event sourcing

It is possible that the user may be using an applet to construct a transaction, rather than wanting to construct it and immediately push it to chain (for example if trying to propose a multisignature transaction).

This means that an applet should not be written to expect a callback when an operation is executed. Instead, psibase blockchains use an event-sourcing model, where an applet should subscribe to the on-chain events that it cares about, and it will be automatically notified when those events are emitted.

Message routing

Whenever we call operation(), action(), or query() inside an applet, Window.postMessage is used to send a specific message to the common-sys window.

According to the Window.postMessage documentation:

After postMessage() is called, the MessageEvent will be dispatched only after all pending execution contexts have finished.

Therefore postMessage in the below diagram does not immediately post to the other window. Instead it schedules a payload to be dispatched after the completion of all remaining execution contexts. Below is a detailed diagram showing how messages are routed from one applet, through the common-sys infrastructure, to another applet, and back.

sequenceDiagram

title Client message routing: Operation

participant applet_1
participant common_sys
participant applet_2
participant chain

applet_1->>applet_1: Schedule postMessage(applet_2:operation1)

note over applet_1: Waiting on IO<br>(Execution Contexts finished)
applet_1->>common_sys: applet_2:operation1

note over common_sys: ops: [<br> applet_1<br>]
common_sys->>chain: request applet 2
chain-->>common_sys: 
note over common_sys: waits for applet_2 to load
common_sys->>common_sys: Schedule postMessage(applet_2:operation1)

note over common_sys: Waiting <br> (Execution Contexts finished)
common_sys->>applet_2: operation1
applet_2->>applet_2: Schedule postMessage(applet_2:action1)

note over applet_2: Waiting on IO<br>(Execution Contexts finished)
applet_2->>common_sys: applet_2:action1
common_sys->>common_sys: Schedule transaction in 100ms: [applet_2:action1]
common_sys->>common_sys: Schedule postMessage(applet_2:action1_reply)

note over common_sys: Waiting <br> (Execution Contexts finished)
common_sys->>applet_2: action1Reply
applet_2->>applet_2: schedule(applet_1:operation1Reply)

note over applet_2: Waiting on IO<br>(Execution Contexts finished)
applet_2->>common_sys: applet_1:operation1Reply
note over common_sys: ops: []
common_sys->>common_sys: Schedule postMessage(applet_1:operation1Reply)

note over common_sys: Waiting<br>(Execution Contexts finished)
common_sys->>applet_1: operation1Reply

note over applet_1,chain: 

note over common_sys: 100ms passed with no operations
common_sys->>chain: pushTransaction(transaction)
chain-->>common_sys: 

note over chain: transaction finished

Queries work the same way as operations, except that the handlers for queries cannot call action() or operation(), but can only call another query().

Composability

The existence of a framework that permits front-ends to communicate with each other in an authenticated way is very powerful. This enabled a level of application composability on the front-end that could greatly simplify the efforts of a developer who is attempting to build a powerful application. The next diagram shows how the author of a TokenCreator applet only needs to implement very minimal code in order to achieve what is ultimately quite a complex operation.

sequenceDiagram

actor Alice
participant TokenCreator applet
participant Common sys
participant Token applet
participant Symbol applet
participant Account applet
participant Psibase blockchain

title Alice using third-party app to create a token with a symbol & custom configuration
Alice->>TokenCreator applet: Submits form with new token<br> characteristics and symbol name


TokenCreator applet->>Common sys: [Operation-1] CreateToken
Common sys->>Token applet: 
activate Token applet
Token applet->>Symbol applet: [Operation-1A] BuySymbol
    activate Symbol applet
    Symbol applet->>Common sys: [Action] token-sys:credit
    Symbol applet->>Common sys: [Action] symbol-sys:buySymbol
    Symbol applet-->>Token applet: end [operation-1A]
    deactivate Symbol applet
Token applet->>Common sys: [Action] symbol-sys:mapToNewToken
Token applet-->>Common sys: end [operation-1]
deactivate Token applet

note over Common sys: 3 actions to submit
Common sys->>Token applet: For token-sys:credit<br>Get permission for Symbol to call<br>credit on Alice's behalf
Token applet-->>Common sys: 
Common sys->>Symbol applet: For symbol-sys:mapToNewToken<br>Get permission for Token to call<br>mapToNewToken transcript on Alice's behalf
Symbol applet-->>Common sys: 
Common sys->>Account applet: Sign transaction
Account applet-->>Common sys: 
Common sys->>Psibase blockchain: Submit transaction
Psibase blockchain-->>Common sys: 
Common sys-->>TokenCreator applet: end [Operation-1]
TokenCreator applet->>TokenCreator applet: Listens for SymbolMapped event
TokenCreator applet-->>Alice: Done

This could have important implications for DeFi and other industries where composability is an important factor.

Action Scripts

• What are action scripts
• Use case
  • Example
• Security
  • Application groups
  • Future extension outside application groups
• Atomicity

What are action scripts

In most blockchains, every function called on an on-chain smart-contract is called a "transaction." In some other blockchains, calling a function on chain requires one "action," and a "transaction" may be comprised of one or many such actions. On these chains, a transaction can be viewed as a kind of "script" with one very basic functionality: execute an ordered list of actions.

In psibase, this concept of a script has been further generalized, and is known as an "Action Script". Transactions in psibase blockchains may contain a list of actions, but some actions may themselves be scripts written to execute an ordered list of other actions. Such scripts are called Action Scripts. What differentiates Action Scripts from regular actions? Action scripts enable intelegent, dynamic transactions that act from the perspective of the user rather than the perspective of an application. Regular actions may only execute inline actions as the running application, whereas an action called from an Action Script appears to originate from the user.

Use case

An applet may pack several actions into a single transaction to execute one overall user-initiated operation. But consider the case where the intermediate value of one action is required as a parameter for a subsequent action. Without Action Scripts, it would be required that the execution of this operation be split into two separate transactions, and the client would need to read the intermediate state from the chain before submitting the second transaction.

In psibase blockchains, an Action Script may be written to bundle all the actions together as inline-actions. Intermediate state or return values of prior actions can be read and fed into subsequent actions within the Action Script, and all inline actions executed may run as the user, as though they were submitted as part of an overall transaction.

Example

Consider an applet that wants to create a new token with a symbol. A user is presented with a form that lets them select the properties of the new token, the name of the symbol, and a button labeled, "Create."

To accomplish this, the following steps would be required:

1. create@token-sys: Create a new token
2. Look up the nft created by the token contract that represents ownership of the new token
3. debit@nft-sys: Debit the token nft
4. Look up how much a new symbol costs
5. credit@token-sys: Send the cost of a new symbol to the symbol contract
6. create@symbol-sys: Tell the symbol contract to create a new symbol (it will debit the tokens sent in the previous step)
7. Look up the NFT generated that represents the ownership of the new symbol
8. debit@nft-sys: Debit the symbol NFT
9. credit@nft-sys: Transfer the symbol NFT back to the symbol contract
10. Look up the ID of the newly created token
11. mapToken@symbol: Map the symbol to the new token ID

Notice that the lookup in steps 2 and 7 each require that prior actions were completed, and are required to run the rest of the actions in this sequence. Regardless of any arrangement of these actions and operations, the minimum number of transactions required to run this sequence without Action Scripts would be 2. The second transaction could not even be constructed until the first one completed. This delay means various user errors could result in partial execution of the sequence, leaving the user with a token without a symbol, or a symbol without a token.

With Action Scripts, the entire sequence can be shrunk down to:

1. Look up how much a new symbol costs
2. credit@token-sys: Send the cost of a new symbol to the symbol contract
3. create@symbol-sys: Create a new symbol
4. createAndMap@tokenSys: Creates a new token, and map it to the newly created symbol

This sequence contains only three actions which can be fit in a single transaction which will execute atomically.

To implement the createAndMap@tokenSys Action Script, the c++ code would be significantly simpler, due to the ability to receive action return values.

// tokenSys.cpp (pseudocode)

void TokenSys::createAndMap(Precision p, Quantity maxSupply, SID symbolId)
{
    // Create new token, and give ownership to sender
    auto tid = create(p, maxSupply);
    auto tokenNft = getToken(tid).ownerNft;
    senderAt<NftSys>().debit(tokenNft, "Debit token ownership NFT");

    // Create new symbol
    auto cost = to<SymbolSys>().getCost(symbolId.str().size());
    senderAt<SymbolSys>().create(symbolId, cost);
    auto symbolNft = to<SymbolSys>().getSymbol(symbolId).ownerNft;
    senderAt<NftSys>().debit(symbolNft, "Take ownership of new symbol");

    // Map symbol to token
    senderAt<NftSys>().credit(symbolNft, SymbolSys::service, "Give symbol to symbol-sys");
    senderAt<SymbolSys>().mapToken(tid, symbolId);
}

Security

Action Scripts are able to run inline actions as the user. In order to prevent Action Scripts from using other applications on behalf of a user without their permission, the user is required to submit the list of applications that the Action Script is permitted to use on their behalf when they submit an Action Script in a transaction. If the Action Script attempts to call an inline action as the user on an application that was not explicitly specified, it will fail the entire transaction.

Application groups

What applications is the user allowed to specify when calling an Action Scripts? If a user could specify anything, then malicious front-ends may be able to trick users into signing an Action Script that acts on their behalf in ways that the user didn't expect (and wouldn't permit).

But if an Action Script could only run actions in its own Application, then its usefulness would be severely limited. To address this, psibase blockchains have a capability similar to that provided in other operating systems, called "Application Groups."

If two or more applications each identify as part of the same application group, then these applications can be trusted by each other, as though they were all part of a single larger application. An Action Script is therefore not only permitted to operate on its own application, but also on any application with whom it shares an application group, as long as the user explicitly specifies each of the required applications when constructing the Action Script.

Future extension outside application groups

In the future, it may be possible to allow end users to be prompted for permission if an Action Script wants access to applications outside of its application group. Due to the implications of allowing such a capability, it's currently not provided until Action Scripts are deemed sufficiently usable and secure.

Atomicity

Action Scripts allow application developers to provide a special interface to applets that abstract the complexity of running transactions that contain actions dependent on the state update generated by the successful execution of prior actions.

The ability to run such sequences atomically means that they will all fail or succeed together, ensuring that the user is not left in a partially completed state. Action Scripts therefore reduce errors and increase developer and user experience.

transact-sys

• SystemService::TransactionSys
• SystemService::ServiceMethod
• SystemService::AuthInterface

SystemService::TransactionSys

struct SystemService::TransactionSys {
    const psibase::AccountNumber service;      // "transact-sys"
    const uint64_t               serviceFlags; // Flags this service must run with

    init(...);           // Only called once during chain initialization
    startBlock(...);     // Called by native code at the beginning of each block
    runAs(...);          // Run `action` using `action.sender's` authority
    getTransaction(...); // Get the currently executing transaction
    currentBlock(...);   // Get the current block header
    headBlock(...);      // Get the head block header
    headBlockTime(...);  // Get the head block time
};

All transactions enter the chain through this service.

This privileged service dispatches top-level actions to other services, checks TAPoS, detects duplicate transactions, and checks authorizations using SystemService::AuthInterface.

Other services use it to get information about the chain, current block, and head block. They also use it to call actions using other accounts' authorities via runAs.

SystemService::TransactionSys::init

void SystemService::TransactionSys::init();

Only called once during chain initialization.

This enables the auth checking system. Before this point, TransactionSys allows all transactions to execute without auth checks. After this point, TransactionSys uses AuthInterface::checkAuthSys to authenticate top-level actions and uses of runAs.

SystemService::TransactionSys::startBlock

void SystemService::TransactionSys::startBlock();

Called by native code at the beginning of each block.

SystemService::TransactionSys::runAs

std::vector<char> SystemService::TransactionSys::runAs(
    psibase::Action            action,
    std::vector<ServiceMethod> allowedActions
);

Run action using action.sender's authority.

Also adds allowedActions to the list of actions that action.service may perform on action.sender's behalf, for as long as this call to runAs is in the call stack. Use "" for service in allowedActions to allow use of any service (danger!). Use "" for method to allow any method.

Returns the action's return value, if any.

This will succeed if any of the following are true:

• getSender() == action.sender's authService
• getSender() == action.sender. Requires action.sender's authService to approve with flag AuthInterface::runAsRequesterReq (normally succeeds).
• An existing runAs is currently on the call stack, getSender() matches action.service on that earlier call, and action matches allowedActions from that same earlier call. Requires action.sender's authService to approve with flag AuthInterface::runAsMatchedReq if allowedActions is empty (normally succeeds), or AuthInterface::runAsMatchedExpandedReq if not empty (normally fails).
• All other cases, requires action.sender's authService to approve with flag AuthInterface::runAsOtherReq (normally fails).

SystemService::TransactionSys::getTransaction

psibase::Transaction SystemService::TransactionSys::getTransaction() const;

Get the currently executing transaction.

SystemService::TransactionSys::currentBlock

psibase::BlockHeader SystemService::TransactionSys::currentBlock() const;

Get the current block header.

SystemService::TransactionSys::headBlock

psibase::BlockHeader SystemService::TransactionSys::headBlock() const;

Get the head block header.

This is not the currently executing block. See currentBlock.

SystemService::TransactionSys::headBlockTime

psibase::TimePointSec SystemService::TransactionSys::headBlockTime() const;

Get the head block time.

This is not the currently executing block time. TODO: remove

SystemService::ServiceMethod

struct SystemService::ServiceMethod {
    psibase::AccountNumber service; 
    psibase::MethodNumber  method;  
};

Identify a service and method.

An empty service or method indicates a wildcard.

SystemService::AuthInterface

struct SystemService::AuthInterface {
    const uint32_t readOnlyFlag;            // See header
    const uint32_t firstAuthFlag;           // See header
    const uint32_t requestMask;             // Bits which identify kind of request
    const uint32_t topActionReq;            // Top-level action
    const uint32_t runAsRequesterReq;       // See header
    const uint32_t runAsMatchedReq;         // See header
    const uint32_t runAsMatchedExpandedReq; // See header
    const uint32_t runAsOtherReq;           // See header

    checkAuthSys(...); // Authenticate a top-level action or a `runAs` action
};

Authenticate actions.

TransactionSys calls into auth services using this interface to authenticate senders of top-level actions and uses of TransactionSys::runAs. Any service may become an auth service by implementing AuthInterface. Any account may select any service to be its authenticator. Be careful; this allows that service to act on the account's behalf and that service to authorize other accounts and services to act on the account's behalf. It can also can lock out that account. See AuthEcSys for a canonical example of implementing this interface.

This interface can't authenticate non-top-level actions other than TransactionSys::runAs actions. Most services shouldn't call or implement AuthInterface; use getSender().

Auth services shouldn't inherit from this struct. Instead, they should define methods with matching signatures.

SystemService::AuthInterface::checkAuthSys

void SystemService::AuthInterface::checkAuthSys(
    uint32_t                    flags,
    psibase::AccountNumber      requester,
    psibase::Action             action,
    std::vector<ServiceMethod>  allowedActions,
    std::vector<psibase::Claim> claims
);

Authenticate a top-level action or a runAs action.

• flags: One of the Req (request) constants, or'ed with 0 or more of the flag constants
• requester: "" if this is a top-level action, or the sender of the runAs action. This is often different from action.sender.
• action: Action to authenticate
• allowedActions: Argument from runAs
• claims: Claims in transaction (e.g. public keys). Empty if runAs

common-sys

SystemService::CommonSys

struct SystemService::CommonSys {
    const psibase::AccountNumber service; 

    serveSys(...);    
    serveCommon(...); 
    storeSys(...);    
};

TODO: doc.

See HTTP and Javascript

SystemService::CommonSys::serveSys

std::optional<psibase::HttpReply> SystemService::CommonSys::serveSys(
    psibase::HttpRequest request
);

SystemService::CommonSys::serveCommon

std::optional<psibase::HttpReply> SystemService::CommonSys::serveCommon(
    psibase::HttpRequest request
);

SystemService::CommonSys::storeSys

void SystemService::CommonSys::storeSys(
    std::string       path,
    std::string       contentType,
    std::vector<char> content
);

proxy-sys

SystemService::ProxySys

struct SystemService::ProxySys {
    const psibase::AccountNumber service; 

    registerServer(...); // Register senders's subdomain
};

The proxy-sys service routes HTTP requests to the appropriate service.

Rule set:

• If the target starts with /common, then route the request to SystemService::CommonSys.
• Else if there's a subdomain and it references a registered service, then route the request to that service.
• Else if the request references an unregistered subdomain, then route the request to psispace-sys.
• Else route the request to CommonSys; this handles the chain's main domain.

See Web Services for more detail, including how to write services which serve HTTP requests.

serve export (not an action)

This service has the following WASM exported function:

extern "C" [[clang::export_name("serve")]] void serve()

psinode calls this function on the proxy-sys service whenever it receives an HTTP request that services may serve. This function does the actual routing. psinode has a local option (TODO: implement) which may choose an alternative routing service instead.

SystemService::ProxySys::registerServer

void SystemService::ProxySys::registerServer(
    psibase::AccountNumber server
);

Register senders's subdomain.

server will handle requests to this subdomain.

psispace-sys

SystemService::PsiSpaceSys

struct SystemService::PsiSpaceSys {
    const psibase::AccountNumber service; 

    serveSys(...);  
    storeSys(...);  
    removeSys(...); 
};

Provide web hosting.

This service provides web hosting to non-service accounts. It supports both an upload UI (TODO) and command-line upload using psibase upload and psibase upload tree.

Uploading a directory tree:

psibase -a $ROOT_URL -s $PVT_KEY upload-tree -S $ACCOUNT psispace-sys / $DIR

Uploading a single file:

psibase -a $ROOT_URL -s $PVT_KEY upload -S $ACCOUNT psispace-sys
        /index.html text/html $PATH_TO_FILE

You don't need the -a and -s options if your running a local test chain at http://psibase.127.0.0.1.sslip.io:8080/ and don't protect the accounts with keypairs.

After files are uploaded, the site is available at either http://$ACCOUNT.$DOMAIN or http://$DOMAIN/applet/$ACCOUNT

SystemService::PsiSpaceSys::serveSys

std::optional<psibase::HttpReply> SystemService::PsiSpaceSys::serveSys(
    psibase::HttpRequest request
);

SystemService::PsiSpaceSys::storeSys

void SystemService::PsiSpaceSys::storeSys(
    std::string       path,
    std::string       contentType,
    std::vector<char> content
);

SystemService::PsiSpaceSys::removeSys

void SystemService::PsiSpaceSys::removeSys(
    std::string path
);

Standard Actions

TODO