Stonesignal User Manual 1.0.0-SNAPSHOT
The stonesignal package provides a server for recording
device location data.
The stonesignal server package is available from the following sources:
Regardless of the distribution method, the stonesignal package will contain a command
named stonesignal that acts as the main entrypoint to all of the server and client
functionality.
The
stonesignal server requires a
PostgreSQL
server. The
stonesignal server will create the required tables and database objects on
first startup, given the name of a running PostgreSQL database, and a PostgreSQL role
and password.
The stonesignal command requires that a Java 21+ compatible JVM be accessible
via /usr/bin/env java.
Verify the integrity of the distribution zip file:
Unzip the zip file, and set up the environment appropriately. The stonesignal command
expects an environment variable named STONESIGNAL_HOME
to be defined that points to the installation directory.
The
stonesignal package uses
PostgreSQL for all persistent data.
Create an
owner
role in PostgreSQL that will own the database objects:
Create a new database owned by the owner role:
The stonesignal package sets up multiple roles during database initialization. The
configured roles have different degrees of privileges in order to allow, for example,
external systems such as
database metrics collectors read-only access to the database. All the defined rules
are declared with
the built-in PostgreSQL restrictions such as nocreatedb,
nocreaterole, etc.
During the startup of the
stonesignal server, the server will connect to the
database using the
owner role and do any
database table initialization and/or schema upgrades necessary. The server will then
disconnect from
the database, and then connect to the database again using the
worker role. The
worker role is then used for normal operation of the server; if this role is somehow
compromised,
the role only has a limited ability to do any damage to the database, and cannot affect
the
audit log at all.
The owner role is the role that owns the database and is permitted to create tables,
create new roles, etc. This role is used by the stonesignal package when creating
the database during the first run of the server, and for upgrading database schemas
later. Administrators
are free to pick the name of the role, although it is recommended that the role be
named
stonesignal_owner to make it clear as to the purpose of the role. The
stonesignal package will create the other roles it uses for normal operation
automatically, and will manage their passwords based on the server configuration.
If the
PostgreSQL OCI image is used,
it is common to have the image create this role automatically using the
POSTGRES_USER and
POSTGRES_PASSWORD variables:
The worker role is the role that is used for normal database operation. It is
a role that has read/write access to all tables (except for the audit log which is
restricted to
being append-only), although it is not granted the ability to create new tables, drop
tables, or do
other schema manipulation. The role is always named stonesignal,
and adminstrators are required to set a password for this role.
The reader role is a role that is permitted read-only access to some of the
database. Specifically, the role does not have access to columns containing secrets
such as device
keys. The role is therefore safe for use by external systems that want to connect
to the database
directly to do data analysis. The role is always named
stonesignal_reader,
and adminstrators are required to set a password for this role.
The device role is a role that is permitted update access to some of the
database. The role is always named stonesignal_device,
and adminstrators are required to set a password for this role.
The server can now be run with stonesignal server:
The server does not fork into the background and is designed to be run under process
supervision.
The
stonesignal server is configured using a single JSON-formatted configuration file.
The format has a fully documented
schema and so
configuration files can be independently validated, and benefit from autocompletion
in most modern IDEs.
Configuration files are allowed to contain line-based // style
comments.
The smallest working configuration file, assuming a database at
db.example.com:
The Database section of the configuration file configures the database.
The
OwnerRole property specifies the name of the role that
owns the database. Conventionally, this should be
stonesignal_owner, but can be set independently by the database administrator.
The OwnerRolePassword property specifies the password of the owner role.
The
WorkerRolePassword property specifies the password of the
worker
role used for normal database operation.
The
ReaderRolePassword property specifies the password of the
reader
role used for read-only database access.
The
DeviceRolePassword property specifies the password of the
device
role.
The Name property specifies the database name.
The Upgrade property specifies that the database schema should be upgraded on
startup.
An example database configuration:
The
OpenTelemetry section of the configuration file configures
Open Telemetry. This section is optional and
telemetry is disabled if the section is not present.
The logical service name should be provided in the
ServiceName
property.
If the
OpenTelemetry object contains a
Traces
object, OTLP
traces
will be sent to a specified endpoint. The
Endpoint property specifies the
endpoint, and the
Protocol property must be
HTTP.
If the
OpenTelemetry object contains a
Metrics
object, OTLP
metrics
will be sent to a specified endpoint. The
Endpoint property specifies the
endpoint, and the
Protocol property must be
HTTP.
If the
OpenTelemetry object contains a
Logs
object, OTLP
logs
will be sent to a specified endpoint. The
Endpoint property specifies the
endpoint, and the
Protocol property property must be
HTTP.
An example Open Telemetry configuration:
The JSON schema for the configuration file is as follows:
The stonesignal package provides a server-based application to record
location data sent from devices.
This section of the documentation describes the internal stonesignal model.
A device is a piece of GPS (or other location technology) enabled electronic
equipment. In the typical use case, a device will be a smartphone.
Devices are assigned device IDs. A device ID
is a public UUID value that identifies the device.
Devices are assigned
device keys. A
device key
is a secret
32-byte random value that the device uses to authenticate
itself to the
device API.
The text encoding for a device key is given by the pattern
[A-F0-9]{64}. So, for example,
the string
02210D83680AC0F4A50C47F51DCCE35610DE59717623A0D9408067279A63D511
is a well-formed device key, whilst
02210d83680ac0f4a50c47f51dcce35610de59717623a0d9408067279a63d511
is not.
The device API is the interface exposed to devices. The device API serves
only one purpose: It accepts location updates from devices and records them into the
database.
The admin API is the interface exposed to administrators. The API allows
for enrolling and editing devices.
The
data API is the interface exposed to data analysis applications. It
provides a read-only view of devices and locations, but it deliberately does not have
access to
secret values such as
device keys.
The server maintains an append-only audit log consisting of a series of
audit events. An audit event has an integer
id, an owner (represented by an account UUID),
a timestamp, a type, and a
message
consisting of a set of key/value pairs.
Each operation that changes the underlying database typically results in an event
being logged to the audit log.
The
stonesignal package is extensively instrumented with
OpenTelemetry
in order to allow for the server to be continually monitored. The package publishes
metrics,
logs, and traces, all of which can be independently enabled or disabled. Most installations
will only want to
enable metrics or logs in production; traces are more useful when trying to diagnose
performance
problems, or for doing actual development on the
stonesignal package.
The package publishes the following metrics that can be used for monitoring:
The package may produce other metrics, however these are undocumented and
should not be relied upon.
The stonesignal package provides a command-line interface for performing tasks such as
starting the server, checking configuration files, and etc. The base
stonesignal
command is broken into a number of subcommands which are documented over the following
sections.
All subcommands accept a --verbose parameter that may be set to one of
trace, debug, info,
warn, or error. This parameter sets the lower bound for
the severity of messages that will be logged. For example, at debug verbosity, only
messages of severity debug and above will be logged. Setting the verbosity to
trace
level effectively causes everything to be logged, and will produce large volumes of
debugging output.
The
stonesignal command-line tool uses
quarrel
to parse command-line arguments, and therefore supports placing command-line arguments
into a file, one argument
per line, and then referencing that file with
@. For example:
All subcommands, unless otherwise specified, yield an exit code of 0 on success, and
a non-zero exit code on failure.
server - Start the server
The server command starts the server.
version - Display the package version
The version command displays the current version of the package.
The command has no parameters.
The admin API is the API used to accept administrative
commands.
The API is designed as a set of HTTP endpoints with which clients exchange
messages in a strictly-defined and versioned JSON (or, optionally, CBOR-encoded)
format.
There is no version negotiation in the protocol. Clients must check to see
which versions are available using the
version document,
and then speak directly to the versioned endpoints using the correct
protocol version.
The API endpoints allow for communicating in a number of different formats that
each use the same underlying data model. If a client specifies a
Content-Type header with a request, the server will
respond using the same format.
All API endpoints will accept a Content-Type header
that must be one of the following values:
If no Content-Type header is provided,
application/stonesignal+json is assumed.
Clients must include an
Authorization header
with a value of
Bearer k, where
k is the configured
API key.
All endpoints require authentication except for the
root
endpoint.
All endpoints return a 200 HTTP status code
on success with an endpoint-specific response message. On errors,
all endpoints return an HTTP status code greater than or equal to
400 with a response conforming to the
following schema:
The
/ endpoint exposes a
ventrad
version document that explains which API versions are available and where they
are hosted.
The
Endpoint property of each protocol defines a
prefix that must be included in order to reach the correct endpoint. So, for
example, if the
Endpoint property is
/1/0, then the
/device-get-by-id
endpoint would be reachable at
/1/0/device-get-by-id.
The endpoint ignores any request data.
The endpoint responds with a
ventrad
version document.
The
/device-get-by-id endpoint is used to retrieve the definition
of a device by its
ID.
The endpoint expects requests of the following type:
The endpoint responds with objects of the following type:
The
/device-get-by-key endpoint is used to retrieve the definition
of a device by its
key.
The endpoint expects requests of the following type:
The endpoint responds with objects of the following type:
The /device-put endpoint is used to update or create the definition
of a device.
The endpoint expects requests of the following type:
The endpoint responds with an empty object.
The
device API is the API used to accept location
data from
devices.
The API is designed as a set of HTTP endpoints with which clients exchange
messages in a strictly-defined and versioned JSON (or, optionally, CBOR-encoded)
format.
There is no version negotiation in the protocol. Clients must check to see
which versions are available using the
version document,
and then speak directly to the versioned endpoints using the correct
protocol version.
The API endpoints allow for communicating in a number of different formats that
each use the same underlying data model. If a client specifies a
Content-Type header with a request, the server will
respond using the same format.
All API endpoints will accept a Content-Type header
that must be one of the following values:
If no Content-Type header is provided,
application/stonesignal+json is assumed.
Clients must include an
Authorization header
with a value of
Bearer k, where
k is the assigned
device key.
All endpoints require authentication except for the
root
endpoint.
All endpoints return a 200 HTTP status code
on success with an endpoint-specific response message. On errors,
all endpoints return an HTTP status code greater than or equal to
400 with a response conforming to the
following schema:
The
/ endpoint exposes a
ventrad
version document that explains which API versions are available and where they
are hosted.
The
Endpoint property of each protocol defines a
prefix that must be included in order to reach the correct endpoint. So, for
example, if the
Endpoint property is
/1/0, then the
/device-location-put
endpoint would be reachable at
/1/0/device-location-put.
The endpoint ignores any request data.
The endpoint responds with a
ventrad
version document.
The /device-location-put is used to submit
a location update from a device.
The endpoint expects requests of the following type:
The endpoint responds with an empty JSON object.
The data API is the API used to retrieve location
data from the server. The data API is considered to
be unprivileged, and therefore will not return any secrets such as device keys.
The API is designed as a set of HTTP endpoints with which clients exchange
messages in a strictly-defined and versioned JSON (or, optionally, CBOR-encoded)
format.
There is no version negotiation in the protocol. Clients must check to see
which versions are available using the
version document,
and then speak directly to the versioned endpoints using the correct
protocol version.
The API endpoints allow for communicating in a number of different formats that
each use the same underlying data model. If a client specifies a
Content-Type header with a request, the server will
respond using the same format.
All API endpoints will accept a Content-Type header
that must be one of the following values:
If no Content-Type header is provided,
application/stonesignal+json is assumed.
Clients must include an
Authorization header
with a value of
Bearer k, where
k is the configured
API key.
All endpoints require authentication except for the
root
endpoint.
All endpoints return a 200 HTTP status code
on success with an endpoint-specific response message. On errors,
all endpoints return an HTTP status code greater than or equal to
400 with a response conforming to the
following schema:
The
/ endpoint exposes a
ventrad
version document that explains which API versions are available and where they
are hosted.
The
Endpoint property of each protocol defines a
prefix that must be included in order to reach the correct endpoint. So, for
example, if the
Endpoint property is
/1/0, then the
/device-get-by-id
endpoint would be reachable at
/1/0/device-get-by-id.
The endpoint ignores any request data.
The endpoint responds with a
ventrad
version document.
The /devices endpoint retrieves the set of defined devices.
The endpoint expects requests of the following type:
The endpoint responds with objects of the following type:
The
/device-get-by-id endpoint is used to retrieve the definition
of a device by its
ID.
The endpoint expects requests of the following type:
The endpoint responds with objects of the following type:
The /device-locations endpoint is used to retrieve the
most recent location of all devices.
The endpoint expects requests of the following type:
The endpoint responds with objects of the following type:
The /device-locations-history endpoint is used to retrieve a
list of locations for a specific device at some given point in the past.
The endpoint expects requests of the following type:
The endpoint responds with objects of the following type:
This section of the manual attempts to describe the security properties of the
stonesignal server.
An attacker with access to the APIs could send specially crafted messages designed
to exhaust
server resources during parsing/validation of the messages.
All APIs exposed by the
stonesignal server are defined in terms
of simple, flat record types. The
jackson parser
is used to deserialize JSON and CBOR requests, and the
dixmont package
is used to prevent the deserialization of anything other than a fixed list of types.
Additionally, requests made to the
device API
and the
data API
have a hardcoded maximum size limit of
32768 octets.