com.io7m.smfj 0.15.0 Specification
Notational Conventions
Overview
This section of the specification documents all notational conventions used throughout.
Unicode
The specification makes reference to the Unicode character set which, at the time of writing, is at version 9.0. The specification often references specific Unicode characters, and does so using the standard notation U+NNNN, where N represents a hexadecimal digit. For example, U+03BB corresponds to the lowercase lambda symbol λ.
EBNF
The specification gives grammar definitions in ISO/IEC 14977:1996 Extended Backus-Naur form.
Haskell
Rather than rely on untyped and ambiguous mathematical notation, this documentation expresses all mathematics and type definitions in strict
Haskell 2010 with no extensions. All Haskell sources are included along with the documentation and can therefore be executed from the command line
GHCi tool in order to interactively check results and experiment with functions.
When used within prose, functions are referred to using fully qualified notation, such as (Vector3f.cross n t). This is the application of the cross function defined in the Vector3f module, to the arguments n and t.
FIX SBE
Definitions of binary structures are given as
FIX SBE definitions. All values are considered to be in big-endian byte order unless otherwise specified.
Requirement Levels
This specification uses terms such as MUST, MUST NOT, SHOULD, SHOULD NOT, etc, according to the definitions given in
RFC 2119.
Model
Overview
The SMF model is a minimalist, portable storage model for
triangle mesh data. SMF files are intended to be consumed by 3D rendering engines directly and therefore do not contain any complex interchange features common to formats such as COLLADA . The
text and
binary encodings of the model are designed to be trivial to parse and to permit the easy construction of extremely fast event-based parsers that do not require loading the entire file into memory for processing.
The format is
versioned, and is specified in a manner that allows for implementations to provide strong guarantees of
forwards and
backwards compatibility as the specification evolves.
The version of the SMF model described by this specification is
(2, 0). See
versioning for details.
Versioning
Overview
The SMF specification is versioned via a restricted subset of the
Semantic Versioning specification. The SMF specification has a
major and
minor version number, with
major version increments denoting incompatible changes, and
minor version increments denoting new functionality. There is no
patch version number. A version of the specification with
major version
m and
minor version
n is denoted as specification version
(m, n).
Forward Compatibility
Assuming a version of the SMF specification m, an update to the specification that yields version n such that n > m is considered to be forwards compatible if a parser that supports format version m can read files that were written using format version n.
Backward Compatibility
Assuming a version of the SMF specification m, an update to the specification that yields version n such that n > m is considered to be backwards compatible if a parser that supports format version n can read files that were written using format version m.
Compatibility
The SMF specification is designed such that a correctly-written parser implementation that supports a major version m is able to support the set of versions ∀n. (m, n). This implies full forwards and backwards compatibility for parsers when the major version is unchanged.
Changes that would cause a parser supporting an older version of the specification to fail to read a file written according to a newer version of the specification MUST imply an increment in the major version of the specification.
Changes that would cause a parser supporting a newer version of the specification to fail to read a file written according to an older version of the specification MUST imply an increment in the major version of the specification.
An implication of the above rules is that new features added to the specification must be added in a manner that allows them to be ignored by older parsers, lest the major version of the specification be incremented on every update.
Mesh
A
mesh in the SMF model is a set of
triangles.
Attributes
An attribute in the SMF model is a uniquely-named array of elements of a given type. All attributes within a particular SMF file have the same number of elements. The type of an attribute is a 3-tuple (c, n, s), where c is a component type, n is a component count, and s is a component size expressed in bits.
This version of the specification defines three basic
component types:
signed integers,
unsigned integers, and
floating point numbers. Floating point numbers are
IEEE 754 floating point values.
Attribute names may be at most 64 characters in length and must conform to the following syntax:
Implementations are required to support attributes of at least the following types:
Vertices
A
vertex is an abstract object consisting of exactly one element taken from each of the defined
attributes. A vertex can essentially be considered to be an array index; The vertex at index
n can be considered to be the aggregation of the
nth elements of all of the defined attributes. Vertices are numbered starting at
0.
Triangles
A triangle is a 3-tuple of
vertices. In the SMF model, a triangle references vertices by their numeric index.
Coordinate System
A coordinate system in the SMF model is a set of three axis names and a triangle winding order. The axis names specify which Cartesian coordinate system axes correspond to the right, up, and forward directions. The winding order specifies whether vertices for triangles are given in clockwise or counter-clockwise order. A coordinate system is only valid if the three chosen axes have different axis names. The axisName function relates an axis to an axis name. The axisValid function gives a formal definition of the validity of sets of axes.
Schema ID
A
schema identifier is an optional identifier that can be inserted into SMF files. Because the SMF model does not predefine any particular
attributes, tools that consume SMF files cannot know ahead of time if the file they have just loaded will actually contain the attributes that they are expecting. A
schema identifier effectively provides a concrete name for a set of attributes so that tools that process SMF files can perform validation of attributes based on the identifier. It is somewhat analogous to XML namespace declarations; The author of a particular document inserts an XML namespace identifier into their document, and validation tools use this identifier to locate schema definitions against which the document is then validated.
A schema identifier consists of a string and two integer values:
The schema_id uniquely identifies the schema, and the schema_version_major and schema_version_minor values identify the version of the schema.
Schema names may be at most 64 characters in length and must conform to the following syntax:
Metadata
The SMF model supports the inclusion of arbitrary metadata. A
metadata value is an opaque data value with a
schema identifier specified in the same manner as the
schema identifier. Applications may use the metadata facilities to associate small amounts of extra information with mesh data. Typical uses include checksums, copyright information, and generating software package version information.
SMF/B - Binary Encoding
Overview
The
SMF/B format is a binary encoding for the
SMF model.
An
SMF/B file begins with a simple fixed-size
header followed by a series of
sections containing data and metadata.
All data in an
SMF/B file is written in
big-endian byte order, with the exception of
triangle and
vertex data which may be electively written in big-endian or little-endian byte order based on the order specified in the
SMF section.
Header
An SMF/B file begins with the following fixed-size structure:
The value of the magicNumber field MUST be 0x89534D460D0A1A0A. Implementations MUST immediately reject any files that do not begin with this magic number. The derivation of this constant is taken almost verbatim from the PNG file format with the characters PNG changed to SMF.
The
versionMajor field specifies the
major version number of the specification to which the data in the file is expected to conform. The
versionMinor field specifies the
minor version number of the specification to which the data in the file is expected to conform.
Sections
Definition
Directly following the
header, an
SMF/B file consists of a series of
sections. A
section begins with the following fixed-size structure:
The
id field of the section header identifies the type of the section. The
size field of the section header specifies the
data size; the size in octets of the data that follows the section header. The
data size necessarily includes any trailing
padding octets that may be inserted in order to guarantee that the start of the
next section (if any) is correctly
aligned. This implies that the
data size of any given section MUST be a multiple of the
alignment size.
Implementations MUST ignore any content inside a section of an unrecognized type . As sections explicitly state the size of their own data, implementations can simply seek forwards in the file by the specified data size to reach the next section.
The first section in an
SMF/B file MUST be an
smf section. The last section in an
SMF/B file MUST be an
end section.
Alignment
Sections MUST be aligned to
16 octet boundaries. As the section
header is defined to be exactly
16 octets, this implies that the data within a section will also be aligned to a
16 octet boundary .
Available Sections
This version of the specification defines the following sections:
Enumerating Sections
Note: This subsection is informative.
The design of sections allows for implementations to quickly enumerate all sections within a file using the following strategy:
The bytesAvailable method is assumed to return true if there are any bytes remaining in the file. The currentOffset method is assumed to return the current read position in octets within the file. The readUnsigned64 method is assumed to read a 64-bit big-endian integer from the current file, advancing the current read position by 8 octets. The readUnsigned32 method is assumed to read a 32-bit big-endian integer from the current file, advancing the current read position by 4 octets.
Section - end
Definition
The end section indicates the end of an SMF/B file.
Cardinality/Ordering
An end section must appear exactly once in an SMF/B file.
The end section MUST be the last section in an SMF/B file. Implementations MUST reject SMF/B files that do not contain a terminating end section.
Magic Number
The end section is identified by the magic number 0x534D465F454E4421.
Data
The
end section contains no data. The specified
data size for the section MUST be
0.
Section - smf
Definition
The smf section specifies information common to all sections in an SMF/B file.
Cardinality/Ordering
An smf section MUST appear exactly once in an SMF/B file.
The smf section MUST be the first section in an SMF/B file. Implementations MUST reject SMF/B files that do not start with an smf section.
Magic Number
The smf section is identified by the magic number 0x534D465F48454144.
Data
The smf section begins with the following structure:
The fieldsSize field specifies the total size of the header fields, after which a list of attributes will be declared. The purpose of this field is to allow for forward compatibility: Future updates to this specification may add extra fields to the end of the header structure, and the value of this field allows older implementations to skip those newer fields.
If the length of the schema_id field is 0, the schema identifier as a whole is ignored.
The
vertexCount field specifies the number of vertices that will be specified in a
vertices-noninterleaved section. Parsers MUST raise an appropriate error if a non-zero vertex count is specified but no vertices are actually provided in any following section.
The
triangleCount field specifies the number of triangles that will be specified in a
triangles section. The
triangleSizeBits field specifies the size in bits of the individual vertex indices within each triangle. Parsers MUST raise an appropriate error if a non-zero triangle count is specified but no triangles are actually provided in any following section.
The
attributeCount field specifies the number of
attributes that will be declared.
The axis and winding order values are related to integer values via the following functions:
The
byteOrder field specifies the
endianness of the values in the
vertex and
triangle.
Directly following the end of the header structure is a series of attribute declarations:
The name field specifies the name of the attribute. The componentKind field specifies the type of the attribute components. The componentCount field specifies the number of components in each value in the attribute, and the componentSizeBits field specifies the size in bits of each component value.
Attribute component types are related to integer values via the following function:
Section - triangles
Definition
The triangles section specifies triangle data.
Cardinality/Ordering
A triangles section can appear [0, 1] times in an SMF/B file.
The triangles section has no requirements on ordering.
Magic Number
The triangles section is identified by the magic number 0x534D465F54524953.
Data
Triangles are specified as an array of 3-element unsigned integer vectors, with each integer component being of the size declared in the preceding
smf section. No padding is inserted between triangles, but padding may need to be added at the end of the array in order to ensure proper section
alignment. All values are stored in either big-endian or little-endian form depending on the order specified in the
SMF section.
Section - vertices-noninterleaved
Definition
The
vertices-noninterleaved section specifies vertex data for all declared
attributes.
Cardinality/Ordering
A vertices-noninterleaved section can appear [0, 1] times in an SMF/B file.
The vertices-noninterleaved section has no requirements on ordering.
Magic Number
The vertices-noninterleaved section is identified by the magic number 0x534D465F56444E49.
Data
The data for each attribute is given in full in the order in which the attributes were declared in the header. Specifically, in a file containing
v vertices and a list of attributes
a of length
n, the data of the section will contain
v values of the type specified in
a !! 0, followed by
v values of the type specified by
a !! 1, and so on up to
a !! (n - 1). All values are stored in either big-endian or little-endian form depending on the order specified in the
SMF section.
The start of the data for each attribute is aligned to the next 16 octet boundary regardless of type. The alignment is achieved by inserting padding octets at the end of the data for each attribute so that the start of the data for the next attribute, or the start of the next section, occurs on a 16 octet boundary.
Section - metadata
Definition
The metadata section specifies arbitrary metadata.
Cardinality/Ordering
A metadata section can appear [0, n] times in an SMF/B file, for any n.
The metadata section has no requirements on ordering.
Magic Number
The metadata section is identified by the magic number 0x534D465F4D455441.
Data
The data of a metadata section starts with the following fixed-size header:
The
metaSchemaIdentifier, field is intended to provide type information for the metadata. It has the same semantics and purpose as
schema identifiers. The
metaSize field specifies the size in octets of the metadata.
Example
A diagram of an example SMF/B file is as follows:
SMF/T - Text Encoding
Overview
The
SMF/T format is a text encoding for the
SMF model. Data specified in the text encoding is separated into explicitly delimited
sections.
The format is line-based, and lines (excluding the very first line in the file) containing only whitespace, or lines beginning with U+0023 # are ignored. In a given line, whitespace between tokens is insignificant.
Sections
Definition
A
section is started with a
section command and terminated with a corresponding
end command. Implementations MUST ignore any content between and including an unrecognized section command and the corresponding end command .
The first line of an
SMF/T file MUST begin an
smf section.
A section is terminated with an end command. Implementations MUST raise errors if an end-of-file condition is encountered before an end command is given for the current section.
The command takes no arguments.
Available Sections
This version of the specification defines the following sections:
Section - smf
Definition
The smf section specifies the major and minor version of the specification to which the rest of the file is expected to conform. The first argument specifies the major version, and the second argument specifies the minor version. Implementations MUST immediately reject files that are of unsupported major versions, and MUST halt processing of the rest of the file if the version numbers cannot be parsed.
The command takes the following arguments:
The contents of an
smf section are a series of
subcommands with at most one subcommand appearing on each line within the section. A
subcommand consists of the subcommand's name and a series of zero or more arguments. Implementations MUST ignore subcommands that have unrecognized names, but SHOULD log warning messages to indicate that unrecognized subcommands have been specified.
Subcommands
This version of the specification defines the following smf subcommands:
Subcommand - attribute
The
attribute subcommand specifies an
attribute that will appear in the model.
The subcommand takes the following arguments:
The subcommand is allowed to appear any number of times, but all specified attribute names must be unique.
Subcommand - coordinates
The
coordinates subcommand specifies the
coordinate space of the mesh data that will appear in the model.
The subcommand takes the following arguments:
If the subcommand does not appear in the section, implementations MUST behave as if the following subcommand had been specified:
Subcommand - endianness
The endianness subcommand specifies the byte order of data that will appear in the file. Because byte ordering is irrelevant in an SMF/T file, the effect of evaluating this subcommand is to set the byte order in the parsed SMF model to the specified value. This will affect subsequent serializations of the model to formats with which byte ordering is relevant (such as SMF/B) .
The subcommand takes the following arguments:
The subcommand is allowed to appear at most once. If the command does not appear, implementations MUST behave as if big-endian byte ordering was specified.
Subcommand - schema
The command takes the following arguments:
Subcommand - triangles
The
triangles subcommand specifies the number of
triangles that will appear in the model, and the size of the vertex indices in bits.
The subcommand takes the following arguments:
If the subcommand does not appear in the section, implementations MUST behave as if the following subcommand had been specified:
Parsers MUST raise an appropriate error if a non-zero triangle count is specified but no triangles are actually provided in any following section.
Subcommand - vertices
The
vertices subcommand specifies the number of
vertices that will appear in the model for each attribute.
The subcommand takes the following arguments:
If the subcommand does not appear in the section, implementations MUST behave as if the following subcommand had been specified:
Parsers MUST raise an appropriate error if a non-zero vertex count is specified but no vertices are actually provided in any following section.
Section - triangles
Definition
The triangles section specifies triangle data that will appear in the model.
The command takes no arguments.
In a file that is specified to contain
k triangles, the next
k non-empty, non-commented lines will contain exactly
3 whitespace-separated unsigned integer values each representing the index of a vertex. Implementations MUST raise errors if there are not exactly
k triangles specified before section's
end command.
Section - vertices-noninterleaved
Definition
The vertices-noninterleaved section specifies the vertex data that will appear in the model.
The contents of a
vertices-noninterleaved section are a series of
subcommands with at most one subcommand appearing on each line within the section. A
subcommand consists of the subcommand's name and a series of zero or more arguments. Implementations MUST ignore subcommands that have unrecognized names, but SHOULD log warning messages to indicate that unrecognized subcommands have been specified.
Subcommands
This version of the specification defines the following vertices-noninterleaved subcommands:
Subcommand - attribute
The attribute subcommand indicates the start of data for an attribute.
The command takes the following arguments:
The named attribute must have been declared in the
smf section; Implementations MUST raise errors when encountering
attribute subcommands that name attributes that were not declared in the
smf section.
For an attribute
a that specifies
c components of type
t, in a file that is specified to contain
v vertices, the next
v non-empty, non-commented lines will contain exactly
c whitespace-separated values of type
t. Implementations MUST raise errors if there are not exactly
v vertex values specified before section's
end command or the start of the next
attribute subcommand, whichever occurs first.
Section - metadata
Definition
The metadata section specifies an item of optional metadata that will be attached to the model.
The command takes the following arguments:
For any
v,
s, and
n, the command
meta v s n will indicate that the next
n non-empty, non-commented lines will contain
RFC4648 base64url encoded data of a type with vendor
v and schema
s. Implementations MUST raise errors if there are not exactly
n lines of data specified before the section's
end command.
Example
The following is a complete example of an SMF/T file:
Specification History
History
This section provides a brief summary of specification changes over time.