• A user interface for managing images and captions for those images.
• A caption categorization system for assisting with keeping captions consistent across large datasets.
• The ability to import captions and images into a dataset from a directory hierarchy.
• The ability to export captions and images into a directory for use in training scripts.
• A persistent undo/redo system that can store every change ever made to a dataset, including the ability to effectively revert to an earlier version at any time.
• A carefully-engineered Java API for manipulating datasets; the command-line tools and user interface are thin shells over this API.
• Datasets are backed by SQLite for reliable, transactional updates, and a file format that is designed to endure for decades to come.
• Command line tools for automating operations such as importing, exporting, and interrogating metadata.
• The application is comprehensively documented; you are currently reading this documentation!

$ java -version
openjdk version "21.0.4" 2024-07-16
OpenJDK Runtime Environment (build 21.0.4+7)
OpenJDK 64-Bit Server VM (build 21.0.4+7, mixed mode, sharing)

$ unzip com.io7m.laurel.distribution-0.99.0-distribution.zip
   creating: laurel/
   creating: laurel/bin/
  inflating: laurel/bin/laurel
  inflating: laurel/bin/laurel-ui
   creating: laurel/lib/
  inflating: laurel/lib/com.io7m.anethum.api-1.1.1.jar
  inflating: laurel/lib/com.io7m.blackthorne.core-2.0.2.jar
  inflating: laurel/lib/com.io7m.blackthorne.jxe-2.0.2.jar
  inflating: laurel/lib/com.io7m.darco.api-1.0.0.jar
  inflating: laurel/lib/com.io7m.darco.sqlite-1.0.0.jar
...

$ chmod +x laurel/bin/*

$ export LAUREL_HOME=$(realpath laurel)

$ laurel/bin/laurel version
com.io7m.laurel 1.0.0-SNAPSHOT 7e810b7cda6e7d8db2032fdb936f9260aaf906f2

laurel: usage: laurel [command] [arguments ...]

  The laurel command-line application.

  Use the "help" command to examine specific commands:

    $ laurel help help.

  Command-line arguments can be placed one per line into a file, and
  the file can be referenced using the @ symbol:

    $ echo help > file.txt
    $ echo help >> file.txt
    $ laurel @file.txt

  Commands:
    export     Export a dataset into a directory.
    help       Show usage information for a command.
    import     Import a directory into a dataset.
    version    Show the application version.

  Documentation:
    https://www.io7m.com/software/laurel/

$ laurel import --input-directory /tmp/data --output-file output.ldb

$ (cat <<EOF
import
--input-directory
/tmp/data
--output-file
output.ldb
EOF
) > args.txt

$ laurel @args.txt

caption ::= [a-z0-9A-Z_-][a-z0-9A-Z_ \-']*

caption_file ::= caption ("," caption)+ [ "," ]

red drapes,
black and white zigzag floor,
red chair,
gold lamp,
coffee cup,

1. Take the set of captions provided for i and combine them into a prompt p. The exact means by which the captions are combined into a prompt is typically a configurable aspect of the training method. In practice, the most significant caption ("rose bush") would be the first caption in the prompt, and all other captions would be randomly shuffled and concatenated onto the prompt.
2. Generate an image g with g = generate(m, p).
3. Compare the images g and i. The differences between the two images are what the fine-tuning of the model will learn.

1. The image i does not contain a red sofa. However, one of the captions provided for i is "red sofa", and so the prompt p contains the caption "red sofa".
2. An image g is generated with g = generate(m, p). Because p contains the caption "red sofa", the generated image g will likely contain a red sofa.
3. The process compares the images g and i. The source image i doesn't contain a red sofa, but the generated image g almost certainly does. The system then, essentially, erroneously learns that it should be adding red sofas to images!

1. The image i contains a red sofa. However, none of the captions provided for i are "red sofa", and so the prompt p does not contain the caption "red sofa".
2. An image g is generated with g = generate(m, p). Because p does not contain the caption "red sofa", the generated image g will probably not contain a red sofa.
3. The process compares the images g and i. The source image i contains a red sofa, but the generated image g almost certainly does not. The system then, essentially, erroneously learns that it should be removing red sofas from images!

• Include a single primary caption at the start of the prompt of every image in the dataset. This primary caption is effectively the name of the concept that you are trying to teach to the model. The reason for this follows from an understanding of the training process: By making the primary caption prominent and ubiquitous, the system should learn to primarily associate the image differences with this caption.
• Caption all elements of an image that you do not want the model to associate with your primary caption. This will help ensure that the captioned objects do not show up as differences in the images that the training process will, as a result, learn.
• Be consistent in your captioning between images with respect to which aspects of the image you caption. For example, if in one of your images, you caption the lighting or the background colour, then you should caption the lighting and background colour in all of the images. This assumes, of course, that you are not trying to teach the model about lighting or background colours! This practice is, ultimately, about reducing false negatives.

category ::= [a-z0-9A-Z_-][a-z0-9A-Z_ \-']*

CREATE TABLE schema_version (
  version_lock            INTEGER NOT NULL DEFAULT 1,
  version_application_id  TEXT    NOT NULL,
  version_number          INTEGER NOT NULL,

  CONSTRAINT check_lock_primary
    PRIMARY KEY (version_lock),

  CONSTRAINT check_lock_locked
    CHECK (version_lock = 1)
)
-- [jooq ignore start]
STRICT
-- [jooq ignore stop];
CREATE TABLE metadata (
  meta_name  TEXT NOT NULL,
  meta_value TEXT NOT NULL

-- [jooq ignore start]
  ,
  CONSTRAINT metadata_primary_key
    PRIMARY KEY (meta_name)
-- [jooq ignore stop]
)
-- [jooq ignore start]
STRICT
-- [jooq ignore stop];
CREATE TABLE image_blobs (
  image_blob_id      INTEGER PRIMARY KEY NOT NULL,
  image_blob_data    BLOB                NOT NULL,
  image_blob_sha256  TEXT                NOT NULL,
  image_blob_type    TEXT                NOT NULL
)
-- [jooq ignore start]
STRICT
-- [jooq ignore stop];
CREATE TABLE images (
  image_id      INTEGER PRIMARY KEY NOT NULL,
  image_name    TEXT                NOT NULL,
  image_file    TEXT,
  image_source  TEXT,
  image_blob    INTEGER             NOT NULL,

-- [jooq ignore start]
  CONSTRAINT images_name_unique
    UNIQUE (image_name),
-- [jooq ignore stop]

  CONSTRAINT images_blob_exists
    FOREIGN KEY (image_blob) REFERENCES image_blobs (image_blob_id)
)
-- [jooq ignore start]
STRICT
-- [jooq ignore stop];
CREATE TABLE captions (
  caption_id    INTEGER PRIMARY KEY NOT NULL,
  caption_text  TEXT                NOT NULL

-- [jooq ignore start]
  ,
  CONSTRAINT captions_text_unique
    UNIQUE (caption_text)
-- [jooq ignore stop]
)
-- [jooq ignore start]
STRICT
-- [jooq ignore stop];
CREATE TABLE global_captions (
  global_caption_id     INTEGER PRIMARY KEY NOT NULL,
  global_caption_text   TEXT                NOT NULL,
  global_caption_order  INTEGER             NOT NULL

-- [jooq ignore start]
  ,
  CONSTRAINT global_captions_text_unique
    UNIQUE (global_caption_text)
-- [jooq ignore stop]
)
-- [jooq ignore start]
STRICT
-- [jooq ignore stop];
CREATE TABLE categories (
  category_id        INTEGER PRIMARY KEY NOT NULL,
  category_text      TEXT                NOT NULL,
  category_required  INTEGER             NOT NULL

-- [jooq ignore start]
  ,
  CONSTRAINT categories_text_unique
    UNIQUE (category_text)
-- [jooq ignore stop]
)
-- [jooq ignore start]
STRICT
-- [jooq ignore stop];
CREATE TABLE caption_categories (
  caption_caption_id       INTEGER NOT NULL,
  caption_category_id  INTEGER NOT NULL,

  CONSTRAINT caption_categories_caption_exists
    FOREIGN KEY (caption_caption_id)
      REFERENCES captions (caption_id)
        ON DELETE CASCADE,

  CONSTRAINT caption_categories_category_exists
    FOREIGN KEY (caption_category_id)
      REFERENCES categories (category_id)
        ON DELETE CASCADE,

  CONSTRAINT caption_categories_primary_key
    PRIMARY KEY (caption_caption_id, caption_category_id)
)
-- [jooq ignore start]
STRICT
-- [jooq ignore stop];
CREATE TABLE image_captions (
  image_caption_image    INTEGER NOT NULL,
  image_caption_caption  INTEGER NOT NULL,

  CONSTRAINT image_captions_image_exists
    FOREIGN KEY (image_caption_image)
      REFERENCES images (image_id)
        ON DELETE CASCADE,

  CONSTRAINT image_captions_caption_exists
    FOREIGN KEY (image_caption_caption)
      REFERENCES captions (caption_id)
        ON DELETE CASCADE,

  CONSTRAINT image_captions_primary_key
    PRIMARY KEY (image_caption_image, image_caption_caption)
)
-- [jooq ignore start]
STRICT
-- [jooq ignore stop];
CREATE VIEW image_captions_counts AS
  SELECT
    image_captions.image_caption_caption         AS count_caption_id,
    count (image_captions.image_caption_caption) AS count_caption_count
  FROM
    image_captions
  GROUP BY image_captions.image_caption_caption;
CREATE TABLE undo (
  undo_id           INTEGER PRIMARY KEY NOT NULL,
  undo_data         BLOB                NOT NULL,
  undo_description  TEXT                NOT NULL,
  undo_time         INTEGER             NOT NULL
)
-- [jooq ignore start]
STRICT
-- [jooq ignore stop];
CREATE TABLE redo (
  redo_id           INTEGER PRIMARY KEY NOT NULL,
  redo_data         BLOB                NOT NULL,
  redo_description  TEXT                NOT NULL,
  redo_time         INTEGER             NOT NULL
)
-- [jooq ignore start]
STRICT
-- [jooq ignore stop];

let Images             = { All images in the dataset }
let CategoriesRequired = { All categories in the dataset that are marked as "required" }

for Image in Images do
  let CaptionsPresent = CaptionsAssigned(Image);
  for Category in CategoriesRequired do
    let CaptionsRequired = CaptionsInCategory(Category);
    if IsEmpty (CaptionsRequired ∩ CaptionsPresent) then
      Fail("At least one caption is required from the category")
    end if;
  done;
done;

Copyright © 2024 Mark Raynsford <code@io7m.com> https://www.io7m.com

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.