Documentation‎ > ‎Development‎ > ‎

Machine Learning Deployment Format (mldef)

The machine learning deployment format (mldef) is meant to fulfill a very simple goal:
  • Allow machine learning models created with state of the art tools such as Tensorflow and Keras to be packaged in a compact format for deployment on mobile or embedded platforms with limited hardware.
In addition the following are sub-goals:
  • Allow the models to be self describing so that it can be installed without technical know how.
  • Allow the models to monitor themselves during run-time and report their health status and performance.
  • Allow the models to be signed by the author for security
The format has been created with simplicity and low threshold of adoption in mind:
  • A model is compressed into a single zip archive much like .jar or .docx files.
  • Metadata is stored in a file called manifest.json in human readable .JSON (UTF-8) format in the root of the zip file.
  • Model weights and other data is either stored in one or more files in textual .CSV (UTF-8) or in raw binary form, optimized for loading into GPU memory without any pre-processing needed.
The standard file extension used for a mldef files are .mldef

Inside the .mldef

The files inside the archive should be arranged in accordance with the  guidelines provided to ensure maximum portability:
  • Keep manifest.json in the root of zip file.
  • Use all lower-case file names without special characters or spaces [0-9a-z\.].
  • Avoid filenames longer than 15 characters.
  • Give all files the correct file extension.
  • Arrange files in a directory structure that reflects the model structure.
  • Avoid folders more than 2 levels deep.
  • Avoid more than 100 files in total.

Inside the manifest.json

The format of manifest.json is arranged in sections as follows:

Model section
The model section describes the model from bird's eye perspective. There should only be one model section.
  • Name (text) - The name of the model.
  • Description (text) - A description of the model.
  • Tags (tags) - A comma separated list of words that correspond to tags. Used when arranging models in a zoo.
  • Author name (text) - The name of the author of this model.
  • Author email (email) - The email of the author of this model.
  • URL (URL) - The URL where this model originated (such as a github page or website of organization behind it etc).
  • License file (file-path) - The path of the license file for this model (as located inside the zip).

Data file sections

Each DataFile section declare the presence of one data file and provides the necessary meta data to load that data file. The data files must be declared before they can be used:
  • Name - The unique nick name by which this file may be referenced later in the manifest.
  • Path - The full path inside the zip where this data-file resides.
  • Storage-bytes (uint) - Number of bytes per data point after parsing this file.
  • Storage-format (enum: int, uint, fixed, float) - Format of each data point after parsing this file.
NOTE: The size of the file determines the number of data points.
NOTE: The format of the file is determined based on the file extension used. Valid extensions are:
  • .csv - Comma Separated Values. Headers should be avoided. UTF-8 without BOM is expected.
  • .float32 - Raw 32bit IEEE floats.
  • .float64 - Raw 64bit IEEE floats.

Node range sections

Each node range section declares one or more input or output nodes in the model:
  • Name (text) - The name of these nodes.
  • Description (text) - The purpose of these nodes.
  • Type (enum: input, output) -The type of node, either input or output.
  • Count (uint) - The number of nodes.
  • Unit (enum: seconds, hertz, meters, grams, ...) - In what unit is the data for this connection point.
  • Normalization pre-offset (float) - What should the original data be offset by for this data-point (for accelerated pre-processing off-loading).
  • Normalization scale (float) - What should the original data be scaled by for this data point before scale (for accelerated pre-processing off-loading).
  • Normalization post-offset (float) - What should the original data be offset by for this data-point after scale (for accelerated pre-processing off-loading).
  • Maximum value (float) - What is the maximum value that data for this point can support.
  • Minimum value (float) - What is the minimum value that data for this point can support.
  • Missing error-value (float) - What value should be sent instead of data when data is missing.
  • Above range error-value (float) - What value should be sent instead of data when data is above range.
  • Below range error-value (float) - What value should be sent instead of data when data is below range.
Layer sections

Each layer section declares a new layer referencing data points from a data file as weights:
  • Name (text) - The unique nick name by which this layer may be referenced later in the manifest.
  • Prior (text) - The layer from which this layer takes input. Must be empty for layers marked as input.
  • Description (text) - The purpose of this layer.
  • Capabilities (text) - A comma separated list of words that correspond to capabilities of the layer. Examples include (feed-forward, dense, pool, convolutional, noisy, residual, input, output).
  • DataSource (text) - The unique name of a datafile as declared previously. Must be empty for layers marked as input.
  • StartIndex (uint) - The index of the first data point that will be used as a weight for this layer. Must be empty for layers marked as input.
  • Count (uint) - The number of data points that will be used as a weight for this layer from the referenced data source, or in the case of inputs, simply the number of inputs.

Validations

There will be made available a tool written in C++ to manage models in this format.

The tool will carry out a set of validation tests to validate the integrity of the format. It will basically ensure the rules above are followed, and making sure that all required data is in place.

It can also be supplied with a list of test data sets. When the model is run on this test data, the "known good" output will be recorded for each set. This can then later be verified on the target device to ensure that the model is functioning properly.

Signing

The model .zip file may contain a file called SIGNATURE that contains a signed checksum of all the files inside the .zip file, one on each line.

If the target device cares, it can require this file to be present, load and parse this file and compare the locally generated signed hashes to verify the authenticity of the files before deciding to trust them.

Commandline tool

Tool usage:

mldef <options> <manifest file>

or

mldef <options> <mldef file>

 Option    Description
-h
--help		Show help
  -v
--verbose		Verbose mode. Will spit more logs while working. Useful for debugging.
 -c
--create		Create a stream in .mldef format from the manifest.json as specified. File will be spit out on stdout. Combine with -f to create an actual file.
 -f <filename>
--file <filename>		Write to named file instead of stdout.
Only valid when combined with -c
-0 -1 ... -9 Specify zip compression ratio where -0 is no compression (fast)  and -9 is max compression (slow)
Only valid when combined with -c
 -s
--signature		When used with -c and manifest, update signature file before creating.
When used without -c and mldef, test the existing signature file.
 -i
--inspect		Inspect mode. Will check validity of the source files. This is implied when using -c  as the tool will refuse to create invalid mldef output, but it can also be useful on it's own.
It can be used both with manifest.json and mldef input.
The only difference is that for manifest.json it will check the manifest outside the zip file, while for mldef it will check them inside.
 -r <test-data-set>
--run <test-data-set>		Load and run the model with supplied test data-set and return the output.
Input is taken from the supplied filename unless the filename is '-' in which case stdin i used.
Output is returned on stdout, unless the -f option is given to specify a file to put ouput into.
When combined with -v it will show useful statistics during the run on stderr.

This is useful for
  • Using the model from a script
  • Regression testing
  • Verifying results across platforms
  • Quickly testing nmodel during development
-o <format>
--output-format <format>		Specify the output format
For -r you can choose between csv or binary
 -i <format>
--input-format <format>		 Specify the input format
For -r you can choose between csv or binary








Comments