GuidesRecipesAPI ReferenceChangelogDiscussions
Log In

Allocating Custom Memory to a Model

When deploying a model to an edge device it's sometimes necessary to adjust the amount of memory allocated to your models so that they can run more effectively. This is usually the case on smaller form-factor devices and single board computers.

📘

model.memory is a shared value

If your device is running more than 1 model, each model will be given the same amount of memory allocated by the --model.memory flag

To define how much memory you'd like to allocate to your model, use the --model.memory flag followed by a positive integer representing the number of megabytes that you'd like to make available to your mode. For example, a value of 200 would be interpreted as 200 MB of memory.

  1. Log into your edge device

  2. Open up a terminal and type sudo vi /etc/systemd/system/modzy-core.service or otherwise prepare to edit the modzy-core.service file.
    This will open up a configuration file that should look something like this

    [Unit]
    Description=Service for Modzy Core
    Requires=docker.service
    Wants=network.target
    After=docker.service
    
    [Service]
    EnvironmentFile=-/etc/environment
    ExecStart=/opt/modzy/bin/modzy-core server --storage.filesystem.path /opt/modzy/core/data
    SyslogIdentifier=modzy-core
    Restart=on-failure
    TimeoutStopSec=60
    Type=exec
    
    [Install]
    WantedBy=multi-user.target
    
  3. Edit the ExecStart row by adding --model.memory [memory amount] to the end of the row
    Replace [memory amount] with the amount of memory you'd like to allocate to each model, defined as an integer number of megabytes. For instance --model.memory 100 would allocate 100 MB to each model running on this device.

  4. Save your edits and close the file (in vi this can be done by hitting the 'esc' key and then typing :wq before finally typing the 'enter' key.

  5. Finally, restart Modzy Core by running the following commands

sudo systemctl daemon-reload
sudo systemctl restart modzy-core

Choosing how much memory to use

Determining the right around of RAM to allocate for your models can be an iterative process, and depends on the device you're using, the size of your model, and the data that you're processing. One helpful starting point is our Hardware Resource Guidance table where we've provided some examples of device, model, and RAM settings that work well.