When deploying a model to an edge device it's sometimes necessary to adjust the amount of memory allocated to your models so that they can run more effectively. This is usually the case on smaller form-factor devices and single board computers.
model.memory is a shared value
If your device is running more than 1 model, each model will be given the same amount of memory allocated by the
To define how much memory you'd like to allocate to your model, use the
--model.memory flag followed by a positive integer representing the number of megabytes that you'd like to make available to your mode. For example, a value of
200 would be interpreted as 200 MB of memory.
Log into your edge device
Open up a terminal and type
sudo vi /etc/systemd/system/modzy-core.serviceor otherwise prepare to edit the
This will open up a configuration file that should look something like this
[Unit] Description=Service for Modzy Core Requires=docker.service Wants=network.target After=docker.service [Service] EnvironmentFile=-/etc/environment ExecStart=/opt/modzy/bin/modzy-core server --storage.filesystem.path /opt/modzy/core/data SyslogIdentifier=modzy-core Restart=on-failure TimeoutStopSec=60 Type=exec [Install] WantedBy=multi-user.target
ExecStartrow by adding
--model.memory [memory amount]to the end of the row
Replace [memory amount] with the amount of memory you'd like to allocate to each model, defined as an integer number of megabytes. For instance
--model.memory 100would allocate 100 MB to each model running on this device.
Save your edits and close the file (in vi this can be done by hitting the 'esc' key and then typing
:wqbefore finally typing the 'enter' key.
Finally, restart Modzy Core by running the following commands
sudo systemctl daemon-reload
sudo systemctl restart modzy-core
Determining the right around of RAM to allocate for your models can be an iterative process, and depends on the device you're using, the size of your model, and the data that you're processing. One helpful starting point is our Hardware Resource Guidance table where we've provided some examples of device, model, and RAM settings that work well.
Updated 3 months ago