Allocating Custom Memory to a Model
When deploying a model to an edge device it's sometimes necessary to adjust the amount of memory allocated to your models so that they can run more effectively. This is usually the case on smaller form-factor devices and single board computers.
To define how much memory you'd like to allocate to your model, use the --model.memory
flag followed by a positive integer representing the number of megabytes that you'd like to make available to your mode. For example, a value of 200
would be interpreted as 200 MB of memory.
Examples
Starting core with 100 MB of memory
$ ./modzy-core server --model.memory 100 --modzy.url MODZY_URL --modzy.token DEVICE_REGISTRATION_TOKEN
Restarting core with 1 GB of memory
$ ./modzy-core server --resume --model.memory 1000
If your device is running more than 1 model, each model will be given the same amount of memory allocated by the
--model.memory
flag
Choosing how much memory to use
Determining the right around of RAM to allocate for your models can be an iterative process, and depends on the device you're using, the size of your model, and the data that you're processing. One helpful starting point is our Hardware Resource Guidance table where we've provided some examples of device, model, and RAM settings that work well.
Updated 7 months ago