Allocating Custom Memory to a Model

When deploying a model to an edge device it's sometimes necessary to adjust the amount of memory allocated to your models so that they can run more effectively. This is usually the case on smaller form-factor devices and single board computers.

To define how much memory you'd like to allocate to your model, use the --model.memory flag followed by a positive integer representing the number of megabytes that you'd like to make available to your mode. For example, a value of 200 would be interpreted as 200 MB of memory.

Examples

Starting core with 100 MB of memory

$ ./modzy-core server --model.memory 100 --modzy.url MODZY_URL --modzy.token DEVICE_REGISTRATION_TOKEN

Restarting core with 1 GB of memory

$ ./modzy-core server --resume --model.memory 1000

📘

If your device is running more than 1 model, each model will be given the same amount of memory allocated by the --model.memory flag

Choosing how much memory to use

Determining the right around of RAM to allocate for your models can be an iterative process, and depends on the device you're using, the size of your model, and the data that you're processing. One helpful starting point is our Hardware Resource Guidance table where we've provided some examples of device, model, and RAM settings that work well.