./modzy-core direct
Start the Modzy Edge model server on a remote device in a faster, disconnected mode
$ modzy-core direct [flags]
sends all inferences directly to the running model and returns the model response in a single request. Nothing is queued or persisted, and there is no observability in this mode.
./modzy-core direct
only supportsinferences
APISince
direct
mode does not store any inference results, it can only be used with the inferences API . Both HTTP and gRPC inferences for this route are supported.
Usage
Enter the following code into the terminal:
$ ./modzy-core direct
The output is similar to the following:
2022-11-17T04:00:04.363Z INFO logger/logger.go:108 acquiring lock
2022-11-17T04:00:04.363Z INFO modzy/registration.go:49 connecting to Modzy...
2022-11-17T04:00:04.366Z INFO modzy/registration.go:271 Device Group is valid -- Verifying Certificates
2022-11-17T04:00:04.366Z INFO modzy/registration.go:300 valid device registration files found...bypassing registration and starting Core
2022-11-17T04:00:04.404Z INFO runner/direct.go:81 starting model: k8cjaqdcgz 1.0.0
2022-11-17T04:00:04.404Z INFO runtime/docker.go:89 Checking if container is running...
2022-11-17T04:00:04.410Z INFO runtime/docker.go:123 creating container...
2022-11-17T04:00:05.440Z INFO runtime/docker.go:182 container is ready: running
2022-11-17T04:00:16.257Z INFO apiserver/server.go:161 Modzy Core server is starting...
2022-11-17T04:00:16.258Z INFO apiserver/server.go:228 Server is starting...
2022-11-17T04:00:16.258Z INFO apiserver/server.go:270 Server is listening at :55000
Direct Flags
Flag | Type | Description |
---|---|---|
--cert | string | TLS cert to use |
-h, --help | Help for direct mode | |
--http.port | string | Http port exposed for core server (default "55000") |
--key | string | TLS key to use |
--model.runtime | string | The model container runtime. Only recommended to use when container requires GPU to run (i.e., nvidia ). (default "docker") |
--model.timeout | float | Length of time (in seconds) to wait while attempting to connect to model container. (default 10) |
--model.memory | int | Model container Memory usage limit (in MB). Applies to all models running on Core |
--modzy.token | string | Registration token |
--modzy.url | string | Url of Modzy Platform |
--resume | Resume running after registering. |
Updated 6 months ago