./modzy-core direct

Start the Modzy Edge model server on a remote device in a faster, disconnected mode

$ modzy-core direct [flags] sends all inferences directly to the running model and returns the model response in a single request. Nothing is queued or persisted, and there is no observability in this mode.

🚧

./modzy-core direct only supports inferences API

Since direct mode does not store any inference results, it can only be used with the inferences API . Both HTTP and gRPC inferences for this route are supported.

Usage

Enter the following code into the terminal:

$ ./modzy-core direct

The output is similar to the following:

2022-11-17T04:00:04.363Z	INFO	logger/logger.go:108	acquiring lock
2022-11-17T04:00:04.363Z	INFO	modzy/registration.go:49	connecting to Modzy...
2022-11-17T04:00:04.366Z	INFO	modzy/registration.go:271	Device Group is valid -- Verifying Certificates
2022-11-17T04:00:04.366Z	INFO	modzy/registration.go:300	valid device registration files found...bypassing registration and starting Core
2022-11-17T04:00:04.404Z	INFO	runner/direct.go:81	starting model: k8cjaqdcgz 1.0.0
2022-11-17T04:00:04.404Z	INFO	runtime/docker.go:89	Checking if container is running...
2022-11-17T04:00:04.410Z	INFO	runtime/docker.go:123	creating container...
2022-11-17T04:00:05.440Z	INFO	runtime/docker.go:182	container is ready: running
2022-11-17T04:00:16.257Z	INFO	apiserver/server.go:161	Modzy Core server is starting...
2022-11-17T04:00:16.258Z	INFO	apiserver/server.go:228	Server is starting...
2022-11-17T04:00:16.258Z	INFO	apiserver/server.go:270	Server is listening at :55000

Direct Flags

FlagTypeDescription
--certstringTLS cert to use
-h, --helpHelp for direct mode
--http.portstringHttp port exposed for core server (default "55000")
--keystringTLS key to use
--model.runtimestringThe model container runtime. Only recommended to use when container requires GPU to run (i.e., nvidia).
(default "docker")
--model.timeoutfloatLength of time (in seconds) to wait while attempting to connect to model container. (default 10)
--model.memoryintModel container Memory usage limit (in MB). Applies to all models running on Core
--modzy.tokenstringRegistration token
--modzy.urlstringUrl of Modzy Platform
--resumeResume running after registering.