|
| 1 | +# Simple Inference Server |
| 2 | + |
| 3 | +This program is created to serve as the easiest tool to bring any request-response application (like inference of the ML-model) into the web. |
| 4 | +It is completely based on [drogon](https://github.com/drogonframework/drogon) http server lib. |
| 5 | + |
| 6 | +It is useful when you have an early-stage prototype, and you want to share it with your manager who doesn't want to use docker :) |
| 7 | + |
| 8 | +## Workflow |
| 9 | + |
| 10 | +- Accepts `POST` request with a single `.zip` file to the root endpoint `/` |
| 11 | +- Displays the URL to track the task if it was accepted (`/track_result?task=...`) |
| 12 | +- When processing is finished, the track page displays the URL to download result (`get_result?task=...`) |
| 13 | + |
| 14 | +## Usage |
| 15 | + |
| 16 | +The actual processing should be defined as an application (or shell script), taking 2 positional arguments: |
| 17 | +```shell |
| 18 | +invoke.sh /path/to/input.zip /path/to/output.zip |
| 19 | +``` |
| 20 | + |
| 21 | +The server doesn't know anything about underlying processing. |
| 22 | +The script is given to the server as a command-line argument `--invokePath`. |
| 23 | + |
| 24 | +A few other options are supported: |
| 25 | +```shell |
| 26 | +$ simple_inference_server --help |
| 27 | +Simple Inference Server: |
| 28 | + -h [ --help ] Display help message |
| 29 | + --invokePath arg Path to the script that will be invoked |
| 30 | + --host arg (=127.0.0.1) Host to bind to |
| 31 | + --port arg (=7654) Port to bind to |
| 32 | + --cert arg (="") Path to SSL certificate |
| 33 | + --key arg (="") Path to SSL key |
| 34 | +``` |
| 35 | + |
| 36 | + |
| 37 | +## Application |
| 38 | + |
| 39 | +The application is lightweight (binary size is ~3 Mb on Arch Linux) and has very few dependencies: |
| 40 | +```shell |
| 41 | +$ ldd simple_inference_server |
| 42 | + linux-vdso.so.1 (0x000072b04509d000) |
| 43 | + libfmt.so.10 => /usr/lib/libfmt.so.10 (0x000072b044d90000) |
| 44 | + libssl.so.3 => /usr/lib/libssl.so.3 (0x000072b044cb6000) |
| 45 | + libcrypto.so.3 => /usr/lib/libcrypto.so.3 (0x000072b044600000) |
| 46 | + libcares.so.2 => /usr/lib/libcares.so.2 (0x000072b044c7f000) |
| 47 | + libjsoncpp.so.25 => /usr/lib/libjsoncpp.so.25 (0x000072b044c43000) |
| 48 | + libuuid.so.1 => /usr/lib/libuuid.so.1 (0x000072b04508d000) |
| 49 | + libz.so.1 => /usr/lib/libz.so.1 (0x000072b044c2a000) |
| 50 | + libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x000072b044200000) |
| 51 | + libm.so.6 => /usr/lib/libm.so.6 (0x000072b044b3b000) |
| 52 | + libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x000072b044b0d000) |
| 53 | + libc.so.6 => /usr/lib/libc.so.6 (0x000072b04400f000) |
| 54 | + /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x000072b04509f000) |
| 55 | +``` |
| 56 | + |
| 57 | +## Example of WebForm |
| 58 | + |
| 59 | +The simplest web form that may serve as an interface to this server (provided that it is running on [localhost](http://localhost:7654)) looks like this: |
| 60 | +```html |
| 61 | +<!DOCTYPE html> |
| 62 | +<html lang=""> |
| 63 | +<head> |
| 64 | +</head> |
| 65 | + |
| 66 | +<body> |
| 67 | +<form method="post" enctype="multipart/form-data" action="http://localhost:7654"> |
| 68 | + <p> |
| 69 | + <input type="file" name="file" accept="application/zip"> |
| 70 | + </p> |
| 71 | + <p> |
| 72 | + <input type="submit"/> |
| 73 | + </p> |
| 74 | +</form> |
| 75 | +</body> |
| 76 | +``` |
0 commit comments