|
| 1 | +# Creating REST Service with MONAI Deploy Application |
| 2 | + |
| 3 | +This application provides an example of how to make a MONAI Deploy app run as a REST service on the [Aidoc](https://www.aidoc.com/) platform. It is compliant with its [third party integration API](https://ai-partner-sdk.aidoc-cloud.com/prod/api/third-parties/doc/#), and the results [callback message schema](https://ai-partner-sdk.aidoc-cloud.com/prod/api/aidoc-callback/doc/#). |
| 4 | + |
| 5 | +This example uses a subset of the callback message attributes, covering only the required ones as well as some common attributes. For the full message definition, please contact Aidoc directly. |
| 6 | + |
| 7 | +## High Level Design |
| 8 | + |
| 9 | +The high-level design of this REST service involves a few key components: |
| 10 | + |
| 11 | +1. **MONAI Deploy Application**: The core AI logic is encapsulated in a standard MONAI Deploy application (e.g., `AISpleenSegApp`), which is built and tested as a regular containerized workload. The application is responsible for generating the inference results using Pydantic classes that are based on Aidoc's callback message schema. It then reports these results as a JSON string via a callback function provided during its construction. |
| 12 | +2. **REST Service**: A lightweight REST application, built using Flask, acts as the front-end. It exposes endpoints to start and check the status of a processing job. |
| 13 | +3. **Request Handling**: |
| 14 | + - When the REST service receives a request to process data, it handles only one request at a time, as per the API specification. |
| 15 | + - It creates an instance of the MONAI Deploy application. |
| 16 | + - It sets the necessary environment variables for the input and output folders for the processing execution. |
| 17 | + - Crucially, it delegates the execution of the MONAI Deploy application to a separate background thread to avoid blocking the web server. |
| 18 | +4. **Callback Mechanism**: |
| 19 | + - The callback message, which includes the AI results and a list of output files, is generated within the MONAI Deploy application at the end of its run. |
| 20 | + - This message is then passed to a callback function that was provided by the REST service during the creation of the MONAI Deploy app instance. |
| 21 | + - The REST service, upon receiving the callback, is then responsible for making the final `POST` request to the external callback endpoint specified by the original caller. |
| 22 | + |
| 23 | +This design separates the core AI application from the web-serving logic, allowing each to be developed and tested independently. |
| 24 | + |
| 25 | +### Component Diagram |
| 26 | + |
| 27 | +This diagram shows the static components of the system and their relationships. |
| 28 | + |
| 29 | +```mermaid |
| 30 | +C4Component |
| 31 | + title Component Diagram for MONAI Deploy REST Service |
| 32 | +
|
| 33 | + Person(client, "External Client", "e.g., Aidoc Platform") |
| 34 | +
|
| 35 | + Container_Boundary(rest_service_container, "REST Service") { |
| 36 | + Component(flask, "Flask App", "Python, Flask", "Handles HTTP requests, manages processing threads, and sends callbacks.") |
| 37 | + } |
| 38 | +
|
| 39 | + Container_Boundary(monai_app_container, "MONAI Deploy Application") { |
| 40 | + Component(monai_app, "AISpleenSegApp", "Python, MONAI Deploy SDK", "Orchestrates the AI inference pipeline and prepares the result message.") |
| 41 | + Component(operators, "MONAI Deploy Operators", "Python, MONAI Deploy SDK", "Perform tasks like data loading, inference, and writing results.") |
| 42 | + } |
| 43 | +
|
| 44 | + System_Ext(fs, "Filesystem", "Stores input/output data.") |
| 45 | +
|
| 46 | + Rel(client, flask, "1. Sends processing request", "JSON/HTTPS") |
| 47 | + Rel(flask, client, "2. Responds 202 Accepted") |
| 48 | + Rel(flask, monai_app, "3. Instantiates & runs in background thread") |
| 49 | + Rel(monai_app, operators, "4. Uses operators to process data") |
| 50 | + Rel(monai_app, fs, "5. Reads from & Writes to") |
| 51 | + Rel(monai_app, flask, "6. Invokes callback on completion") |
| 52 | + Rel(flask, client, "7. Sends final results", "JSON/HTTPS") |
| 53 | +``` |
| 54 | + |
| 55 | +### Sequence Diagram |
| 56 | + |
| 57 | +This diagram illustrates the sequence of interactions for a processing job, including status checks. |
| 58 | + |
| 59 | +```mermaid |
| 60 | +sequenceDiagram |
| 61 | + actor Client |
| 62 | + participant REST Service |
| 63 | + participant "MONAI Deploy App Thread" as AppThread |
| 64 | + participant AISpleenSegApp |
| 65 | +
|
| 66 | + Client->>+REST Service: POST /process (payload) |
| 67 | + REST Service-->>-Client: HTTP 202 Accepted |
| 68 | + REST Service->>+AppThread: Spawn thread(run_processing) |
| 69 | +
|
| 70 | + opt While processing is busy |
| 71 | + Client->>+REST Service: POST /process (payload) |
| 72 | + REST Service-->>-Client: HTTP 409 Conflict |
| 73 | +
|
| 74 | + Client->>+REST Service: GET /status |
| 75 | + REST Service-->>-Client: HTTP 200 OK ("status": "PROCESSING") |
| 76 | + end |
| 77 | +
|
| 78 | + AppThread->>+AISpleenSegApp: Create instance(status_callback) |
| 79 | + AppThread->>AISpleenSegApp: run() |
| 80 | + Note over AISpleenSegApp: Executes processing pipeline... |
| 81 | + AISpleenSegApp->>AISpleenSegApp: Formats success message |
| 82 | + AISpleenSegApp->>AppThread: status_callback (message) |
| 83 | +
|
| 84 | + AISpleenSegApp-->>AppThread: run() completes successfully |
| 85 | + deactivate AISpleenSegApp |
| 86 | + AppThread->>AppThread: Formats final message |
| 87 | + AppThread->>+Client: POST callback_url (Final Results) |
| 88 | + Client-->>-AppThread: HTTP 200 OK |
| 89 | +
|
| 90 | + Note over REST Service: Processing status set to IDLE. |
| 91 | + deactivate AppThread |
| 92 | +
|
| 93 | + Client->>+REST Service: GET /status |
| 94 | + REST Service-->>-Client: HTTP 200 OK ("status": "IDLE") |
| 95 | +``` |
| 96 | + |
| 97 | +## How to Run in Development Environment |
| 98 | + |
| 99 | +Change your working directory to the one containing this README file and the `restful_app` folder. |
| 100 | + |
| 101 | +1. **Install Dependencies** |
| 102 | + |
| 103 | + Create and activate a Python virtual environment. |
| 104 | + |
| 105 | + ```bash |
| 106 | + pip install -r restful_app/requirements.txt |
| 107 | + ``` |
| 108 | +2. **Download Test Data and Set Environment Variables** |
| 109 | + The model and test DICOM series are shared on Google Drive, which requires gaining access permission first. The zip file is available [here](https://drive.google.com/uc?id=1IwWMpbo2fd38fKIqeIdL8SKTGvkn31tK). |
| 110 | + |
| 111 | + Please make a request so that it can be shared with a specific Gmail account. |
| 112 | + |
| 113 | + `gdown` may also work: |
| 114 | + ``` |
| 115 | + pip install gdown |
| 116 | + gdown https://drive.google.com/uc?id=1IwWMpbo2fd38fKIqeIdL8SKTGvkn31tK |
| 117 | + ``` |
| 118 | + |
| 119 | + Unzip the file to local folders. If deviating from the path noted below, please adjust the env var values. |
| 120 | + |
| 121 | + ``` |
| 122 | + unzip -o "ai_spleen_seg_bundle_data.zip" |
| 123 | + rm -rf models && mkdir -p models/model && mv model.ts models/model && ls models/model |
| 124 | + ``` |
| 125 | + |
| 126 | + Set the environment variables so that the model can be found by the Spleen Seg app. These settings are also consolidated in the `env_settings.sh` script. |
| 127 | + |
| 128 | + ``` |
| 129 | + export HOLOSCAN_MODEL_PATH=models |
| 130 | + ``` |
| 131 | + |
| 132 | +3. **Run the Web Application** |
| 133 | + |
| 134 | + ```bash |
| 135 | + python restful_app/app.py |
| 136 | + ``` |
| 137 | + |
| 138 | + The application will start on `http://127.0.0.1:5000`. |
| 139 | + |
| 140 | +## Test API Endpoints |
| 141 | + |
| 142 | +A simple test client is provided, which makes calls to the endpoint, as well as providing a callback endpoint to receive message content at the specified port. |
| 143 | + |
| 144 | +Open another console window and change directory to the same as this file. |
| 145 | + |
| 146 | +Set the environment vars so that the test script can get the input DCM and write the callback contents. Also, once the REST app completes each processing, the Spleen Seg app's output will also be saved in the output folder specified below (the script passes the output folder via the REST API). |
| 147 | +
|
| 148 | +``` |
| 149 | +export HOLOSCAN_INPUT_PATH=dcm |
| 150 | +export HOLOSCAN_OUTPUT_PATH=output_restful_app |
| 151 | +``` |
| 152 | +
|
| 153 | +Run the test script, and examine its console output. |
| 154 | +
|
| 155 | +``` |
| 156 | +source test_endpoints.sh |
| 157 | +``` |
| 158 | +
|
| 159 | +Once the script completes, examine the `output` folder, which should contain the following (the DICOM file name will be different): |
| 160 | +
|
| 161 | +``` |
| 162 | +output |
| 163 | +├── 1.2.826.0.1.3680043.10.511.3.22611096892439837402906545708809852.dcm |
| 164 | +└── stl |
| 165 | + └── spleen.stl |
| 166 | +``` |
| 167 | +
|
| 168 | +The script can be run multiple times or modified to loop with different output folder settings. |
| 169 | +
|
| 170 | +### Check Status |
| 171 | +
|
| 172 | +- **URL**: `/status` |
| 173 | +- **Method**: `GET` |
| 174 | +- **Description**: Checks the current status of the processor. |
| 175 | +- **Success Response**: |
| 176 | + - **Code**: 200 OK |
| 177 | + - **Content**: `{ "status": "IDLE" }` or `{ "status": "PROCESSING" }` |
| 178 | +
|
| 179 | +### Process Data |
| 180 | +
|
| 181 | +- **URL**: `/process` |
| 182 | +- **Method**: `POST` |
| 183 | +- **Description**: Starts a new processing job. |
| 184 | +- **Body**: |
| 185 | +
|
| 186 | + ```json |
| 187 | + { |
| 188 | + "input_folder": "/path/to/your/input/data", |
| 189 | + "output_folder": "/path/to/your/output/folder", |
| 190 | + "callback_url": "http://your-service.com/callback" |
| 191 | + } |
| 192 | + ``` |
| 193 | +
|
| 194 | +- **Success Response**: |
| 195 | + - **Code**: 202 ACCEPTED |
| 196 | + - **Content**: `{ "message": "Processing started." }` |
| 197 | +- **Error Response**: |
| 198 | + - **Code**: 409 CONFLICT |
| 199 | + - **Content**: `{ "error": "Processor is busy." }` |
| 200 | + - **Code**: 400 BAD REQUEST |
| 201 | + - **Content**: `{ "error": "Missing required fields." }` |
| 202 | +
|
| 203 | +### Callback |
| 204 | +
|
| 205 | +When processing is complete, the application will send a `POST` request to the `callback_url` provided in the process request. The body of the callback will be similar to this: |
| 206 | +
|
| 207 | +```json |
| 208 | +{ |
| 209 | + "run_success": true, |
| 210 | + "output_files": ["output_spleen/1.2.826.0.1.3680043.10.511.3.13787585732573161684951883631909444.dcm", "output_spleen/stl/spleen.stl"], |
| 211 | + "error_message": null, |
| 212 | + "error_code": null, |
| 213 | + "result": { |
| 214 | + "aggregated_results": { |
| 215 | + "name": "Spleen Segmentation", |
| 216 | + "algorithm_class": ["Measurement"] |
| 217 | + }, |
| 218 | + "detailed_results":{ |
| 219 | + "Spleen Segmentation": { |
| 220 | + "detection": null, |
| 221 | + "measurement": { |
| 222 | + "measurements_text": "Spleen segmentation completed successfully.", "key_slice_instance_uid": null, |
| 223 | + "key_measurement": null |
| 224 | + }, |
| 225 | + "classification": null |
| 226 | + } |
| 227 | + } |
| 228 | + } |
| 229 | +} |
| 230 | +``` |
| 231 | +
|
| 232 | +Or in case of an error: |
| 233 | +
|
| 234 | +```json |
| 235 | +{ |
| 236 | + "run_success": false, |
| 237 | + "error_message": "E.g., Model network is not load and model file not found.", |
| 238 | + "error_code": 500 |
| 239 | +} |
| 240 | +``` |
| 241 | +
|
| 242 | +Please note: The test script uses a simple `nc` command to emulate the callback service. This lightweight approach may sometimes lead to timeout errors on the client side (the REST service), preventing the test script from capturing the callback message. If this occurs, running the script again is a known workaround. |
| 243 | +
|
| 244 | +## Packaging and Testing the REST Service Container |
| 245 | +
|
| 246 | +### Packaging the Application |
| 247 | +
|
| 248 | +To package the REST service application into a MONAI App Package (MAP) container, you can use the MONAI Deploy CLI. The following is an example command, run with the current working directory as the parent of `restful_app`: |
| 249 | +
|
| 250 | +```bash |
| 251 | +monai-deploy package restful_app -m models/spleen_ct -c restful_app/app.yaml -t monai-rest:1.0 --platform x86_64 -l DEBUG |
| 252 | +``` |
| 253 | +
|
| 254 | +This command packages the `restful_app` directory, includes the specified model, uses `app.yaml` for configuration, and tags the resulting Docker image as `monai-rest-x64-workstation-dgpu-linux-amd64:1.0`, which includes the target platform name. |
| 255 | +
|
| 256 | +Note that the model folder should contain only the model file (e.g., `model.ts`) or subfolders that each contain only a model file. |
| 257 | +
|
| 258 | +
|
| 259 | +### Running the MAP Container |
| 260 | +
|
| 261 | +While you can run MAPs with the `monai-deploy run` command, it currently has limitations regarding the mapping of arbitrary volumes and passing extra environment variables that are necessary for this REST service. Therefore, it's required to use the `docker run` command directly (or a platform specific equivalent) to have full control over the container's execution environment. |
| 262 | +
|
| 263 | +```bash |
| 264 | +docker run --gpus=all --network host --name my_monai_rest_service -t --rm \ |
| 265 | +-v <host folder for staging input folder>:/var/holoscan/input/ \ |
| 266 | +-v <host folder for saving output folder>:/var/holoscan/output/ \ |
| 267 | +-v <host folder for saving temp files>:/var/holoscan/ \ |
| 268 | +-e FLASK_HOST="0.0.0.0" \ |
| 269 | +-e FLASK_PORT="5000" \ |
| 270 | +--entrypoint /bin/bash monai-rest-x64-workstation-dgpu-linux-amd64:1.0 -c "python3 -u /opt/holoscan/app/" |
| 271 | +``` |
| 272 | +
|
| 273 | +**Command parameters** |
| 274 | +
|
| 275 | +- `--gpus=all`: Exposes all available host GPUs to the container, which is necessary for CUDA-based inference. A specific CUDA device ID can also be used. |
| 276 | +- `--network host`: The container shares the host's network stack, making the Flask server directly accessible on the host's IP address and port (e.g., `http://localhost:5000`). |
| 277 | +- `--name my_monai_rest_service`: Assigns a convenient name to the running container. |
| 278 | +- `-t --rm`: Allocates a pseudo-terminal and automatically removes the container when it stops. |
| 279 | +- `-v <host folder for staging input folder>:/var/holoscan/input/`: Mounts a host directory into the container as `/var/holoscan/input/`. This allows the REST service to access input files using an internal container path. For example, the inference input (e.g., a DICOM study's instance files) should be staged in a subfolder on the host, e.g. `my_test_study`, and the client request message must use the corresponding internal container path (e.g., `/var/holoscan/input/my_test_study`). |
| 280 | +- `-v <host folder for saving output folder>:/var/holoscan/output/`: Mounts a host directory into the container as `/var/holoscan/output/`, allowing the REST service to save the inference result files. |
| 281 | +- `-e FLASK_HOST="0.0.0.0"` and `-e FLASK_PORT="5000"`: These environment variables configure the Flask-based REST application to be accessible from outside the container on the specified port. |
| 282 | +- `--entrypoint /bin/bash ... -c "python3 -u /opt/holoscan/app/"`: This overrides the default entrypoint of the MAP container. Instead of running the MONAI Deploy application directly, it starts a bash shell that executes the command to run the Flask application, effectively starting the REST service. |
| 283 | + |
| 284 | +The simple test client, `test_endpoints.sh`, can be used to test the REST service container. It requires a couple of simple changes to use the container's internal folder paths for I/O. For example: |
| 285 | +
|
| 286 | +```bash |
| 287 | +# Get the absolute path to the input and output directories |
| 288 | +INPUT_DIR="/var/holoscan/input/spleen_ct_tcia" |
| 289 | +OUTPUT_DIR="/var/holoscan/output/output_spleen_rest" |
| 290 | +``` |
0 commit comments