Skip to content

Commit c3c1da7

Browse files
ospillingerospillinger
authored
Update docs (#1765)
1 parent 324efd1 commit c3c1da7

File tree

6 files changed

+94
-70
lines changed

6 files changed

+94
-70
lines changed

‎README.md

Lines changed: 23 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -51,33 +51,34 @@ cortex is ready!
5151
#### Define an API
5252

5353
```python
54-
class PythonPredictor:
55-
def __init__(self, config):
56-
from transformers import pipeline
54+
# predictor.py
5755

58-
self.model = pipeline(task="text-generation")
56+
from transformers import pipeline
5957

60-
def predict(self, payload):
61-
return self.model(payload["text"])[0]
58+
class PythonPredictor:
59+
def __init__(self, config):
60+
self.model = pipeline(task="text-generation")
6261

63-
requirements = ["tensorflow", "transformers"]
62+
def predict(self, payload):
63+
return self.model(payload["text"])[0]
6464
```
6565

6666
#### Configure an API
6767

68-
```python
69-
api_spec = {
70-
"name": "text-generator",
71-
"kind": "RealtimeAPI",
72-
"compute": {
73-
"gpu": 1,
74-
"mem": "8Gi"
75-
},
76-
"autoscaling": {
77-
"min_replicas": 1,
78-
"max_replicas": 10
79-
}
80-
}
68+
```yaml
69+
# text_generator.yaml
70+
71+
- name: text-generator
72+
kind: RealtimeAPI
73+
predictor:
74+
type: python
75+
path: predictor.py
76+
compute:
77+
gpu: 1
78+
mem: 8Gi
79+
autoscaling:
80+
min_replicas: 1
81+
max_replicas: 10
8182
```
8283
8384
<br>
@@ -92,11 +93,8 @@ api_spec = {
9293
9394
#### Deploy to your cluster
9495
95-
```python
96-
import cortex
97-
98-
cx = cortex.client("aws")
99-
cx.create_api(api_spec, predictor=PythonPredictor, requirements=requirements)
96+
```bash
97+
$ cortex deploy text_generator.yaml
10098

10199
# creating http://example.com/text-generator
102100
```

‎docs/workloads/realtime/example.md

Lines changed: 43 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -10,47 +10,73 @@ Deploy realtime APIs that can respond to prediction requests on demand.
1010
* Metrics and log aggregation
1111
* Rolling updates
1212

13-
## How it works
14-
15-
### Install cortex
13+
## Install Cortex
1614

1715
```bash
1816
$ pip install cortex
1917
```
2018

21-
### Spin up a cluster on AWS
19+
## Create a cluster on AWS
20+
21+
```yaml
22+
# cluster.yaml
23+
24+
region: us-east-1
25+
instance_type: g4dn.xlarge
26+
min_instances: 1
27+
max_instances: 3
28+
spot: true
29+
```
2230
2331
```bash
24-
$ cortex cluster up
32+
$ cortex cluster up --config cluster.yaml
2533
```
2634

27-
### Define a realtime API
35+
## Deploy a realtime API
36+
37+
### Implement your API
38+
39+
```bash
40+
$ mkdir text-generator && cd text-generator
41+
$ touch predictor.py requirements.txt text_generator.yaml
42+
```
2843

2944
```python
30-
# text_generator.py
45+
# predictor.py
3146

32-
import cortex
47+
from transformers import pipeline
3348

3449
class PythonPredictor:
3550
def __init__(self, config):
36-
from transformers import pipeline
3751
self.model = pipeline(task="text-generation")
3852

3953
def predict(self, payload):
4054
return self.model(payload["text"])[0]
55+
```
4156

42-
requirements = ["tensorflow", "transformers"]
57+
```python
58+
# requirements.txt
59+
60+
transformers
61+
torch
62+
```
4363

44-
api_spec = {"name": "text-generator", "kind": "RealtimeAPI"}
64+
```yaml
65+
# text_generator.yaml
4566

46-
cx = cortex.client("aws")
47-
cx.create_api(api_spec, predictor=PythonPredictor, requirements=requirements)
67+
- name: text-generator
68+
kind: RealtimeAPI
69+
predictor:
70+
type: python
71+
path: predictor.py
72+
compute:
73+
gpu: 1
4874
```
4975
50-
### Deploy to AWS
76+
### Deploy
5177
5278
```bash
53-
$ python text_generator.py
79+
$ cortex deploy text_generator.yaml
5480
```
5581

5682
### Monitor
@@ -68,10 +94,10 @@ $ cortex logs text-generator
6894
### Make a request
6995

7096
```bash
71-
$ curl https://***.execute-api.us-west-2.amazonaws.com/text-generator -X POST -H "Content-Type: application/json" -d '{"text": "hello world"}'
97+
$ curl http://***.elb.us-west-2.amazonaws.com/text-generator -X POST -H "Content-Type: application/json" -d '{"text": "hello world"}'
7298
```
7399

74-
### Delete the API
100+
### Delete
75101

76102
```bash
77103
$ cortex delete text-generator

‎docs/workloads/realtime/predictors.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -285,7 +285,7 @@ Here are some examples:
285285
#### Making the request
286286

287287
```bash
288-
$ curl https://***.amazonaws.com/my-api \
288+
$ curl http://***.amazonaws.com/my-api \
289289
-X POST -H "Content-Type: application/json" \
290290
-d '{"key": "value"}'
291291
```
@@ -308,7 +308,7 @@ class PythonPredictor:
308308
#### Making the request
309309

310310
```bash
311-
$ curl https://***.amazonaws.com/my-api \
311+
$ curl http://***.amazonaws.com/my-api \
312312
-X POST -H "Content-Type: application/octet-stream" \
313313
--data-binary @object.pkl
314314
```
@@ -349,7 +349,7 @@ class PythonPredictor:
349349
#### Making the request
350350

351351
```bash
352-
$ curl https://***.amazonaws.com/my-api \
352+
$ curl http://***.amazonaws.com/my-api \
353353
-X POST \
354354
-F "text=@text.txt" \
355355
-F "object=@object.pkl" \
@@ -384,7 +384,7 @@ class PythonPredictor:
384384
#### Making the request
385385

386386
```bash
387-
$ curl https://***.amazonaws.com/my-api \
387+
$ curl http://***.amazonaws.com/my-api \
388388
-X POST \
389389
-d "key=value"
390390
```
@@ -407,7 +407,7 @@ class PythonPredictor:
407407
#### Making the request
408408

409409
```bash
410-
$ curl https://***.amazonaws.com/my-api \
410+
$ curl http://***.amazonaws.com/my-api \
411411
-X POST -H "Content-Type: text/plain" \
412412
-d "hello world"
413413
```

‎test/apis/batch/image-classifier/README.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -159,7 +159,7 @@ $ cortex get image-classifier --env aws
159159
160160
no submitted jobs
161161
162-
endpoint: https://abcdefg.execute-api.us-west-2.amazonaws.com/image-classifier
162+
endpoint: http://***.elb.us-west-2.amazonaws.com/image-classifier
163163
```
164164

165165
<br>
@@ -181,7 +181,7 @@ $ export CORTEX_DEST_S3_DIR=<YOUR_S3_DIRECTORY> # e.g. export CORTEX_DEST_S3_DI
181181
Now that you've deployed a Batch API, you are ready to submit jobs. You can provide image urls directly in the request by specifying the urls in `item_list`. The curl command below showcases how to submit image urls in the request.
182182

183183
```bash
184-
$ export BATCH_API_ENDPOINT=<BATCH_API_ENDPOINT> # e.g. export BATCH_API_ENDPOINT=https://abcdefg.execute-api.us-west-2.amazonaws.com/image-classifier
184+
$ export BATCH_API_ENDPOINT=<BATCH_API_ENDPOINT> # e.g. export BATCH_API_ENDPOINT=https://***.elb.us-west-2.amazonaws.com/image-classifier
185185
$ export CORTEX_DEST_S3_DIR=<YOUR_S3_DIRECTORY> # e.g. export CORTEX_DEST_S3_DIR=s3://my-bucket/dir
186186
$ curl $BATCH_API_ENDPOINT \
187187
-X POST -H "Content-Type: application/json" \
@@ -223,23 +223,23 @@ $ cortex get image-classifier --env aws
223223
job id status progress start time duration
224224
69d6faf82e4660d3 running 0/3 20 Jul 2020 01:07:44 UTC 3m26s
225225
226-
endpoint: https://abcdefg.execute-api.us-west-2.amazonaws.com/image-classifier
226+
endpoint: http://***.elb.us-west-2.amazonaws.com/image-classifier
227227
```
228228

229229
### Get the job status with an HTTP request
230230

231-
You can make a GET request to your `<BATCH_API_ENDPOINT>/JOB_ID` to get the status of your job.
231+
You can make a GET request to your `<BATCH_API_ENDPOINT>?JOB_ID` to get the status of your job.
232232

233233
```bash
234-
$ curl https://abcdefg.execute-api.us-west-2.amazonaws.com?jobID=69d6faf82e4660d3
234+
$ curl http://***.elb.us-west-2.amazonaws.com/image-classifier?jobID=69d6faf82e4660d3
235235
236236
{
237237
"job_status":{
238238
"job_id":"69d6faf82e4660d3",
239239
"api_name":"image-classifier",
240240
...
241241
},
242-
"endpoint":"https://abcdefg.execute-api.us-west-2.amazonaws.com/image-classifier"
242+
"endpoint":"https://***.elb.us-west-2.amazonaws.com/image-classifier"
243243
}
244244
```
245245

@@ -265,7 +265,7 @@ worker stats
265265
requested initializing running failed succeeded
266266
1 1 0 0 0
267267
268-
job endpoint: https://abcdefg.execute-api.us-west-2.amazonaws.com/image-classifier/69d6faf82e4660d3
268+
job endpoint: http://***.elb.us-west-2.amazonaws.com/image-classifier/69d6faf82e4660d3
269269
```
270270

271271
### Stream logs
@@ -328,7 +328,7 @@ Before we submit the job, let's perform a dry run to ensure that only the desire
328328
Get the endpoint from `cortex get image-classifier` if you haven't done so already.
329329

330330
```bash
331-
$ export BATCH_API_ENDPOINT=<BATCH_API_ENDPOINT> # e.g. export BATCH_API_ENDPOINT=https://abcdefg.execute-api.us-west-2.amazonaws.com/image-classifier
331+
$ export BATCH_API_ENDPOINT=<BATCH_API_ENDPOINT> # e.g. export BATCH_API_ENDPOINT=https://***.elb.us-west-2.amazonaws.com/image-classifier
332332
$ export CORTEX_DEST_S3_DIR=<YOUR_S3_DIRECTORY> # e.g. export CORTEX_DEST_S3_DIR=s3://my-bucket/dir
333333
$ curl $BATCH_API_ENDPOINT?dryRun=true \
334334
-X POST -H "Content-Type: application/json" \
@@ -366,7 +366,7 @@ When you submit a job specifying `delimited_files`, your Batch API will get all
366366
In this example `urls_0.json` and `urls_1.json` each contain 8 urls. Let's classify the images from the URLs listed in those 2 files.
367367

368368
```bash
369-
$ export BATCH_API_ENDPOINT=<BATCH_API_ENDPOINT> # e.g. export BATCH_API_ENDPOINT=https://abcdefg.execute-api.us-west-2.amazonaws.com/image-classifier
369+
$ export BATCH_API_ENDPOINT=<BATCH_API_ENDPOINT> # e.g. export BATCH_API_ENDPOINT=https://***.elb.us-west-2.amazonaws.com/image-classifier
370370
$ export CORTEX_DEST_S3_DIR=<YOUR_S3_DIRECTORY> # e.g. export CORTEX_DEST_S3_DIR=s3://my-bucket/dir
371371
$ curl $BATCH_API_ENDPOINT \
372372
-X POST -H "Content-Type: application/json" \
@@ -438,7 +438,7 @@ We'll classify the 16 images that can be found here `s3://cortex-examples/image-
438438
Let's do a dry run to make sure the correct list of images will be submitted to the job.
439439

440440
```bash
441-
$ export BATCH_API_ENDPOINT=<BATCH_API_ENDPOINT> # e.g. export BATCH_API_ENDPOINT=https://abcdefg.execute-api.us-west-2.amazonaws.com/image-classifier
441+
$ export BATCH_API_ENDPOINT=<BATCH_API_ENDPOINT> # e.g. export BATCH_API_ENDPOINT=https://***.elb.us-west-2.amazonaws.com/image-classifier
442442
$ export CORTEX_DEST_S3_DIR=<YOUR_S3_DIRECTORY> # e.g. export CORTEX_DEST_S3_DIR=s3://my-bucket/dir
443443
$ curl $BATCH_API_ENDPOINT?dryRun=true \
444444
-X POST -H "Content-Type: application/json" \
@@ -475,7 +475,7 @@ validations passed
475475
Let's actually submit the job now. Your Batch API will get all of the input S3 files based on `s3_paths` and will apply the filters specified in `includes` and `excludes`.
476476

477477
```bash
478-
$ export BATCH_API_ENDPOINT=<BATCH_API_ENDPOINT> # e.g. export BATCH_API_ENDPOINT=https://abcdefg.execute-api.us-west-2.amazonaws.com/image-classifier
478+
$ export BATCH_API_ENDPOINT=<BATCH_API_ENDPOINT> # e.g. export BATCH_API_ENDPOINT=https://***.elb.us-west-2.amazonaws.com/image-classifier
479479
$ export CORTEX_DEST_S3_DIR=<YOUR_S3_DIRECTORY> # e.g. export CORTEX_DEST_S3_DIR=s3://my-bucket/dir
480480
$ curl $BATCH_API_ENDPOINT \
481481
-X POST -H "Content-Type: application/json" \
@@ -541,7 +541,7 @@ You can download the aggregated results file with `aws s3 cp $CORTEX_DEST_S3_DIR
541541
You can stop a running job by sending a DELETE request to `<BATCH_API_ENDPOINT>/<JOB_ID>`.
542542

543543
```bash
544-
$ export BATCH_API_ENDPOINT=<BATCH_API_ENDPOINT> # e.g. export BATCH_API_ENDPOINT=https://abcdefg.execute-api.us-west-2.amazonaws.com/image-classifier
544+
$ export BATCH_API_ENDPOINT=<BATCH_API_ENDPOINT> # e.g. export BATCH_API_ENDPOINT=https://***.elb.us-west-2.amazonaws.com/image-classifier
545545
$ curl -X DELETE $BATCH_API_ENDPOINT?jobID=69d96a01ea55da8c
546546
547547
stopped job 69d96a01ea55da8c

‎test/apis/pytorch/text-generator/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,7 @@ $ cortex get text-generator --env aws
8585
status up-to-date requested last update avg request 2XX
8686
live 1 1 1m - -
8787

88-
endpoint: https://***.execute-api.us-west-2.amazonaws.com/text-generator
88+
endpoint: http://***.elb.us-west-2.amazonaws.com/text-generator
8989
```
9090

9191
## Run on GPUs

‎test/apis/traffic-splitter/README.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -41,30 +41,30 @@ iris-classifier-onnx 30 live 1 1m -
4141
iris-classifier-tf 70 live 1 1m - - -
4242

4343
last updated: 1m
44-
endpoint: https://abcedefg.execute-api.us-west-2.amazonaws.com/iris-classifier
45-
example curl: curl https://abcedefg.execute-api.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
44+
endpoint: http://***.elb.us-west-2.amazonaws.com/iris-classifier
45+
example curl: curl http://***.elb.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
4646
...
4747
```
4848

4949
## Make multiple requests
5050

5151
```bash
52-
$ curl https://abcedefg.execute-api.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
52+
$ curl http://***.elb.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
5353
setosa
5454

55-
$ curl https://abcedefg.execute-api.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
55+
$ curl http://***.elb.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
5656
setosa
5757

58-
$ curl https://abcedefg.execute-api.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
58+
$ curl http://***.elb.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
5959
setosa
6060

61-
$ curl https://abcedefg.execute-api.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
61+
$ curl http://***.elb.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
6262
setosa
6363

64-
$ curl https://abcedefg.execute-api.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
64+
$ curl http://***.elb.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
6565
setosa
6666

67-
$ curl https://abcedefg.execute-api.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
67+
$ curl http://***.elb.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
6868
setosa
6969
```
7070

@@ -83,8 +83,8 @@ iris-classifier-onnx 30 live 1 4m 6.00791 ms
8383
iris-classifier-tf 70 live 1 4m 5.81867 ms 5 -
8484

8585
last updated: 4m
86-
endpoint: https://comtf6hs64.execute-api.us-west-2.amazonaws.com/iris-classifier
87-
example curl: curl https://comtf6hs64.execute-api.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
86+
endpoint: http://***.elb.us-west-2.amazonaws.com/iris-classifier
87+
example curl: curl http://***.elb.us-west-2.amazonaws.com/iris-classifier -X POST -H "Content-Type: application/json" -d @sample.json
8888
...
8989
```
9090

0 commit comments

Comments
 (0)