v0.18.0

New features

• Support Inferentia instances #1119 #654 (RobertLucian)
• Automatically provision HTTPS API Gateway endpoints for Cortex APIs #1108 #1077 (tthebst)
• Support multi-model endpoints for TensorFlow and ONNX predictors #1107 #890 (RobertLucian)
• Support local Docker images in the local environment #1114 #1094 (RobertLucian)
• Support replica parallelism fields (processes_per_replica and threads_per_process) in the local environment #1158 #960 #1090 (RobertLucian)
• Support a .env file to export environment variables in the API container #1154 #1147 (RobertLucian, spentaur)

Breaking changes

• autoscaling.workers_per_replica and autoscaling.threads_per_worker have been moved/renamed to predictor.processes_per_replica and predictor.threads_per_process (see API configuration docs)
• endpoint and local_port have been moved to a new sub-field called networking (see API configuration docs)
• model has been renamed to model_path in TensorFlow and ONNX predictors (see API configuration docs)

Bug fixes

• Prevent GPU overprovisioning during autoscaling #1111 #1085 (vishalbollu)

New examples

• tensorflow/image-classifier-resnet50 and pytorch/image-classifier-resnet50 to demonstrate using Inferentia #1119 (RobertLucian)
• pytorch/multi-model-text-analyzer, tensorflow/multi-model-classifier, and onnx/multi-model-classifier to demonstrate multi-model APIs #1107 (RobertLucian)

New guides

• Multi-model endpoints using TensorFlow and ONNX predictors #1107 (RobertLucian)

Docs

• Add API architecture diagram #1126 (deliahu)
• Add documentation for configuring the CLI on a new machine #1127 (deliahu, javithe7)

Misc

• Call predictor __init__() from the request's threadpool to avoid mutlithreadding issues in some ML frameworks when using 1 thread #1146 (deliahu)
• Allow changing an API's endpoint to not trigger a rolling replica update #1155 #1116 (deliahu)
• Set the default shell to bash in Predictor Dockerfiles (which simplifies using them as base images for custom-built images) #1104 #1086 (RobertLucian)
• Move endpoint and local_port to networking API config #1151 #1091 (deliahu)
• Rename model to model_path in API config #1150 #1115 (deliahu)
• Use cluster name for cloudwatch metrics namespace #1138 (deliahu)
• Misc UI improvements #1159 #1084 #1152 #1136 #1128 a52f0a0 #1096 (deliahu)

v0.17.1

Bug fixes

• Fix bug which caused installation of packages from conda-packages.txt to get stuck when running Cortex locally #1101 #1100 (dsuess)

New guides

• Multi-model endpoints using the Python predictor #1081 (deliahu)

Docs

• Improve tutorial #1099 #1098 (ssami)
• Misc docs improvements #1095 e15cc62 8023ff9 (deliahu, vishalbollu)

v0.17.0

New features

• Support arbitrary API request payload content types, including raw bytes and form fields (not just JSON) #1062, #332, #917 (deliahu)
• Support custom SSL certificates for the API load balancer #1069, #326, #1066 (vishalbollu)
• Add a cloudwatch dashboard to show metrics for each running API #1054, #855 (tthebst)
• Allow for custom tagging of AWS resources created by cortex (and add the cortex.dev/cluster-name tag by default) #1031, #854, #856 (vishalbollu)
• Expose request query parameters to the predictor's predict() function #1062, #546 (deliahu)
• Expose request headers to the predictor's predict() function #1062 (deliahu)
• Allow users to change the Python version via conda-packages.txt #1052, #1051 (RobertLucian)

Bug fixes

• Fix bug which caused a validation error when running TensorFlow or ONNX locally with locally saved models #1075 (RobertLucian)
• Enable tty on local docker API containers to avoid strange characters showing up in cortex logs #1067 (deliahu)
• Allow cluster's min_instances and max_instances to be updated at the same time #1050, #840 (deliahu)

New examples

• Update pytorch/sentiment-analyzer to use the transformers library #1064 (deliahu)

New guides

• Viewing API metrics on the CloudWatch dashboard (deliahu)

Docs

• Add architecture diagram #1042, #1013 (deliahu)
• Document how to install packages from private PyPI indexes #1072 (RobertLucian)

Misc

• Add zsh completion #1024, #1020 (deliahu)
• Rename cortex cluster update to cortex cluster configure #1035, #887 (zouyee)
• Add instance and pricing information to cortex cluster info output #1053, #835, #935 (deliahu)
• Rename tracker to monitoring #1041, #869 (deliahu)
• Change the default cortex region to us-east-1 #1063 (deliahu)
• Disable cluster logging #1029, #888 (deliahu)
• Add git to API images #1068 (RobertLucian)
• Add upper bound validation for max_replica_concurrency #1025 (zouyee)
• Misc UI/UX improvements #1032, #1023, #1033, #1036, #1045, #1047, #1049, #1044, 93032e2, #1065, #726, #1048, #894, #1043, #1028, #933, #1027, #934, #1026 (deliahu, vishalbollu)

Breaking changes

• cortex cluster update has been renamed to cortex cluster configure (so it won't be misinterpreted as updating the Cortex version of the cluster)
• the tracker field in API configuration has been renamed to monitoring (to be consistent with the other field names)

v0.16.1

Bug fixes

• Read and validate TensorFlow and ONNX models from buckets in any region #1059 (vishalbollu)

v0.16.0

New features

• Support deploying APIs locally #973 #109 (vishalbollu)
• Enable private networking: private subnets for instances, internal API load balancer, and internal operator load balancer #978 #965 #832 #964 (deliahu)
• Support installing system packages via dependencies.sh #880 #852 (RobertLucian)
• Support installing conda packages via conda-packages.txt #880 #844 (RobertLucian)
• Allow for spot instances to be used with a single instance type #979 #886 (RobertLucian)
• Support specifying serving images in API configuration (on a per-API basis) #948 #900 (RobertLucian)
• Add cortex commands to list and remove CLI environments #973 #730 (deliahu)
• Support bytes and starlette.responses.Response response types from predict() #915 #913 (RobertLucian)
• Add slim predictor base images #992 #781 (deliahu)
• Support configuring instance volume type and provisioned IOPS #982 #592 (tthebst)
• Support highly available NAT Gateway #978 #963 (deliahu)
• Add --yes flag to skip prompts on cluster CLI commands #980 #929 (deliahu)

New Examples

• Bart summarizer (using PyTorch) #907 (ismaelc)
• Named entity recognizer (using spacy) #924 (aced125)
• Lite version of the license plate reader #994 (RobertLucian)

New Guides

• Set up AWS API gateway (deliahu)
• Plot response code counts (deliahu)
• Plot API request time (deliahu)
• Plot in-flight requests (deliahu)
• Set up VPC peering (deliahu)
• SSH into AWS instance (RobertLucian)

Docs

• Add environment documentation (deliahu)

Misc

• Use rolling updates for daemonsets when running cortex cluster update #972 #630 (tthebst)
• Switch from Classic Elastic Load Balancer to Network Load Balancer #978 #966 (deliahu)
• Show the original error message when encountering "invalid AWS credentials" #918 (deliahu)
• Log unexpected TensorFlow Serving gRPC errors #949 (deliahu)

Breaking Changes

• Previously, custom serving images were configured in the cluster configuration file (e.g. cluster.yaml would have a line for image_python_serve: my-repo/python-serve:latest). Now, custom images are specified inside the API configuration (e.g. cortex.yaml should have image: my-repo/python-serve:latest in the predictor section of your API configuration). Here's the full documentation for API configuration.
• The names of the serving base images have been updated to be more descriptive, and "slim" images have been added (they are more appropriate to use as base images when building custom images). Here is the full documentation for custom Docker images.
• The debug query parameter to APIs (which caused the input and output to predict() to be logged) has been removed (#985)

v0.15.1

Bug fixes

• Fix admin check for root AWS users #940 (deliahu)
• Fix CORS #942 (vishalbollu)

Misc

• Print panic stack traces #921 (deliahu)

v0.15.0

Bug fixes

• Improve availability zone selection and validation #885 #891 (deliahu)
• Validate predictor implementation using getfullargspec() #902 (vishalbollu)
• Do not remove cluster configuration cache if refresh fails #893 (vishalbollu)

New Examples

• Keras autoencoder for filtering out the noise from text documents #834 (RobertLucian)

Misc

• Check cluster status before executing cluster commands #881 #879 #892 (vishalbollu)
• Check for AWS Administrator IAM access in cluster up and cluster down commands #878 (deliahu)
• Wait for cloudformation stacks to delete during cluster down #876 (vishalbollu)
• Verify cortex operator url during cortex configure #877 (vishalbollu)
• Allow blank bucket values in cluster.yaml #875 (vishalbollu)
• Improve various error messages #895 #896 #897 #899 #905 (deliahu)
• Improve documentation #861 #853 #851 #868 #870 #871, #872, ac481b9 3ad3903 (deliahu, vishalbollu, RobertLucian)

Breaking changes

• Remove json_tricks for encoding API responses (responses from predict() must now be json serializable) #908 (vishalbollu)

v0.14.1

Bug fixes

• Fix JSON parsing before it gets passed to predict() #865 (vishalbollu)
• Support setup.py packages in requirements.txt #864 (deliahu)
• Run TensorFlow Predictor's model validations in the region that contains the bucket #866 (deliahu)

Misc

• Check that cluster image versions match the CLI version #862 (deliahu)
• Add Cython to Python predictor image 4a463ca (deliahu)

v0.14.0

New features

• Support request-based autoscaling #815 #838 #573 (vishalbollu, deliahu)
• Support fine-grained configuration for autoscaling algorithm behavior #815 (deliahu)
• Support configurable in-replica parallelism (i.e. workers, threads) #838 #590 (vishalbollu, deliahu)
• Support configurable request queue length #838 #646 (vishalbollu)
• Support .cortexignore file to exclude files/directories from Cortex project zip #800 #723 (wingkwong)

Bug fixes

• Ensure previous logs are never shown after showing newer ones #792 (deliahu)
• Skip service quota validation in unsupported regions #825 (deliahu)
• Fix prediction metrics when specifying tracker.key #793 (deliahu)

New Examples

• Real-Time License Plate Detector Example Project (YOLOv3, CRAFT, CRNN) #803 (RobertLucian)

Misc

• Show a warning if AWS session token is detected #842 (chrisranderson, vishalbollu)
• Disable NAT gateway #808 (deliahu)
• Add debug information to cluster error messages 5a1a2bc #850 (deliahu)
• Add cluster costs to README #807 #806 (bcjordan, deliahu)
• Document which system packages are installed in Docker images #847 #822 (deliahu)
• Update pytorch examples to use GPU #849 (vishalbollu)
• Install libsndfile1 in API Docker images #826 (deliahu)

Breaking API Changes

• min_replicas, max_replicas, and init_replicas have been moved from the compute configuration key to autoscaling
• max_surge and max_unavailable have been moved from the compute configuration key to update_strategy
• target_cpu_utilization has been removed in favor of the request-based scaling configuration parameters (see the autoscaling docs for a detailed explanation of the new parameters)

v0.13.1

Bug fixes

• Fix cortex cluster update when using spot instances with no on-demand backup nodegroup #787 (vishalbollu)

Misc

• Set locale to en_US.UTF-8 #784 (deliahu, RobertLucian)
• Cause replica to error when pip install fails 394862b (deliahu)
• Query EKS price from AWS Pricing API #783 (deliahu)
• Assert API version before inspecting args in python #789 (vishalbollu)
• Improve healthcheck #788 (vishalbollu)