Skip to content

Traffic for all deployments set to zero during any deployment #182

@LittleColin

Description

@LittleColin

The begin_create_or_update call for the endpoint is always made during deployment. If there are existing deployments, because these are not being specified as part of this update call, this call causes the traffic to be set to 0% for existing deployments. This is shortly followed up by a call to set the traffic to 100% again, but there is a window during deployment where traffic is 0%.

https://github.com/microsoft/llmops-promptflow-template/blob/d8ee083a730f00e9cd23a4ec39c2f884eab2e56f/llmops/common/deployment/provision_endpoint.py#L74

Propose that the endpoint is not updated if it already exists since there are no properties on the endpoint that need updating on each deployment, except perhaps for the build id tag, but it's likely to be more appropriate to leave that tag pointing to the build that caused the endpoint to be created anyway.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions