Skip to content

MOE with JetStream #277

@patrick-toulme

Description

@patrick-toulme

Could someone explain or point to a doc that explains how MOE is implemented on Jetstream? Specifically, the all-to-all communications, static vs dynamic, sparse matmuls.

I would like to understand how XLA compiles MOE.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions