[feature] help us implement unified attention

[sayakpaul]

Currently, we support ring and ulysses attention backends for context parallelism. More info: https://huggingface.co/docs/diffusers/main/en/api/parallel

Unified CP is a generalization of performing Ulysses and Ring together. Both those methods become special subcases of Unified attention. Paper: https://arxiv.org/abs/2405.07719v3

Before proceeding with a PR, please use this thread to discuss your proposal and we can take it from there.

[tolgacangoz]

I could handle this after our MAGI-1 PR, if no one else will work on it before the PR's completion. MAGI-1 proposes to include another CP-specialized attention module: MagiAttention.

[yiyixuxu]

hey @tolgacangoz
Thanks for offering! However, we'd like to give other community members an opportunity to work on this feature as you already have many PR open.
Appreciate your understanding!