Skip to content

Conversation

sageyou
Copy link
Collaborator

@sageyou sageyou commented Nov 27, 2023

Motivation

  1. add pretrain and finetune pipeline of mae
  2. update vit: initialization method and pre-scale of attention

Test Plan

(How should this PR be tested? Do you require special setup to run the test or repro the fixed bug?)

Related Issues and PRs

(Is this PR part of a group of changes? Link the other relevant PRs and Issues here. Use https://help.github.com/en/articles/closing-issues-using-keywords for help on GitHub syntax)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant