-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Constraint on split_k
on m * n
#8404
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…t callable to determine split_k.
…t callable to determine split_k. Update constraints dict to accept callable for split_k and invoke that callable to determine split_k.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting
split_k
on m * n
Update constraints dict to accept callable for split_k and invoke that callable to determine split_k.
Spoke to Philippe Tillet offline, there are cases where we need to decide
split_k
dynamically based on the shapes of the operands.Notably, in our case we want batch size invariance, but want to restrict the amount of memory that's otherwise allocated, i.e.
[split_k, m, n]
allocation for the scratch memory to do thebmk,kn->bmn
multiplication, so pass that as an additional constraintmax_allowable_mn
which drop down to split_k = 1 ifm * n > max_allowable_mn
.New contributor declaration
I am not making a trivial change, such as fixing a typo in a comment.
I have written a PR description following these
rules.
I have run
pre-commit run --from-ref origin/main --to-ref HEAD
.Select one of the following.
/test
forlit
tests/unittest
for C++ tests/python/test
for end-to-end testsADDED
.Select one of the following.
lit
tests.lit
tests I have added follow these best practices,including the "tests should be minimal" section. (Usually running Python code
and using the instructions it generates is not minimal.)