Skip to content

Conversation

YZP17121579
Copy link

GptJ & GptNeoX may generate random outputs when using batch inference mode and no prefix prompt.
The problem is caused by the nullptr check
in

if (tiled_prefix_prompt_lengths != nullptr) {

@BasicCoder
Copy link

BasicCoder commented Aug 12, 2023

I think this is a duplicate solution of #716 which is more elegant and efficient.

@RobotGF RobotGF mentioned this pull request Sep 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants