Skip to content

Problem with draftretriever.search function behavior #29

@qingyiaaaaa

Description

@qingyiaaaaa

Issue description:

I am using the draftretriever.search function, and I have encountered a few issues.I'm sorry, I'm not very familiar with Rust, so it's difficult for me to understand the source code of the search function.

Here is my code

Image

And the print result

[[6, 13, 14, -2], [5, 7, -2, -2], [6, 13, 14, 15], [6, 13, -2, -2], [5, 7, 8, 9], [5, 7, 8, -2]]

[[1, 0, 0, 0, 0, 0, 0, 0, 0], [1, 1, 0, 0, 0, 0, 0, 0, 0], [1, 0, 1, 0, 0, 0, 0, 0, 0], [1, 1, 0, 1, 0, 0, 0, 0, 0], [1, 0, 1, 0, 1, 0, 0, 0, 0], [1, 1, 0, 1, 0, 1, 0, 0, 0], [1, 0, 1, 0, 1, 0, 1, 0, 0], [1, 1, 0, 1, 0, 1, 0, 1, 0], [1, 0, 1, 0, 1, 0, 1, 0, 1]]

[0, 1, 2, 3, 4, 5, 6, 7, 8]

[0, 1, 1, 2, 2, 3, 3, 4, 4]

[[0, 2, 4, 6, 8], [0, 1, 3, 5, 7]]

1.Why does the retrieved_token_list have a maximum length of only 4, even though I set the long parameter to 32?When I set the long parameter to 32, I expect the returned sequences in retrieved_token_list to have a maximum length of 32, but each sequence in the list has a maximum length of only 4. How can I retrieve sequences with a longer length?

2.Why does the sequence such as [1, 2, 3, 6, 13, 14, 15, 16, 17, 18] will be matched when my query is [2, 2, 3]?I have run several experiments, and it seems like only the last two tokens of the query ([2, 3]) are being used for matching.

3.In addition to retrieved_token_list, the search function returns four other objects: _draft_attn_mask, _tree_indices, _draft_position_ids, and _retrieve_indices. What do these objects represent, and how should they be used?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions