You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Fix exception when estimating token count of tool execution request (#85)
## Issue
Closeslangchain4j/langchain4j#3409
The HuggingFaceTokenCountEstimator does not properly check for null text
in AiMessages which may be tool execution requests. This results in a
stacktrace similar to the following:
```
java.util.concurrent.CompletionException: java.lang.NullPointerException: text cannot be null
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
at java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:687)
at java.base/java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:662)
at java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2200)
...
Caused by: java.lang.NullPointerException: text cannot be null
at ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.encode(HuggingFaceTokenizer.java:237)
at dev.langchain4j.model.embedding.onnx.HuggingFaceTokenCountEstimator.estimateTokenCountInText(HuggingFaceTokenCountEstimator.java:99)
at dev.langchain4j.model.embedding.onnx.HuggingFaceTokenCountEstimator.estimateTokenCountInMessage(HuggingFaceTokenCountEstimator.java:110)
at dev.langchain4j.model.embedding.onnx.HuggingFaceTokenCountEstimator.estimateTokenCountInMessages(HuggingFaceTokenCountEstimator.java:122)
at dev.langchain4j.memory.chat.TokenWindowChatMemory.ensureCapacity(TokenWindowChatMemory.java:95)
at dev.langchain4j.memory.chat.TokenWindowChatMemory.add(TokenWindowChatMemory.java:71)
at dev.langchain4j.service.tool.ToolService.executeInferenceAndToolsLoop(ToolService.java:128)
...
```
## Change
We check for null text and return 0 for the estimated token count.
## General checklist
- [X] There are no breaking changes
- [ ] I have added unit and/or integration tests for my change
- [ ] The tests cover both positive and negative cases
- [ ] I have manually run all the unit and integration tests in the
module I have added/changed, and they are all green
- [ ] I have added/updated the
[documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs)
- [ ] I have added an example in the [examples
repo](https://github.com/langchain4j/langchain4j-examples) (only for
"big" features)
- [ ] I have added/updated [Spring Boot
starter(s)](https://github.com/langchain4j/langchain4j-spring) (if
applicable)
Co-authored-by: Dmytro Liubarskyi <langchain4j@gmail.com>
Copy file name to clipboardExpand all lines: langchain4j-embeddings/src/main/java/dev/langchain4j/model/embedding/onnx/HuggingFaceTokenCountEstimator.java
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -107,7 +107,7 @@ public int estimateTokenCountInMessage(ChatMessage message) {
0 commit comments