Skip to content

Commit 53ebad7

Browse files
Fix exception when estimating token count of tool execution request (#85)
## Issue Closes langchain4j/langchain4j#3409 The HuggingFaceTokenCountEstimator does not properly check for null text in AiMessages which may be tool execution requests. This results in a stacktrace similar to the following: ``` java.util.concurrent.CompletionException: java.lang.NullPointerException: text cannot be null at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315) at java.base/java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:687) at java.base/java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:662) at java.base/java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2200) ... Caused by: java.lang.NullPointerException: text cannot be null at ai.djl.huggingface.tokenizers.HuggingFaceTokenizer.encode(HuggingFaceTokenizer.java:237) at dev.langchain4j.model.embedding.onnx.HuggingFaceTokenCountEstimator.estimateTokenCountInText(HuggingFaceTokenCountEstimator.java:99) at dev.langchain4j.model.embedding.onnx.HuggingFaceTokenCountEstimator.estimateTokenCountInMessage(HuggingFaceTokenCountEstimator.java:110) at dev.langchain4j.model.embedding.onnx.HuggingFaceTokenCountEstimator.estimateTokenCountInMessages(HuggingFaceTokenCountEstimator.java:122) at dev.langchain4j.memory.chat.TokenWindowChatMemory.ensureCapacity(TokenWindowChatMemory.java:95) at dev.langchain4j.memory.chat.TokenWindowChatMemory.add(TokenWindowChatMemory.java:71) at dev.langchain4j.service.tool.ToolService.executeInferenceAndToolsLoop(ToolService.java:128) ... ``` ## Change We check for null text and return 0 for the estimated token count. ## General checklist - [X] There are no breaking changes - [ ] I have added unit and/or integration tests for my change - [ ] The tests cover both positive and negative cases - [ ] I have manually run all the unit and integration tests in the module I have added/changed, and they are all green - [ ] I have added/updated the [documentation](https://github.com/langchain4j/langchain4j/tree/main/docs/docs) - [ ] I have added an example in the [examples repo](https://github.com/langchain4j/langchain4j-examples) (only for "big" features) - [ ] I have added/updated [Spring Boot starter(s)](https://github.com/langchain4j/langchain4j-spring) (if applicable) Co-authored-by: Dmytro Liubarskyi <langchain4j@gmail.com>
1 parent a77a5e3 commit 53ebad7

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

langchain4j-embeddings/src/main/java/dev/langchain4j/model/embedding/onnx/HuggingFaceTokenCountEstimator.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -107,7 +107,7 @@ public int estimateTokenCountInMessage(ChatMessage message) {
107107
} else if (message instanceof UserMessage userMessage) {
108108
return estimateTokenCountInText(userMessage.singleText());
109109
} else if (message instanceof AiMessage aiMessage) {
110-
return estimateTokenCountInText(aiMessage.text());
110+
return aiMessage.text() == null ? 0 : estimateTokenCountInText(aiMessage.text());
111111
} else if (message instanceof ToolExecutionResultMessage toolExecutionResultMessage) {
112112
return estimateTokenCountInText(toolExecutionResultMessage.text());
113113
} else {

0 commit comments

Comments
 (0)