Skip to content

Conversation

@b41sh
Copy link
Member

@b41sh b41sh commented Nov 13, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

The PR fix the bug copy into infinite number to Variant field caused panic

for example:
we have the following numbers.ndjson file, the number 1.0e999999 is a infinite number

{"id": 1, "value": 100, "score": 1.0e38}
{"id": 2, "value": 200, "score": -1.0e77}
{"id": 3, "value": 300, "score": 1.0e999999}
CREATE OR REPLACE TABLE number (id Int32, value Int64, score Variant);

COPY INTO number FROM @data/ndjson/numbers.ndjson file_format = (type = NDJSON) ON_ERROR=continue;
error: APIError: QueryFailed: [1104]called `Option::unwrap()` on a `None` value

Since the infinite number exceeds the range of f64, we convert it to a null value

CREATE OR REPLACE TABLE number (id Int32, value Int64, score Variant);

COPY INTO number FROM @data/ndjson/numbers.ndjson file_format = (type = NDJSON) ON_ERROR=continue;
╭─────────────────────────────────────────────────────────────────────────────────────────╮
│          File         │ Rows_loaded │ Errors_seen │    First_error   │ First_error_line │
│         String        │    Int32    │    Int32    │ Nullable(String) │  Nullable(Int32) │
├───────────────────────┼─────────────┼─────────────┼──────────────────┼──────────────────┤
│ ndjson/numbers.ndjson30NULLNULL │
╰─────────────────────────────────────────────────────────────────────────────────────────╯
1 row read in 0.581 sec. Processed 3 row, 128 B (5.16 rows/s, 220 B/s)

SELECT * FROM number;
╭───────────────────────────────────────────────────────╮
│        id       │      value      │       score       │
│ Nullable(Int32) │ Nullable(Int64) │ Nullable(Variant) │
├─────────────────┼─────────────────┼───────────────────┤
│               1100 │ 1e38              │
│               2200-1e77             │
│               3300null              │
╰───────────────────────────────────────────────────────╯
3 rows read in 0.091 sec. Processed 3 rows, 113 B (32.97 rows/s, 1.21 KiB/s)
  • fixes: #[Link the issue here]

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-bugfix this PR patches a bug in codebase label Nov 13, 2025
@b41sh b41sh marked this pull request as ready for review November 14, 2025 03:45
@b41sh b41sh requested review from sundy-li and youngsofun November 14, 2025 03:45
@BohuTANG BohuTANG merged commit ac0fb76 into databendlabs:main Nov 14, 2025
87 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-bugfix this PR patches a bug in codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants