jsondecode error calling `cat` in condense_decoded_json_recursive

[siko1056]

Some days ago, I ran a benchmark including your package and observed running this code

pkg install https://github.com/apjanke/octave-jsonstuff/releases/download/v0.3.3/jsonstuff-0.3.3.tar.gz
pkg load jsonstuff

urlwrite ( ...
  'https://github.com/json-iterator/test-data/raw/master/large-file.json', ...
  'large-file.json');

json_str = fileread ('large-file.json');

which jsondecode

octave_obj = jsondecode (json_str);

the following error message in Octave 5.2.0

'jsondecode' is a function from the file /home/siko1056/.local/share/octave/5.2.0/jsonstuff-0.3.3/jsondecode.m

error: orderfields: structs must have same fields up to order
error: cat: field names mismatch in concatenating structs
error: called from
    jsondecode>condense_decoded_json_recursive at line 116 column 9
    jsondecode>condense_decoded_json at line 67 column 7
    jsondecode at line 63 column 7
    blub at line 12 column 12

and the following error message in Octave 7.0.0

'jsondecode' is a function from the file /home/siko1056/.local/share/octave/7.0.0/jsonstuff-0.3.3/jsondecode.m

error: cat: field names mismatch in concatenating structs
error: called from
    jsondecode>condense_decoded_json_recursive at line 116 column 9
    jsondecode>condense_decoded_json at line 67 column 7
    jsondecode at line 63 column 7
    blub at line 12 column 12

I did no further investigation yet.

octave-jsonstuff/inst/jsondecode.m

Lines 115 to 119 in 68b209a

	if is_condensable
	out = cat (1, x{:});
	else
	out = x;
	endif

Thank you for your package @apjanke, it helped our GSoC student Abdallah a lot 👍

[apjanke]

Ah, cool. Thanks for the nice, full repro code! Looks like is_condensable() is not correctly checking for heterogeneous JSON objects inside arrays. I'll get on this.

Glad this package has been useful!

[apjanke]

Well, that's interesting. I added a repro based on your code at scratch/repro_jsoniterator_largefile_failure.m, and when I run this in Octave 4.4.1 or 5.2.0 on macOS, from master on jsonstuff, it straight up crashes Octave. That's not good.

[apjanke]

Updated this to High Priority since it's a crasher.

[siko1056]

Oh, maybe there is an Octave bug lingering below this crash? 😨

[apjanke]

Oh, maybe there is an Octave bug lingering below this crash?

Unlikely. My jsonstuff implementation is largely done inside an oct-file, so it's a lot more probable that there's a bug in my oct-file that's crashing Octave. :) Once you're in oct-file territory, you're in C, and there's nothing Octave can do to protect itself from client bugs.