cannot mix struct and non-struct, non-null values

[javey9110]

08:47:38,384 graphrag.index.emit.parquet_table_emitter INFO emitting parquet table create_final_community_reports.parquet
08:47:38,387 graphrag.index.emit.parquet_table_emitter ERROR Error while emitting parquet table
Traceback (most recent call last):
File "/repo/eluxjiw/.local/raghub/lib/python3.11/site-packages/graphrag/index/emit/parquet_table_emitter.py", line 40, in emit
await self._storage.set(filename, data.to_parquet())
^^^^^^^^^^^^^^^^^
File "/repo/eluxjiw/.local/raghub/lib/python3.11/site-packages/pandas/util/decorators.py", line 333, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/repo/eluxjiw/.local/raghub/lib/python3.11/site-packages/pandas/core/frame.py", line 3113, in to_parquet
return to_parquet(
^^^^^^^^^^^
File "/repo/eluxjiw/.local/raghub/lib/python3.11/site-packages/pandas/io/parquet.py", line 480, in to_parquet
impl.write(
File "/repo/eluxjiw/.local/raghub/lib/python3.11/site-packages/pandas/io/parquet.py", line 190, in write
table = self.api.Table.from_pandas(df, **from_pandas_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pyarrow/table.pxi", line 3874, in pyarrow.lib.Table.from_pandas
File "/repo/eluxjiw/.local/raghub/lib/python3.11/site-packages/pyarrow/pandas_compat.py", line 611, in dataframe_to_arrays
arrays = [convert_column(c, f)
^^^^^^^^^^^^^^^^^^^^^
File "/repo/eluxjiw/.local/raghub/lib/python3.11/site-packages/pyarrow/pandas_compat.py", line 611, in
arrays = [convert_column(c, f)
^^^^^^^^^^^^^^^^^^^^
File "/repo/eluxjiw/.local/raghub/lib/python3.11/site-packages/pyarrow/pandas_compat.py", line 598, in convert_column
raise e
File "/repo/eluxjiw/.local/raghub/lib/python3.11/site-packages/pyarrow/pandas_compat.py", line 592, in convert_column
result = pa.array(col, type=type, from_pandas=True, safe=safe)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pyarrow/array.pxi", line 339, in pyarrow.lib.array
File "pyarrow/array.pxi", line 85, in pyarrow.lib._ndarray_to_array
File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: ('cannot mix struct and non-struct, non-null values', 'Conversion failed for column findings with type object')

This ERROR will cause miss create_final_communities_report.parquet, even though the result display as "All workflows completed successfully"

[javey9110]

Oddly enough, this works fine with a small number of input files, but runs into problems as you increase the number of files.

[javey9110]

My venv info:
graphrag 0.3.6
pyarrow 15.0.2
pandas 2.2.3

[saber1111lll]

Have you solved this problem?I’m also encountering this problem now