You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Pandas recently, i think, got an option called future.infer_string, currently set to False, which will read the sequence of str objects as pyarrow string dtype. Since this is going to be the default in Pandas version 3.0, I have pandas.options.future.infer_string=True at the top of my script. With this option set i get an inscrutable error during some of my interactive sessions. A minimum reproducible example of my repl interaction is listed:
In [1]: importduckdb, pandasaspdIn [2]: s= ["string"for_inrange(5)]
In [3]: pd.options.future.infer_string=True## changed to be prepared for upcoming update of pandasIn [4]: df=pd.DataFrame({"s":s})
In [5]: duckdb.sql("select * from df")
---------------------------------------------------------------------------NotImplementedExceptionTraceback (mostrecentcalllast)
CellIn[5], line1---->1duckdb.sql("select * from df")
NotImplementedException: NotimplementedError: Datatype'str'notrecognizedIn [6]: pd.options.future.infer_string=False## default behaviorIn [7]: df1=pd.DataFrame({"s":s})
In [8]: duckdb.sql("select * from df1")
Out[8]:
┌─────────┐
│ s │
│ varchar │
├─────────┤
│ string │
│ string │
│ string │
│ string │
│ string │
└─────────┘
In [9]:
I have two questions:
Since pyarrow strings are going to become the default for collection of strings, is it possible to be make replacement scan not raise this error with str dtype?
Regardless of how this pandas dtype evolution is addressed, can this error message be improved? I would've never been able to distill this error had my table been bigger than just a few columns!
I love using duckdb from within the python repl and i believe that replacement scans are just so brilliant!
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
Pandas recently, i think, got an option called
future.infer_string
, currently set toFalse
, which will read the sequence of str objects as pyarrow string dtype. Since this is going to be the default in Pandas version 3.0, I havepandas.options.future.infer_string=True
at the top of my script. With this option set i get an inscrutable error during some of my interactive sessions. A minimum reproducible example of my repl interaction is listed:I have two questions:
str
dtype?I love using duckdb from within the python repl and i believe that replacement scans are just so brilliant!
Thanks in advance!
Beta Was this translation helpful? Give feedback.
All reactions