Support for Arrow 55

[bazooka720]

We are using your connector to work with datafusion for one of our needs and use arrow-schema v55.0.0. However, the connectorx version seems built against older arrow crates (v54.3.1). Can you advise?

[EricFecteau]

Can you confirm that you are using the GitHub version with Arrow 54 and not the Crates.io version with Arrow 53?

You can see the issue here: #801 54 should work with Arrow 55, but if we update to 55 it will break compatibility with any crate that uses 53.

[kesavkolla]

Now arrow is on 56. Is it possible to bump the arrow dependency to 56?

[EricFecteau]

As explained here, we should stay with Arrow 54 for as long as possible, as a bridge between crates that have updated and those who have not. This is not a "outputs are incompatible" issue it's a "you can't compile if a project has both Arrow 53 and Arrow 55". Some crates are still on Arrow 53 or lower.

If you need Arrow 56 output from Connector X, you can use df-interchange to convert Arrow 54 to Arrow 56.

[kesavkolla]

For me it's not just the format. I need to use arrow crate 56 because in that version they have made changes to FlightSQL.
Since connectorx is using 54 I can't use arrow 56 in my project

[EricFecteau]

Did you look at df-interchange that I linked to above? It should solve your problem by converting 54 to 56.

Personally, moving Connector X to 56 would be an ecosystem mistake right now.

[surister]

@kesavkolla by any chance, are you only using postgres?

[kesavkolla]

No. My use case for now is Oracle