Support dtype conversion from processor output

[donin1129]

Hello team,

First off, I want to express my sincere appreciation for the excellent work on the library! Our team has genuinely enjoyed using it, and we find it incredibly valuable.

We've recently encountered a specific scenario where we believe an enhancement to the TransformerMultiModal class could significantly benefit users, particularly concerning dtype handling.

The Challenge:
Currently, within the _prepare_model_inputs method (located here: https://github.com/dottxt-ai/outlines/blob/main/outlines/models/transformers.py#L658), we've identified a need for a mechanism to convert the data type (dtype) of inputs. While we understand that general dtype handling is typically managed by the processor, certain hardware-specific data types, such as bfloat16 (which is crucial for our team's current setup), ideally shouldn't be part of the processor's call function due to performance or architectural considerations.

We'd love to hear your thoughts and support on this. Would you be open to integrating a feature that allows for explicit dtype conversion at this stage, or perhaps you have an alternative approach in mind that would address this need effectively?

Thank you.

[RobinPicard]

Hi @donin1129! Thanks for creating a discussion, I'm really glad to hear that outlines has been useful to you and enjoyable to work with.

I have never encountered this issue, but that sounds like something important and we're definitely willing to look into it. Do you think you could open a draft PR with a super basic implementation (no tests or docs needed) of what you'd like the library to do? From there we could discuss how we can make it fit best in our current architecture and what interface we should propose.