Add support for MultiHeadAttention layers in tensorflow.js #8574
Description
I am working on creating and and mode in the python then converting the model over to tfjs to run inference. It does not appear that this is currently not supported.
System information
- TensorFlow.js version (you are using): tensorflow_node 4.22
- Are you willing to contribute it (Yes/No): don't have the skillset
Describe the feature and the current behavior/state.
Models created in python with a MultiHeadAttention layer will successfully be able to transfer to tfjs. Currently MultiHeadAttention layers are not supported.
Will this change the current api? How?
add support for MultiHeadAttention Layers
Who will benefit with this feature?
Anyone who is trying to use modern transformer architectures
Any Other info.