When using Lambda for L2 regularization in Keras, there is an error saying that output_shape cannot be found?

[20220317]

我的keras版本是3.10.0，

[dhantule]

Hi @20220317, Could you please provide some reproducible code to reproduce the issue?

[Shoumik-Gandre]

Reproduced issue with this:

# repro_keras_lambda_shape_error.py
# Reproduces: "We could not automatically infer the shape of the Lambda's output.
# Please specify the `output_shape` argument for this Lambda layer."

import numpy as np

# Keras 3 API (works the same with tf.keras imports below)
from keras.layers import Input, Dense, Lambda
from keras.models import Model
from keras import backend as K  # using K.l2_normalize like in the image

# ----- Model -----
inp = Input(shape=(128,), name="inp")               # <KerasTensor shape=(None, 128)>
x = Dense(units=128, activation=None)(inp)          # Dense layer
# Lambda layer without output_shape — this is what triggers the error
x = Lambda(lambda t: K.l2_normalize(t, axis=1))(x)  # same as your code

model = Model(inp, x)
model.compile(optimizer="adam", loss="mse")

# ----- Dummy data -----
x_data = np.random.randn(8, 128).astype("float32")
y_data = np.random.randn(8, 128).astype("float32")

# ----- Train (fails here with the message in your screenshot) -----
model.fit(x_data, y_data, epochs=1, batch_size=4)

[Shoumik-Gandre]

Approach 1 (will work): Keras.backend.l2_normalize is OLD API and no longer available in V3. Just use the modern syntax here:

import numpy as np

# Keras 3 API (works the same with tf.keras imports below)
from keras.layers import Input, Dense, Lambda
from keras.models import Model
import tensorflow as tf  # THIS


# ----- Model -----
inp = Input(shape=(128,), name="inp")               # <KerasTensor shape=(None, 128)>
x = Dense(units=128, activation=None)(inp)          # Dense layer
# Lambda layer without output_shape — this is what triggers the error
x = Lambda(lambda t: tf.nn.l2_normalize(t, axis=1))(x)  # Use the modern syntax

model = Model(inp, x)
model.compile(optimizer="adam", loss="mse")

# ----- Dummy data -----
x_data = np.random.randn(8, 128).astype("float32")
y_data = np.random.randn(8, 128).astype("float32")

# ----- Train (fails here with the message in your screenshot) -----
model.fit(x_data, y_data, epochs=1, batch_size=4)

Approach 2 (Less likely to work): Pass in output_shape parameter in Lambda

x = Lambda(lambda t: K.l2_normalize(t, axis=1), output_shape=lambda s:s)(x)  # same as your code

Honestly, just do Approach 1. Here's whats going on. Lambda() tries to find the output shape of the lambda function you passed in. It is most likely None because keras.backend.l2_normalize is not a thing anymore.

[dhantule]

Hi @Shoumik-Gandre, thank you for looking into this issue. I've tested your approach and it worked fine.

@20220317, please refer this attached gist that uses tf.nn.l2_normalize with Keras 3 instead of K.l2_normalize, you may also refer to Approach 2 in the above comment.

Let me know if this helps.