"ValueError: Unknown Layer: ... " When Calling Copy.deepcopy(network) Using Tensorflow

I am currently designing a NoisyNet in Tensorflow, for which I need to define a custom layer. When copying a model containing that custom layer, python raises the error ValueError:

Solution 1:

Found a solution:

The problem, again, was that Tensorflow/Keras didn't know how to interpret the custom layer. So, to provide the information how to interpret a layer, one can use Keras's CustomObjectScope and copy the model within that scope as follows:

# Import
import copy
from keras.utils import CustomObjectScope

# Copy
with CustomObjectScope({"MyCustomLayer":MyCustomLayer}):
        model_copy = copy.deepcopy(model)

This takes care of the copying part. However, this is only going to work out of the box as long as there is no custom inputs specified as parameters to the custom layer's constructor (__init(...)).

I guess this is case since behind the scenes the copy() function seems to temporarily save and then load again the original model using some pickle-functionality or so, such that one has to declare values for further constructor-parameters as well as follows:

If the beginning of the custom class looks as follows, where output_dim is one of the custom parameters referred to above:

class MyCustomLayer(keras.layers.Layer):

    def __init__(self, output_dim, **kwargs):
        self.output_dim = output_dim
        super(MyCustomLayer, self).__init__(**kwargs)

then one has to add a function to the class MyCustomLayer that also takes care of making the custom constructor parameters persistent for saving and loading (while copying):

def get_config(self):
        config = super(MyCustomLayer, self).get_config()

        # Specify here all the values for the constructor's parameters
        config['output_dim'] = self.output_dim

        return config

These two steps solved the problem in my case.