How To Perform Multiclass Multioutput Classification Using Lstm

I have multiclass multioutput classification (see https://scikit-learn.org/stable/modules/multiclass.html for details). In other words, my dataset looks as follows. node_name, time

Solution 1:

If I understand correctly, label_1 is binary, whereas label_2 is a multiclass problem, so we need the model to have two outputs with separate loss functions; binary and categorical crossentropy respectively.

However, Sequential API does not allow multiple input/output.

The Sequential API allows you to create models layer-by-layer for most problems. It is limited in that it does not allow you to create models that share layers or have multiple inputs or outputs.

You can use the functional API to create two output layers, and compile the model with required loss functions.

X=Input(input_shape)
X=Layer(X)
''''
out1=Dense(1, activation='sigmoid')(X)
out2=Dense(3, activation='softmax')(X)
model = Model(inputs = input, outputs = [out1,out2])
model.compile(loss = ['binary_crossentropy','categorical_crossentropy'], loss_weights = [l1,l2], ...)

model.fit(input,[label_1, label_2_toCategotical]

The loss that the network will minimize will be the weighted sum of the 2 losses, weighted by l1 and l2.

Hope this helps :)

Solution 2:

This is a somewhat complicated problem, since the Scikit-Learn API and Keras API for multiclass multi-output are not directly compatible. Further, there are even differences in how TensorFlow v1 and v2 handle things. The existing Keras wrappers don't really work for more complex cases.

I created an extension of KerasClassifier that is able to deal with these situations, the package and documentation are here (GitHub). Full disclosure: I am the the creator of the package, but I have no financial interests, it's open source.

With these extended versions, you can easily handle multiclass multi-output problems. I think for your situation it should work out of the box, but if not you can just inherit from KerasClassifier and overwrite target_encoder to transform from the Scikit-Learn data format to whatever your Keras Model uses. More details here (docs).

Hope this helps!