Htydjtk

I want to compute the precision, recall and F1-score for my binary KerasClassifier model, but don't find any solution.

Here's my actual code:

# Split dataset in train and test data 

X_train, X_test, Y_train, Y_test = train_test_split(normalized_X, Y, test_size=0.3, random_state=seed)



# Build the model

model = Sequential()

model.add(Dense(23, input_dim=45, kernel_initializer='normal', activation='relu'))

model.add(Dense(1, kernel_initializer='normal', activation='sigmoid'))



# Compile model

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])





tensorboard = TensorBoard(log_dir="logs/{}".format(time.time()))



time_callback = TimeHistory()



# Fit the model

history = model.fit(X_train, Y_train, validation_split=0.3, epochs=200, batch_size=5, verbose=1, callbacks=[tensorboard, time_callback]) 

And then I am predicting on new test data, and getting the confusion matrix like this:

y_pred = model.predict(X_test)

y_pred =(y_pred>0.5)

list(y_pred)



cm = confusion_matrix(Y_test, y_pred)

print(cm)

But is there any solution to get the accuracy-score, the F1-score, the precision, and the recall? (If not complicated, also the cross-validation-score, but not necessary for this answer)

Thank you for any help!

Metrics have been removed from Keras core. You need to calculate them manually. They removed them on 2.0 version. Those metrics are all global metrics, but Keras works in batches. As a result, it might be more misleading than helpful.

However, if you really need them, you can do it like this

from keras import backend as K



def recall_m(y_true, y_pred):

        true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))

        possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))

        recall = true_positives / (possible_positives + K.epsilon())

        return recall



def precision_m(y_true, y_pred):

        true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))

        predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))

        precision = true_positives / (predicted_positives + K.epsilon())

        return precision



def f1_m(y_true, y_pred):

    precision = precision_m(y_true, y_pred)

    recall = recall_m(y_true, y_pred)

    return 2*((precision*recall)/(precision+recall+K.epsilon()))



# compile the model

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc',f1_m,precision_m, recall_m])



# fit the model

history = model.fit(Xtrain, ytrain, validation_split=0.3, epochs=10, verbose=0)



# evaluate the model

loss, accuracy, f1_score, precision, recall = model.evaluate(Xtest, ytest, verbose=0)

You could use the scikit-learn classification report. To convert your labels into a numerical or binary format take a look at the scikit-learn label encoder.

y_pred = model.predict(x_test, batch_size=64, verbose=1)

y_pred_bool = np.argmax(y_pred, axis=1)



print(classification_report(y_test, y_pred_bool))

which gives you (output copied from the scikit-learn example):

             precision  recall   f1-score    support



 class 0       0.50      1.00      0.67         1

 class 1       0.00      0.00      0.00         1

 class 2       1.00      0.67      0.80         3

Try this: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_fscore_support.html with Y_test, y_pred as parameters.