Validation loss increases and validation accuracy decreases
$begingroup$
I have an issue with my model. I'm trying to use the most basic Conv1D model to analyze review data and output a rating of 1-5 class, therefore the loss is categorical_crossentropy. Model structure is as below
# define model
model = Sequential()
model.add(Embedding(vocab_size, 100, input_length=max_length))
model.add(Conv1D(filters=32, kernel_size=8, activation='relu'))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(10, activation='relu'))
model.add(Dense(5, activation='softmax'))
# compile network
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit network
model.fit(final_X_train, final_Y_train, epochs=5, batch_size=50, validation_data=(final_X_val, final_Y_val), callbacks=callback)
Total params: 14,977,717
Trainable params: 14,977,717
Non-trainable params: 0
Train on 212135 samples, validate on 69472 samples
Epoch 1/5
loss: 0.9452 - acc: 0.5781 - val_loss: 0.8440 - val_acc: 0.6309
Epoch 2/5
loss: 0.7707 - acc: 0.6711 - val_loss: 0.8234 - val_acc: 0.6433
Epoch 3/5
loss: 0.5807 - acc: 0.7657 - val_loss: 0.9144 - val_acc: 0.6345
Epoch 4/5
loss: 0.3736 - acc: 0.8575 - val_loss: 1.1982 - val_acc: 0.6194
Epoch 5/5
loss: 0.2285 - acc: 0.9173 - val_loss: 1.5770 - val_acc: 0.6073
Training acc increases and loss decreases as expected. But validation loss and validation acc decrease straight after the 2nd epoch itself. The overall testing after training gives an accuracy around 60s.
The total accuracy is : 0.6046845041714888
I've already cleaned, shuffled, down-sampled (all classes have 42427 number of data samples) and split the data properly to training(70%) / validation(10%) / testing(20%).
If you see any improvements to fix this problem, please let me know. :)
python deep-learning classification keras overfitting
New contributor
$endgroup$
add a comment |
$begingroup$
I have an issue with my model. I'm trying to use the most basic Conv1D model to analyze review data and output a rating of 1-5 class, therefore the loss is categorical_crossentropy. Model structure is as below
# define model
model = Sequential()
model.add(Embedding(vocab_size, 100, input_length=max_length))
model.add(Conv1D(filters=32, kernel_size=8, activation='relu'))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(10, activation='relu'))
model.add(Dense(5, activation='softmax'))
# compile network
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit network
model.fit(final_X_train, final_Y_train, epochs=5, batch_size=50, validation_data=(final_X_val, final_Y_val), callbacks=callback)
Total params: 14,977,717
Trainable params: 14,977,717
Non-trainable params: 0
Train on 212135 samples, validate on 69472 samples
Epoch 1/5
loss: 0.9452 - acc: 0.5781 - val_loss: 0.8440 - val_acc: 0.6309
Epoch 2/5
loss: 0.7707 - acc: 0.6711 - val_loss: 0.8234 - val_acc: 0.6433
Epoch 3/5
loss: 0.5807 - acc: 0.7657 - val_loss: 0.9144 - val_acc: 0.6345
Epoch 4/5
loss: 0.3736 - acc: 0.8575 - val_loss: 1.1982 - val_acc: 0.6194
Epoch 5/5
loss: 0.2285 - acc: 0.9173 - val_loss: 1.5770 - val_acc: 0.6073
Training acc increases and loss decreases as expected. But validation loss and validation acc decrease straight after the 2nd epoch itself. The overall testing after training gives an accuracy around 60s.
The total accuracy is : 0.6046845041714888
I've already cleaned, shuffled, down-sampled (all classes have 42427 number of data samples) and split the data properly to training(70%) / validation(10%) / testing(20%).
If you see any improvements to fix this problem, please let me know. :)
python deep-learning classification keras overfitting
New contributor
$endgroup$
add a comment |
$begingroup$
I have an issue with my model. I'm trying to use the most basic Conv1D model to analyze review data and output a rating of 1-5 class, therefore the loss is categorical_crossentropy. Model structure is as below
# define model
model = Sequential()
model.add(Embedding(vocab_size, 100, input_length=max_length))
model.add(Conv1D(filters=32, kernel_size=8, activation='relu'))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(10, activation='relu'))
model.add(Dense(5, activation='softmax'))
# compile network
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit network
model.fit(final_X_train, final_Y_train, epochs=5, batch_size=50, validation_data=(final_X_val, final_Y_val), callbacks=callback)
Total params: 14,977,717
Trainable params: 14,977,717
Non-trainable params: 0
Train on 212135 samples, validate on 69472 samples
Epoch 1/5
loss: 0.9452 - acc: 0.5781 - val_loss: 0.8440 - val_acc: 0.6309
Epoch 2/5
loss: 0.7707 - acc: 0.6711 - val_loss: 0.8234 - val_acc: 0.6433
Epoch 3/5
loss: 0.5807 - acc: 0.7657 - val_loss: 0.9144 - val_acc: 0.6345
Epoch 4/5
loss: 0.3736 - acc: 0.8575 - val_loss: 1.1982 - val_acc: 0.6194
Epoch 5/5
loss: 0.2285 - acc: 0.9173 - val_loss: 1.5770 - val_acc: 0.6073
Training acc increases and loss decreases as expected. But validation loss and validation acc decrease straight after the 2nd epoch itself. The overall testing after training gives an accuracy around 60s.
The total accuracy is : 0.6046845041714888
I've already cleaned, shuffled, down-sampled (all classes have 42427 number of data samples) and split the data properly to training(70%) / validation(10%) / testing(20%).
If you see any improvements to fix this problem, please let me know. :)
python deep-learning classification keras overfitting
New contributor
$endgroup$
I have an issue with my model. I'm trying to use the most basic Conv1D model to analyze review data and output a rating of 1-5 class, therefore the loss is categorical_crossentropy. Model structure is as below
# define model
model = Sequential()
model.add(Embedding(vocab_size, 100, input_length=max_length))
model.add(Conv1D(filters=32, kernel_size=8, activation='relu'))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(10, activation='relu'))
model.add(Dense(5, activation='softmax'))
# compile network
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit network
model.fit(final_X_train, final_Y_train, epochs=5, batch_size=50, validation_data=(final_X_val, final_Y_val), callbacks=callback)
Total params: 14,977,717
Trainable params: 14,977,717
Non-trainable params: 0
Train on 212135 samples, validate on 69472 samples
Epoch 1/5
loss: 0.9452 - acc: 0.5781 - val_loss: 0.8440 - val_acc: 0.6309
Epoch 2/5
loss: 0.7707 - acc: 0.6711 - val_loss: 0.8234 - val_acc: 0.6433
Epoch 3/5
loss: 0.5807 - acc: 0.7657 - val_loss: 0.9144 - val_acc: 0.6345
Epoch 4/5
loss: 0.3736 - acc: 0.8575 - val_loss: 1.1982 - val_acc: 0.6194
Epoch 5/5
loss: 0.2285 - acc: 0.9173 - val_loss: 1.5770 - val_acc: 0.6073
Training acc increases and loss decreases as expected. But validation loss and validation acc decrease straight after the 2nd epoch itself. The overall testing after training gives an accuracy around 60s.
The total accuracy is : 0.6046845041714888
I've already cleaned, shuffled, down-sampled (all classes have 42427 number of data samples) and split the data properly to training(70%) / validation(10%) / testing(20%).
If you see any improvements to fix this problem, please let me know. :)
python deep-learning classification keras overfitting
python deep-learning classification keras overfitting
New contributor
New contributor
New contributor
asked 14 hours ago
strangerstranger
112
112
New contributor
New contributor
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
What you are experiencing is known as overfitting, and it’s a common problem in machine learning and data science.
Overfitting happens when a model begins to focus on the noise in the training dataset and extracts features based on it. This helps the model to improve its performance on the training set, but hurts its ability to generalize, so the accuracy of the validation set decreases.
To deal with overfitting, you need to use regularization during the training. You ca try adding dropout layers or batch-normalization layers, adding weight regularization, or you can artificially increase the size of your training set by performing some data augmentation.
You can read more about it in the following post:
What are the possible approaches to fixing Overfitting on a CNN?
$endgroup$
$begingroup$
Thanks, I tried adding regularizers to Conv1D and Dense layers as below.Conv1D(filters=64, kernel_size=8, activation='relu', kernel_regularizer=l2(0.001), bias_regularizer=l2(0.001))
ANDDense(10, activation='relu', kernel_regularizer=l2(0.001))
. But unfortunately none of them did any changes to val_acc and val_loss.
$endgroup$
– stranger
11 hours ago
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
stranger is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47720%2fvalidation-loss-increases-and-validation-accuracy-decreases%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
What you are experiencing is known as overfitting, and it’s a common problem in machine learning and data science.
Overfitting happens when a model begins to focus on the noise in the training dataset and extracts features based on it. This helps the model to improve its performance on the training set, but hurts its ability to generalize, so the accuracy of the validation set decreases.
To deal with overfitting, you need to use regularization during the training. You ca try adding dropout layers or batch-normalization layers, adding weight regularization, or you can artificially increase the size of your training set by performing some data augmentation.
You can read more about it in the following post:
What are the possible approaches to fixing Overfitting on a CNN?
$endgroup$
$begingroup$
Thanks, I tried adding regularizers to Conv1D and Dense layers as below.Conv1D(filters=64, kernel_size=8, activation='relu', kernel_regularizer=l2(0.001), bias_regularizer=l2(0.001))
ANDDense(10, activation='relu', kernel_regularizer=l2(0.001))
. But unfortunately none of them did any changes to val_acc and val_loss.
$endgroup$
– stranger
11 hours ago
add a comment |
$begingroup$
What you are experiencing is known as overfitting, and it’s a common problem in machine learning and data science.
Overfitting happens when a model begins to focus on the noise in the training dataset and extracts features based on it. This helps the model to improve its performance on the training set, but hurts its ability to generalize, so the accuracy of the validation set decreases.
To deal with overfitting, you need to use regularization during the training. You ca try adding dropout layers or batch-normalization layers, adding weight regularization, or you can artificially increase the size of your training set by performing some data augmentation.
You can read more about it in the following post:
What are the possible approaches to fixing Overfitting on a CNN?
$endgroup$
$begingroup$
Thanks, I tried adding regularizers to Conv1D and Dense layers as below.Conv1D(filters=64, kernel_size=8, activation='relu', kernel_regularizer=l2(0.001), bias_regularizer=l2(0.001))
ANDDense(10, activation='relu', kernel_regularizer=l2(0.001))
. But unfortunately none of them did any changes to val_acc and val_loss.
$endgroup$
– stranger
11 hours ago
add a comment |
$begingroup$
What you are experiencing is known as overfitting, and it’s a common problem in machine learning and data science.
Overfitting happens when a model begins to focus on the noise in the training dataset and extracts features based on it. This helps the model to improve its performance on the training set, but hurts its ability to generalize, so the accuracy of the validation set decreases.
To deal with overfitting, you need to use regularization during the training. You ca try adding dropout layers or batch-normalization layers, adding weight regularization, or you can artificially increase the size of your training set by performing some data augmentation.
You can read more about it in the following post:
What are the possible approaches to fixing Overfitting on a CNN?
$endgroup$
What you are experiencing is known as overfitting, and it’s a common problem in machine learning and data science.
Overfitting happens when a model begins to focus on the noise in the training dataset and extracts features based on it. This helps the model to improve its performance on the training set, but hurts its ability to generalize, so the accuracy of the validation set decreases.
To deal with overfitting, you need to use regularization during the training. You ca try adding dropout layers or batch-normalization layers, adding weight regularization, or you can artificially increase the size of your training set by performing some data augmentation.
You can read more about it in the following post:
What are the possible approaches to fixing Overfitting on a CNN?
answered 14 hours ago
Mark.FMark.F
9661418
9661418
$begingroup$
Thanks, I tried adding regularizers to Conv1D and Dense layers as below.Conv1D(filters=64, kernel_size=8, activation='relu', kernel_regularizer=l2(0.001), bias_regularizer=l2(0.001))
ANDDense(10, activation='relu', kernel_regularizer=l2(0.001))
. But unfortunately none of them did any changes to val_acc and val_loss.
$endgroup$
– stranger
11 hours ago
add a comment |
$begingroup$
Thanks, I tried adding regularizers to Conv1D and Dense layers as below.Conv1D(filters=64, kernel_size=8, activation='relu', kernel_regularizer=l2(0.001), bias_regularizer=l2(0.001))
ANDDense(10, activation='relu', kernel_regularizer=l2(0.001))
. But unfortunately none of them did any changes to val_acc and val_loss.
$endgroup$
– stranger
11 hours ago
$begingroup$
Thanks, I tried adding regularizers to Conv1D and Dense layers as below.
Conv1D(filters=64, kernel_size=8, activation='relu', kernel_regularizer=l2(0.001), bias_regularizer=l2(0.001))
AND Dense(10, activation='relu', kernel_regularizer=l2(0.001))
. But unfortunately none of them did any changes to val_acc and val_loss.$endgroup$
– stranger
11 hours ago
$begingroup$
Thanks, I tried adding regularizers to Conv1D and Dense layers as below.
Conv1D(filters=64, kernel_size=8, activation='relu', kernel_regularizer=l2(0.001), bias_regularizer=l2(0.001))
AND Dense(10, activation='relu', kernel_regularizer=l2(0.001))
. But unfortunately none of them did any changes to val_acc and val_loss.$endgroup$
– stranger
11 hours ago
add a comment |
stranger is a new contributor. Be nice, and check out our Code of Conduct.
stranger is a new contributor. Be nice, and check out our Code of Conduct.
stranger is a new contributor. Be nice, and check out our Code of Conduct.
stranger is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47720%2fvalidation-loss-increases-and-validation-accuracy-decreases%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown