Is a Java collection guaranteed to be in a valid, usable state after a ConcurrentModificationException?
I am writing a GUI application using the Immediate Mode GUI pattern, and the UI runs on a thread separate to the engine that powers the actual functionality of the application. The GUI thread ends up iterating over many lists of objects that are conceptually "owned" by the engine thread, and these lists change extremely infrequently. The GUI thread is vsync'ed, meaning it runs at about 60Hz, and the engine thread runs at about 200Hz.
Sometimes, actions in the UI will change the contents of the collections in the engine, and I have a message-passing system to post Runnables to the engine thread to do these mutations to ensure that these mutations don't collide with what's happening in the engine. That way, I can ensure that the engine always sees a consistent view of the data, which for my application is very important.
Because the engine is in charge of all data mutations, though, it sometimes happens that the engine changes the contents of a collection while the GUI is iterating through it, and because these collections are standard Java collections, this predictably and correctly throws a ConcurrentModificationException
. I can think of a few high-level ways to deal with this:
- locking, either by using a synchronized collection or read-write locks
- double-buffer the data that the GUI thread reads, and have the GUI thread flip the double-buffer when it's done drawing a frame
- ignore the CME and abort drawing the rest of the frame, which will draw partial information for the frame in which the "bad" mutation happens, and just continue on to the next frame
Locking comes with a significant performance penalty, and while it would be fine for the GUI to sometimes stall while waiting to acquire the lock from the engine thread, it is very important for the engine thread to run at a consistent speed, and even an R/W lock would cause stalls in the engine thread. Double-buffering comes with significant complexity, as there is a lot of data that is read by the GUI on each frame.
I give you all of this background because I know that option 3 is ugly, and that my question is in some sense "not the right question". The Javadoc for ConcurrentModificationException
even says:
Note that fail-fast behavior cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast operations throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: ConcurrentModificationException should be used only to detect bugs.
But! I am not concerned with the correctness of the GUI for the single frame that would be marred by the CME. I am only concerned with what happens on the next frame. Which leads to my question: is it safe to continue using a Java collection (I'm most interested in the answer for ArrayList
and HashMap
) after a ConcurrentModificationException
has been thrown from its iterator? It seems logical that it would be, but I can't find a piece of documentation that says that the object will still be in a usable state after the CME is thrown. Obviously the iterator is toast at that point, but I'd like to swallow the exception and continue using the collection.
java multithreading collections
add a comment |
I am writing a GUI application using the Immediate Mode GUI pattern, and the UI runs on a thread separate to the engine that powers the actual functionality of the application. The GUI thread ends up iterating over many lists of objects that are conceptually "owned" by the engine thread, and these lists change extremely infrequently. The GUI thread is vsync'ed, meaning it runs at about 60Hz, and the engine thread runs at about 200Hz.
Sometimes, actions in the UI will change the contents of the collections in the engine, and I have a message-passing system to post Runnables to the engine thread to do these mutations to ensure that these mutations don't collide with what's happening in the engine. That way, I can ensure that the engine always sees a consistent view of the data, which for my application is very important.
Because the engine is in charge of all data mutations, though, it sometimes happens that the engine changes the contents of a collection while the GUI is iterating through it, and because these collections are standard Java collections, this predictably and correctly throws a ConcurrentModificationException
. I can think of a few high-level ways to deal with this:
- locking, either by using a synchronized collection or read-write locks
- double-buffer the data that the GUI thread reads, and have the GUI thread flip the double-buffer when it's done drawing a frame
- ignore the CME and abort drawing the rest of the frame, which will draw partial information for the frame in which the "bad" mutation happens, and just continue on to the next frame
Locking comes with a significant performance penalty, and while it would be fine for the GUI to sometimes stall while waiting to acquire the lock from the engine thread, it is very important for the engine thread to run at a consistent speed, and even an R/W lock would cause stalls in the engine thread. Double-buffering comes with significant complexity, as there is a lot of data that is read by the GUI on each frame.
I give you all of this background because I know that option 3 is ugly, and that my question is in some sense "not the right question". The Javadoc for ConcurrentModificationException
even says:
Note that fail-fast behavior cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast operations throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: ConcurrentModificationException should be used only to detect bugs.
But! I am not concerned with the correctness of the GUI for the single frame that would be marred by the CME. I am only concerned with what happens on the next frame. Which leads to my question: is it safe to continue using a Java collection (I'm most interested in the answer for ArrayList
and HashMap
) after a ConcurrentModificationException
has been thrown from its iterator? It seems logical that it would be, but I can't find a piece of documentation that says that the object will still be in a usable state after the CME is thrown. Obviously the iterator is toast at that point, but I'd like to swallow the exception and continue using the collection.
java multithreading collections
add a comment |
I am writing a GUI application using the Immediate Mode GUI pattern, and the UI runs on a thread separate to the engine that powers the actual functionality of the application. The GUI thread ends up iterating over many lists of objects that are conceptually "owned" by the engine thread, and these lists change extremely infrequently. The GUI thread is vsync'ed, meaning it runs at about 60Hz, and the engine thread runs at about 200Hz.
Sometimes, actions in the UI will change the contents of the collections in the engine, and I have a message-passing system to post Runnables to the engine thread to do these mutations to ensure that these mutations don't collide with what's happening in the engine. That way, I can ensure that the engine always sees a consistent view of the data, which for my application is very important.
Because the engine is in charge of all data mutations, though, it sometimes happens that the engine changes the contents of a collection while the GUI is iterating through it, and because these collections are standard Java collections, this predictably and correctly throws a ConcurrentModificationException
. I can think of a few high-level ways to deal with this:
- locking, either by using a synchronized collection or read-write locks
- double-buffer the data that the GUI thread reads, and have the GUI thread flip the double-buffer when it's done drawing a frame
- ignore the CME and abort drawing the rest of the frame, which will draw partial information for the frame in which the "bad" mutation happens, and just continue on to the next frame
Locking comes with a significant performance penalty, and while it would be fine for the GUI to sometimes stall while waiting to acquire the lock from the engine thread, it is very important for the engine thread to run at a consistent speed, and even an R/W lock would cause stalls in the engine thread. Double-buffering comes with significant complexity, as there is a lot of data that is read by the GUI on each frame.
I give you all of this background because I know that option 3 is ugly, and that my question is in some sense "not the right question". The Javadoc for ConcurrentModificationException
even says:
Note that fail-fast behavior cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast operations throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: ConcurrentModificationException should be used only to detect bugs.
But! I am not concerned with the correctness of the GUI for the single frame that would be marred by the CME. I am only concerned with what happens on the next frame. Which leads to my question: is it safe to continue using a Java collection (I'm most interested in the answer for ArrayList
and HashMap
) after a ConcurrentModificationException
has been thrown from its iterator? It seems logical that it would be, but I can't find a piece of documentation that says that the object will still be in a usable state after the CME is thrown. Obviously the iterator is toast at that point, but I'd like to swallow the exception and continue using the collection.
java multithreading collections
I am writing a GUI application using the Immediate Mode GUI pattern, and the UI runs on a thread separate to the engine that powers the actual functionality of the application. The GUI thread ends up iterating over many lists of objects that are conceptually "owned" by the engine thread, and these lists change extremely infrequently. The GUI thread is vsync'ed, meaning it runs at about 60Hz, and the engine thread runs at about 200Hz.
Sometimes, actions in the UI will change the contents of the collections in the engine, and I have a message-passing system to post Runnables to the engine thread to do these mutations to ensure that these mutations don't collide with what's happening in the engine. That way, I can ensure that the engine always sees a consistent view of the data, which for my application is very important.
Because the engine is in charge of all data mutations, though, it sometimes happens that the engine changes the contents of a collection while the GUI is iterating through it, and because these collections are standard Java collections, this predictably and correctly throws a ConcurrentModificationException
. I can think of a few high-level ways to deal with this:
- locking, either by using a synchronized collection or read-write locks
- double-buffer the data that the GUI thread reads, and have the GUI thread flip the double-buffer when it's done drawing a frame
- ignore the CME and abort drawing the rest of the frame, which will draw partial information for the frame in which the "bad" mutation happens, and just continue on to the next frame
Locking comes with a significant performance penalty, and while it would be fine for the GUI to sometimes stall while waiting to acquire the lock from the engine thread, it is very important for the engine thread to run at a consistent speed, and even an R/W lock would cause stalls in the engine thread. Double-buffering comes with significant complexity, as there is a lot of data that is read by the GUI on each frame.
I give you all of this background because I know that option 3 is ugly, and that my question is in some sense "not the right question". The Javadoc for ConcurrentModificationException
even says:
Note that fail-fast behavior cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast operations throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: ConcurrentModificationException should be used only to detect bugs.
But! I am not concerned with the correctness of the GUI for the single frame that would be marred by the CME. I am only concerned with what happens on the next frame. Which leads to my question: is it safe to continue using a Java collection (I'm most interested in the answer for ArrayList
and HashMap
) after a ConcurrentModificationException
has been thrown from its iterator? It seems logical that it would be, but I can't find a piece of documentation that says that the object will still be in a usable state after the CME is thrown. Obviously the iterator is toast at that point, but I'd like to swallow the exception and continue using the collection.
java multithreading collections
java multithreading collections
edited 4 hours ago
Haldean Brown
asked 4 hours ago
Haldean BrownHaldean Brown
8,09443250
8,09443250
add a comment |
add a comment |
4 Answers
4
active
oldest
votes
For most collections, as long as you only have one writer, you are probably ok, but there are no guarantees. If you have multiple writers on a HashMap you can end up with a corrupt map which has an infinite loop in it's structure.
I suggest you use ReentrantLock
which has a tryLock()
method if you want to support only-obtaining-a-lock-if-it's-not-being-used approach to minimise blocking.
Another way to feed data from one thread to another is to Queue
modification tasks which the engine can pick up when not otherwise busy. This allows all access through one thread only.
BTW: A lock/unlock takes around 1 micro-second which is unlikely to make much difference to you unless you do this a lot.
2
Having multiple writers will not causeConcurrentModificationException
. Although the name makes it sound like a multi-threading exception, it is not. Single-threaded code can cause the exception too. --- E.g. having multiple writers on a HashMap, which may corrupt the map, will not cause the exception unless you're iterating it too, and lots of maps are never iterated. --- I believe your answer is missing the point ofConcurrentModificationException
.
– Andreas
4 hours ago
1
I am queueing the modifications, but the issue is that both threads are reading heavily from this shared data, so one of them is always going to lose. TheReentrantLock
is an interesting idea, because the engine could just defer the modification task to the next loop if the GUI thread is currently using the collection.
– Haldean Brown
4 hours ago
@HaldeanBrown You need to queue the entire task, both the reading and writing should be performed by only one thread if you want to avoid locking, or you have to use locking, or your data must be read-only.
– Peter Lawrey
4 hours ago
2
@Andreas I am trying to solve the OP design problem rather than explaining what ConcurrentModificationException means, though you are correct and have a point, I don't think it helps the OP.
– Peter Lawrey
4 hours ago
I think the reentrant lock to "reject" mutations until they can go through without blocking is the way to go, if locking is that cheap. When I prototyped a solution with locks I saw the engine get much slower, but now I'm thinking I must have made a mistake when I did that. Thank you Peter!
– Haldean Brown
4 hours ago
|
show 2 more comments
Quoting part of the quote:
Therefore, it would be wrong to write a program that depended on this exception for its correctness: ConcurrentModificationException should be used only to detect bugs.
Conclusion: Your code is buggy and needs to be fixed, so it doesn't matter what state the collection is in.
You should never get that error in valid code. It's an exception that should never be caught and acted on.
This is not an answer to my question, but am I correct in inferring that your actual answer is that the collection is in an undefined state, and thus should be considered unusable?
– Haldean Brown
4 hours ago
2
@HaldeanBrown I'm saying that you should never code around aConcurrentModificationException
. The exception it a non-guaranteed fast-fail that simply tells you, the programmer, that you've written bad code and the code needs to be fixed. The only correct action when receiving that error is to fix the code, not to add hacky code to work around it. The fact that you're even contemplating doing so mean that you missed the highlighted part of the quote, and that's what this answer is trying to teach you.
– Andreas
4 hours ago
I think what Andreas is saying is that we don't actually know if the collection is unstable or unusable, or not. All we know for sure is that it's an error, and the code needs to be corrected. I'm pretty sure it's implementation dependent. If you look at the code for a collection today, the data might be fine, and then tomorrow Oracle decides to change their implementation and it completely messes you up.
– markspace
4 hours ago
2
@markspace Actually,ConcurrentModificationException
only says that the collection was modified directly, while it was being iterated. It in no way, shape, or form is about whether that modification was incorrectly done. It just says that theIterator
is no longer valid. Multi-threaded update of a collection that is not thread-safe will not cause that exception, it will just silently corrupt the collection. The exception is not about thread-safety, regardless of the exception name.
– Andreas
4 hours ago
That's completely right, and this is a thing I should add to my question; it is true that what I'm talking about is not about thread-safety, because my issue would still crop up if I were using synchronized collections. I could do all of my reads and writes in a thread-safe way but still have the problem with iterators that I'm having now.
– Haldean Brown
3 hours ago
add a comment |
The GUI thread ends up iterating over many lists of objects that are conceptually "owned" by the engine thread
Since the engine owns the data I argue that it should not openly share that data with the GUI.
Consider having the engine push data to the GUI rather than the GUI pulling (or reading directly) from the engines data structures. That way the engine has full and exclusive control of its data, as it probably should.
The downside is that this may take a significant redesign.
But the benefits may be significant enough to warrant it:
- No more
ConcurrentModificationException
- No need for locks
- No need to constantly scan data and redraw, only when an update says to.
The last point could be significant depending on how much data you are dealing with or how complex your redrawing is. Pushing updates to your GUI will give you much more flexibility to optimize if profiling reveals the need to do so. For example, you could make better use of caches on the GUI side and invalidate those caches through updates pushed from the engine.
add a comment |
Locking comes with a significant performance penalty
Does it really? Have you seen if putting in a rudimentary gate/guard/lock/semaphore kills performance? At 60Hz/200Hz it's unlikely to do so.
My first pass at using locking came with a ~10% performance penalty; the problem is that the engine is a complex piece of code that accesses all of these collections in a bunch of different places, so I end up doing a lot of locking and unlocking. I could probably do a cleverer thing with fewer, larger blocks to reduce the performance penalty, but I got sticker shock after seeing the 10%.
– Haldean Brown
4 hours ago
I just saw Peter's edit saying that (un)locking should be near-instantaneous, which makes me wonder if I did a dumb thing when I wrote my locking implementation..
– Haldean Brown
4 hours ago
Just for your edification, try creating a simple driver class with a main method that leverages a bunch of different locking paradigms/primitives (synchronized, the lock types etc). A 10% performance penalty smells like something isn't quite right on the implementation side. Did you run your code through a profiler? Is your use case write-heavy?
– Not a JD
4 hours ago
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55286705%2fis-a-java-collection-guaranteed-to-be-in-a-valid-usable-state-after-a-concurren%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
For most collections, as long as you only have one writer, you are probably ok, but there are no guarantees. If you have multiple writers on a HashMap you can end up with a corrupt map which has an infinite loop in it's structure.
I suggest you use ReentrantLock
which has a tryLock()
method if you want to support only-obtaining-a-lock-if-it's-not-being-used approach to minimise blocking.
Another way to feed data from one thread to another is to Queue
modification tasks which the engine can pick up when not otherwise busy. This allows all access through one thread only.
BTW: A lock/unlock takes around 1 micro-second which is unlikely to make much difference to you unless you do this a lot.
2
Having multiple writers will not causeConcurrentModificationException
. Although the name makes it sound like a multi-threading exception, it is not. Single-threaded code can cause the exception too. --- E.g. having multiple writers on a HashMap, which may corrupt the map, will not cause the exception unless you're iterating it too, and lots of maps are never iterated. --- I believe your answer is missing the point ofConcurrentModificationException
.
– Andreas
4 hours ago
1
I am queueing the modifications, but the issue is that both threads are reading heavily from this shared data, so one of them is always going to lose. TheReentrantLock
is an interesting idea, because the engine could just defer the modification task to the next loop if the GUI thread is currently using the collection.
– Haldean Brown
4 hours ago
@HaldeanBrown You need to queue the entire task, both the reading and writing should be performed by only one thread if you want to avoid locking, or you have to use locking, or your data must be read-only.
– Peter Lawrey
4 hours ago
2
@Andreas I am trying to solve the OP design problem rather than explaining what ConcurrentModificationException means, though you are correct and have a point, I don't think it helps the OP.
– Peter Lawrey
4 hours ago
I think the reentrant lock to "reject" mutations until they can go through without blocking is the way to go, if locking is that cheap. When I prototyped a solution with locks I saw the engine get much slower, but now I'm thinking I must have made a mistake when I did that. Thank you Peter!
– Haldean Brown
4 hours ago
|
show 2 more comments
For most collections, as long as you only have one writer, you are probably ok, but there are no guarantees. If you have multiple writers on a HashMap you can end up with a corrupt map which has an infinite loop in it's structure.
I suggest you use ReentrantLock
which has a tryLock()
method if you want to support only-obtaining-a-lock-if-it's-not-being-used approach to minimise blocking.
Another way to feed data from one thread to another is to Queue
modification tasks which the engine can pick up when not otherwise busy. This allows all access through one thread only.
BTW: A lock/unlock takes around 1 micro-second which is unlikely to make much difference to you unless you do this a lot.
2
Having multiple writers will not causeConcurrentModificationException
. Although the name makes it sound like a multi-threading exception, it is not. Single-threaded code can cause the exception too. --- E.g. having multiple writers on a HashMap, which may corrupt the map, will not cause the exception unless you're iterating it too, and lots of maps are never iterated. --- I believe your answer is missing the point ofConcurrentModificationException
.
– Andreas
4 hours ago
1
I am queueing the modifications, but the issue is that both threads are reading heavily from this shared data, so one of them is always going to lose. TheReentrantLock
is an interesting idea, because the engine could just defer the modification task to the next loop if the GUI thread is currently using the collection.
– Haldean Brown
4 hours ago
@HaldeanBrown You need to queue the entire task, both the reading and writing should be performed by only one thread if you want to avoid locking, or you have to use locking, or your data must be read-only.
– Peter Lawrey
4 hours ago
2
@Andreas I am trying to solve the OP design problem rather than explaining what ConcurrentModificationException means, though you are correct and have a point, I don't think it helps the OP.
– Peter Lawrey
4 hours ago
I think the reentrant lock to "reject" mutations until they can go through without blocking is the way to go, if locking is that cheap. When I prototyped a solution with locks I saw the engine get much slower, but now I'm thinking I must have made a mistake when I did that. Thank you Peter!
– Haldean Brown
4 hours ago
|
show 2 more comments
For most collections, as long as you only have one writer, you are probably ok, but there are no guarantees. If you have multiple writers on a HashMap you can end up with a corrupt map which has an infinite loop in it's structure.
I suggest you use ReentrantLock
which has a tryLock()
method if you want to support only-obtaining-a-lock-if-it's-not-being-used approach to minimise blocking.
Another way to feed data from one thread to another is to Queue
modification tasks which the engine can pick up when not otherwise busy. This allows all access through one thread only.
BTW: A lock/unlock takes around 1 micro-second which is unlikely to make much difference to you unless you do this a lot.
For most collections, as long as you only have one writer, you are probably ok, but there are no guarantees. If you have multiple writers on a HashMap you can end up with a corrupt map which has an infinite loop in it's structure.
I suggest you use ReentrantLock
which has a tryLock()
method if you want to support only-obtaining-a-lock-if-it's-not-being-used approach to minimise blocking.
Another way to feed data from one thread to another is to Queue
modification tasks which the engine can pick up when not otherwise busy. This allows all access through one thread only.
BTW: A lock/unlock takes around 1 micro-second which is unlikely to make much difference to you unless you do this a lot.
edited 4 hours ago
answered 4 hours ago
Peter LawreyPeter Lawrey
448k56574977
448k56574977
2
Having multiple writers will not causeConcurrentModificationException
. Although the name makes it sound like a multi-threading exception, it is not. Single-threaded code can cause the exception too. --- E.g. having multiple writers on a HashMap, which may corrupt the map, will not cause the exception unless you're iterating it too, and lots of maps are never iterated. --- I believe your answer is missing the point ofConcurrentModificationException
.
– Andreas
4 hours ago
1
I am queueing the modifications, but the issue is that both threads are reading heavily from this shared data, so one of them is always going to lose. TheReentrantLock
is an interesting idea, because the engine could just defer the modification task to the next loop if the GUI thread is currently using the collection.
– Haldean Brown
4 hours ago
@HaldeanBrown You need to queue the entire task, both the reading and writing should be performed by only one thread if you want to avoid locking, or you have to use locking, or your data must be read-only.
– Peter Lawrey
4 hours ago
2
@Andreas I am trying to solve the OP design problem rather than explaining what ConcurrentModificationException means, though you are correct and have a point, I don't think it helps the OP.
– Peter Lawrey
4 hours ago
I think the reentrant lock to "reject" mutations until they can go through without blocking is the way to go, if locking is that cheap. When I prototyped a solution with locks I saw the engine get much slower, but now I'm thinking I must have made a mistake when I did that. Thank you Peter!
– Haldean Brown
4 hours ago
|
show 2 more comments
2
Having multiple writers will not causeConcurrentModificationException
. Although the name makes it sound like a multi-threading exception, it is not. Single-threaded code can cause the exception too. --- E.g. having multiple writers on a HashMap, which may corrupt the map, will not cause the exception unless you're iterating it too, and lots of maps are never iterated. --- I believe your answer is missing the point ofConcurrentModificationException
.
– Andreas
4 hours ago
1
I am queueing the modifications, but the issue is that both threads are reading heavily from this shared data, so one of them is always going to lose. TheReentrantLock
is an interesting idea, because the engine could just defer the modification task to the next loop if the GUI thread is currently using the collection.
– Haldean Brown
4 hours ago
@HaldeanBrown You need to queue the entire task, both the reading and writing should be performed by only one thread if you want to avoid locking, or you have to use locking, or your data must be read-only.
– Peter Lawrey
4 hours ago
2
@Andreas I am trying to solve the OP design problem rather than explaining what ConcurrentModificationException means, though you are correct and have a point, I don't think it helps the OP.
– Peter Lawrey
4 hours ago
I think the reentrant lock to "reject" mutations until they can go through without blocking is the way to go, if locking is that cheap. When I prototyped a solution with locks I saw the engine get much slower, but now I'm thinking I must have made a mistake when I did that. Thank you Peter!
– Haldean Brown
4 hours ago
2
2
Having multiple writers will not cause
ConcurrentModificationException
. Although the name makes it sound like a multi-threading exception, it is not. Single-threaded code can cause the exception too. --- E.g. having multiple writers on a HashMap, which may corrupt the map, will not cause the exception unless you're iterating it too, and lots of maps are never iterated. --- I believe your answer is missing the point of ConcurrentModificationException
.– Andreas
4 hours ago
Having multiple writers will not cause
ConcurrentModificationException
. Although the name makes it sound like a multi-threading exception, it is not. Single-threaded code can cause the exception too. --- E.g. having multiple writers on a HashMap, which may corrupt the map, will not cause the exception unless you're iterating it too, and lots of maps are never iterated. --- I believe your answer is missing the point of ConcurrentModificationException
.– Andreas
4 hours ago
1
1
I am queueing the modifications, but the issue is that both threads are reading heavily from this shared data, so one of them is always going to lose. The
ReentrantLock
is an interesting idea, because the engine could just defer the modification task to the next loop if the GUI thread is currently using the collection.– Haldean Brown
4 hours ago
I am queueing the modifications, but the issue is that both threads are reading heavily from this shared data, so one of them is always going to lose. The
ReentrantLock
is an interesting idea, because the engine could just defer the modification task to the next loop if the GUI thread is currently using the collection.– Haldean Brown
4 hours ago
@HaldeanBrown You need to queue the entire task, both the reading and writing should be performed by only one thread if you want to avoid locking, or you have to use locking, or your data must be read-only.
– Peter Lawrey
4 hours ago
@HaldeanBrown You need to queue the entire task, both the reading and writing should be performed by only one thread if you want to avoid locking, or you have to use locking, or your data must be read-only.
– Peter Lawrey
4 hours ago
2
2
@Andreas I am trying to solve the OP design problem rather than explaining what ConcurrentModificationException means, though you are correct and have a point, I don't think it helps the OP.
– Peter Lawrey
4 hours ago
@Andreas I am trying to solve the OP design problem rather than explaining what ConcurrentModificationException means, though you are correct and have a point, I don't think it helps the OP.
– Peter Lawrey
4 hours ago
I think the reentrant lock to "reject" mutations until they can go through without blocking is the way to go, if locking is that cheap. When I prototyped a solution with locks I saw the engine get much slower, but now I'm thinking I must have made a mistake when I did that. Thank you Peter!
– Haldean Brown
4 hours ago
I think the reentrant lock to "reject" mutations until they can go through without blocking is the way to go, if locking is that cheap. When I prototyped a solution with locks I saw the engine get much slower, but now I'm thinking I must have made a mistake when I did that. Thank you Peter!
– Haldean Brown
4 hours ago
|
show 2 more comments
Quoting part of the quote:
Therefore, it would be wrong to write a program that depended on this exception for its correctness: ConcurrentModificationException should be used only to detect bugs.
Conclusion: Your code is buggy and needs to be fixed, so it doesn't matter what state the collection is in.
You should never get that error in valid code. It's an exception that should never be caught and acted on.
This is not an answer to my question, but am I correct in inferring that your actual answer is that the collection is in an undefined state, and thus should be considered unusable?
– Haldean Brown
4 hours ago
2
@HaldeanBrown I'm saying that you should never code around aConcurrentModificationException
. The exception it a non-guaranteed fast-fail that simply tells you, the programmer, that you've written bad code and the code needs to be fixed. The only correct action when receiving that error is to fix the code, not to add hacky code to work around it. The fact that you're even contemplating doing so mean that you missed the highlighted part of the quote, and that's what this answer is trying to teach you.
– Andreas
4 hours ago
I think what Andreas is saying is that we don't actually know if the collection is unstable or unusable, or not. All we know for sure is that it's an error, and the code needs to be corrected. I'm pretty sure it's implementation dependent. If you look at the code for a collection today, the data might be fine, and then tomorrow Oracle decides to change their implementation and it completely messes you up.
– markspace
4 hours ago
2
@markspace Actually,ConcurrentModificationException
only says that the collection was modified directly, while it was being iterated. It in no way, shape, or form is about whether that modification was incorrectly done. It just says that theIterator
is no longer valid. Multi-threaded update of a collection that is not thread-safe will not cause that exception, it will just silently corrupt the collection. The exception is not about thread-safety, regardless of the exception name.
– Andreas
4 hours ago
That's completely right, and this is a thing I should add to my question; it is true that what I'm talking about is not about thread-safety, because my issue would still crop up if I were using synchronized collections. I could do all of my reads and writes in a thread-safe way but still have the problem with iterators that I'm having now.
– Haldean Brown
3 hours ago
add a comment |
Quoting part of the quote:
Therefore, it would be wrong to write a program that depended on this exception for its correctness: ConcurrentModificationException should be used only to detect bugs.
Conclusion: Your code is buggy and needs to be fixed, so it doesn't matter what state the collection is in.
You should never get that error in valid code. It's an exception that should never be caught and acted on.
This is not an answer to my question, but am I correct in inferring that your actual answer is that the collection is in an undefined state, and thus should be considered unusable?
– Haldean Brown
4 hours ago
2
@HaldeanBrown I'm saying that you should never code around aConcurrentModificationException
. The exception it a non-guaranteed fast-fail that simply tells you, the programmer, that you've written bad code and the code needs to be fixed. The only correct action when receiving that error is to fix the code, not to add hacky code to work around it. The fact that you're even contemplating doing so mean that you missed the highlighted part of the quote, and that's what this answer is trying to teach you.
– Andreas
4 hours ago
I think what Andreas is saying is that we don't actually know if the collection is unstable or unusable, or not. All we know for sure is that it's an error, and the code needs to be corrected. I'm pretty sure it's implementation dependent. If you look at the code for a collection today, the data might be fine, and then tomorrow Oracle decides to change their implementation and it completely messes you up.
– markspace
4 hours ago
2
@markspace Actually,ConcurrentModificationException
only says that the collection was modified directly, while it was being iterated. It in no way, shape, or form is about whether that modification was incorrectly done. It just says that theIterator
is no longer valid. Multi-threaded update of a collection that is not thread-safe will not cause that exception, it will just silently corrupt the collection. The exception is not about thread-safety, regardless of the exception name.
– Andreas
4 hours ago
That's completely right, and this is a thing I should add to my question; it is true that what I'm talking about is not about thread-safety, because my issue would still crop up if I were using synchronized collections. I could do all of my reads and writes in a thread-safe way but still have the problem with iterators that I'm having now.
– Haldean Brown
3 hours ago
add a comment |
Quoting part of the quote:
Therefore, it would be wrong to write a program that depended on this exception for its correctness: ConcurrentModificationException should be used only to detect bugs.
Conclusion: Your code is buggy and needs to be fixed, so it doesn't matter what state the collection is in.
You should never get that error in valid code. It's an exception that should never be caught and acted on.
Quoting part of the quote:
Therefore, it would be wrong to write a program that depended on this exception for its correctness: ConcurrentModificationException should be used only to detect bugs.
Conclusion: Your code is buggy and needs to be fixed, so it doesn't matter what state the collection is in.
You should never get that error in valid code. It's an exception that should never be caught and acted on.
answered 4 hours ago
AndreasAndreas
78.7k464129
78.7k464129
This is not an answer to my question, but am I correct in inferring that your actual answer is that the collection is in an undefined state, and thus should be considered unusable?
– Haldean Brown
4 hours ago
2
@HaldeanBrown I'm saying that you should never code around aConcurrentModificationException
. The exception it a non-guaranteed fast-fail that simply tells you, the programmer, that you've written bad code and the code needs to be fixed. The only correct action when receiving that error is to fix the code, not to add hacky code to work around it. The fact that you're even contemplating doing so mean that you missed the highlighted part of the quote, and that's what this answer is trying to teach you.
– Andreas
4 hours ago
I think what Andreas is saying is that we don't actually know if the collection is unstable or unusable, or not. All we know for sure is that it's an error, and the code needs to be corrected. I'm pretty sure it's implementation dependent. If you look at the code for a collection today, the data might be fine, and then tomorrow Oracle decides to change their implementation and it completely messes you up.
– markspace
4 hours ago
2
@markspace Actually,ConcurrentModificationException
only says that the collection was modified directly, while it was being iterated. It in no way, shape, or form is about whether that modification was incorrectly done. It just says that theIterator
is no longer valid. Multi-threaded update of a collection that is not thread-safe will not cause that exception, it will just silently corrupt the collection. The exception is not about thread-safety, regardless of the exception name.
– Andreas
4 hours ago
That's completely right, and this is a thing I should add to my question; it is true that what I'm talking about is not about thread-safety, because my issue would still crop up if I were using synchronized collections. I could do all of my reads and writes in a thread-safe way but still have the problem with iterators that I'm having now.
– Haldean Brown
3 hours ago
add a comment |
This is not an answer to my question, but am I correct in inferring that your actual answer is that the collection is in an undefined state, and thus should be considered unusable?
– Haldean Brown
4 hours ago
2
@HaldeanBrown I'm saying that you should never code around aConcurrentModificationException
. The exception it a non-guaranteed fast-fail that simply tells you, the programmer, that you've written bad code and the code needs to be fixed. The only correct action when receiving that error is to fix the code, not to add hacky code to work around it. The fact that you're even contemplating doing so mean that you missed the highlighted part of the quote, and that's what this answer is trying to teach you.
– Andreas
4 hours ago
I think what Andreas is saying is that we don't actually know if the collection is unstable or unusable, or not. All we know for sure is that it's an error, and the code needs to be corrected. I'm pretty sure it's implementation dependent. If you look at the code for a collection today, the data might be fine, and then tomorrow Oracle decides to change their implementation and it completely messes you up.
– markspace
4 hours ago
2
@markspace Actually,ConcurrentModificationException
only says that the collection was modified directly, while it was being iterated. It in no way, shape, or form is about whether that modification was incorrectly done. It just says that theIterator
is no longer valid. Multi-threaded update of a collection that is not thread-safe will not cause that exception, it will just silently corrupt the collection. The exception is not about thread-safety, regardless of the exception name.
– Andreas
4 hours ago
That's completely right, and this is a thing I should add to my question; it is true that what I'm talking about is not about thread-safety, because my issue would still crop up if I were using synchronized collections. I could do all of my reads and writes in a thread-safe way but still have the problem with iterators that I'm having now.
– Haldean Brown
3 hours ago
This is not an answer to my question, but am I correct in inferring that your actual answer is that the collection is in an undefined state, and thus should be considered unusable?
– Haldean Brown
4 hours ago
This is not an answer to my question, but am I correct in inferring that your actual answer is that the collection is in an undefined state, and thus should be considered unusable?
– Haldean Brown
4 hours ago
2
2
@HaldeanBrown I'm saying that you should never code around a
ConcurrentModificationException
. The exception it a non-guaranteed fast-fail that simply tells you, the programmer, that you've written bad code and the code needs to be fixed. The only correct action when receiving that error is to fix the code, not to add hacky code to work around it. The fact that you're even contemplating doing so mean that you missed the highlighted part of the quote, and that's what this answer is trying to teach you.– Andreas
4 hours ago
@HaldeanBrown I'm saying that you should never code around a
ConcurrentModificationException
. The exception it a non-guaranteed fast-fail that simply tells you, the programmer, that you've written bad code and the code needs to be fixed. The only correct action when receiving that error is to fix the code, not to add hacky code to work around it. The fact that you're even contemplating doing so mean that you missed the highlighted part of the quote, and that's what this answer is trying to teach you.– Andreas
4 hours ago
I think what Andreas is saying is that we don't actually know if the collection is unstable or unusable, or not. All we know for sure is that it's an error, and the code needs to be corrected. I'm pretty sure it's implementation dependent. If you look at the code for a collection today, the data might be fine, and then tomorrow Oracle decides to change their implementation and it completely messes you up.
– markspace
4 hours ago
I think what Andreas is saying is that we don't actually know if the collection is unstable or unusable, or not. All we know for sure is that it's an error, and the code needs to be corrected. I'm pretty sure it's implementation dependent. If you look at the code for a collection today, the data might be fine, and then tomorrow Oracle decides to change their implementation and it completely messes you up.
– markspace
4 hours ago
2
2
@markspace Actually,
ConcurrentModificationException
only says that the collection was modified directly, while it was being iterated. It in no way, shape, or form is about whether that modification was incorrectly done. It just says that the Iterator
is no longer valid. Multi-threaded update of a collection that is not thread-safe will not cause that exception, it will just silently corrupt the collection. The exception is not about thread-safety, regardless of the exception name.– Andreas
4 hours ago
@markspace Actually,
ConcurrentModificationException
only says that the collection was modified directly, while it was being iterated. It in no way, shape, or form is about whether that modification was incorrectly done. It just says that the Iterator
is no longer valid. Multi-threaded update of a collection that is not thread-safe will not cause that exception, it will just silently corrupt the collection. The exception is not about thread-safety, regardless of the exception name.– Andreas
4 hours ago
That's completely right, and this is a thing I should add to my question; it is true that what I'm talking about is not about thread-safety, because my issue would still crop up if I were using synchronized collections. I could do all of my reads and writes in a thread-safe way but still have the problem with iterators that I'm having now.
– Haldean Brown
3 hours ago
That's completely right, and this is a thing I should add to my question; it is true that what I'm talking about is not about thread-safety, because my issue would still crop up if I were using synchronized collections. I could do all of my reads and writes in a thread-safe way but still have the problem with iterators that I'm having now.
– Haldean Brown
3 hours ago
add a comment |
The GUI thread ends up iterating over many lists of objects that are conceptually "owned" by the engine thread
Since the engine owns the data I argue that it should not openly share that data with the GUI.
Consider having the engine push data to the GUI rather than the GUI pulling (or reading directly) from the engines data structures. That way the engine has full and exclusive control of its data, as it probably should.
The downside is that this may take a significant redesign.
But the benefits may be significant enough to warrant it:
- No more
ConcurrentModificationException
- No need for locks
- No need to constantly scan data and redraw, only when an update says to.
The last point could be significant depending on how much data you are dealing with or how complex your redrawing is. Pushing updates to your GUI will give you much more flexibility to optimize if profiling reveals the need to do so. For example, you could make better use of caches on the GUI side and invalidate those caches through updates pushed from the engine.
add a comment |
The GUI thread ends up iterating over many lists of objects that are conceptually "owned" by the engine thread
Since the engine owns the data I argue that it should not openly share that data with the GUI.
Consider having the engine push data to the GUI rather than the GUI pulling (or reading directly) from the engines data structures. That way the engine has full and exclusive control of its data, as it probably should.
The downside is that this may take a significant redesign.
But the benefits may be significant enough to warrant it:
- No more
ConcurrentModificationException
- No need for locks
- No need to constantly scan data and redraw, only when an update says to.
The last point could be significant depending on how much data you are dealing with or how complex your redrawing is. Pushing updates to your GUI will give you much more flexibility to optimize if profiling reveals the need to do so. For example, you could make better use of caches on the GUI side and invalidate those caches through updates pushed from the engine.
add a comment |
The GUI thread ends up iterating over many lists of objects that are conceptually "owned" by the engine thread
Since the engine owns the data I argue that it should not openly share that data with the GUI.
Consider having the engine push data to the GUI rather than the GUI pulling (or reading directly) from the engines data structures. That way the engine has full and exclusive control of its data, as it probably should.
The downside is that this may take a significant redesign.
But the benefits may be significant enough to warrant it:
- No more
ConcurrentModificationException
- No need for locks
- No need to constantly scan data and redraw, only when an update says to.
The last point could be significant depending on how much data you are dealing with or how complex your redrawing is. Pushing updates to your GUI will give you much more flexibility to optimize if profiling reveals the need to do so. For example, you could make better use of caches on the GUI side and invalidate those caches through updates pushed from the engine.
The GUI thread ends up iterating over many lists of objects that are conceptually "owned" by the engine thread
Since the engine owns the data I argue that it should not openly share that data with the GUI.
Consider having the engine push data to the GUI rather than the GUI pulling (or reading directly) from the engines data structures. That way the engine has full and exclusive control of its data, as it probably should.
The downside is that this may take a significant redesign.
But the benefits may be significant enough to warrant it:
- No more
ConcurrentModificationException
- No need for locks
- No need to constantly scan data and redraw, only when an update says to.
The last point could be significant depending on how much data you are dealing with or how complex your redrawing is. Pushing updates to your GUI will give you much more flexibility to optimize if profiling reveals the need to do so. For example, you could make better use of caches on the GUI side and invalidate those caches through updates pushed from the engine.
edited 3 hours ago
answered 4 hours ago
xtraticxtratic
2,5741824
2,5741824
add a comment |
add a comment |
Locking comes with a significant performance penalty
Does it really? Have you seen if putting in a rudimentary gate/guard/lock/semaphore kills performance? At 60Hz/200Hz it's unlikely to do so.
My first pass at using locking came with a ~10% performance penalty; the problem is that the engine is a complex piece of code that accesses all of these collections in a bunch of different places, so I end up doing a lot of locking and unlocking. I could probably do a cleverer thing with fewer, larger blocks to reduce the performance penalty, but I got sticker shock after seeing the 10%.
– Haldean Brown
4 hours ago
I just saw Peter's edit saying that (un)locking should be near-instantaneous, which makes me wonder if I did a dumb thing when I wrote my locking implementation..
– Haldean Brown
4 hours ago
Just for your edification, try creating a simple driver class with a main method that leverages a bunch of different locking paradigms/primitives (synchronized, the lock types etc). A 10% performance penalty smells like something isn't quite right on the implementation side. Did you run your code through a profiler? Is your use case write-heavy?
– Not a JD
4 hours ago
add a comment |
Locking comes with a significant performance penalty
Does it really? Have you seen if putting in a rudimentary gate/guard/lock/semaphore kills performance? At 60Hz/200Hz it's unlikely to do so.
My first pass at using locking came with a ~10% performance penalty; the problem is that the engine is a complex piece of code that accesses all of these collections in a bunch of different places, so I end up doing a lot of locking and unlocking. I could probably do a cleverer thing with fewer, larger blocks to reduce the performance penalty, but I got sticker shock after seeing the 10%.
– Haldean Brown
4 hours ago
I just saw Peter's edit saying that (un)locking should be near-instantaneous, which makes me wonder if I did a dumb thing when I wrote my locking implementation..
– Haldean Brown
4 hours ago
Just for your edification, try creating a simple driver class with a main method that leverages a bunch of different locking paradigms/primitives (synchronized, the lock types etc). A 10% performance penalty smells like something isn't quite right on the implementation side. Did you run your code through a profiler? Is your use case write-heavy?
– Not a JD
4 hours ago
add a comment |
Locking comes with a significant performance penalty
Does it really? Have you seen if putting in a rudimentary gate/guard/lock/semaphore kills performance? At 60Hz/200Hz it's unlikely to do so.
Locking comes with a significant performance penalty
Does it really? Have you seen if putting in a rudimentary gate/guard/lock/semaphore kills performance? At 60Hz/200Hz it's unlikely to do so.
answered 4 hours ago
Not a JDNot a JD
3248
3248
My first pass at using locking came with a ~10% performance penalty; the problem is that the engine is a complex piece of code that accesses all of these collections in a bunch of different places, so I end up doing a lot of locking and unlocking. I could probably do a cleverer thing with fewer, larger blocks to reduce the performance penalty, but I got sticker shock after seeing the 10%.
– Haldean Brown
4 hours ago
I just saw Peter's edit saying that (un)locking should be near-instantaneous, which makes me wonder if I did a dumb thing when I wrote my locking implementation..
– Haldean Brown
4 hours ago
Just for your edification, try creating a simple driver class with a main method that leverages a bunch of different locking paradigms/primitives (synchronized, the lock types etc). A 10% performance penalty smells like something isn't quite right on the implementation side. Did you run your code through a profiler? Is your use case write-heavy?
– Not a JD
4 hours ago
add a comment |
My first pass at using locking came with a ~10% performance penalty; the problem is that the engine is a complex piece of code that accesses all of these collections in a bunch of different places, so I end up doing a lot of locking and unlocking. I could probably do a cleverer thing with fewer, larger blocks to reduce the performance penalty, but I got sticker shock after seeing the 10%.
– Haldean Brown
4 hours ago
I just saw Peter's edit saying that (un)locking should be near-instantaneous, which makes me wonder if I did a dumb thing when I wrote my locking implementation..
– Haldean Brown
4 hours ago
Just for your edification, try creating a simple driver class with a main method that leverages a bunch of different locking paradigms/primitives (synchronized, the lock types etc). A 10% performance penalty smells like something isn't quite right on the implementation side. Did you run your code through a profiler? Is your use case write-heavy?
– Not a JD
4 hours ago
My first pass at using locking came with a ~10% performance penalty; the problem is that the engine is a complex piece of code that accesses all of these collections in a bunch of different places, so I end up doing a lot of locking and unlocking. I could probably do a cleverer thing with fewer, larger blocks to reduce the performance penalty, but I got sticker shock after seeing the 10%.
– Haldean Brown
4 hours ago
My first pass at using locking came with a ~10% performance penalty; the problem is that the engine is a complex piece of code that accesses all of these collections in a bunch of different places, so I end up doing a lot of locking and unlocking. I could probably do a cleverer thing with fewer, larger blocks to reduce the performance penalty, but I got sticker shock after seeing the 10%.
– Haldean Brown
4 hours ago
I just saw Peter's edit saying that (un)locking should be near-instantaneous, which makes me wonder if I did a dumb thing when I wrote my locking implementation..
– Haldean Brown
4 hours ago
I just saw Peter's edit saying that (un)locking should be near-instantaneous, which makes me wonder if I did a dumb thing when I wrote my locking implementation..
– Haldean Brown
4 hours ago
Just for your edification, try creating a simple driver class with a main method that leverages a bunch of different locking paradigms/primitives (synchronized, the lock types etc). A 10% performance penalty smells like something isn't quite right on the implementation side. Did you run your code through a profiler? Is your use case write-heavy?
– Not a JD
4 hours ago
Just for your edification, try creating a simple driver class with a main method that leverages a bunch of different locking paradigms/primitives (synchronized, the lock types etc). A 10% performance penalty smells like something isn't quite right on the implementation side. Did you run your code through a profiler? Is your use case write-heavy?
– Not a JD
4 hours ago
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55286705%2fis-a-java-collection-guaranteed-to-be-in-a-valid-usable-state-after-a-concurren%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown