Use pv with find -exec
I use the following command to verify ~700 GiB of backed-up files:
$ find -type f -exec md5sum {} + | sort > ~/checksums
This takes many hours, so I would like to integrate pv
into the command to show the progress.
I could do this:
$ find -type f -exec pv {} + | md5sum
But it concatenates all of the files, resulting in only one checksum.
So how could I include pv
and still get a text file full of checksums at the end?
find checksum pv
add a comment |
I use the following command to verify ~700 GiB of backed-up files:
$ find -type f -exec md5sum {} + | sort > ~/checksums
This takes many hours, so I would like to integrate pv
into the command to show the progress.
I could do this:
$ find -type f -exec pv {} + | md5sum
But it concatenates all of the files, resulting in only one checksum.
So how could I include pv
and still get a text file full of checksums at the end?
find checksum pv
add a comment |
I use the following command to verify ~700 GiB of backed-up files:
$ find -type f -exec md5sum {} + | sort > ~/checksums
This takes many hours, so I would like to integrate pv
into the command to show the progress.
I could do this:
$ find -type f -exec pv {} + | md5sum
But it concatenates all of the files, resulting in only one checksum.
So how could I include pv
and still get a text file full of checksums at the end?
find checksum pv
I use the following command to verify ~700 GiB of backed-up files:
$ find -type f -exec md5sum {} + | sort > ~/checksums
This takes many hours, so I would like to integrate pv
into the command to show the progress.
I could do this:
$ find -type f -exec pv {} + | md5sum
But it concatenates all of the files, resulting in only one checksum.
So how could I include pv
and still get a text file full of checksums at the end?
find checksum pv
find checksum pv
edited 6 hours ago
EmmaV
asked 7 hours ago
EmmaVEmmaV
1,1431332
1,1431332
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Your first command should not be able to run at all as you can't use a pipe in an -exec
like that (this was apparently a typo in the original question).
Instead:
find . -type f -exec md5sum {} + | sort -o ~/checksums
or, with pv
,
find . -type f -exec md5sum {} + | pv | sort -o ~/checksums
In both of the above, md5sum
would be called with as many pathnames as possible in batches. sort
would take the output of find
(which is the output of md5sum
) and sort it into the given filename. The second variation additionally inserts pv
between the find
and the sort
.
You can't use -exec pv {} | md5sum
for individual files as the pipe would need to be embedded in a in-line shell script that you call from -exec
for each file. But even the correct -exec sh -c 'pv "$1" | md5sum' sh {} ;
would discard the filename from the generated md5sum
output, so that can't be used either. The pv
utility acts like a drop-in replacement for cat
.
I wrote the first command wrong. I'll change it.
– EmmaV
6 hours ago
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f495440%2fuse-pv-with-find-exec%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Your first command should not be able to run at all as you can't use a pipe in an -exec
like that (this was apparently a typo in the original question).
Instead:
find . -type f -exec md5sum {} + | sort -o ~/checksums
or, with pv
,
find . -type f -exec md5sum {} + | pv | sort -o ~/checksums
In both of the above, md5sum
would be called with as many pathnames as possible in batches. sort
would take the output of find
(which is the output of md5sum
) and sort it into the given filename. The second variation additionally inserts pv
between the find
and the sort
.
You can't use -exec pv {} | md5sum
for individual files as the pipe would need to be embedded in a in-line shell script that you call from -exec
for each file. But even the correct -exec sh -c 'pv "$1" | md5sum' sh {} ;
would discard the filename from the generated md5sum
output, so that can't be used either. The pv
utility acts like a drop-in replacement for cat
.
I wrote the first command wrong. I'll change it.
– EmmaV
6 hours ago
add a comment |
Your first command should not be able to run at all as you can't use a pipe in an -exec
like that (this was apparently a typo in the original question).
Instead:
find . -type f -exec md5sum {} + | sort -o ~/checksums
or, with pv
,
find . -type f -exec md5sum {} + | pv | sort -o ~/checksums
In both of the above, md5sum
would be called with as many pathnames as possible in batches. sort
would take the output of find
(which is the output of md5sum
) and sort it into the given filename. The second variation additionally inserts pv
between the find
and the sort
.
You can't use -exec pv {} | md5sum
for individual files as the pipe would need to be embedded in a in-line shell script that you call from -exec
for each file. But even the correct -exec sh -c 'pv "$1" | md5sum' sh {} ;
would discard the filename from the generated md5sum
output, so that can't be used either. The pv
utility acts like a drop-in replacement for cat
.
I wrote the first command wrong. I'll change it.
– EmmaV
6 hours ago
add a comment |
Your first command should not be able to run at all as you can't use a pipe in an -exec
like that (this was apparently a typo in the original question).
Instead:
find . -type f -exec md5sum {} + | sort -o ~/checksums
or, with pv
,
find . -type f -exec md5sum {} + | pv | sort -o ~/checksums
In both of the above, md5sum
would be called with as many pathnames as possible in batches. sort
would take the output of find
(which is the output of md5sum
) and sort it into the given filename. The second variation additionally inserts pv
between the find
and the sort
.
You can't use -exec pv {} | md5sum
for individual files as the pipe would need to be embedded in a in-line shell script that you call from -exec
for each file. But even the correct -exec sh -c 'pv "$1" | md5sum' sh {} ;
would discard the filename from the generated md5sum
output, so that can't be used either. The pv
utility acts like a drop-in replacement for cat
.
Your first command should not be able to run at all as you can't use a pipe in an -exec
like that (this was apparently a typo in the original question).
Instead:
find . -type f -exec md5sum {} + | sort -o ~/checksums
or, with pv
,
find . -type f -exec md5sum {} + | pv | sort -o ~/checksums
In both of the above, md5sum
would be called with as many pathnames as possible in batches. sort
would take the output of find
(which is the output of md5sum
) and sort it into the given filename. The second variation additionally inserts pv
between the find
and the sort
.
You can't use -exec pv {} | md5sum
for individual files as the pipe would need to be embedded in a in-line shell script that you call from -exec
for each file. But even the correct -exec sh -c 'pv "$1" | md5sum' sh {} ;
would discard the filename from the generated md5sum
output, so that can't be used either. The pv
utility acts like a drop-in replacement for cat
.
edited 6 hours ago
answered 7 hours ago
KusalanandaKusalananda
125k16236389
125k16236389
I wrote the first command wrong. I'll change it.
– EmmaV
6 hours ago
add a comment |
I wrote the first command wrong. I'll change it.
– EmmaV
6 hours ago
I wrote the first command wrong. I'll change it.
– EmmaV
6 hours ago
I wrote the first command wrong. I'll change it.
– EmmaV
6 hours ago
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f495440%2fuse-pv-with-find-exec%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown