How does the MQTT server output a single raw file from multiple client publications?
I'm trying my first IoT project whereby I want to:
- have an electret microphone capture audio
- have an ESP8266 NodeMCU 12-E board submit captured audio to a remote server
- have a remote server receive the audio data using an MQTT server
- publish the audio data as a .WAV file on the server
I saw someone online demonstrate something similar in this YouTube video
https://www.youtube.com/watch?v=rU_Pw9Jb_PM
The author shared the project on github here
https://github.com/hjltu/esp8266-wifi-microphone
When I study the code, I think what I see is the author taking the value of analogRead(A)
and appending it to some kind of string as a payload, which is then published to an MQTT server.
I see that the author expects the MQTT server or some other software to process the ESP8266 microphone audio data and output it as a .RAW file. This RAW file is eventually converted to a .WAV file with the help of ffmpeg.
My question is this: What command allows MQTT server to generate the .RAW file? Or is this done by an entirely different software? And it appears to me that for a single recording/audio file, the my_record()
of esp8266-wifi-mic.ino
file will send multiple payloads to the MQTT server. So how does the MQTT server know which published transmission belongs to which RAW file?
mqtt esp8266
New contributor
add a comment |
I'm trying my first IoT project whereby I want to:
- have an electret microphone capture audio
- have an ESP8266 NodeMCU 12-E board submit captured audio to a remote server
- have a remote server receive the audio data using an MQTT server
- publish the audio data as a .WAV file on the server
I saw someone online demonstrate something similar in this YouTube video
https://www.youtube.com/watch?v=rU_Pw9Jb_PM
The author shared the project on github here
https://github.com/hjltu/esp8266-wifi-microphone
When I study the code, I think what I see is the author taking the value of analogRead(A)
and appending it to some kind of string as a payload, which is then published to an MQTT server.
I see that the author expects the MQTT server or some other software to process the ESP8266 microphone audio data and output it as a .RAW file. This RAW file is eventually converted to a .WAV file with the help of ffmpeg.
My question is this: What command allows MQTT server to generate the .RAW file? Or is this done by an entirely different software? And it appears to me that for a single recording/audio file, the my_record()
of esp8266-wifi-mic.ino
file will send multiple payloads to the MQTT server. So how does the MQTT server know which published transmission belongs to which RAW file?
mqtt esp8266
New contributor
add a comment |
I'm trying my first IoT project whereby I want to:
- have an electret microphone capture audio
- have an ESP8266 NodeMCU 12-E board submit captured audio to a remote server
- have a remote server receive the audio data using an MQTT server
- publish the audio data as a .WAV file on the server
I saw someone online demonstrate something similar in this YouTube video
https://www.youtube.com/watch?v=rU_Pw9Jb_PM
The author shared the project on github here
https://github.com/hjltu/esp8266-wifi-microphone
When I study the code, I think what I see is the author taking the value of analogRead(A)
and appending it to some kind of string as a payload, which is then published to an MQTT server.
I see that the author expects the MQTT server or some other software to process the ESP8266 microphone audio data and output it as a .RAW file. This RAW file is eventually converted to a .WAV file with the help of ffmpeg.
My question is this: What command allows MQTT server to generate the .RAW file? Or is this done by an entirely different software? And it appears to me that for a single recording/audio file, the my_record()
of esp8266-wifi-mic.ino
file will send multiple payloads to the MQTT server. So how does the MQTT server know which published transmission belongs to which RAW file?
mqtt esp8266
New contributor
I'm trying my first IoT project whereby I want to:
- have an electret microphone capture audio
- have an ESP8266 NodeMCU 12-E board submit captured audio to a remote server
- have a remote server receive the audio data using an MQTT server
- publish the audio data as a .WAV file on the server
I saw someone online demonstrate something similar in this YouTube video
https://www.youtube.com/watch?v=rU_Pw9Jb_PM
The author shared the project on github here
https://github.com/hjltu/esp8266-wifi-microphone
When I study the code, I think what I see is the author taking the value of analogRead(A)
and appending it to some kind of string as a payload, which is then published to an MQTT server.
I see that the author expects the MQTT server or some other software to process the ESP8266 microphone audio data and output it as a .RAW file. This RAW file is eventually converted to a .WAV file with the help of ffmpeg.
My question is this: What command allows MQTT server to generate the .RAW file? Or is this done by an entirely different software? And it appears to me that for a single recording/audio file, the my_record()
of esp8266-wifi-mic.ino
file will send multiple payloads to the MQTT server. So how does the MQTT server know which published transmission belongs to which RAW file?
mqtt esp8266
mqtt esp8266
New contributor
New contributor
New contributor
asked 2 hours ago
JohnJohn
1084
1084
New contributor
New contributor
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
You are correct about the microphone input, void my_record()
samples the microphone output level 1000 times, appending each reading to a string variable and publishes the resulting string to an MQTT broker.
This process repeats 11 times every time my_record() is called.
note: You are sort of misunderstanding about the .RAW file. It is a raw file, meaning that it is unprocessed and unformatted .... just a stream of bytes. Using the term .RAW implies a file name extension.
What command allows MQTT server to generate the .RAW file?
The MQTT broker (server) does not generate the raw file, the file is published to the MQTT broker by an outside source, the ESP8266 in this instance.
So how does the MQTT server know which published transmission belongs to which RAW file?
It does not know. All it does is to relay messages. It is up to the publisher
to send to the correct topic
and it is up to the subscriber
to watch the data at the correct topic.
The messages could arrive to the subscriber out of sequence, so a sequence number needs to be included with the message if a correct data sequence is desired.
Have a look at these for a visual demo of MQTT messages.
https://shiftr.io/shiftr-io/demo ... you can only subscribe to this one
https://shiftr.io/try or https://shiftr.io/shiftr-io/try ... you can publish to this one (and subscribe)
You can get your own account and watch your own messages being sent and received without the clutter of other messages.
New contributor
Oh....I see. So that means I need to create another client application to read subscription data from MQTT server, and save content to a .raw file?
– John
1 hour ago
1
exactly correct ..... the MQTT server is just a long pipe for your data
– jsotola
1 hour ago
data can arrive out of sequence though ..... updated answer
– jsotola
1 hour ago
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "666"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
John is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fiot.stackexchange.com%2fquestions%2f3838%2fhow-does-the-mqtt-server-output-a-single-raw-file-from-multiple-client-publicati%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
You are correct about the microphone input, void my_record()
samples the microphone output level 1000 times, appending each reading to a string variable and publishes the resulting string to an MQTT broker.
This process repeats 11 times every time my_record() is called.
note: You are sort of misunderstanding about the .RAW file. It is a raw file, meaning that it is unprocessed and unformatted .... just a stream of bytes. Using the term .RAW implies a file name extension.
What command allows MQTT server to generate the .RAW file?
The MQTT broker (server) does not generate the raw file, the file is published to the MQTT broker by an outside source, the ESP8266 in this instance.
So how does the MQTT server know which published transmission belongs to which RAW file?
It does not know. All it does is to relay messages. It is up to the publisher
to send to the correct topic
and it is up to the subscriber
to watch the data at the correct topic.
The messages could arrive to the subscriber out of sequence, so a sequence number needs to be included with the message if a correct data sequence is desired.
Have a look at these for a visual demo of MQTT messages.
https://shiftr.io/shiftr-io/demo ... you can only subscribe to this one
https://shiftr.io/try or https://shiftr.io/shiftr-io/try ... you can publish to this one (and subscribe)
You can get your own account and watch your own messages being sent and received without the clutter of other messages.
New contributor
Oh....I see. So that means I need to create another client application to read subscription data from MQTT server, and save content to a .raw file?
– John
1 hour ago
1
exactly correct ..... the MQTT server is just a long pipe for your data
– jsotola
1 hour ago
data can arrive out of sequence though ..... updated answer
– jsotola
1 hour ago
add a comment |
You are correct about the microphone input, void my_record()
samples the microphone output level 1000 times, appending each reading to a string variable and publishes the resulting string to an MQTT broker.
This process repeats 11 times every time my_record() is called.
note: You are sort of misunderstanding about the .RAW file. It is a raw file, meaning that it is unprocessed and unformatted .... just a stream of bytes. Using the term .RAW implies a file name extension.
What command allows MQTT server to generate the .RAW file?
The MQTT broker (server) does not generate the raw file, the file is published to the MQTT broker by an outside source, the ESP8266 in this instance.
So how does the MQTT server know which published transmission belongs to which RAW file?
It does not know. All it does is to relay messages. It is up to the publisher
to send to the correct topic
and it is up to the subscriber
to watch the data at the correct topic.
The messages could arrive to the subscriber out of sequence, so a sequence number needs to be included with the message if a correct data sequence is desired.
Have a look at these for a visual demo of MQTT messages.
https://shiftr.io/shiftr-io/demo ... you can only subscribe to this one
https://shiftr.io/try or https://shiftr.io/shiftr-io/try ... you can publish to this one (and subscribe)
You can get your own account and watch your own messages being sent and received without the clutter of other messages.
New contributor
Oh....I see. So that means I need to create another client application to read subscription data from MQTT server, and save content to a .raw file?
– John
1 hour ago
1
exactly correct ..... the MQTT server is just a long pipe for your data
– jsotola
1 hour ago
data can arrive out of sequence though ..... updated answer
– jsotola
1 hour ago
add a comment |
You are correct about the microphone input, void my_record()
samples the microphone output level 1000 times, appending each reading to a string variable and publishes the resulting string to an MQTT broker.
This process repeats 11 times every time my_record() is called.
note: You are sort of misunderstanding about the .RAW file. It is a raw file, meaning that it is unprocessed and unformatted .... just a stream of bytes. Using the term .RAW implies a file name extension.
What command allows MQTT server to generate the .RAW file?
The MQTT broker (server) does not generate the raw file, the file is published to the MQTT broker by an outside source, the ESP8266 in this instance.
So how does the MQTT server know which published transmission belongs to which RAW file?
It does not know. All it does is to relay messages. It is up to the publisher
to send to the correct topic
and it is up to the subscriber
to watch the data at the correct topic.
The messages could arrive to the subscriber out of sequence, so a sequence number needs to be included with the message if a correct data sequence is desired.
Have a look at these for a visual demo of MQTT messages.
https://shiftr.io/shiftr-io/demo ... you can only subscribe to this one
https://shiftr.io/try or https://shiftr.io/shiftr-io/try ... you can publish to this one (and subscribe)
You can get your own account and watch your own messages being sent and received without the clutter of other messages.
New contributor
You are correct about the microphone input, void my_record()
samples the microphone output level 1000 times, appending each reading to a string variable and publishes the resulting string to an MQTT broker.
This process repeats 11 times every time my_record() is called.
note: You are sort of misunderstanding about the .RAW file. It is a raw file, meaning that it is unprocessed and unformatted .... just a stream of bytes. Using the term .RAW implies a file name extension.
What command allows MQTT server to generate the .RAW file?
The MQTT broker (server) does not generate the raw file, the file is published to the MQTT broker by an outside source, the ESP8266 in this instance.
So how does the MQTT server know which published transmission belongs to which RAW file?
It does not know. All it does is to relay messages. It is up to the publisher
to send to the correct topic
and it is up to the subscriber
to watch the data at the correct topic.
The messages could arrive to the subscriber out of sequence, so a sequence number needs to be included with the message if a correct data sequence is desired.
Have a look at these for a visual demo of MQTT messages.
https://shiftr.io/shiftr-io/demo ... you can only subscribe to this one
https://shiftr.io/try or https://shiftr.io/shiftr-io/try ... you can publish to this one (and subscribe)
You can get your own account and watch your own messages being sent and received without the clutter of other messages.
New contributor
edited 1 hour ago
New contributor
answered 1 hour ago
jsotolajsotola
1366
1366
New contributor
New contributor
Oh....I see. So that means I need to create another client application to read subscription data from MQTT server, and save content to a .raw file?
– John
1 hour ago
1
exactly correct ..... the MQTT server is just a long pipe for your data
– jsotola
1 hour ago
data can arrive out of sequence though ..... updated answer
– jsotola
1 hour ago
add a comment |
Oh....I see. So that means I need to create another client application to read subscription data from MQTT server, and save content to a .raw file?
– John
1 hour ago
1
exactly correct ..... the MQTT server is just a long pipe for your data
– jsotola
1 hour ago
data can arrive out of sequence though ..... updated answer
– jsotola
1 hour ago
Oh....I see. So that means I need to create another client application to read subscription data from MQTT server, and save content to a .raw file?
– John
1 hour ago
Oh....I see. So that means I need to create another client application to read subscription data from MQTT server, and save content to a .raw file?
– John
1 hour ago
1
1
exactly correct ..... the MQTT server is just a long pipe for your data
– jsotola
1 hour ago
exactly correct ..... the MQTT server is just a long pipe for your data
– jsotola
1 hour ago
data can arrive out of sequence though ..... updated answer
– jsotola
1 hour ago
data can arrive out of sequence though ..... updated answer
– jsotola
1 hour ago
add a comment |
John is a new contributor. Be nice, and check out our Code of Conduct.
John is a new contributor. Be nice, and check out our Code of Conduct.
John is a new contributor. Be nice, and check out our Code of Conduct.
John is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Internet of Things Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fiot.stackexchange.com%2fquestions%2f3838%2fhow-does-the-mqtt-server-output-a-single-raw-file-from-multiple-client-publicati%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown