Latent variables with thresholds
$begingroup$
There are many ML techniques to estimate latent variables such as the EM algorithm. Is there a technique that allows for thresholds for each of the latent variables?
I have a feature space with 10 variables $(X_1,dots,X_{10})$ and the outcome $Y$. 7 of the $X$ features are known (I have their observations) and 3 are unknown. Each of the unknown can be within a range from 0 up to a positive constant number.
What ML technique would you recommend for estimating the above latent variables with the setup described above?
machine-learning estimators
edited 17 mins ago
Stephen Rauch♦
1,53551330
asked Jun 28 '18 at 13:45
mrtmrt
11
$endgroup$
add a comment |
$begingroup$
There are many ML techniques to estimate latent variables such as the EM algorithm. Is there a technique that allows for thresholds for each of the latent variables?
I have a feature space with 10 variables $(X_1,dots,X_{10})$ and the outcome $Y$. 7 of the $X$ features are known (I have their observations) and 3 are unknown. Each of the unknown can be within a range from 0 up to a positive constant number.
What ML technique would you recommend for estimating the above latent variables with the setup described above?
machine-learning estimators
edited 17 mins ago
Stephen Rauch♦
1,53551330
asked Jun 28 '18 at 13:45
mrtmrt
11
$endgroup$
$begingroup$
Please clarify "allows for thresholds". For example do you mean the latent variables are discrete values lying on a continuum or that they are disjoint categories. For the latter you'd usually encode as "one-hot" for the former you could relax to continuous range.
$endgroup$
– jayprich
Jun 28 '18 at 14:02
$begingroup$
My miss. I mean that a variable can take any real value between 0 and 10 for instance. Or between -3 and 20.
$endgroup$
– mrt
Jun 28 '18 at 15:04
add a comment |
$begingroup$
There are many ML techniques to estimate latent variables such as the EM algorithm. Is there a technique that allows for thresholds for each of the latent variables?
I have a feature space with 10 variables $(X_1,dots,X_{10})$ and the outcome $Y$. 7 of the $X$ features are known (I have their observations) and 3 are unknown. Each of the unknown can be within a range from 0 up to a positive constant number.
What ML technique would you recommend for estimating the above latent variables with the setup described above?
machine-learning estimators
edited 17 mins ago
Stephen Rauch♦
1,53551330
asked Jun 28 '18 at 13:45
mrtmrt
11
$endgroup$
There are many ML techniques to estimate latent variables such as the EM algorithm. Is there a technique that allows for thresholds for each of the latent variables?
I have a feature space with 10 variables $(X_1,dots,X_{10})$ and the outcome $Y$. 7 of the $X$ features are known (I have their observations) and 3 are unknown. Each of the unknown can be within a range from 0 up to a positive constant number.
What ML technique would you recommend for estimating the above latent variables with the setup described above?
machine-learning estimators
machine-learning estimators
edited 17 mins ago
Stephen Rauch♦
1,53551330
asked Jun 28 '18 at 13:45
mrtmrt
11
edited 17 mins ago
Stephen Rauch♦
1,53551330
asked Jun 28 '18 at 13:45
mrtmrt
11
edited 17 mins ago
Stephen Rauch♦
1,53551330
edited 17 mins ago
Stephen Rauch♦
1,53551330
edited 17 mins ago
Stephen Rauch♦
1,53551330
1,53551330
asked Jun 28 '18 at 13:45
mrtmrt
11
asked Jun 28 '18 at 13:45
mrtmrt
11
asked Jun 28 '18 at 13:45
mrtmrt
11
11
$begingroup$
Please clarify "allows for thresholds". For example do you mean the latent variables are discrete values lying on a continuum or that they are disjoint categories. For the latter you'd usually encode as "one-hot" for the former you could relax to continuous range.
$endgroup$
– jayprich
Jun 28 '18 at 14:02
$begingroup$
My miss. I mean that a variable can take any real value between 0 and 10 for instance. Or between -3 and 20.
$endgroup$
– mrt
Jun 28 '18 at 15:04
add a comment |
$begingroup$
Please clarify "allows for thresholds". For example do you mean the latent variables are discrete values lying on a continuum or that they are disjoint categories. For the latter you'd usually encode as "one-hot" for the former you could relax to continuous range.
$endgroup$
– jayprich
Jun 28 '18 at 14:02
$begingroup$
My miss. I mean that a variable can take any real value between 0 and 10 for instance. Or between -3 and 20.
$endgroup$
– mrt
Jun 28 '18 at 15:04
$begingroup$
Please clarify "allows for thresholds". For example do you mean the latent variables are discrete values lying on a continuum or that they are disjoint categories. For the latter you'd usually encode as "one-hot" for the former you could relax to continuous range.
$endgroup$
– jayprich
Jun 28 '18 at 14:02
$begingroup$
Please clarify "allows for thresholds". For example do you mean the latent variables are discrete values lying on a continuum or that they are disjoint categories. For the latter you'd usually encode as "one-hot" for the former you could relax to continuous range.
$endgroup$
– jayprich
Jun 28 '18 at 14:02
$begingroup$
My miss. I mean that a variable can take any real value between 0 and 10 for instance. Or between -3 and 20.
$endgroup$
– mrt
Jun 28 '18 at 15:04
$begingroup$
My miss. I mean that a variable can take any real value between 0 and 10 for instance. Or between -3 and 20.
$endgroup$
– mrt
Jun 28 '18 at 15:04
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
re. "estimate latent variables"
Quantities that are trained in order to fit a "best" model within a family of models are called hyper-parameters. To any instance of the model they are fixed. To the optimisation routine they are an index into search space. Adding constraints on the range of a hyper-parameter both reduces the search space of the optimisation and requires extra "feasibility" checks during typical gradient descent.
A variable is "latent" when it is purely internal to the model, i.e. not an observable. The meaning of its scale would depend on the context and on your interpretation, since it cannot be compared to anything observed. You rarely want to constrain that range inside the model.
I would suggest leaving the hyper-parameters and latent variables unconstrained and if you want to read an output train a "neuron"-like response to get what you want out : e.g. sigmoid / tanh / softmax
answered Jun 28 '18 at 17:24
jayprichjayprich
1262
$endgroup$
$begingroup$
Do you see this as an application to NNs? My (limited) experience with NNs tells me that the latent variables and the parameters will be lost in the complexity of the network with the multiple layers and neurons.
$endgroup$
– mrt
Jul 2 '18 at 15:01
$begingroup$
The learned connection strengths, biases and what have you are the "parameter" .. the conventional train/predict scenario they do not change once found. Online learning that adapts to new data clearly blurs that line. A "latent variable" in a NN is any state information that's not an input or an output .. it changes all the time in response to neighbours etc .. nothing is "lost" it's just not clear to me how you think the word "latent" applies to your stated scenario .. seems you're really talking about constraining an input or an output .. but perhaps I misunderstand
$endgroup$
– jayprich
Jul 3 '18 at 16:40
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f33769%2flatent-variables-with-thresholds%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
re. "estimate latent variables"
Quantities that are trained in order to fit a "best" model within a family of models are called hyper-parameters. To any instance of the model they are fixed. To the optimisation routine they are an index into search space. Adding constraints on the range of a hyper-parameter both reduces the search space of the optimisation and requires extra "feasibility" checks during typical gradient descent.
A variable is "latent" when it is purely internal to the model, i.e. not an observable. The meaning of its scale would depend on the context and on your interpretation, since it cannot be compared to anything observed. You rarely want to constrain that range inside the model.
I would suggest leaving the hyper-parameters and latent variables unconstrained and if you want to read an output train a "neuron"-like response to get what you want out : e.g. sigmoid / tanh / softmax
answered Jun 28 '18 at 17:24
jayprichjayprich
1262
$endgroup$
$begingroup$
Do you see this as an application to NNs? My (limited) experience with NNs tells me that the latent variables and the parameters will be lost in the complexity of the network with the multiple layers and neurons.
$endgroup$
– mrt
Jul 2 '18 at 15:01
$begingroup$
The learned connection strengths, biases and what have you are the "parameter" .. the conventional train/predict scenario they do not change once found. Online learning that adapts to new data clearly blurs that line. A "latent variable" in a NN is any state information that's not an input or an output .. it changes all the time in response to neighbours etc .. nothing is "lost" it's just not clear to me how you think the word "latent" applies to your stated scenario .. seems you're really talking about constraining an input or an output .. but perhaps I misunderstand
$endgroup$
– jayprich
Jul 3 '18 at 16:40
add a comment |
$begingroup$
re. "estimate latent variables"
Quantities that are trained in order to fit a "best" model within a family of models are called hyper-parameters. To any instance of the model they are fixed. To the optimisation routine they are an index into search space. Adding constraints on the range of a hyper-parameter both reduces the search space of the optimisation and requires extra "feasibility" checks during typical gradient descent.
A variable is "latent" when it is purely internal to the model, i.e. not an observable. The meaning of its scale would depend on the context and on your interpretation, since it cannot be compared to anything observed. You rarely want to constrain that range inside the model.
I would suggest leaving the hyper-parameters and latent variables unconstrained and if you want to read an output train a "neuron"-like response to get what you want out : e.g. sigmoid / tanh / softmax
answered Jun 28 '18 at 17:24
jayprichjayprich
1262
$endgroup$
$begingroup$
Do you see this as an application to NNs? My (limited) experience with NNs tells me that the latent variables and the parameters will be lost in the complexity of the network with the multiple layers and neurons.
$endgroup$
– mrt
Jul 2 '18 at 15:01
$begingroup$
The learned connection strengths, biases and what have you are the "parameter" .. the conventional train/predict scenario they do not change once found. Online learning that adapts to new data clearly blurs that line. A "latent variable" in a NN is any state information that's not an input or an output .. it changes all the time in response to neighbours etc .. nothing is "lost" it's just not clear to me how you think the word "latent" applies to your stated scenario .. seems you're really talking about constraining an input or an output .. but perhaps I misunderstand
$endgroup$
– jayprich
Jul 3 '18 at 16:40
add a comment |
$begingroup$
re. "estimate latent variables"
Quantities that are trained in order to fit a "best" model within a family of models are called hyper-parameters. To any instance of the model they are fixed. To the optimisation routine they are an index into search space. Adding constraints on the range of a hyper-parameter both reduces the search space of the optimisation and requires extra "feasibility" checks during typical gradient descent.
A variable is "latent" when it is purely internal to the model, i.e. not an observable. The meaning of its scale would depend on the context and on your interpretation, since it cannot be compared to anything observed. You rarely want to constrain that range inside the model.
I would suggest leaving the hyper-parameters and latent variables unconstrained and if you want to read an output train a "neuron"-like response to get what you want out : e.g. sigmoid / tanh / softmax
answered Jun 28 '18 at 17:24
jayprichjayprich
1262
$endgroup$
re. "estimate latent variables"
Quantities that are trained in order to fit a "best" model within a family of models are called hyper-parameters. To any instance of the model they are fixed. To the optimisation routine they are an index into search space. Adding constraints on the range of a hyper-parameter both reduces the search space of the optimisation and requires extra "feasibility" checks during typical gradient descent.
A variable is "latent" when it is purely internal to the model, i.e. not an observable. The meaning of its scale would depend on the context and on your interpretation, since it cannot be compared to anything observed. You rarely want to constrain that range inside the model.
I would suggest leaving the hyper-parameters and latent variables unconstrained and if you want to read an output train a "neuron"-like response to get what you want out : e.g. sigmoid / tanh / softmax
answered Jun 28 '18 at 17:24
jayprichjayprich
1262
answered Jun 28 '18 at 17:24
jayprichjayprich
1262
answered Jun 28 '18 at 17:24
jayprichjayprich
1262
answered Jun 28 '18 at 17:24
jayprichjayprich
1262
1262
$begingroup$
Do you see this as an application to NNs? My (limited) experience with NNs tells me that the latent variables and the parameters will be lost in the complexity of the network with the multiple layers and neurons.
$endgroup$
– mrt
Jul 2 '18 at 15:01
$begingroup$
The learned connection strengths, biases and what have you are the "parameter" .. the conventional train/predict scenario they do not change once found. Online learning that adapts to new data clearly blurs that line. A "latent variable" in a NN is any state information that's not an input or an output .. it changes all the time in response to neighbours etc .. nothing is "lost" it's just not clear to me how you think the word "latent" applies to your stated scenario .. seems you're really talking about constraining an input or an output .. but perhaps I misunderstand
$endgroup$
– jayprich
Jul 3 '18 at 16:40
add a comment |
$begingroup$
Do you see this as an application to NNs? My (limited) experience with NNs tells me that the latent variables and the parameters will be lost in the complexity of the network with the multiple layers and neurons.
$endgroup$
– mrt
Jul 2 '18 at 15:01
$begingroup$
The learned connection strengths, biases and what have you are the "parameter" .. the conventional train/predict scenario they do not change once found. Online learning that adapts to new data clearly blurs that line. A "latent variable" in a NN is any state information that's not an input or an output .. it changes all the time in response to neighbours etc .. nothing is "lost" it's just not clear to me how you think the word "latent" applies to your stated scenario .. seems you're really talking about constraining an input or an output .. but perhaps I misunderstand
$endgroup$
– jayprich
Jul 3 '18 at 16:40
$begingroup$
Do you see this as an application to NNs? My (limited) experience with NNs tells me that the latent variables and the parameters will be lost in the complexity of the network with the multiple layers and neurons.
$endgroup$
– mrt
Jul 2 '18 at 15:01
$begingroup$
Do you see this as an application to NNs? My (limited) experience with NNs tells me that the latent variables and the parameters will be lost in the complexity of the network with the multiple layers and neurons.
$endgroup$
– mrt
Jul 2 '18 at 15:01
$begingroup$
The learned connection strengths, biases and what have you are the "parameter" .. the conventional train/predict scenario they do not change once found. Online learning that adapts to new data clearly blurs that line. A "latent variable" in a NN is any state information that's not an input or an output .. it changes all the time in response to neighbours etc .. nothing is "lost" it's just not clear to me how you think the word "latent" applies to your stated scenario .. seems you're really talking about constraining an input or an output .. but perhaps I misunderstand
$endgroup$
– jayprich
Jul 3 '18 at 16:40
$begingroup$
The learned connection strengths, biases and what have you are the "parameter" .. the conventional train/predict scenario they do not change once found. Online learning that adapts to new data clearly blurs that line. A "latent variable" in a NN is any state information that's not an input or an output .. it changes all the time in response to neighbours etc .. nothing is "lost" it's just not clear to me how you think the word "latent" applies to your stated scenario .. seems you're really talking about constraining an input or an output .. but perhaps I misunderstand
$endgroup$
– jayprich
Jul 3 '18 at 16:40
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f33769%2flatent-variables-with-thresholds%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Please clarify "allows for thresholds". For example do you mean the latent variables are discrete values lying on a continuum or that they are disjoint categories. For the latter you'd usually encode as "one-hot" for the former you could relax to continuous range.
$endgroup$
– jayprich
Jun 28 '18 at 14:02
$begingroup$
My miss. I mean that a variable can take any real value between 0 and 10 for instance. Or between -3 and 20.
$endgroup$
– mrt
Jun 28 '18 at 15:04