Reference request: initializing big neural networks with small neural networks [on hold]
$begingroup$
I am currently trying some meta-algorithms on training neural networks. Start with a small but expressive enough network for training and after several epochs, initialize a larger neural network with the trained weight in the small neural network. It seems to me that we can gain marginal improvement in the optimization speed in terms of time.
I believe there are already lots of paper in this direction but can find any paper related possibly because I am using a wrong key word. I wonder where I can find relevant papers, thanks in advance.
machine-learning optimization
asked yesterday
Lind AxiaoLind Axiao
1
New contributor
Lind Axiao is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
put on hold as too broad by Stephen Rauch, Sean Owen♦ yesterday
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |
$begingroup$
I am currently trying some meta-algorithms on training neural networks. Start with a small but expressive enough network for training and after several epochs, initialize a larger neural network with the trained weight in the small neural network. It seems to me that we can gain marginal improvement in the optimization speed in terms of time.
I believe there are already lots of paper in this direction but can find any paper related possibly because I am using a wrong key word. I wonder where I can find relevant papers, thanks in advance.
machine-learning optimization
asked yesterday
Lind AxiaoLind Axiao
1
New contributor
Lind Axiao is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
put on hold as too broad by Stephen Rauch, Sean Owen♦ yesterday
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |
$begingroup$
I am currently trying some meta-algorithms on training neural networks. Start with a small but expressive enough network for training and after several epochs, initialize a larger neural network with the trained weight in the small neural network. It seems to me that we can gain marginal improvement in the optimization speed in terms of time.
I believe there are already lots of paper in this direction but can find any paper related possibly because I am using a wrong key word. I wonder where I can find relevant papers, thanks in advance.
machine-learning optimization
asked yesterday
Lind AxiaoLind Axiao
1
New contributor
Lind Axiao is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I am currently trying some meta-algorithms on training neural networks. Start with a small but expressive enough network for training and after several epochs, initialize a larger neural network with the trained weight in the small neural network. It seems to me that we can gain marginal improvement in the optimization speed in terms of time.
I believe there are already lots of paper in this direction but can find any paper related possibly because I am using a wrong key word. I wonder where I can find relevant papers, thanks in advance.
machine-learning optimization
machine-learning optimization
asked yesterday
Lind AxiaoLind Axiao
1
New contributor
Lind Axiao is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked yesterday
Lind AxiaoLind Axiao
1
New contributor
Lind Axiao is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked yesterday
Lind AxiaoLind Axiao
1
New contributor
Lind Axiao is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked yesterday
Lind AxiaoLind Axiao
1
asked yesterday
Lind AxiaoLind Axiao
1
1
New contributor
Lind Axiao is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Lind Axiao is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Lind Axiao is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
put on hold as too broad by Stephen Rauch, Sean Owen♦ yesterday
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
put on hold as too broad by Stephen Rauch, Sean Owen♦ yesterday
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |
add a comment |
0
active
oldest
votes
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
