PleaseKillMeNowOkay
PleaseKillMeNowOkay OP t1_iqu4rs2 wrote
Reply to comment by sydjashim in Neural network that models a probability distribution by PleaseKillMeNowOkay
Same initialization but not the exact weights. However, I've run the experiments enough times with the same result for me to be sure that the initial weights aren't an issue.
PleaseKillMeNowOkay OP t1_iqscxo9 wrote
Reply to comment by SimulatedAnnealing in Neural network that models a probability distribution by PleaseKillMeNowOkay
The simpler model had lower training loss with the same number of epochs. I tried training the second model until it had the same training loss as the first model, which took much longer. The validation did not improve and had a slight upward trend, which I know means that it's overfitting.
PleaseKillMeNowOkay OP t1_iqqz3lp wrote
Reply to comment by thebear96 in Neural network that models a probability distribution by PleaseKillMeNowOkay
I could add more linear layers and based on my experiments it would probably help but my intention is to compare my new model with the old one for which I presume the architecture should be as close as possible.
PleaseKillMeNowOkay OP t1_iqqxw6h wrote
Reply to comment by thebear96 in Neural network that models a probability distribution by PleaseKillMeNowOkay
I wouldn't call it a bigger network necessarily. The second network has two more output neurons compared to the first. Rest are the same. How much difference that makes. Idk
PleaseKillMeNowOkay OP t1_iqqxd7o wrote
Reply to comment by thebear96 in Neural network that models a probability distribution by PleaseKillMeNowOkay
Yes, I trained until the validation loss stopped improving, and then some more just to make sure.
PleaseKillMeNowOkay OP t1_iqqwpem wrote
Reply to comment by thebear96 in Neural network that models a probability distribution by PleaseKillMeNowOkay
That's what I thought but I haven't been able to get the second model to even match the performance of the first one. I tried regularization methods without much success.
PleaseKillMeNowOkay OP t1_iqqw34u wrote
Reply to comment by UsernameRelevant in Neural network that models a probability distribution by PleaseKillMeNowOkay
I did. The second model performed worse. I didn't think that was possible.
Submitted by PleaseKillMeNowOkay t3_xtadfd in deeplearning
PleaseKillMeNowOkay OP t1_iqufvaa wrote
Reply to comment by sydjashim in Neural network that models a probability distribution by PleaseKillMeNowOkay
This seems interesting. I'll give this a shot. Thanks!