In my deep NN with 3 layer, . In the second iteration of GD, The activation of Layer 1 and Layer 2 output all 0 due to ReLU as all the input are smaller than 0. And L3 output some value with high floating point which is opposite to first forward_ propagation . Is this how it should work ? Submitted by Emotional-Fox-4285 t3_yoauod on November 7, 2022 at 2:59 AM in deeplearning 12 comments 0
Intelligent-Aioli-43 t1_ivepfnl wrote on November 7, 2022 at 12:53 PM Could you try Leaky ReLu? Permalink 1
Viewing a single comment thread. View all comments