I can’t say for certain without the code but it looks like the loss is being applied to every hidden unit (as a scalar) rather than being distributed based off of each units contribution to the loss (as a vector). Check the shape of your loss as it moves through each layer?

Edit: also, are you applying the total loss or the mean loss? It should be the latter.