Submitted by atomsinmove t3_10jhn38 in singularity
red75prime t1_j5kx5ha wrote
Reply to comment by No_Ninja3309_NoNoYes in Steelmanning AI pessimists. by atomsinmove
Backpropagation is a tool that takes care of servicemen not getting the orders. There's the vanishing gradient problem affecting deep networks, but RELUs and residual connections seem to take care of it just fine. Mitigation of the problem in recurrent networks is harder though.
As for the brain... The brain architecture most likely is not the one and only architecture suitable for general intelligence. And taking into account that researchers get similar results when scaling up different architectures, there are quite a few of them.
Viewing a single comment thread. View all comments