Submitted by cthorrez t3_xsq40j in MachineLearning
DataMonkey2 t1_iqmmmrz wrote
For machine learning, is anyone else seeing the tide change. At first it was a step(useless for optimising obv), then sigmoid/logistic in the 80s, then tanh going into the 00s, which was generally replaced by Relu. Whilst there's a lot more leaky relu now, is anyone seeing more significant uptake of newer activations like mish?
Viewing a single comment thread. View all comments