Viewing a single comment thread. View all comments

resented_ape t1_j3kiocc wrote

Have you looked at other related papers? In particular, hypergradient descent and papers that cite that work (which you can find via google scholar)?

7

LahmacunBear OP t1_j3l5ub2 wrote

Oh damn, that paper almost does exactly what I do. Huh. Oh well. Slightly different implementation though. I in contrast, use both grads from the same timestep and have an accumulated Ct.

5

SatoshiNotMe t1_j3lhi10 wrote

Are either of these open source and easily usable as a PyTorch optimizer ?

1

LahmacunBear OP t1_j3mrexi wrote

Mine’s in Tensorflow 2.11 — I’m sure writing a PyTorch version wouldn’t be hard. The extra lines of the algorithm are three lines in my paper. I can share my code though?

1

SatoshiNotMe t1_j3n5p3v wrote

Thanks! I was just curious for future reference. I’ll need to first read the papers to see if it can help with my projects.

2