This is my preferred interpretation of RESNETs too.

You messed up the text in box 12, it's a duplicate of box 11.

The reason Google doesn't bother is that they are aggressive about acquisitions. They're outsourcing the difficult risky work.

Learned index functions are similar to compression algorithms and might be of interest here, but I think I agree with your argument anyway because they're very overparameterized.

What is there in this genre other than learned indices and surrogate modeling for physics problems?

Linear algebra can be skipped IMO.

Deep neural networks are a specific type of statistical model that is basically made up of repeatedly stapled together basic statistical models. This makes statistics the broadest field.

Essentially all machine learning methods come from probability, statistics, and signal processing.

Machine learning contains other methods than deep neural networks, but they are not as popular and I wouldn't worry about them at first.

Understand linear regression, probability and statistics first, then try to understand neural networks. Focus on statistical modeling, not null hypothesis significance testing.

Focal loss is typical for class imbalanced data.

My brain prefers solutions that modify the training objective to resampling based solutions on the grounds of elegance. I'm not sure if that's a good attitude to have or not, if anyone has any thoughts.

Some of us are in it for the betterment of mankind.

With a background in OR and fluid dynamics, once you get going you should check out Kidger's work on Neural Differential Equations.

Useful in what are now operator learning contexts maybe.

I guess you have no choice but to add support for PDEs and rebrand to TorchNDE.

Monte Carlo Dropout during the forward pass can be used for variance estimation.

Kingma's Reparameterization Trick.

Minsky on how single layer Perceptrons can't solve xor, and whoever did the follow-up on how MLPs can.

Wolpert on No Free Lunch in search and optimization.

I think it's fine the way it is now. ML models are very statistical physics-y. Variation from run to run is extremely low.

There's some work in forgery detection, but real artists are mostly angry about it and skeptical.

It's easier to break down the subjective experience of a fish into mechanical subcomponents than it is to do so for higher intelligences.

Does category theory continue to be as insanely mind-blowing once you actually understand some of it?

To operationalize the question a bit and hopefully make it more interesting, let's consider whether 2032 will have AI models that are equally as conscious as fish, in whatever sense fish might be said to have consciousness.

Interpretability, causal ML, cost of training, out-of-distribution detection.

Also inherits every other problem that can plague statistical modeling.

Musk is too popular for liking him to provide high quality diagnostic information about a person.

I want more comments like this.

On the other hand, bioethics is filled with people who understand philosophy but not the subject material, and has a very counterproductive do-nothing bias.

Method of adjoints.

http://www.argmin.net/2016/05/18/mates-of-costate/

Sensor fusion