I don’t have much experience with that specific problem but I would tend to think it’s hard to generalize like this to “models that hit the bottom” without knowing what the validation loss actually looked like and what that new data looks like. Chances are, this data is not just perfectly sampled from the first dataset and the features have some idiosyncratic/new statistical properties. In that case, by feeding them in some way to your pre-trained model, the model loss is mechanically not in that minima it supposedly reached in the first training run anymore.

Kinda random but if you think the size of the input really matters for the model to learn well (which frankly I’m not convinced is an issue) you could add one or two hidden layers of decreasing sizes behind the large input size layers, before you concatenate them with the smaller size ones.

Never realized AWS was subsidizing killing retail. If we all switch to GCP we can have corner stores again?

Why avatars would be so important?

I was just told that toddlers are attracted to faces with high contrast too? Maybe related if true

Very nice!