The first things that came to mind are things that just weren't taught much in school - data quality/relevance/sourcing, designing proxies for business metrics, how good is good enough, privacy of training data, etc.

The deviations from theory that have come to mind:

• In theory, deep learning learns its own feature representation so that sounds like the best path. In practice, if the whole system is a black box it's very hard to debug and may run afoul of regulation, so the dream of "it just learns everything from raw inputs" isn't always the answer
• Overparameterize and regularize sounds like a great strategy, but then it can take way longer to train and may limit the number of things you can try before any deadlines
• I haven't had as much success with deep networks as wide networks