While defining fixed weights defeats the purpose of 'learning', you may add some filters and set them non-trainable if you have domain experience. In fact, transfer learning can be seen as fixing primary/coarse filters and learning finer ones.