Not really direct answer to your question, but the general problem you are trying to solve is called image anomaly detection, there are well know approaches that try to solve this problem, some of them in an unsupervised manner (assuming that you have a significant dataset of images without anomalies - learn inlier look for outliers) - check https://towardsdatascience.com/an-effective-approach-for-image-anomaly-detection-7b1d08a9935b for some ideas and pointers (sry if pointing obvious things)