A system for removing a noise artifact from an image of a document extracts a first set of features from the image, where the first set of features represents items on the image. The system identifies noise artifact features from the first set of features representing pixel values of the noise artifact. The system generates a second set of features by removing the noise artifact features from the first set of features. The system generates a test clean image of the document based on the second set of features as an input. The system determines whether a portion of the test clean image that previously displayed the noise artifact corresponds to a counterpart portion of the training clean image. If it is determined that the portion of the test clean image corresponds to the counterpart portion of the training clean image, the system outputs the test clean image.