Saliency Models as Gamut-Mapping Artifact Detectors
CC Attribution - NoDerivatives 2.0 UK: England & Wales:
You are free to use, copy, distribute and transmit the work or content in unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Cao, Guanqun
Pedersen, Marius
Baranczuk, Zofia
When an image is reproduced with a device different artificats can occur. These artifacts, if dectectable by observers, will reduce the quality of the image. If these artifacts occur in salient regions (regions of interest) or or if the artifacts introduce salient regions regions they contribute to reduce the quality of the reproduction. In this paper we CGIV 2010 and MCS’10 Final Program and Proceedings xxv propose a novel method for the detection of artifacts based on saliency models. The method is evaluated against a set of gamut mapped images containing the artifacts, which have have been marked by a group of group experts. The results have shown that the proposed metrics are promising to detect the artifacts through the reproduction.
 Good morning everyone, I'm Guan Chun-sao from the Eurison Wundus program, CMED program. This is a course project that I did last time in the Norwegian color lab. It has been carried out with Maru's Patterson and Sofia Bernschuk. So the title is, Sailing Sea Models as Government Mapping Artifact Detectives. The outline of this presentation is as follows. First I give a short introduction of artifacts on the Sailing Sea map. And then I will discuss about our proposed experimental framework. After we obtain the results, I will talk about the experiments and the evaluation. And finally I conclude. So it has been years that we are trying to have an accurate color reproduction. This goal has not been achieved due to many reasons. Artifacts are one of the main factors affecting image quality in color reproduction. Here I give an illustration of the Government Mapping Artifacts. The original image is on the left and the Government Mapping image is in the middle. And you can see that the artifacts marked by one of our color experts lies around the contour of the blob. And how can we do with it? One of the solution is to have an inspection. That's very time consuming. So for the traditional objective matrix such as MSC and PSNI, they are not sufficient. Because there is a lack of correspondence to the human visual system. So the idea behind this project is that the artifacts modify the saliency images. Just imagine if there is an artifact occurring in the image. If it does, it will capture our attention. So the aim of this paper is to propose a novel objective approach to detect the Government Mapping Artifacts based on visual saliency. So I'm just, uh, some words about artifacts. A previous study has shown that artifacts strongly influence Government Mapping images. And the hard book at all also has found that lots of shadow details is one of the main, one of the important criteria of image quality. Also contouring effect is common in Government Mapping images. So our focus is to find lots of shadow details and contouring effects. Also some, a few words about saliency map. Visual saliency is a very hot topic nowadays. And according to ETH, they define saliency as perceptual quality that makes an object, person, or pixel region stand out relative to its neighbors. And that captures our attention. So we adopt a very simple but effective model. It outperforms, it is this recent model outperforms a rest in that it is able to efficiently output full resolution saliency maps. It establishes well defined boundaries of salient objects. It rescars high frequency arising from texture, noise, and the blocking artifacts. So a short summary of this adopted model. Firstly, we transform the original image into the slab space and then apply a different Gaussian filter on it. So how do we apply that? After we obtain the original image in slab space, we calculate the mean of each channel. And then also for the original image, for the same image, we apply the Gaussian blur on it. And that derives the difference between each channels and combine them to form the final saliency map. Here we can see that the one below that the yellow flower is darkly captures our attention. And the adopted method is correctly indicates the same region. So how do we use it? Firstly, as I said before, when there is an artifact occur, it will capture our attention. So there will be a saliency difference on it. So what we did is that for the original image and the gamut mapped image, we calculate the saliency maps on both of them. And then derive the difference between them. So here are some results. For the ego image, we can see that it correctly captures the ego's body, which is mainly the loss of shadow details occur. But for the second one, there is a problem. Because the heart of the flower strongly captures our attention, even though there is a small difference between the original one and the gamut mapped one in the heart of the flower. But it indicates that that region is mainly where the artifacts occur. So we found that this map that is not working so well, so we decided to go deeper to detect the local regions. That's how we carried it out. Firstly, for the original image, we apply a mean shift clustering on it so that we can separate the image into several parts, and according to the lightness. And because of the shadow details and the loss of shadow details and the contouring effect, mainly occur in the low lightness region. So we have more focus on those regions. So for the same region, we calculate the saliency map on it onto the segmented regions and also, finally, we calculate the absolute difference on it. So, and then we can show that where the artifacts are. And after we obtain the results, oh, sorry, and there is a mathematical representation of the previous one. So for the original image in slap space, we segmented by the mean shift clustering. And then calculate the saliency map onto both local original image and that one of the gamemaps image, and then derive the difference. So after we obtain the results and we are trying to evaluate it. So we perform the psychovisual test by inviting 12 expert observers to mark the artifacts. The experimental setup follows the CIE guidelines for the gamemap mapping. And we prepare 20 gamemap image for the psychovisual test. So here is a screenshot of the software. The observers are instructed to mark where, according to his opinion, where the artifacts occur. And the software automatically generates a mask, including the artifacts in the observers' opinion. So for the 20 observers, we obtain 20 masks for each image. And then we are trying to calculate the 2D coefficients between the proposed approach and the meanmax from the observers. So how do we calculate the meanmax? We collect all the masks from the observers and then divide by the 20. And in order to see if there is an agreement between the observers to the artifacts. So we also calculate the 2D coefficients between the meanmax and each observers' mask. And also, that's the main part, to evaluate our proposed mask, proposed method. So we also calculate the correlation coefficients between the meanmax and the proposed mask. And for comparison, we also have, we use SSM and also calculate the correlation coefficients. So here are the results. So for the first image, we can see that in the shadow regions around the baby's face, there is quite many loss of details there. And our proposed method has a very high agreement with the observers' opinion. And for the second image, there's a problem. It seems that the observers think the artifacts are occurring on the food region. But our model also captures the low-loss regions around the corner. But for the rest of the image, we can see that also there are some, there are very high correlation between the proposed method and the observers' opinion. And also for the one that I have shown before, the ego image and the flower. And we can see that our proposed method works quite well. So here are the results of the 20 images. Please note that if there is a high variation between the observers' opinion on the artifacts, then we do not expect that our proposed method is going to work well. So we can see that for most of the image, the proposed method is able to indicate where the artifacts occur. So now I conclude. We have proposed a novel method based on saliency for the detection of gamut mapping artifacts. And experiment with 12 expert observers who are carried out. And the regions marked by the observers were compared against the proposed method. And the results are promising and show that our proposed method can be used to detect gamut mapping artifacts. Thank you. Thank you very much. I think we have time for, we will take the time for one quick question before going to cover break. It's the first one over there. No? Okay. Do you differentiate between creating an artifact and contributing and having an artifact and then making it less or more visible? Because sometimes you have already marked, please mark the artifacts on the right-hand side that assumes you might have artifacts in the original already. Do you differentiate between those? No. We mainly focus on the artifacts by the gamut mapping, not what we deliberately, I would say, to generate the artifacts. Thank you. Thank you very much.
 Good morning everyone, I'm Guan Chun-sao from the Eurison Wundus program, CMED program.  This is a course project that I did last time in the Norwegian color lab.  It has been carried out with Maru's Patterson and Sofia Bernschuk.  So the title is, Sailing Sea Models as Government Mapping Artifact Detectives.  The outline of this presentation is as follows.  First I give a short introduction of artifacts on the Sailing Sea map.  And then I will discuss about our proposed experimental framework.  After we obtain the results, I will talk about the experiments and the evaluation.  And finally I conclude.  So it has been years that we are trying to have an accurate color reproduction.  This goal has not been achieved due to many reasons.  Artifacts are one of the main factors affecting image quality in color reproduction.  Here I give an illustration of the Government Mapping Artifacts.  The original image is on the left and the Government Mapping image is in the middle.  And you can see that the artifacts marked by one of our color experts lies around the contour of the blob.  And how can we do with it?  One of the solution is to have an inspection.  That's very time consuming.  So for the traditional objective matrix such as MSC and PSNI, they are not sufficient.  Because there is a lack of correspondence to the human visual system.  So the idea behind this project is that the artifacts modify the saliency images.  Just imagine if there is an artifact occurring in the image.  If it does, it will capture our attention.  So the aim of this paper is to propose a novel objective approach to detect the Government Mapping Artifacts based on visual saliency.  So I'm just, uh, some words about artifacts.  A previous study has shown that artifacts strongly influence Government Mapping images.  And the hard book at all also has found that lots of shadow details is one of the main, one of the important criteria of image quality.  Also contouring effect is common in Government Mapping images.  So our focus is to find lots of shadow details and contouring effects.  Also some, a few words about saliency map.  Visual saliency is a very hot topic nowadays.  And according to ETH, they define saliency as perceptual quality that makes an object, person, or pixel region stand out relative to its neighbors.  And that captures our attention.  So we adopt a very simple but effective model.  It outperforms, it is this recent model outperforms a rest in that it is able to efficiently output full resolution saliency maps.  It establishes well defined boundaries of salient objects.  It rescars high frequency arising from texture, noise, and the blocking artifacts.  So a short summary of this adopted model.  Firstly, we transform the original image into the slab space and then apply a different Gaussian filter on it.  So how do we apply that?  After we obtain the original image in slab space, we calculate the mean of each channel.  And then also for the original image, for the same image, we apply the Gaussian blur on it.  And that derives the difference between each channels and combine them to form the final saliency map.  Here we can see that the one below that the yellow flower is darkly captures our attention.  And the adopted method is correctly indicates the same region.  So how do we use it?  Firstly, as I said before, when there is an artifact occur, it will capture our attention.  So there will be a saliency difference on it.  So what we did is that for the original image and the gamut mapped image, we calculate the saliency maps on both of them.  And then derive the difference between them.  So here are some results.  For the ego image, we can see that it correctly captures the ego's body, which is mainly the loss of shadow details occur.  But for the second one, there is a problem.  Because the heart of the flower strongly captures our attention, even though there is a small difference between the original one and the gamut mapped one in the heart of the flower.  But it indicates that that region is mainly where the artifacts occur.  So we found that this map that is not working so well, so we decided to go deeper to detect the local regions.  That's how we carried it out.  Firstly, for the original image, we apply a mean shift clustering on it so that we can separate the image into several parts, and according to the lightness.  And because of the shadow details and the loss of shadow details and the contouring effect, mainly occur in the low lightness region.  So we have more focus on those regions.  So for the same region, we calculate the saliency map on it onto the segmented regions and also, finally, we calculate the absolute difference on it.  So, and then we can show that where the artifacts are.  And after we obtain the results, oh, sorry, and there is a mathematical representation of the previous one.  So for the original image in slap space, we segmented by the mean shift clustering.  And then calculate the saliency map onto both local original image and that one of the gamemaps image, and then derive the difference.  So after we obtain the results and we are trying to evaluate it.  So we perform the psychovisual test by inviting 12 expert observers to mark the artifacts.  The experimental setup follows the CIE guidelines for the gamemap mapping.  And we prepare 20 gamemap image for the psychovisual test.  So here is a screenshot of the software.  The observers are instructed to mark where, according to his opinion, where the artifacts occur.  And the software automatically generates a mask, including the artifacts in the observers' opinion.  So for the 20 observers, we obtain 20 masks for each image.  And then we are trying to calculate the 2D coefficients between the proposed approach and the meanmax from the observers.  So how do we calculate the meanmax?  We collect all the masks from the observers and then divide by the 20.  And in order to see if there is an agreement between the observers to the artifacts.  So we also calculate the 2D coefficients between the meanmax and each observers' mask.  And also, that's the main part, to evaluate our proposed mask, proposed method.  So we also calculate the correlation coefficients between the meanmax and the proposed mask.  And for comparison, we also have, we use SSM and also calculate the correlation coefficients.  So here are the results.  So for the first image, we can see that in the shadow regions around the baby's face, there is quite many loss of details there.  And our proposed method has a very high agreement with the observers' opinion.  And for the second image, there's a problem.  It seems that the observers think the artifacts are occurring on the food region.  But our model also captures the low-loss regions around the corner.  But for the rest of the image, we can see that also there are some, there are very high correlation between the proposed method and the observers' opinion.  And also for the one that I have shown before, the ego image and the flower.  And we can see that our proposed method works quite well.  So here are the results of the 20 images.  Please note that if there is a high variation between the observers' opinion on the artifacts, then we do not expect that our proposed method is going to work well.  So we can see that for most of the image, the proposed method is able to indicate where the artifacts occur.  So now I conclude.  We have proposed a novel method based on saliency for the detection of gamut mapping artifacts.  And experiment with 12 expert observers who are carried out.  And the regions marked by the observers were compared against the proposed method.  And the results are promising and show that our proposed method can be used to detect gamut mapping artifacts.  Thank you.  Thank you very much. I think we have time for, we will take the time for one quick question before going to cover break.  It's the first one over there.  No? Okay.  Do you differentiate between creating an artifact and contributing and having an artifact and then making it less or more visible?  Because sometimes you have already marked, please mark the artifacts on the right-hand side that assumes you might have artifacts in the original already.  Do you differentiate between those?  No.  We mainly focus on the artifacts by the gamut mapping, not what we deliberately, I would say, to generate the artifacts.  Thank you.  Thank you very much.
Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_0_59.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_60_707.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_708_1307.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_1308_2459.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_2460_2723.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_2724_3107.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_3108_3503.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_3504_3731.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_3732_4559.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_4560_5783.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_5784_7427.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_7428_8183.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_8184_9455.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_9456_10775.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_10776_11603.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_11604_12323.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_12324_13127.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_13128_13175.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_13176_14903.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_14904_16595.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_16596_16979.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_16980_17879.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_17880_18527.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_18528_18623.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_18624_20279.png Saliency_Models_as_Gamut-Mapping_Artifact_Detectors_20280_20308.png