# SImProv: scalable image provenance framework for robust content attribution

Alexander Black Tu Bui Simon Jenni Zhifei Zhang Viswanathan Swaminanthan John
Collomosse CVSSP, University of Surrey, UK  Adobe Research

###### Abstract

We present SImProv – a scalable image provenance framework to match a query
image back to a trusted database of originals and identify possible
manipulations on the query. SImProv consists of three stages: a scalable
search stage for retrieving top-k most similar images; a re-ranking and near-
duplicated detection stage for identifying the original among the candidates;
and finally a manipulation detection and visualization stage for localizing
regions within the query that may have been manipulated to differ from the
original. SImProv is robust to benign image transformations that commonly
occur during online redistribution, such as artifacts due to noise and
recompression degradation, as well as out-of-place transformations due to
image padding, warping, and changes in size and shape. Robustness towards out-
of-place transformations is achieved via the end-to-end training of a
differentiable warping module within the comparator architecture. We
demonstrate effective retrieval and manipulation detection over a dataset of
100 million images.

††journal: Computer Vision and Image Understanding

##  1 Introduction

Images are a great way to share stories and spread information. However,
images can be easily manipulated to tell altered or even completely false
stories. As both the number of images shared online each day and the ease of
image manipulation grow, the need for tools to provide content provenance
information rises. This is addressed in the recently introduced C2PA standards
(Coalition for Content Provenance and Authenticity, 2021) which specifies how
provenance information can be encapsulated as meta-data alongside the image
content. If an image follows the C2PA standards, users can extract the entire
edit story via its secondary stream meta-data.

This paper addresses a common scenario where meta-data is striped from an
image during its online redistribution. It contributes a technique for
robustly matching a query (without meta-data) to an original from a trusted
database (with full meta-data), followed by an intuitive visualization of the
image regions that have been manipulated to differ from the original.

Robust image matching poses many challenges. Images spread online are often
subject to benign transformations such as changes to quality, resolution,
aspect ratio, format _etc_. Additionally, we aim to match images that have
been manipulated for editorial reasons that alter or falsify their stories (we
also call this editorial changes, as opposed to benign changes). We note that
cryptographic (bit-level) hashes cannot be relied for matching, nor can simple
pixel difference operations be used to visualize changes due solely to
manipulation. We propose SImProv - a robust and scalable content provenance
framework that compliments C2PA. SImProv has two technical contributions:

Robust Near-Duplicate Image Search. We learn a visual search embedding that is
robust to both benign transformations and content manipulations. We train a
convolutional neural network (CNN) using a contrastive learning approach. We
use a dataset of original photographs modified in Adobe PhotoshopTM, combined
with data augmentations simulating benign image modifications. This yields a
search embedding for robustly matching a near-duplicate query image
circulating ‘in the wild’ to a trusted database of original images (hereon, we
use the term ‘near-duplicate’ to refer to images that undergo certain
transformations regardless of such transformations being benign or editorial
changes).

An earlier version of SImProv was proposed at the CVPR workshop on Media
Forensics 2021 (Black et al., 2021). The proposed method improves upon this
using instance-level feature pooling methods to improve near-duplicate image
search. We show that incorporating these into our image fingerprinting
descriptor improves performance scalability, using a corpus of up to 100
million diverse photographic and artistic images from Behance.Net. These
adaptations demonstrate the utility of our approach for web-scale content
authenticity applications.

Pairwise Image Comparison. We propose a novel CNN architecture for pairwise
image comparison that learns a joint image pair representation. We use this
architecture to train two models for near-duplicated detection and editorial
change localization respectively. In the near-duplicated detection model, the
pair representation is used in conjunction with the individual visual search
embeddings of both images to decide whether the two input images are two
versions of the same image or completely unrelated distinct images. In the
editorial change visualization model, the pair representation is used to
produce a heatmap that localizes visual discrepancies due to editorial
manipulation. The network incorporates both a de-warping and image correlation
module, and is trained end-to-end to ignore out-of-place transformation of
content e.g. due to padding or warping as well as in-place corruption due to
noise. In this extension of the earlier proposed pair-wise approach (Black et
al., 2021) we show that fusing end-to-end features from the image embedding
together with the pair-wise embedding model improves the performance of the
near-duplicate detection and re-ranking. These tasks were previously trained
and applied as two sequential, entirely decoupled processes.

##  2 Related Work

The issue of visual content authenticity has been extensively studied from two
main perspectives: detection and attribution.

Detection typically involves identifying instances of visual tampering or
generative content – ‘deep fakes’ (Dolhansky et al., 2020). This usually
requires ”blind” detection, where the image in question is the only available
information. Different statistical approaches have been explored to localize
manipulated regions (Wang et al., 2019, 2010). Other methods can identify
whether a generative adversarial networks (GANs) was used to create the
content (Wang et al., 2020; Nguyen et al., 2022) or even specify which GAN,
using GAN fingerprints (Yu et al., 2019). Most frequently, these methods focus
on detection of fake faces in particular (Guo et al., 2021; Rössler et al.,
2019).

Image Attribution methods aim to link image to data on its provenance, using
embedded metadata ((2020), CAI; Aythora et al., 2020), watermarking (K.
Hameed, 2006; Devi et al., 2019; Profrock et al., 2006; Baba et al., 2019), or
perceptual hashing (Pan, 2019; Liu et al., 2016; Cao et al., 2017; Khelifi and
Bouridane, 2017). Emerging standards securely transport a cryptographically
signed edit history within image metadata ((2020), CAI; Aythora et al., 2020;
Coalition for Content Provenance and Authenticity, 2021). However, social
media platforms often strip metadata from uploaded images, and may even
replace it with false information to misattribute the image (Council, 2020).

CSQ (Yuan et al., 2020) approaches hashing as a retrieval/attribution
optimization task. Deep Supervised Hashing (DSH)(Liu et al., 2016) and
HashNet(Cao et al., 2017) train a siamese convolutional neural network to
learn visual hashes. They use a ranking loss, which is commonly used in visual
search (Gordo et al., 2016). DSDH (Li et al., 2017) learns metric ranking and
classification directly from the hash code. Our approach uses deep metric
learning as well, but differs in that we use contrastive training (Chen et
al., 2020) and data augmentation to learn invariances relevant to benign and
editorial image transformation. Compared to the image similarity detection
challenge and dataset (Douze et al., 2021), which focuses on large-scale
retrieval of images subjected to benign transformations, our approach
addresses the more complex problem of detecting editorial changes.

Localization of image manipulation focuses on identifying image splicing
(M.Huh et al., 2018) or the use of photo-retouching tools (Wang et al., 2019)
in blind detection tasks. We tackle the problem by combining perceptual
hashing and pair-wise comparison. Our image comparator, the second
contribution of this paper, assumes that a trusted ”original” image can be
found using visual search (the first contribution of this paper). The
comparator is able to ignore differences caused by benign image
transformations but is sensitive to editorial manipulations. This is achieved
through the use of a differential optical flow (Teed and Deng, 2020) and
dewarping module in our two-stream architecture. Two-stream networks have
previously been used to predict the types of editing operations applied to
pairs of images (Jenni and Favaro, 2018). Our approach is different because it
produces a heatmap of editing operations that is insensitive to specific
transformation classes. Another feature of our method is a classification
score that can be used at inference to determine whether an image is a benign
or manipulated version, or a different image altogether.

---

()

()

Fig. 1: Architecture diagrams of each of second and third stages of the
proposed SImProv framework. Stage 2 (left) performs re-ranking of the top-100
retrieved results, utilizing a pairwise embedding $z$ to re-order the results
and identify a correct match to the query. Stage 3 (right) uses the Pairwise
Embedding Network (PEN) to identify whether the query image has been
manipulated and localizes the manipulation with a heatmap.

##  3 Method

Our approach for image provenance assumes the existence of a trusted database
$\mathcal{D}=\\{I_{1},I_{2},...,I_{N}\\}$ containing N original images and
their associated provenance information (_e.g_. curated by a trusted
publisher, or via a decentralized immutable data-store such as a blockchain).
Given a query image $q$, our goals are: (i) determining whether there exists
an original version of $q$ in $\mathcal{D}$; and (ii) localizing editorial
changes if a match is found. The two goals appear to conflict each other since
the former requires robustness to both benign and editorial changes while the
latter should be sensitive to editorial manipulations. Learning a single model
to achieve our goals is therefore extremely challenging. We instead propose a
multi-stage framework. Our SImProv consists of 3 stages: (i) a visual search
stage followed by (ii) re-ranking and near-duplicated detection, and finally
(iii) detection and visualization of editorial changes (Fig. 1).

Firstly, in LABEL:method:search we describe the representation learning
process, used for near-duplicate image search (stage 1). We develop a model
that learns 256-D representations of images that are further binarized into a
128-bit hash for scalable search (Johnson et al., 2019). The search is used to
identify the most similar images to a users’ query image.

Secondly, 3.2 describes the Pairwise Embedding Network (PEN) to deliver a
pairwise representation of the query and a candidate image. PEN is our core
design for the later stages of SImProv. In 3.3, PEN is integrated to our
stage-2 Pairwise Similarity Evaluation Network (PSEN) to re-rank the top k
images (k=100) and identify the likelihood of candidate image being a near-
duplicated version of the query, as opposed to being just another distinct
image.

Finally, 3.4 describes how PEN is leveraged to identify whether the query
image is a manipulated or benignly transformed version of the original (stage
3). If the query is identified as manipulated, we visualize a heatmap of the
manipulated region on top of the image.

###  3.1 Near-Duplicate Image Search

We train a CNN model $f_{r}(.)$ to encode an image $I$ into a compact
embedding space $r=f_{r}(I)\in\mathbb{R}^{256}$. Additionally, we perform
KMeans quantization with 1024 clusters, followed by product quantization,
resulting in a 128-bit descriptor of the 256-dimensional embedding (Johnson et
al., 2019). We employ the ResNet50 (He et al., 2016) backbone for $f_{r}$,
replacing the final layer with a 256-D fully connected (fc) layer as the
embedding. We use DeepAugMix (Hendrycks et al., 2020) as the pretrained
weight. We finetune with a multiple-positives contrastive loss as described in
(Black et al., 2021).

During inference, we find our model benefits from geometric pooling (GeM)
(Radenović et al., 2018) in two ways. Firstly, model becomes resolution
agnostic and can take larger images as input, capturing more information.
Secondly, it allows to focus on local features, which is more beneficial for
matching out of place transformed images.

For a set of $K$ spatial feature map activations
$\Phi=[\phi_{1},\ldots\phi_{K}]$, the GeM (Radenović et al., 2018) pooling
operation $G$ is defined as $G(\Phi)=[g_{1},\ldots g_{k},\ldots g_{K}]$;
$g_{k}=(\frac{1}{|\phi_{k}|}\sum_{x\in\phi_{k}}x^{p_{k}})^{\frac{1}{p_{k}}}$,
where $p_{k}$ is a hyper-parameter ($p_{k}$ is fixed at a default value of $3$
in our experiments).

###  3.2 Pairwise Embedding Network

Fig. 2: Architecture of the proposed Pairwise Embedding Network (PEN). A
candidate match to the user queried image is obtained from near-duplicate
search (LABEL:method:search, not shown). Image alignment is performed via
differentiable de-warping unit (DWU) based on a dense optical flow estimate
provided by the flow estimator. The resulting image pair are separately
encoded via a feature extractor $f_{E}(.)$ and the concatenated features
passed through $f_{S}(.)$ to obtain the combined feature $z$.

We propose an Pairwise Embedding Network (PEN) that learns a joint
representation of two input images (Fig. 2). This architecture is later
utilized for two purposes: near-duplicated detection (3.3.1) and localization
of editorial change (3.4), which corresponds to stage 2 and 3 of SImProv
respectively.

The PEN accepts a pair of query-candidate images as input and outputs a
$n$-dimensional ($n=256$ in our experiments) representation of the image pair.
The PEN architecture consists of 2 modules: a geometrical alignment module,
$\mathcal{F}_{A}$, followed by a projection module, $\mathcal{F}_{P}$ (Fig.
2). Below we describe our designs for $\mathcal{F}_{A}$ and $\mathcal{F}_{P}$.

Geometric Alignment Module is used to account for the fact that the query $q$
may undergo through geometric transformations which alter the pixel placement.
We correct its alignment prior to joint representation learning. In this work
we use RAFT (Teed and Deng, 2020), but any flow estimation network that can be
trained end-to-end is suitable.

Our DWU then uses the predicted optical flow to align the query image with the
candidate image:

$\displaystyle M\;:\;(x,y)$
$\displaystyle\mapsto(x+\rho^{x}(x),y+\rho^{y}(y))$ (1)
$\displaystyle\textrm{DWU}\,(q|\rho^{x},\rho^{y})$
$\displaystyle=\mathcal{S}(M)\in\mathbb{R}^{H\times W}$ (2)

where $(x,y)$ refers to the pixel coordinates in the query image $q$ which are
mapped to their corresponding coordinates $M$ in the candidate image according
to the optical flow ${\rho^{x},\rho^{y}}$. $\mathcal{S}(.)$ is a bilinear
sampler that effectively fits a local grid around $M$:
$\mathcal{S}(M)={M+\Delta M|\Delta M\in\mathbb{R}^{2},\left|\Delta
M\right|<=1}$ where output coordinates are computed through linear
interpolation.

Projection Module takes the candidate $I$ and the aligned query,
$q^{\prime}=\mathcal{F}_{A}(q|I)$ and outputs a single feature $z$. We first
extract local features of each image using a shared CNN module:
$z_{q}=f_{E}(q^{\prime});\;z_{I}=f_{E}(I)\in\mathbb{R}^{H^{\prime}\times
W^{\prime}\times C}$, where $H^{\prime}$, $W^{\prime}$ and $C$ are the new
height, width and feature dimension respectively. Our feature extractor
$f_{E}(.)$ is a 3-layer convolutional neural network (CNN) with ReLU
activations, batch normalization, and max pooling layers. It outputs features
at $\frac{1}{4}$ resolution ($H^{\prime}=H/4,W^{\prime}=W/4$ and we set
$C=128$). The combined features are then fed into another CNN to learn a
fusion representation $z=f_{S}([z_{q},z_{I}])\in\mathbb{R}^{256}$, where $[,]$
is concatenation, and $f_{S}(.)$ is made up of 4 ResNet residual blocks (He et
al., 2016) followed by average pooling and a fully-connected (FC) layer that
outputs 256-dimensional features. The PEN output is used for re-ranking, near-
duplicate detection, and manipulation localization, as described below.

###  3.3 Near-duplicate detection and Re-ranking

Our near-duplicate image search method is designed to produce compact
descriptors to enable interactive speeds in search through millions of images.
However, the increase in speed comes at a cost of precision. The correct image
could end up near the top of retrieval results, but in many cases might not in
the first place. We propose a re-ranking model based on pairwise comparison of
the query image with each of the top-k retrieval candidates. Such pairwise
comparison is much slower and is not feasible for search through millions of
images, but allows to identify the most likely match within a shortlist. The
re-ranking consists of two steps: pairwise similarity evaluation and final
reordering. Similarity evaluation produces a similarity confidence score for
each of the 100 query-candidate image pairs. Re-ordering looks at the full
list of 100 confidence scores and decides which of the candidate images is the
most likely match to the query. On a single GTX 1080 Ti GPU initial search
takes 40ms and reordering 400ms on average. Below we describe similarity
evaluation (3.3.1) and reordering (3.3.2).

####  3.3.1 Image Similarity Evaluation

We propose a Pairwise Similarity Evaluation Network (PSEN) that uses two
images as input: the query image $q$ and a candidate image $c$, retrieved by
the near-duplicate search model (LABEL:method:search). The PSEN uses the
previously obtained individual embeddings of the images, as well as a PEN
joint embedding learned from stack of two images together (Fig. 1).

The final output of the model is a confidence score $s$, indicating the
likelihood that the query and candidate images are the same image under
different transformations:

$s=S([f_{r}(q),f_{r}(c),PEN([q,c])])\in[0,1]$ (3)

where $[,]$ is concatenation, $f_{r}(.)$ is the search embedding (3.1),
$PEN([.,.])$ is the pair representation (3.2) and $S(.)$ is a binary
classification fully connected layer. The model is trained with binary cross-
entropy loss.

####  3.3.2 Re-ordering

The role of re-orderer is to decide which of the candidate images is the most
likely match to the query, based on two pieces of information: the initial
raking of near-duplicate image search (distance between candidate and query
embeddings) and similarity confidence scores. The index $n$ of the most likely
match is defined as $n=R(s_{0},\ldots,s_{99})\in[0,100]$, where
$s_{i}=S([f_{r}(q),f_{r}(c_{i}),PEN([q,c_{i}])])$ is the similarity confidence
score between the query image and $i$-th retrieved candidate image; $R(.)$ is
a neural network, consisting of three fully connected layers of sizes
$[8192,1024,101]$, respectively. Re-orderer is trained as a 101-way classifier
with cross-entropy loss. First 100 classes correspond to indices of candidate
images and the final class indicates that the correct match to the query is
not present within the candidate images.

###  3.4 Detecting and Localizing Editorial Change

This stage assumes a near-duplicated image to a query has been found after
phase 2 (3.3). In order to predict the benign-manipulated relationship and
visualize the possible manipulated regions, we train a second PEN model using
a combination of two different loss functions (Fig. 1). The first loss is a
binary cross-entropy loss $\mathcal{L}_{C}(.)$ that predicts whether the pair
is benign (_i.e_., the query image $q$ is either identical to or a benign
transformed version of the candidate image $I$) or manipulated (_i.e_., $z$ is
a manipulated version of $I$). The second loss function is used to minimize
the cosine distance $\mathcal{L}_{T}(.)$ between the manipulation heatmap
derived from $z$ and the ground truth heatmap. This heatmap is generated at a
resolution of $t\times t$ using a fully-connected layer
$E_{t}(z)\in\mathbb{R}^{t^{2}}$.

The total loss is
$\mathcal{L}(.)=w_{c}\mathcal{L}_{C}(.)+w_{t}\mathcal{L}_{T}(.)$ where loss
weight $w_{c}=w_{t}=0.5$ is set empirically.

##  4 Experiments and Discussion

###  4.1 Datasets

The PSBattles (Heller et al., 2018) dataset is used to train and evaluate our
models. We use the splits and edit localization annotations from (Black et
al., 2021). The dataset is split into a training set (PSBat-Train) and a test
set (PSBat-Test), with 6,364/21,197 and 807/2,960 original/manipulated images
in the training and test sets, respectively. The PSBat-Train set is used to
train our image retrieval, similarity evaluation, and edit localization
models, while the PSBat-Test set is used for the two benchmarks we evaluate.

Additionally, we also evaluate the retrieval efficacy of SImProv on BAM-100M,
a large scale dataset consisting of 100M artworks from Behance. BAM-100M is
significantly larger and more diverse than ImageNet, since its collection
spans many fields beyond photography, such as paintings, graphic designs,
advertising and graffiti. We note this is the largest experiment in term of
dataset size for image provenance to date. We create two query sets, BAM-Q-Res
and BAM-Q-Aug from 1K images sampled at random from BAM-100M. To make BAM-Q-
Res, we downscale images at random ratio in range 0.1-0.9 (up to 10x
downscaling) with bilinear interpolation keeping aspect ratio. To make BAM-Q-
Aug, we apply the same augmentation strategy as in PSBat-Ret.

###  4.2 Metrics

To evaluate near duplicate search, we measure the ratio of queries that return
the relevant images within the top-$k$ retrieval using the Instance Retrieval
IR@$k$ metric. Formally,
$IR@k=\frac{1}{Q}\sum_{i=1}^{Q}{\sum_{j=1}^{k}{r(q_{i},j)}}$ where Q is number
of queries, relevance function $r(q_{i},j)=1$ if the returned image at rank
$j$ is relevant to the query $q_{i}$ (there is only one such image in PSBat-
Ret), otherwise 0.

We use Average Precision (AP) to measure the accuracy of both classifiers: the
same/different similarity evaluation network and benign/manipulated classifier
branch of the edit localization network.

For the generated heatmap, we compute Intersection over Union (IoU) with the
ground truth by up-sampling the 7x7 heatmap to the image resolution $H\times
W$, converting it to binary with a threshold, and then calculating the
intersection and union. This is expressed as
$\mathrm{IoU}=\frac{1}{Q}\sum_{i=1}^{Q}{\frac{S(U_{i})\cap T_{i}}{S(U_{i})\cup
T_{i}}}$ where $T_{i}$ is the $H\times W$ binary ground truth heatmap, $U_{i}$
is the predicted heatmap after interpolation and thresholding, and $S(U_{i})$
is the set of values in $U_{i}$. We improve the heatmap using the image pair
classification result, setting $S(U_{i})=U_{i}$ if the query is classified as
manipulated, ${0}^{H\times W}$ if benign, and ${1}^{H\times W}$ if distinct.

###  4.3 Evaluating Near-Duplicate Search

We compare our retrieval method (both before and after re-ranking) against 9
baselines. ICN (Black et al., 2021) is our initial workshop version of
SImProv. ImageNet fine. and MSResNet fine. are the finetuned models on PSBat-
Train using our training strategy. All methods produce 128-bit hash code
except pHash (64-bit).

Table 1: Retrieval performance (on 2M images, PSBat-Ret) reported as IR score
at ranks [1,10,100], for query images subjected to benign transforms,
manipulation, or both. Stage 1 refers to nearest-neighbor search only.

Method | Benign | Manip | Manip+Benign | Average
---|---|---|---|---
| IR@1 | IR@10 | IR@100 | IR@1 | IR@10 | IR@100 | IR@1 | IR@10 | IR@100 | IR@1 | IR@10 | IR@100
SImProv (stage 1 + 2) | 0.9746 | 0.9849 | 0.9849 | 0.9170 | 0.9564 | 0.9564 | 0.9142 | 0.9271 | 0.9271 | 0.9353 | 0.9561 | 0.9561
SImProv (stage 1) | 0.9450 | 0.9749 | 0.9849 | 0.8838 | 0.9307 | 0.9564 | 0.8064 | 0.8845 | 0.9271 | 0.8784 | 0.9300 | 0.9561
ICN (Black et al., 2021) (stage 1 + 2) | 0.9423 | 0.9811 | 0.9867 | 0.9154 | 0.9453 | 0.9453 | 0.8705 | 0.9099 | 0.9164 | 0.9094 | 0.9454 | 0.9494
ICN (Black et al., 2021) (stage 1) | 0.9305 | 0.9725 | 0.9867 | 0.8557 | 0.9061 | 0.9453 | 0.7662 | 0.8582 | 0.9164 | 0.8508 | 0.9123 | 0.9494
MSResNet fine. | 0.7532 | 0.7931 | 0.8258 | 0.9199 | 0.9500 | 0.9655 | 0.6754 | 0.7406 | 0.7884 | 0.7828 | 0.8279 | 0.8599
ImageNet fine. | 0.7709 | 0.8988 | 0.9577 | 0.7791 | 0.8689 | 0.9159 | 0.5641 | 0.7348 | 0.8420 | 0.7047 | 0.8342 | 0.9052
DeepAugMix (Hendrycks et al., 2020) | 0.6944 | 0.7743 | 0.8336 | 0.8956 | 0.9385 | 0.9615 | 0.5629 | 0.6725 | 0.7553 | 0.7176 | 0.7951 | 0.8501
CSQ (Yuan et al., 2020) | 0.1390 | 0.1803 | 0.3292 | 0.2095 | 0.2584 | 0.4122 | 0.0628 | 0.0957 | 0.2214 | 0.1371 | 0.1781 | 0.3209
pHash (Zauner, 2010) | 0.3674 | 0.3731 | 0.3768 | 0.3693 | 0.3753 | 0.3764 | 0.1739 | 0.1821 | 0.1857 | 0.3035 | 0.3102 | 0.3130

Tab. 1 compares retrieval performance. The two online hashing methods, CSQ
(Yuan et al., 2020) and HashNet (Cao et al., 2017), are among the worst
performers. CSQ and HashNet struggle to cope with strong ImageNet-C
transformations present during training and test, resulting in lower
performance than the classical pHash. ImageNet (He et al., 2016), MSResNet
(Lenyk and Park, 2021) and DeepAugMix (Hendrycks et al., 2020) perform
strongly on the Manip set but poorly when they undergo benign transformations.
When trained via our contrastive loss (eq. LABEL:eq:simclr), all models gain
with our proposed SImProv (stage 1) achieving 25% improvement on Benign IR@1
and 24% on Manip+Benign versus the pretrained DeepAugMix model. SImProv (stage
1) also outperforms the finetuned ImageNet/MSResNet by a large margin on all
top-k scores and query sets. The improvement of SImProv compared to the one
presented in the workshop paper (ICN (stage 1)) (Black et al., 2021) can be
attributed to geometric pooling, which allows higher resolution input. We
demonstrate significant performance improvement of the proposed re-ranking
method (SImProv (stage 1+2)) compared to the naive re-ranking approach used in
ICN (stage 1+2).

###  4.4 Feature Pooling

Fig. 3: Top-1 performance of GeM features on different input image
resolutions. Dash-lines represent the performance without using feature
pooling.

We show the superiority of GeM features against the traditional output
features from the last FC layer of the retrieval model (Black et al., 2021) as
well as its dependence on input image resolution in Fig. 3. GeM features work
best at 384x384 resolution, outperforming (Black et al., 2021) by 4% on the
challenging Manip.+Benign test set. The presence of benign transformations
hampers GeM performance as the resolution increases, underperforming (Black et
al., 2021) from 512x512 resolution on Benign set and from 640x640 on the
Manip.+Benign set.

Tab. 2 compare GeM with (Black et al., 2021) and a similar feature pooling
method - RMAC (Tolias et al., 2016), at two pooling levels, L=3 and L=4. It
can be seen that the pooling level does not affect much the performance of
both GeM and RMAC. Additionally, RMAC is comparable to GeM, slightly
outperforming GeM on the Benign set at L=3 but underperforming on the Manip.
set at L=4. However, we choose GeM as the proposed method since it is
significantly faster than RMAC. It takes RMAC $28.72$ seconds to perform
$1000$ iterations, while GeM is $\sim 18$ times faster, with just $1.63$ for
the same setup.

Table 2: Top-1 retrieval performance of GeM and RMAC features and output from
the last FC layer on the 384x384 resolution test sets.

| RMAC | GeM | CNN
---|---|---|---
| L=3 | L=4 | L=3 | L=4 |
Benign | 0.9503 | 0.9444 | 0.9450 | 0.9441 | 0.9272
Manip. | 0.8777 | 0.8777 | 0.8838 | 0.8926 | 0.8520
Manip.+Benign | 0.8025 | 0.8027 | 0.8064 | 0.8068 | 0.7570

###  4.5 Evaluating Localization of Editorial Changes

Fig. 4: Heatmap results. Left col.: Original image. Middle col.: Manipulated
image also subjected to benign transformation. Right col.: Heatmap output
(green) ignoring benign transformation and highlighting manipulation (ground
truth in yellow). Table 3: Evaluating heatmap accuracy and intrepretability
for baseline methods. Our proposed SImProv method is compared against
baselines both objectively for accuracy (IoU) and subjectively via users to
determine which exhibits best intrepretability (% method preference).
$\mathcal{F}_{A}$ indicates geometric alignment module applied.

Method | Accuracy (IoU) | Interp. (%)
---|---|---
Ours | 0.565 | 84.6
ResNetConv+Geo. Align. $\mathcal{F}_{A}$ | 0.243 | 5.50
SSD+Geo. Align. $\mathcal{F}_{A}$ | 0.231 | 3.30
MantraNet+Geo. Align. $\mathcal{F}_{A}$ | 0.061 | 2.75

The proposed method is compared with four baselines in terms of localization
performance. The first baseline, Sum of Squared Distances (SSD), calculates
SSD between two images at the pixel level, resizes it to $7\times 7$, and then
resizes it back before thresholding to create continuity in the detected
heatmap. The second baseline, ResNetConv, extracts $7\times 7\times 2048$
features from a pre-trained ImageNet ResNet50 model for both query and
original images. These are averaged across channels to produce a $7\times 7$
heatmap. ErrAnalysis \- inspired from the blind detection technique in (Wang
et al., 2010), we perform JPEG compression on the query image and compare with
itself. MantraNet \- is a supervised blind detection method (Wu et al., 2019)
that detects anomalous regions. Additionally we evaluate baselines with images
passed through our alignment module.

We compare the heatmaps generated by our SImProv with baseline methods.
Heatmaps are produced by upsampling the $7\times 7$ heatmap output of the
SImProv to the size of the image using bicubic interpolation. Heatmaps may be
presented on false-colour scale (_e.g_. jet) in this form, or thresholded to
produce an outline of the predicted manipulated region. In our experiments, we
threshold the normalized heatmaps at 0.35 determined empirically. Tab. 3
(first column) reports the IoU metric between the predicted heatmap and the
ground truth, both with and without the thresholding. Whilst most baselines
are improved through use of our geometric alignment ($\mathcal{F}_{A}$)
process, our SImProv significantly exceeds baseline performances by at least
0.30. Change localization examples for SImProv are shown in Fig. 4.

The effectiveness of heatmap interpretability is compared to baseline methods
using a crowd-sourced study on Amazon Mechanical Turk (MTurk). Participants
are shown an original image and an image that has been altered, along with the
ground truth for the altered image. The altered image is also accompanied by a
grid of heatmaps generated by nine different methods. The 9 methods included
our own, 4 baselines (SSD, MantraNet, ErrAnalysis, and ResNetConv), and 4
warp-corrected baselines that used $\mathcal{F}_{A}$ for geometric alignment.
Each of the 200 tasks was annotated by 5 different participants.

Tab. 3 (final col.) presents the results, which favor our proposed method,
even when the image pair are pre-aligned.

###  4.6 Large Scale Retrieval

We evaluate the scalability of our method by indexing the BAM-100M database.
We compare the $IR@k$ performance of SImProv to its earlier version ICN (Black
et al., 2021). Fig. 6 shows $IR@k$ versus database size curves of SImProv and
ICN on BAM-100M with BAM-Q-Res as the query set. We demonstrates that
SImProv’s performance does not degrade nearly as much as ICN with increase in
database size. For SImProv, the $IR@k$ remains nearly $1.0$ at all image
database sizes, dipping to $0.999$ for the most challenging case of $IR@1$ for
database sizes above 30M. ICN, on the other hand, is much greater affected by
database size, with $IR@1$ dropping from $0.997$ at 1M images to $0.985$ at
100M images. Results for the more challenging query set BAM-Q-Aug are depicted
in Fig. 5. SImProv outperforms ICN by a large margin at early k values, on
both BAM-100M and a subset of 1M images. The performance drop when increasing
the database size from 1M to 100M for SImProv is also lower than ICN. The IR@k
curves converge as k value reaches 100, and saturated performance is achieved
at IR@100 for both methods regardless of database size, which justifies our
design choice of selecting top-100 images for SImProv subsequent stages.

Fig. 5: Retrieval performance comparison of ICN (Black et al., 2021) and
SImProv on a 1M subset and fullset of BAM-100M, using BAM-Q-Aug queries. Fig.
6: Top-k retrieval performance comparison of ICN (Black et al., 2021) and
SImProv versus database size, using BAM-Q-Res queries.

###  4.7 Evaluating Classification

Table 4: SImProv stage 2 - Same / different classifier Test | Average Precision (AP)
---|---
ICN (Black et al., 2021) | Ours
Original | 1.000 | 1.000
Benign | 0.9996 | 1.000
Manip. | 0.9976 | 0.9922
Benign+Manip. | 0.9962 | 0.9909
Distinct | 0.9895 | 0.9973
Table 5: SImProv stage 3 - Benign / Manip classifier Test | Average Precision (AP)
---|---
ICN (Black et al., 2021) | Ours
Original | 1.000 | 1.000
Benign | 0.9635 | 0.9800
Manip. | 0.9726 | 0.9932
Benign+Manip. | 0.8807 | 0.9793

We evaluate the classification performance of two classifiers: same/different
in SImProv stage 2 and benign/manipulated in stage 3. Same/different
classification is an output of PSEN 3.3.2, which classifies a pair of images
as either being two entirely different images, or the same image, potentially
under different transformations. Benign/manipulated classification is an
output of the change localization network 3.4, which assumes that the input
images are not distinct and focuses on classifying whether the differences
between them are benign or editorial. We compare the performance of our
approach with ICN (Black et al., 2021), which has a single 3-way (benign,
manipulated, distinct) classifier. In case of same/different evaluation, we
combine the confidences of ‘benign’ and ‘manipulated’ to count as ‘same’.

We evaluate the performance of the 2-way same/different classification by
comparing each original image in the test set with: itself, benign transformed
version of itself, manipulated version, manipulated as well as benign
transformed and an entirely different image, chosen at random. All of the
cases except the last are expected to be classified as ‘same’ and the last one
as ‘distinct’. Tab. 4 shows the Average Precision (AP) scores achieved for
each case. A non-modified original-original pair is always correctly
classified as the same image by both both methods. Introduction of benign
transformations reduces the accuracy of ICN slightly, but does not affect our
approach. The most challenging case is queries that are both manipulated and
benign transformed, however both methods maintain AP near $0.99$ in all of the
cases.

The bigger difference in performance can be seen in Tab. 5 which shows the AP
scores for benign/manipulated classification. Here, our approach outperforms
ICN by $~{}2\%$ in the cases where the query image is either just benignly
transformed or just manipulated. The difference in performance grows to
$~{}9.9\%$ when the query is both manipulated and benign transformed.

##  5 Conclusion

We presented a Scalable Image Provenance (SImProv) framework for large-scale
retrieval and visual comparison of a pair of images in order to detect and
localize manipulated regions. SImProv enables users to match images
circulating ‘in the wild’ to a trusted database of original images. When a
query image is matched to an original, SImProv generates a heatmap that
highlights areas of manipulation while ignoring benign transformations that
can occur when images are shared online. We introduced two main architecture
changes compared to an earlier version of the work (Black et al., 2021):
incorporation of instance-level feature pooling for image retrieval and
combination of individual and pairwise descriptors for near-duplicate
detection, followed by re-ranking. We show that feature pooling improves
retrieval performance by enabling the use of queries of larger resolution. We
use a large corpus of 100 million diverse images to demonstrate that these
changes improve retrieval performance and make our approach applicable to the
web-scale content authenticity problem.

## References

  * Aythora et al. (2020) Aythora, J., Burke-Agüero, R., Chamayou, A., Clebsch, S., Costa, M., Earnshaw, N., Ellis, L., Eng, P., 2020.  Multi-stakeholder media provenance management to counter synthetic media risks in news publishing, in: Proc. Intl. Broadcasting Convention (IBC).
  * Baba et al. (2019) Baba, S., Krekor, L., Arif, T., Shaaban, Z., 2019\.  Watermarking scheme for copyright protection of digital images.  IJCSNS 9.
  * Black et al. (2021) Black, A., Bui, T., Jin, H., Swaminathan, V., Collomosse, J., 2021\.  Deep image comparator: Learning to visualize editorial change, in: Proc. CVPR WS, IEEE. pp. 972–980.
  * (4) (CAI), C.A.I., 2020.  Setting the Standard for Content Attribution.  Technical Report. Adobe Inc.
  * Cao et al. (2017) Cao, Z., Long, M., Wang, J., Yu, P.S., 2017.  Hashnet: Deep learning to hash by continuation, in: Proc. CVPR, pp. 5608–5617.
  * Chen et al. (2020) Chen, T., Kornblith, S., Norouzi, M., Hinton, G., 2020\.  A simple framework for contrastive learning of visual representations, in: Proc. ICML, pp. 1597–1607.
  * Coalition for Content Provenance and Authenticity (2021) Coalition for Content Provenance and Authenticity, 2021\.  Draft Technical Specification 0.7.  Technical Report. C2PA.
  * Council (2020) Council, I., 2020.  Social media sites photo metadata test results.  http://embeddedmetadata.org/social-media-test-results.php.
  * Devi et al. (2019) Devi, P., Venkatesan, M., Duraiswamy, K., 2019.  A fragile watermarking scheme for image authentication with tamper localization using integer wavelet transform.  J. Computer Science 5, 831–837.
  * Dolhansky et al. (2020) Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., Ferrer, C.C., 2020.  The deepfake detection challenge (DFDC) dataset.  CoRR abs/2006.07397.  arXiv:2006.07397.
  * Douze et al. (2021) Douze, M., Tolias, G., Pizzi, E., Papakipos, Z., Chanussot, L., Radenovic, F., Jenicek, T., Maximov, M., Leal-Taixé, L., Elezi, I., Chum, O., Ferrer, C.C., 2021\.  The 2021 image similarity dataset and challenge.  doi:10.48550/ARXIV.2106.09672.
  * Gordo et al. (2016) Gordo, A., Almazán, J., Revaud, J., Larlus, D., 2016\.  Deep image retrieval: Learning global representations for image search, in: Proc. ECCV, pp. 241--257.
  * Guo et al. (2021) Guo, Z., Yang, G., Chen, J., Sun, X., 2021.  Fake face detection via adaptive manipulation traces extraction network.  Computer Vision and Image Understanding 204, 103170.
  * He et al. (2016) He, K., Zhang, X., Ren, S., Sun, J., 2016.  Deep residual learning for image recognition, in: Proc. CVPR, pp. 770--778.
  * Heller et al. (2018) Heller, S., Rossetto, L., Schuldt, H., 2018.  The PS-Battles Dataset -- an Image Collection for Image Manipulation Detection.  CoRR abs/1804.04866.
  * Hendrycks et al. (2020) Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al., 2020\.  The many faces of robustness: A critical analysis of out-of-distribution generalization.  arXiv preprint arXiv:2006.16241 .
  * Jenni and Favaro (2018) Jenni, S., Favaro, P., 2018\.  Self-supervised feature learning by learning to spot artifacts, in: Proc. CVPR.
  * Johnson et al. (2019) Johnson, J., Douze, M., Jegou, H., 2019.  Billion-scale similarity search with gpus.  IEEE Transactions on Big Data .
  * K. Hameed (2006) K. Hameed, A. Mumtax, S.G., 2006.  Digital image watermarking in the wavelet transform domain.  WASET 13, 86--89.
  * Khelifi and Bouridane (2017) Khelifi, F., Bouridane, A., 2017\.  Perceptual video hashing for content identification and authentication.  IEEE TCSVT 1.
  * Lenyk and Park (2021) Lenyk, Z., Park, J., 2021.  Microsoft vision model resnet-50 combines web-scale data and multi-task learning to achieve state of the art.  https://pypi.org/project/microsoftvision/.
  * Li et al. (2017) Li, Q., Sun, Z., He, R., Tan, T., 2017.  Deep supervised discrete hashing, in: Proc. NeurIPS, pp. 2482--2491.
  * Liu et al. (2016) Liu, H., Wang, R., Shan, S., Chen, X., 2016.  Deep supervised hashing for fast image retrieval, in: Proc. CVPR, pp. 2064--2072.
  * M.Huh et al. (2018) M.Huh, Liu, A., Owens, A., Efros, A., 2018.  Fighting fake news: Image splice detection via learned self-consistency, in: Proc. ECCV.
  * Nguyen et al. (2022) Nguyen, T.T., Nguyen, Q.V.H., Nguyen, D.T., Nguyen, D.T., Huynh-The, T., Nahavandi, S., Nguyen, T.T., Pham, Q.V., Nguyen, C.M., 2022.  Deep learning for deepfakes creation and detection: A survey.  Computer Vision and Image Understanding 223, 103525.
  * Pan (2019) Pan, T., 2019.  Digital-content-based identification: Similarity hashing for content identification in decentralized environments, in: Proc. Blockchain for Science.
  * Profrock et al. (2006) Profrock, D., Schlauweg, M., Muller, E., 2006.  Content-based watermarking by geometric wrapping and feature- based image segmentation, in: Proc. SITIS, pp. 572--581.
  * Radenović et al. (2018) Radenović, F., Tolias, G., Chum, O., 2018.  Fine-tuning cnn image retrieval with no human annotation.  IEEE TPAMI 41, 1655--1668.
  * Rössler et al. (2019) Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nießner, M., 2019\.  FaceForensics++: Learning to detect manipulated facial images, in: International Conference on Computer Vision (ICCV).
  * Teed and Deng (2020) Teed, Z., Deng, J., 2020.  Raft: Recurrent all-pairs field transforms for optical flow, in: Proc. ECCV, Springer. pp. 402--419.
  * Tolias et al. (2016) Tolias, G., Sicre, R., Jégou, H., 2016.  Particular object retrieval with integral max-pooling of cnn activations, in: Proc. ICLR, pp. 1--12.
  * Wang et al. (2019) Wang, S.Y., Wang, O., Owens, A., Zhang, R., Efros, A., 2019.  Detecting photoshopped faces by scripting photoshop, in: Proc. ICCV.
  * Wang et al. (2020) Wang, S.Y., Wang, O., Zhang, R., Owens, A., Efros, A., 2020.  Cnn-generated images are surprisingly easy to spot... for now, in: Proc. CVPR.
  * Wang et al. (2010) Wang, W., Dong, J., Tan, T., 2010.  Tampered region localization of digital color images based on jpeg compression noise, in: Intl. WS Digital Watermarking, Springer. pp. 120--133.
  * Wu et al. (2019) Wu, Y., AbdAlmageed, W., Natarajan, P., 2019.  Mantra-net: Manipulation tracing network for detection and localization of image forgeries with anomalous features, in: Proc. CVPR, pp. 9543--9552.
  * Yu et al. (2019) Yu, N., Davis, L., Fritz, M., 2019.  Attributing fake images to gans: Learning and analyzing gan fingerprints, in: Proc. ICCV.
  * Yuan et al. (2020) Yuan, L., Wang, T., Zhang, X., Tay, F., Jie, Z., Liu, W., Feng, J., 2020.  Central similarity quantization for efficient image and video retrieval, in: Proc. CVPR, pp. 3083--3092.
  * Zauner (2010) Zauner, C., 2010.  Implementation and Benchmarking of Perceptual Image Hash Functions.  Master’s thesis. Upper Austria University of Applied Sciences, Hagenberg.


CAHIER DU LAMSADE _403_ June, 2022Is there an Ethical Operational
Research?Odile Bellenguez, Nadia Brauner, Alexis Tsoukiàs _Laboratoire
d’Analyse et Modélisation_ _de Systèmes pour l’Aide à la Décision_ _UMR 7243_

# Is there an ethical Operational Research?

Odille Bellenguez${\dagger}$, Nadia Brauner${\ddagger}$, Alexis Tsoukiàs∗
${\dagger}$IMT Atlantique, LS2N
${\ddagger}$Université Grenoble Alpes, CNRS, Grenoble INP, G-SCOP
∗CNRS-LAMSADE, PSL, Université Paris Dauphine

###### Abstract

The ethical dimension of Operational Research and Decision Aiding, although
not a new subject, turns to be a concern, both for the large public and the OR
community, because of the wide spread of autonomous artefacts endowed with
decision capacity thanks to the use of models, methods and tools developed
within our field. The paper addresses the question of whether there exists an
“Ethical Operational Research”, identifies the ethical questions which are
specific to our professional community and suggests research topics which,
although independently developed, are relevant for handling such questions.

##  1 Introduction

There is an increasing interest and discussion about “Ethical Operational
Research” and more generally about “Ethical” or “Responsible” Decision Support
(see [57]). The topic is not really a new one: there is a EURO Working Group
on Ethics111See https://www.euro-online.org/websites/ethicsandor/ since 2002
and there is a “Prometheus Oath” written by J.P. Brans, founder of this
Working Group (see [14]). Interested readers can see two excellent surveys in
[15] and [43] of the literature on this topic.

Under such a perspective this contribution just continues an ongoing
discussion already started in the 60s and early 70s (see [2], [19]) and
continued since then (see [27], [28], [29], [37], [60]). The reason for which
these topics turn to be discussed is related to the increasing diffusion of
“autonomous artefacts” with augmented decision capacity. Both the wide public,
but also scientists and policy makers are concerned by the wide spread of
devices and processes which decide or recommend decisions using “algorithms”
or “methods” which are felt to be non-controllable, dangerous, biased, unfair,
inexplicable with unknown long-term impacts ([17]).

We share such concerns with a near discipline, Artificial Intelligence222See
the topics discussed at the conference: https://facctconference.org/ or the
Mechanism Design for Social Good Working Group: https://www.md4sg.com/. See
also the High-Level Expert Group for Trustworthy Artificial Intelligence
recommendation to the European Commission: https://digital-
strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai. and more
precisely with Computational Social Choice. Most of these concerns are not
really new (see the discussion in [57]), but it pays to continue the
discussion as we do with this paper.

The paper is organised as follows: in section 2 we introduce a general
framework for the paper discussion. In section 3 we identify the type of
“ethical questions” that could be of interest for this paper. In section 4 we
discuss the questions which can be raised within our professional practice as
decision analysts (or designers of decision support devices). In section 5 we
briefly detail some research topics which although stand alone are at the same
time useful in order to improve how we handle the topics discussed in section
4. We summarise the discussion in section 6 where we ultimately provide a
reply to the question in the title.

##  2 Setting

In order to frame our discussion we are going to use a precise setting
introduced and discussed in [38], [39] and [55].

We consider a situation where a “client” (an entity implied in a decision
process), asks an advice to a “decision analyst” about how to improve his/her
conduct with respect to that decision process. We also consider situations in
which the “client” does not seek an advice for a precise decision process, but
for a class of decision processes, the advice coming under the form of a
device/system/software which is supposed to elaborate recommendations; the
decision analyst in such cases being the designer of such system.s. Under such
a perspective the decision support activities can be seen as:
\- either the direct interactions between client and analyst in order to
elaborate a recommendation;
\- or the design of an appropriate system which on its turn will compute or
help to compute a recommendation.
Clearly many combinations between these two extreme cases are possible in the
real world.

As already partially mentioned in [56], the use of a formal decision aiding
methodology implies considering three different dimensions.

  1. 1.

An axiomatic dimension, establishing the conditions under which it is possible
to use protocols/algorithms/models in a meaningful and useful way.

  2. 2.

An algorithmic dimension, considering the size of the solution space, the
necessary data (availability, accessibility, storage, quality) as well as the
necessary computing resources.

  3. 3.

A pragmatic dimension related to the conditions under which a decision aiding
process is valid and legitimate (see [34], [35], [39]).

##  3 Ethics for whom?

First of all we need to identify different categories of concerned
individuals. To be more precise: as Operational Researchers or Decision
Analysts we may raise ethical questions for different purposes and under
different perspectives. Not all of them are necessarily of interest for a
scientific investigation.

We maybe raise ethical questions just because we are conscious citizens. These
are the typical questions which all of us some day need to consider, but
generally are related to our own individual values. These cannot be matter of
study, analysis or guidelines and principles of conduct within our discipline
if not respecting the very general values our societies consider relevant. But
there is nothing specific to the fact that we are decision analysts.

Just to be more precise: accepting to provide support and models for military
operations can raise ethical and/or moral questions to any among us, but the
positive or negative reply is a matter of personal choices. Some of us will be
happy to do it, others not and others might be indifferent. We cannot
establish any ethical guideline on how to handle such issues.

Some among us, besides being citizens are also teachers or researchers (in
OR). Independently from our specific discipline and research field, there
exist ethical questions related to our precise role of scientists. Such
ethical questions concern our conduct as teachers (with respect to our
students and colleagues) and researchers (with respect to our near scientific
communities and the science in large).

Such issues are generally handled through deontology charts (how to behave
appropriately with the students, how to conduct experiments, how to write
papers, how to quote the existing literature etc.), or specific debates in
philosophy of sciences (see e.g. [23]), but are not specific to our discipline
and research areas (see e.g. the Practical guide “Integrity and responsibility
in research practises” from the CNRS ethics committee [21], the Singapore
Statement on Research Integrity [59], the OECD Best Practices for Ensuring
Scientific Integrity and Preventing Misconduct [42], The European Code of
Conduct for Research Integrity by the European Academies [1]).

There are instead two areas of ethical concerns which are specific to our
domain and role of Decision Analysts. These are related to our profession
(providing advice to decision makers or designing tools to be used by decision
makers) and to our research in the broad area of Operational Research and
Decision Analysis.

##  4 Professional Ethics

It is interesting to note that large part of the debate and the literature
about the ethical dimension of our discipline and the practice related to it,
originate from discussions about professionalism and deontology in our
profession started at the late 60s. At that time the idea of creating a
professional body of “chartered” operational researchers or decision analysts
(later on called “OR Fellows” by the ORS in UK) ignited a debate which lasted
several decades (and still persists) before reaching any practical conclusion
([3], [4], [20], [33], [44], [51]). This discussion moved beyond the UK and
USA professional bodies (see [46]) and we can try to summarise under a
specific perspective (the one of conducting rationally decision aiding
processes).

As decision analysts we provide support to clients. We are not the only
professionals who provide decision support: lawyers, accountants, physicians,
psychotherapists, engineers, just to give some examples, help their clients to
handle their problems and they do so using some scientific knowledge and
approach, thus distinguishing their profession from just informal intuitive
advice to friends and relatives. There are two topics we need to consider
here:
\- What does make our decision support different from other equally
scientifically based decision support activities? In other terms why decision
analysts are not psychotherapists?
\- Since we nevertheless share some features with other professions, who
already considered the problem of deontology, compliance, unsatisfied clients,
young professional training etc., why our profession should not establish
similar protocols, practices, training modules etc.?

We are not going to pursue further this discussion because it is out of the
scope of this paper, but we can identify the ethical problems to handle within
our profession: under which conditions we can claim that our professional
advice to a client satisfies appropriate ethical standards and who establishes
such standards?

We need to separate two distinct cases raising potentially different types of
“ethical questions”. The first one is the case where analysts directly provide
at a client some advice on how to handle a problem within a decision process,
a typical case being organising the shifts of the personnel at the emergency
department of a hospital or managing a large call for tenders for software
COTS for an IT industry.

The second case consists in designing generic methods, protocols or software
aimed at being used for a precise class of decision problems, possibly
customising such products for and with specific clients. Typical examples here
include, supply management packages, flow-shop scheduling procedures, generic
recommender systems etc.

There are certainly mixed cases as well as cases where specific applications
become generic ones (a well known example being the yield management
procedures originally designed for a specific client and then developed as
stand alone customisable methods; see [52]).

Despite the apparent differences between these two cases we will develop a
unique argumentation based on two aspects:
\- the use of a decision aiding methodology;
\- the unveiling of hidden or implicit hypotheses and assumptions within our
models and/or methods.

###  4.1 Decision Aiding Methodology

We start considering our profession as being characterised by the use of a
formal language, the pretention of using/introducing a rationality model
within the client’s decision process and the use of algorithms (see [56]). As
introduced in [55], a decision aiding process can be defined as the
interactions between a “client” (asking for advice) and an “analyst”
(providing the advice) and can be represented through a set of “deliverables”
such as:
\- a description of the problem situation;
\- a problem formulation;
\- an evaluation model;
\- a final recommendation.

In constructing such deliverables we make choices (as analysts). For instance:
\- in order to represent the likelihood of an event we may adopt a probability
measure (while other measures are possible);
\- in order to compute a majority for a voting procedure we may adopt the
“Borda” rule (but others are possible);
\- in order to model the impact of the combined realisation of some decision
variables we assume this impact being linearly defined with respect to the
variables (but other choices are possible).
Such “technical” choices are most of the time uncontrollable by the client and
we (the analysts) are the only able to measure the consequences and to
guarantee the meaningfulness and usefulness of their use.

We also do further hypotheses which are less technical, but nonetheless
important. For instance:
\- in calculating the economic impact of a given infrastructure we consider a
“territory”, but how this territory is chosen/defined?
\- in designing a supply chain we consider the client’s costs and time
constraints, but are these the only constraints we should take into account?
\- in order to set up a vendor rating procedure should we compare the
suppliers between them or only with respect to quality standards?
Despite such choices being agreed with the client(s), it is unlikely the
client(s) really realise the extension to which modifying any of these
hypotheses can modify the outcome of the decision aiding process. In other
terms, beyond any generic deontological constraint due to the fact that we
have a professional relationship, we need to consider the specific constraints
our condition sets. We can try to summarise these through the following
points.

  * •

_Are we sufficiently critical?_ The fact we work for a client does not mean we
cannot or we should not have a personal and independent perspective about what
the client claims to be the problem to consider. We need to be able to show to
the client aspects of the problem or other problems she does not see. At the
same time we should be ready to modify our perspective and learn from the
client’s claims, values and beliefs.

  * •

_Where does rationality come from?_ This topic is extensively discussed in
[38]. The point to raise in this discussion is that there are several sources
of “rationality”, from external norms and standards to subjective behaviours
and argued beliefs. What we need is understanding which among such different
sources we use with that precise client for that precise decision process we
are involved to. The pretention of introducing one or more dimensions of
rationality to the client’s reasoning for her problem, is what, most of the
times, legitimates our action as analysts. Having a clear idea about where
such dimensions of rationality come from is essential in establishing an
appropriate professional relationship with the client.

  * •

_Can we explain, justify and easily revise or update?_ Most of the times we
deliver both a model and the results of applying a set of methods to the
constructed model. This implies choosing among what our technical knowledge
and skills provide, following what the client claims being her problem.
However, we do many technical choices which not always are “obvious”, at least
for the client and/or the other involved stakeholders. It might not be always
necessary, but we need to ask ourselves: should I be asked, am I able to
explain why we did such a technical choice, to completely justify such a
choice against an appeal to a court and to defend the choice against an
“adversary” analyst? Moreover, since modelling for decision support purposes
is always a learning process, we also need to ask ourselves: how easy is it to
revise and update the model and the methods in case the data change, the
values and the opinions change, the problem setting (and formulation) can
change.

  * •

_Is the result convincing for us, for the clients and for the involved
stakeholders?_ Providing decision support means constructing convincing
arguments for some potential action to undertake. Such convictions concern
three different categories of stakeholders. We first need to convince
ourselves that our advice is sound with respect to our technical knowledge and
our methodology. We then need to convince the client that our advice is
appropriate with respect to the problem the client has, the decision process
for which the advice has been asked: the client needs to feel owner of the
advice received. Finally we need to convince the rest of the stakeholders that
the advice to the client was legitimately designed, that we have been critical
and that the impacts of our advice being adopted have been understood.

###  4.2 Hidden hypotheses

Although in a professional setting we deploy a formal decision aiding
methodology, we are always induced in assuming a number of hypotheses as
granted or given. Some of these can become explicit through an appropriate use
of our methodological knowledge. The fact that using a linear (additive,
separable) utility function in order to aggregate the impacts of decisions
along different attributes implies assuming that such impacts are
commensurable and can be traded among them, is part of our methodological
knowledge which will consistently impede us to use such a method in case this
hypothesis does not hold (i.e. the client does not accept it). However, there
are potential misuses and errors which can occur under certain implicit
hypotheses, for which we are not really trained and prepared to handle. Such
implicit assumptions (often ignored exactly because implicit) need to be
explicitly identified and handled if we want to build a trusted relationship
with our clients or in case we want a trustworthy use of any autonomous
artefact using our models and methods.

  * •

_Cognitive biases._ Decision Analysts are subject to the same cognitive
limitations as other humans. They have personal values, personal cognitive
limits, personal habits, culture and feelings. How much these influence the
way through which the client’s problem and the information provided are
modelled? Other professional bodies impose specific training in order to
handle such questions or establish specific protocols and external assessors
to be used by those clients who may doubt about the analysts’ biases. It is
certainly true that our profession does not consider any specific training or
appealing procedure, although these problems exist and should at least be
discussed with our clients.

  * •

_Exceptional circumstances._ Consider a classic risk management model and a
situation characterised by extreme risks (extremely unlikely to occur events,
but with extremely high impacts in case they actually occur; see [41]). Would
our model still work appropriately? Consider a model of extreme risk theory:
would this still apply for emerging risks management (see [10])? Consider a
supply chain model. Would this work and be robust under any possible
circumstances? And would that model still apply if the supply chain problem
concerns humanitarian logistics (see [53])?
The above examples are just cases where mainstream methods and models have
been proven to be inappropriate when exceptional circumstances occur. This
leads to a general question: given any method or model we suggest using for
advising a client, will this advice still hold if such exceptional
circumstances occur and if not what do we suggest the client to do? Which
raises the question of whether we know the application limits of
generalisation of whatever we suggest as recommendation to our clients.

  * •

_Data_. All our methods and algorithms require data. Not only the ones
provided by the client, but also data about the “territory”, the “landscape”,
the “culture”, the “economical and social context”, the “organisation” where
the decision process for which our help is requested is going to be used. Data
are collected, stored, transferred, transformed, manipulated, along
“pipelines” which are far from being with no impact upon the final outcome.
Moreover, data, although they belong to nobody, can be protected by “rights”,
private or collective, social, economic or cultural. In designing methods and
models we need to consider both the data pipeline quality as well as the
rights protection issue (see [18]).

  * •

_Algorithms_. Most of the times our methods require efficient algorithms. Most
of the times we need to trade-off between efficiency and accuracy or even
optimality. Most of the times we also need to take into account other features
of the algorithms such as manipulability, strategic proofness, security,
robustness to adversarial attacks, black-box effect etc. Most of the times our
clients are not aware of what is the impact of choosing an algorithm instead
of another. Clients are also usually unaware of the software differences when
algorithms are coded and of the computing resources necessary to run them. It
is unlikely our clients will ever be tempted to learn all such topics, but is
our “ethical” obligation to know them and let the client understand which are
the stakes at play when choosing a precise algorithm and a precise software
implementation.

  * •

_Impossibilities_. Not all methods fit to any type of decision problem.
Generally speaking we know that most of the times, given a set of properties
to be satisfied by the outcome of a potential algorithm, these are
inconsistent. In other terms there is no algorithm able to satisfy at the same
time all the desired properties (see [7], [13], [58]). This is not really a
problem in terms of computing solutions, but we need to know which properties
are satisfied by which algorithms and we need to be able to explain that to
our clients. In other terms we need to be able to explain to our clients what
an impossibility theorem means for her problem and which are the different
partial solutions we can adopt (and at which “price” in terms of satisfied
properties).

  * •

_Long term consequences_. When American Airlines started studying yield
management methods in order to manage the ticket pricing (see [52]), nobody
(within the company and in the broad Operational Research community) could
ever imagine the impact these methods will have on the travelling industry and
the travelling habits within our societies. Today potentially any operator
running a travelling business (including trains and buses) uses a yield
management method in order to price dynamically tickets. The whole industry in
this field changed its business model and each single consumer modified its
willingness to pay for a travelling ticket (independently from business or
leisure travelling). We are not going to discuss here whether this had a long
term positive or negative impact, although some may discuss the consequences
on the tourism industry, the house renting industry, the environmental impact
etc. The “ethical” question is that all such impacts have never been discussed
neither within the company nor within the society. Nobody anticipated,
discussed or even questioned the new business model underestimating the impact
of a simple optimal pricing method.

Providing models, methods, tools, aiding to improve decision making has
impacts which can go far beyond the client and the other involved
stakeholders. Most of the ethical questions we rose in the previous paragraphs
are related to a precise decision aiding process, the actors involved and the
immediate impacts. But we need to also consider impacts which will occur on a
long, very long term and for stakeholders, citizens, territories and biomes
who never ever thought that changing the optimal production policy of a
company could change their lives half a century later.

##  5 Ethics in OR research

We have seen that most of the “ethical questions” about our discipline concern
its use in the real life and the way through which we handle the relations
with our clients, the relevant stakeholders and the use of our decision aiding
methodology. The reader may note that many of such questions are related to
topics addressed in our research (mostly indirectly) already since the 70s
such as the axiomatic analysis of voting procedures, the analysis of
behavioural biases or the development of problem structuring methods. The
question we raise at this point is: _“are there specific research topics in
Operational Research related to our ethical questions”_? Being more precise:
probably any research topic in our discipline could be relevant for our
ethical questions, but are there some new or more relevant ones?

  1. 1.

Are we aware? The first class of research questions concerns awareness. In
section 4 we raised several ethical questions concerning the use of models,
methods, algorithms, protocols, etc. The fundamental remark is that none among
such tools (which we use in order to advise our clients in their decision
processes) are “neutral”. Using one instead of another can have short or long
term consequences which are independent from what the client asks or the
situation requires. The question is: are we aware of such consequences? And
the consequent research question is: Do we know how to choose appropriately
our tools? In other terms, do we know what each tool can do, cannot do, the
conditions under which they can be used and provide meaningful results?

Axiomatic characterisations, representation theorems, simulations, are
certainly research fields in our discipline which provide results usable in
order to reply to the above demands. Under such a perspective we may emphasise
that using numerical simulations and experimental settings (testing protocols,
behavioural biases and modelling hypotheses) result in very useful tools
helping to undercover hidden behaviours and tacit assumptions which could be
concealed during the use of any among our tools.

We may also emphasise that the more we use the “open science” paradigm,
sharing data, software codes, results and findings, greatly improves our
capacity to increase awareness about what, when and how works (or does not
work).

  2. 2.

Do we help others becoming aware? Being ourselves (as analysts) aware of what
our tools and our methodology can (and cannot) do is necessary, but not
sufficient. Both our clients and the involved stakeholders (possibly the
society as a whole?) need to develop awareness of what our tools can (and
cannot) do.

This of course raises a far larger topic about how scientists can and should
communicate their findings to a “non-scientific” audience in a way that
increases and strengths awareness and autonomy (see [9] or [16]), but remains
relevant for our discipline. Large part of the general public gets used in
misusing statistics and other quantitative information, in using
inappropriately averages, indexes, protocols and codes, producing totally
meaningless results and conclusions (see [30] or for more fun [8]). Even large
and prestigious institutions, not to talk about public agencies and
governments misuse such information in order to justify regulations and
policies (for a famous example see the incredible diffusion of a meaningless
index such as the Shanghai ranking of the Universities: [11]). Under such a
perspective it pays learning to use simple heuristics facilitating
communicating quantitative information in a meaningful way (see [31]).

At the same time developing general frameworks which allow to unify a field of
models and provide a unique frame within which interpret, explain and justify
methods and protocols helps increasing awareness for any stakeholder (and the
general public) involved in a decision process. The reader can see the impact
of measurement theory, [47], in establishing a rigorous notion of
meaningfulness: [40] [48], or the impact of conjoint measurement theory in
unifying the field of multiple criteria decision analysis, [12].

Last, but not least, assuming a problem driven decision support attitude,
instead of a method driven one, generally allows to improve communication with
the client and enhance awareness about why certain methods will not fit in
that precise problem situation, while others might be more appropriate. The
result is adopting an “horizontal” or “methodological” view of our discipline
and not seeing Operational Research as just a collection of methods (see
[56]).

  3. 3.

Are we critical? A decision aiding process is certainly a set of activities
involving the client (who asks for advice) and the analyst (who provides the
advice). However, we already observed that the choices we do, while conducting
the decision aiding process, have impacts far beyond these two stakeholders.
Moreover, remaining confined within the client’s demand and/or the analyst’s
perception can result in missing other opportunities the decision aiding
process offers.

Most problem structuring methods (see [32], [49]) emphasise that decision
problems are constructed (and not identified) and that solutions critically
depend on how problems are formulated. More recently the problem of
constructing the set of alternatives (which is at the centre of the decision
model) has turned to be a research topic (see [22], [25]).

Under such a perspective, research in the following three areas is extremely
important in order to improve and expand our capacity to interact with the
clients and the other stakeholders developing our critical view of the
decision aiding process:
\- problem structuring methods in general, since they provide a general
framework for supporting the whole decision aiding process ([50]);
\- design theory as a formal tool for developing “out-of-the-box” alternatives
beyond the dominant designs usually suggested as solutions ([5], [36]);
\- preference learning because whatever we use in order to elaborate an advice
is learned through interaction with the clients and/or accessing data and
information ([26]).

  4. 4.

Do we help others becoming critical? Keeping a critical perspective with
respect to the problem situation as it appears to be (or as the client makes
it appearing) is certainly important for our ethical questions. However, it is
not sufficient. The client and the involved stakeholders also need to develop
a critical perspective about what happens both within the decision process
they are involved in and the decision aiding process.

Such a process is very much a matter of “convincing”: first of all ourselves
(we remain within standards of meaningfulness), then our clients (they get
something they feel it helps them) and then the rest of the world (the whole
process was legitimate; see [38], [39]).

Under such a perspective our clients and the other stakeholders need to be
able to reply positively to the question: is this advice going to resist to
any arguing against it, arguing grounded on data, procedures, protocols, and
authority? A formal framework where such problems are discussed is formal
argumentation theory (see [6], [24], [45], [54]).

##  6 Conclusions

Let us summarise the discussion and the claims we introduced. Ethical
questions arise in our everyday life as well as in our professional life
independently from what our profession is. The questions we are interested in
this paper derive from our specific profession as decision analysts. As a side
effect we need to consider which research topics, while independently
developed, can help us in handling such ethical questions.

Under such a perspective we need to remember that decisions (what we are
supposed to help making) are value driven and not data driven: data are
necessary, but not sufficient for making decisions. It is part of our
profession to make understand these values, for us, our clients and the
involved stakeholders. At the same time we need to remember that aiding to
decide is problem driven and not method driven, which means we first need to
understand the problem and then we need to think about solving it.

As we show in the paper, ethical questions have been introduced in our
discipline since the very beginning. Our discussion emphasises two parallel
issues we need to consider when we try to handle such questions. The first
concerns awareness of what methods are, can do, cannot do and how choosing any
among them is not neutral with respect to the solution computed and the
recommendation provided. The second is the development of a critical attitude
about the consequences of our modelling choices which goes beyond the usual
relation analyst/client.

A first point to make is that there are no universal procedures, protocols,
algorithms or methods. Any of them will unfit for some reason and for some
kind of problem situation and for some type of client and we need to know how
to handle this. Keeping a critical attitude with respect to the clients’
demand and to our profession helps on a long run both our clients and our
profession. There are many hidden hypotheses and implicit choices in modelling
and solving a decision problem and these need to be clear to us (as analysts),
to our clients and to the stakeholders who might be involved in the problem
situation.

Is there an ethical Operational Research? If ethics consists in applying to
our profession standards of morality (in whatever way these have been
established) then our reply is negative. But if ethics consists in assuming
our responsibility for what we offer to our clients then yes. Although we may
not be liable for what our clients decide using our advice and/or our tools,
we are responsible for many (avoidable) consequences which can occur. We have
a power and we need to use it with responsibility.

## Acknowledgements

The paper reflects discussions hold within the Special Interest Group on
Ethics in Operational Research of the CNRS funded National Research Network on
Operational Research (GDR-RO, http://gdrro.lip6.fr/). We specially thank the
participants to the workshop hold at the LAMSADE, Université Paris Dauphine
the 26/11/2011, http://gdrro.lip6.fr/?q=node/254. Christine Solnon read an
early version of this document and her comments greatly improved it.

## References

  * [1] All European Academies.  The european code of conduct for research integrity, 2017.  https://allea.org/wp-content/uploads/2017/05/ALLEA-European-Code-of-Conduct-for-Research-Integrity-2017.pdf.
  * [2] R.L. Ackoff.  The social responsibility of operational research.  Operational Research Quarterly, 25:361–371, 1974.
  * [3] R.L Ackoff.  The future of operational research is past.  Journal of Operational Research Society, 30:93–104, 1979.
  * [4] R.L. Ackoff.  Resurrecting the future of operational research.  Journal of the Operational Research Society, 30:189–199, 1979.
  * [5] E.R.. Alexander.  Design in the decision making process.  Policy Sciences, 14:279–292, 1982.
  * [6] L. Amgoud and H. Prade.  Using arguments for making and explaining decisions.  Artificial Intelligence, 173:413–436, 2009.
  * [7] K.J. Arrow.  Social choice and individual values.  J. Wiley, New York, 1951.  2nd edition, 1963.
  * [8] H. Barker.  Lying Numbers: How Maths and Statistics Are Twisted and Abused.  Little, Brown Book Group, London, 2020.
  * [9] M. Bauer, N. Allum, and S. Miller.  What can we learn from 25 years of pus survey research? liberating and expanding the agenda.  Public Understanding of Science, 16:79 – 95, 2007.
  * [10] V.M. Bier.  The Gower handbook of extreme risk : assessment, perception and management of extreme events.  Routledge, London, 2016.
  * [11] J.-Ch. Billaut, D. Bouyssou, and Ph. Vincke.  Should you believe in the shanghai ranking?  Scientometrics, 84:237 – 263, 2010.
  * [12] D. Bouyssou and M. Pirlot.  Conjoint measurement models for preference relations.  In D. Bouyssou, D. Dubois, M. Pirlot, and H. Prade, editors, Decision Making Process, pages 617 – 672. J. Wiley, Chichester, 2009.
  * [13] F. Brandt, V. Conitzer, U. Endriss, J. Lang, and A.D. Procaccia.  Handbook of Computational Social Choice.  Cambridge University Press, Cambridge, 2016.
  * [14] J-P. Brans.  OR ethics and decision: the OATH of PROMETHEUS.  European Journal of Operational Research, 140:191–196, 2002.
  * [15] J-P. Brans and G. Gallo.  Ethics in OR/MS: past, present and future.  Annals of Operations Research, 153(1):165 – 178, 2007.
  * [16] M. Bucchi and B. Trench.  Handbook of Public Communication of Science and Technology.  Routledge, London, 2014.  2nd edition.
  * [17] C. Casteluccia and D. Le Métayer.  Understanding algorithmic decision making: opportunities and challenges.  EPRS study, European Parlament, 2019.  104 pages.
  * [18] V. Christophides, V. Efthymiou, Th. Palpanas, G. Papadakis, and K. Stefanidis.  An overview of end-to-end entity resolution for big data.  ACM Computing Surveys, 53:1 – 42, 2021.
  * [19] C.W. Churchman.  Challenge to Reason.  McGraw-Hill, New York, 1968.
  * [20] C.W. Churchman.  Operations research as a profession.  Management Science, 17:B37–B53, 1970.
  * [21] CNRS ethical committee COMETS.  Practical guide “integrity and responsibility in research practises”, 2017.  https://comite-ethique.cnrs.fr/en/practical-guide/.
  * [22] A. Colorni and A. Tsoukiàs.  Designing alternatives for decision problems.  Journal of Multi-Criteria Decision Analysis, 27:150 – 158, 2020\.
  * [23] L. Coutellec.  Penser l’indissociabilité de l’éthique de la recherche, de l’intégrité scientifique et de la responsabilité sociale des sciences.  Revue d’anthropologie des connaissances, 13, 2:381 – 398, 2019\.
  * [24] Ph.M. Dung.  On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games.  Artificial Intelligence, 77:321–358, 1995.
  * [25] V. Ferretti, I. Pluchinotta, and A. Tsoukiàs.  Studying the generation of alternatives in public policy making processes.  European Journal of Operational Research, 273:353 – 363, 2019.  https://doi.org/10.1016/j.ejor.2018.07.054.
  * [26] J. Fürnkranz and E. Hüllermeier.  Preference Learning.  Springer Verlag, Berlin, 2010.
  * [27] G. Gallo.  Operations research and ethics: Responsibility, sharing and cooperation.  European Journal of Operational Research, 153(2):468–476, 2004\.
  * [28] S. Gass.  Ethical guidelines and codes in operations research.  Omega, 37:1044–1050, 2009.
  * [29] S.I. Gass.  On ethics in operational research.  Journal of the Operational Research Society, 45:965–966, 1994.
  * [30] G. Gigerenzer.  Calculated risks: How to know when numbers deceive you.  Simon and Schuster, New York, 2002.
  * [31] G. Gigerenzer and P.M. Todd.  Simple heuristics that make us smart.  Oxford University Press, New York, 1999.
  * [32] R.L. Keeney.  Value-Focused Thinking. A Path to Creative Decision Making.  Harvard University Press, Cambridge, 1992.
  * [33] M.W. Kirby.  A festering sore: the issue of professionalism in the history of the operational research society.  Journal of the Operational Research Society, 57:161 – 172, 2006\.
  * [34] M. Landry, C. Banville, and M. Oral.  Model legitimisation in operational research.  European Journal of Operational Research, 92:443–457, 1996.
  * [35] M. Landry, J.L. Malouin, and M. Oral.  Model validation in operations research.  European Journal of Operational Research, 14:207–220, 1983.
  * [36] P. Le Masson, K. Dorst, and E. Subrahmanian.  Design theory: History, state of the art and advancements.  Research in Engineering Design, 24:97–103, 2013.
  * [37] M. Le Menestrel and L.N. Van Wassenhove.  Ethics in operations research and management sciences: a never-ending effort to combine rigor and passion.  International Journal in Operational Research, 37:1030–1043, 2009\.
  * [38] Y. Meinard and A. Tsoukiàs.  On the rationality of decision aiding processes.  European Journal of Operational Research, 273:1074 – 1084, 2019\.
  * [39] Y. Meinard and A. Tsoukiàs.  What is legitimate decision support?  Cahier du LAMSADE, 401, PSL, Université Paris Dauphine, 2022.  https://arxiv.org/abs/2201.12071.
  * [40] L. Narens and R.D. Luce.  Meaningfulness and invariance.  In Eatwell J., Milgate M., and Newman P, editors, Time Series and Statistics, pages 140 – 148. Palgrave Macmillan, London, 1990.
  * [41] J. Nott.  Extreme Events: A Physical Reconstruction and Risk Assessment.  Cambridge University Press, Cambirdge, 2006.
  * [42] OECD.  Best practices for ensuring scientific integrity and preventing misconduct, 2007.  https://www.oecd.org/science/inno/40188303.pdf.
  * [43] R.J. Ormerod and W. Ulrich.  Operational research and ethics: A literature review.  European Journal of Operational Research, 228(2):291–307, 2013\.
  * [44] ORSA.  Guidelines for the practice of operations research.  Operations Research, 19:1123–1148, 1971.
  * [45] W. Ouerdane, N. Maudet, and A. Tsoukiàs.  Argumentation theory and decision aiding.  In J. Figueira, S. Greco, and M. Ehrgott, editors, Trends in Multiple Criteria Decision Analysis, pages 177 – 208. Springer Verlag, New York, 2010.
  * [46] F. Rauschmayer, I. Kavathatzopoulos, P.L. Kunsch, and M. Le Menestrel.  Why good practice is not enough - ethical challenges for the or practitioner.  Omega, 37:1089–1099, 2009.
  * [47] F.S. Roberts.  Measurement theory, with applications to Decision Making, Utility and the Social Sciences.  Addison-Wesley, Boston, 1979.
  * [48] F.S. Roberts.  On Luce’s theory of meaningfulness.  Philosophy of Science, 47(3):424–433, 1980.
  * [49] J. Rosenhead.  Rational analysis of a problematic world.  J. Wiley, New York, 1989.  2nd revised edition in 2001.
  * [50] J. Rosenhead.  Past, present and future of problem structuring methods.  Journal of the Operational Research Society, 57:759–765, 2006.
  * [51] J. Rosenhead and G.H. Mitchell.  Report of the commission on the future practice of operational research.  Journal of the Operational Research Society, 37:831–886, 1986.
  * [52] B.C. Smith, J.F. Leimkuhler, and R.M. Darrow.  Yield management at American Airlines.  Journal on Applied Analytics, 22:8 – 31, 1992.
  * [53] R. Tomasini and L. van Wassenhove.  Humanitarian Logistics.  Palgrave, London, 2009.
  * [54] J. Trevor and M. Bench-Capon.  Persuasion in practical argument using value-based argumentation frameworks.  Journal of Logic and Computation, 13:429 – 448, 2003.
  * [55] A. Tsoukiàs.  On the concept of decision aiding process.  Annals of Operations Research, 154:3 – 27, 2007.
  * [56] A. Tsoukiàs.  From decision theory to decision aiding methodology.  European Journal of Operational Research, 187:138 – 161, 2008.
  * [57] A. Tsoukiàs.  Social responsibility of algorithms: an overview.  In J. Papathanasiou, P. Zaraté, and J. Freire de Sousa, editors, EURO Working Group on DSS, pages 153 – 166. Springer Nature, 2021.
  * [58] Ph. Vincke.  Exploitation of a crisp relation in a ranking problem.  Theory and Decision, 32(3):221–240, 1992.
  * [59] WCRIF.  Singapore statement on research integrity, 2010.  https://wcrif.org/guidance/singapore-statement.
  * [60] F. Wenstøp.  Operations research and ethics: development trends 1996-2009.  International Transactions in Operational Research, 17:413–426, 2010.