Social desirability of people has been researched in various contexts. Characteristics that have been attributed to people with socially desirable qualities include willingness to help others, keeping the interest of others in mind while embarking on any task, being a good listener, willingness to admit a mistake and being embedded in the larger social matrix. Each vignette was rated to be on a person deemed to be 'socially desirable' or 'socially neutral' by the entire study team, including the psycho-oncologists and oncology clinicians, e.g. being an elderly man who is active and looks after his grandchildren, a young man who is a footballer, a lady who was previously fit and recently struggling with her household chores, a middle-aged lady with breast cancer who is feeling better in between chemotherapy sessions were all deemed to be socially desirable. Examples of 'socially neutral' vignettes were 'a 67-year-old man with lung cancer who is breathless', 'a 77-year-old retired teacher who spends his time in reading and takes a driver to drive him to the market' etc. No one was deemed to be socially undesirable but only rated as socially desirable or neutral. This is in keeping with the professional non-judgmental attitude that clinicians ought to take in ideal circumstances. A consensus rating on social desirability was made by the two psycho-oncologists and all the oncologists who were part of the study team. Each of them rated the vignettes as socially desirable or neutral and any discordance in the ratings was resolved in the second round of feedback and discussions.
A comprehensive list of all doctors and nurses working in the clinical departments of the hospital who came into direct patient contact was obtained from the department of human resources. Potential participants were approached individually by the researchers. Following a sequential design, a proportion of the respondents who participated in the quantitative part of the study were purposively chosen for qualitative in-depth interviews.
The clinicians working in the hospital were approached for being included in the study. They were interviewed in private, in their own offices at a time suitable to them. Clinicians were requested to rate the vignettes on the ECOG PS. In order to avoid the rare problem of being unsure of any of the items of ECOG rating scale during the data collection, we had the ECOG rating scale (vide Appendix I) with scoring criteria in front of the participant while he/she rated each vignette. This ensured that the participants could fully concentrate on the rating of the vignette rather than feel they were being evaluated on their knowledge of the ECOG rating scale as this was not the purpose of the study.
On completion of the ECOG rating, some clinicians were interviewed in-depth. They were interviewed in a relaxed environment. Clinicians who specialised in various clinical specialties of oncology were invited to participate. Participants were interviewed in their own offices as per their convenience. Respondents were asked about their decision-making processes and how they usually processed the information regarding a patient's health, disease and daily life in order to come up with the ECOG PS rating in a busy clinic. The qualitative part of the study adhered to the Consolidated Criteria for Reporting Qualitative Research (COREQ) guidelines for research. All qualitative interviews were recorded and transcribed verbatim. The interviewers were trained in the techniques of qualitative in-depth interviewing by a senior researcher of the team.
All doctors and nurses who had been employed in the hospital and were in direct patient contact were eligible to participate in the study. All clinicians in the study hospital who offered treatment to patients were familiar with ECOG rating and used it routinely. We specifically included only those clinicians who were involved in face-to-face decision-making for patients with cancer. Non-clinical specialties such as laboratory-based disciplines like biochemistry, histopathology and others that were not involved directly in making decisions for treatment based on ECOG rating and were therefore not included in the study.
A predesigned set of 12 clinical case vignettes of variable complexity was designed by senior oncologists who were part of the study team. Participants were required to rate each of the cases on the ECOG scale. The case scenarios were presented with the standard ECOG scale which is a categorical scale (0, 1, 2, 3, and so on). Usually, it is easy to identify a PS ECOG grade '0', as well as a grade '4'. The difficulty appears to be differentiating between grades 1, 2 or 3. The sample case vignettes were designed to study results of ECOG assessment in patients in all PS grades but more specifically within the grades 1, 2 and 3. Socio-demographic variables were collected using a socio-demographic sheet given to all participants.
Interviewers started with a predetermined set of interview items related to decision-making in oncology and the use of the ECOG scale. The interviewer also probed clinicians about decision-making, the algorithms they used and the thought process they engage in while planning and discussing oncology treatments among peers, colleagues and with patients. Qualitative data analysis and data collection went on concurrently in order to incorporate newly emerging themes from earlier interviews until data saturation was achieved.
Quantitative analysis comprised of calculation of agreement indices for the entire population and for subsets of the population. Graphical exploration of the agreement was performed using heatmaps. Prespecified subsets for which agreement was calculated included age group, gender, profession type and years of experience in the field of oncology. Additionally, the questionnaire also included a question about the participant's opinion on the ECOG rating being perceived as a subjective or objective measure. We calculated agreement metrics for this question as well. As multiple raters were involved, a model-based kappa was calculated using the methodology proposed by Nelson et al. A kappa of 1 is interpreted as perfect agreement and 0 being absolute disagreement between raters. In order to circumvent the problems of Cohen's kappa value being influenced by small number values that are different from the other values specially with symmetrical data (kappa paradox), Gwet proposed first-order agreement coefficient or the AC1 statistic, which adjusts for the fact that multiple raters may agree on a rating by chance and addresses the kappa paradox. To describe in brief, this method uses a generalised linear mixed model to calculate the observed agreement between raters while minimising the impact of a chance agreement. Unlike Cohen's kappa and other variations, this method is not affected by prevalence rates. Furthermore, measure of association was also calculated using the same model-based approach. The function for calculation of the model based kappa was provided by Nelson et al.
In order to investigate the effect of patient-related factors on the ECOG PS rating, a cumulative link mixed model (CLMM) was constructed for the data. All the patients described in the vignettes were a priori assigned to having socially desirable characteristics (vignettes 1, 2, 6 & 7) or neutral characteristics (rest of the vignettes). To give an example:'a grandmother who played with her grandchildren' was deemed to have socially desirable characteristics by the researchers and 'a politician who has lost his voice' was deemed to be neutral or not specifically having any socially desirable quality in the description in the vignette. Three characteristics of the patient, namely, age of the patient in the vignette, gender of the patient and the perceived social desirability of the patient were fitted into the CLMM. In the same way, CLMM allowed us to explore the effect of individual raters on the PS rating and to explore the way groups of raters tend to rate the PS. Dot plot of the effect of the rater characteristics on the PS rating was also calculated using CLMM faceted as per rater categories. Dots are coloured as per the job role, for example, consultant, trainee doctor or nursing staff. Maximum likelihood estimates of the coefficients were calculated using adaptive Gauss-Hermite quadrature method using 10 quadrature nodes. The exponents of the coefficients allowed us to get the odds ratio of rating being rated for the aforementioned vignettes.
The second part of the analysis focused on the agreement with the gold standard ratings. In this analysis, the proportion of raters who gave the correct rating was the endpoint. Given that ECOG PS is an example of ordinal rating scale, no efforts were made to calculate the distance of disagreement. We also explored if the proportion of agreement with the golden rating varied as per the rater characteristics using the variables mentioned above.
Qualitative data analysis went side by side with data collection as is recommended for qualitative research. Qualitative data analysis followed the principles of thematic analysis as suggested by Braun and Clarke. The various steps we used included becoming familiar with the data, generating initial codes, searching for themes or charting data, review of themes and synthesis, generating basic and organising themes and finally, arriving at global themes which defines the data. According to experts, thematic analysis is a method of identifying, analysing and reporting patterns or themes within qualitative data and is able to provide a rich description of the data. They also suggest that one of the advantages of this approach is its flexibility and that thematic analysis should be seen as a foundational method for qualitative analysis.