To maximize information retrieval through the catalogue, one another enhancement is to gather all the MeSH terms that are related to a given specialty, since they can be dispersed among the 16 MeSH branches. On the other hand, the use of multiple terminologies is recommended [29] to increase the number of the lexical and graphical forms of a biomedical term recognized by a search engine. Since 2007, the CISMeF resources are indexed using the vocabulary of 23 other terminologies and classifications, most of them being bilingual (English and French). To supply health information seekers with the terminologies available in French, these terminologies are accessible through the Health Multiple Terminologies and

The MeSH thesaurus is partitioned at its upper level into 16 branches (*e.g*. Anatomy, Diseases). The core of MeSH thesaurus is a hierarchical structure that consists of sets of descriptors. However, these hierarchies do not allow a complete view concerning a specialty. The main headings and subheadings in the CISMeF controlled vocabulary are gathered under meta-terms (*e.g.* cardiology) (Figure 4). Meta-terms (n=73) concern medical specialties and it is possible by browsing to know sets of MeSH main headings and subheadings which are semantically related to the same specialty but dispersed in several trees. Meta-terms have been created to optimize information retrieval in CISMeF and to overcome the relatively restrictive nature of MeSH headings. For example a search on "guidelines" or "virology", where cardiology and virology are descriptors, yield few answers. Introducing cardiology and virology as meta-terms is an efficient strategy to obtain more results because instead of exploding one single MeSH tree, the use of meta-terms results in an automatic expansion of the queries by exploding other related MeSH trees besides the current tree, using the well-known automatic query expansion process. In other words, a query using a meta-term corresponds to the union of all the queries for all the terms semantically linked to it. A comparison of the results of MeSH term-based queries and

SC-based queries showed an increased recall with no decrease in precision [33].

The use of multiple terminologies is recommended [29] to increase the number of the lexical and graphical forms of a biomedical term recognized by a search engine. For this reason, CISMeF evolved recently from a single terminology approach using the MeSH main headings and subheadings to a multiple terminologies paradigm using, in addition to the MeSH thesaurus, vocabularies and classifications that deal with various aspects of health. Among them, the Systematized NOmenclature of MEDicine (SNOMED 3.5), the French CCAM for procedures [34], Orphanet for rare diseases2 and some classifications from the World Health Organization : the 10th revision of the International Classification of Diseases3

Our aim is to compare the precision and recall of multiple terminologies meta-terms (mt-mt) to MeSH meta-terms (M-mt) in CISMeF. Since mt-mt are based on M-mt plus semantic links to some terms in other terminologies, the query results for M-mt are all included in the query results for mt-mt, which became the gold standard for recall. We have then to evaluate the precision of the query retrieving resources indexed by a term linked to M-mt (MeSH metaterm query), on the one hand, and by a term linked to mt-mt and not to M-mt ( query) on the other hand. For this purpose, we build Boolean queries using the meta-terms themselves. For example, for the "*surgery*" meta-term, the MeSH meta-term (M-mt) query is "*surgery[M-mt]*". The query is: "*surgery[mt-mt] NOT surgery[M-mt]*". Retrieved resources returned were assessed for relevance. We detail in the next section the criteria we have used for evaluation.

#### *4.2.2. Evaluations*

The resources returned by the CISMeF's search tool using automatic query expansion were assessed for relevance according to a three modality scale used in other standard Information Retrieval test sets: irrelevant (0), partly relevant (1) or fully relevant (2). A physician manually assigned relevance scores (0;1;2) to the top 20 resources returned for

4 http://www.meddramsso.com

each meta-term query. The results of the evaluation are given in the Table 4. We chose to assign relevance scores to the top twenty resources returned because 95% of the end-users do not go beyond this limit when using a general search engine [35]. For the purpose of assessing meta-terms for Information Retrieval, we have developed a test collection comprising relevance judgments for the top 20 resources returned for a selection of 20 etaterms queries. Table 4 shows that the queries yielded 118,772 resources, of which 708 were assessed for relevance (0.6%). Weighted precisions for MeSH meta-terms queries and for queries were computed given the level of relevance considered and compared using χ² test. Indexing methods and meta-terms were compared too. Relative recall for MeSH meta-terms queries were computed given the level of relevance considered.

Supporting E-Health Information Seekers: From Simple Strategies to Knowledge-Based Methods 51

indexing method: manual (precision of 0.50 and 0.81 for, respectively, full and partial relevance) perform better than automatic (precision of 0.38 and 0.48 for, respectively, full

Meta-Term Query Type Nb of documents Relevance on 20 doc

Diagnosis MeSH 13,132 0 2 15

Toxicology MeSH 11,980 0 0 20

Neurology MeSH 9,325 8 4 8

Paediatrics MeSH 7,560 4 4 12

Cardiology MeSH 5,288 1 0 18

Oncology MeSH 5,626 0 1 18

Surgery MeSH 5,504 17 0 3

Rheumatology MeSH 4,408 3 8 9

Gastroenterology MeSH 4,069 0 0 20

Metabolism MeSH 3,797 14 2 4

Dermatology MeSH 3,196 7 0 13

Nutrition MeSH 3,455 0 1 19

Pneumology MeSH 3,466 0 7 12

Gynaecology MeSH 3,186 6 1 12

Obstetrics MeSH 3,063 5 1 12

Virology MeSH 3,122 1 11 6

**Figure 5.** Relevance of resources retrieved by 18 meta-terms queries on top 20 documents.

**Total MeSH 101,332 67 59 223**

Allergies and Immunology MeSH 4,598 1 17 2

Infectious Diseases MeSH 6,557 0 0 20

Delta 350 14 1 5

Delta 482 16 1 3

Delta 2,168 11 5 4

Delta 2,573 3 16 1

Delta 251 2 4 13

Delta 2,388 4 10 6

Delta 1,063 2 14 4

Delta 320 5 0 15

Delta 856 11 5 4

Delta 1,106 8 11 1

Delta 573 2 17 1

Delta 849 0 2 18

Delta 1,427 0 4 16

Delta 1,027 0 9 11

Delta 584 0 14 6

Delta 850 0 1 19

Delta 316 20 0 0

Delta 257 0 20 0

**Delta 17,440 98 134 127**

Not Partially Totally

and partial relevance), and to the studied meta-term.

**Figure 4.** Gathering MeSH *main headings* and *subheadings* under *meta-terms*. *Resource types* are modelled to describe the nature of a resource because of the heterogeneity of resources.

The mean weighted precision of queries was 0.33 and 0.76 for, respectively, full and partial relevance. The mean precision of MeSH meta-terms queries was 0.66 and 0.80 for, respectively, full and partial relevance. The difference between MeSH meta-terms and multiple terminologies meta-terms was significant for full relevance (0.66 *vs* 0.61; p<10-4, χ²) but not for partial relevance (both 0.80; p=0.3, χ²). The mean recall of MeSH meta-terms queries was 0.92 and 0.86 for, respectively, full and partial relevance. Table 5 shows that, whatever the relevance considered was, results varied significantly according to the indexing method: manual (precision of 0.50 and 0.81 for, respectively, full and partial relevance) perform better than automatic (precision of 0.38 and 0.48 for, respectively, full and partial relevance), and to the studied meta-term.

50 eHealth and Remote Monitoring

each meta-term query. The results of the evaluation are given in the Table 4. We chose to assign relevance scores to the top twenty resources returned because 95% of the end-users do not go beyond this limit when using a general search engine [35]. For the purpose of assessing meta-terms for Information Retrieval, we have developed a test collection comprising relevance judgments for the top 20 resources returned for a selection of 20 etaterms queries. Table 4 shows that the queries yielded 118,772 resources, of which 708 were assessed for relevance (0.6%). Weighted precisions for MeSH meta-terms queries and for queries were computed given the level of relevance considered and compared using χ² test. Indexing methods and meta-terms were compared too. Relative recall for MeSH meta-terms

**Figure 4.** Gathering MeSH *main headings* and *subheadings* under *meta-terms*. *Resource types* are modelled

The mean weighted precision of queries was 0.33 and 0.76 for, respectively, full and partial relevance. The mean precision of MeSH meta-terms queries was 0.66 and 0.80 for, respectively, full and partial relevance. The difference between MeSH meta-terms and multiple terminologies meta-terms was significant for full relevance (0.66 *vs* 0.61; p<10-4, χ²) but not for partial relevance (both 0.80; p=0.3, χ²). The mean recall of MeSH meta-terms queries was 0.92 and 0.86 for, respectively, full and partial relevance. Table 5 shows that, whatever the relevance considered was, results varied significantly according to the

to describe the nature of a resource because of the heterogeneity of resources.

queries were computed given the level of relevance considered.

**Figure 5.** Relevance of resources retrieved by 18 meta-terms queries on top 20 documents.

Supporting E-Health Information Seekers: From Simple Strategies to Knowledge-Based Methods 53

 *T /Tt I t*

(10)

*(Supp II )II(Supp) 12 21* (11)

(12)

1 2

1

is possible to calculate the support of an association rule from the support of an itemset. Supp(*Ik*) the support of the itemset *Ik* is defined as the probability of finding *Ik* in a

> *(Supp I ) k k*

The confidence of an association rule represents its precision. This measure corresponds to the proportion of objects that contains the consequent rule among those containing the antecedent. The confidence of the rule *I1*→*I2*, written as Conf(*I1*→*I2*) is calculated as

> ( ) ( ) ( ) *Supp I I Conf I I Supp I*

Two types of rules are distinguished: exact association rules that have a confidence equal to 100%, *i.e.* verified in all the objects of the database and approximate association rules that

Several methods are used to extract all of the association rules from a database. The simplest method consists of enumerating all the itemsets from which all the possible association rules could be generated. The total number of itemsets for a database that contains n Boolean attributes is 2n. This naïve method is inapplicable to real-life databases. A more efficient method involves computing itemsets that have a support higher than a given threshold. They are called *frequent itemsets*. The association rules extraction time depends on the frequent itemsets extraction time. Several accesses to the database are necessary to compute the number of database objects in which each frequent itemset candidate is contained. The association rules algorithms by level consider in each iteration a set of itemsets of a particular size, *i.e.* a set of itemsets in a level of the itemsets lattice. The following properties are used by these algorithms to limit the number of the itemsets candidates: all of the super-sets of an infrequent itemset are infrequent, and all the subsets of a frequent itemset are frequents [37]. This method is founded on the two-stepped model that finds all of the rules that satisfy user-specified minimum support and confidence: (i) Generate all large itemsets that satisfy minimum support and (ii) From large itemests generate all association rules that satisfy minimum confidence. Apriori algorithm [37] realizes a number of database accesses equal to the size of the larger

1 2

The support of the rule *I1*→*I2* written as Supp(*I1*→*I2*) is calculated as follows:

transaction of *T*:

*5.1.3. Confidence*

confidence<100%.

**5.2. Data-mining algorithms**

follows:

**Figure 6.** Determinants of relevance; χ² test.

To complete the information retrieval process and to allow interactive query expansion with the health information seeker, we propose in the next section to use "new" knowledge represented as association rules extracted by data-mining process.

## **5. Knowledge extraction**

The knowledge-approach is based upon a data-mining process, called association rules, which can infer "new" relations between medical concepts. A data-mining system may generate several thousands and even several millions frequent association rules, and only some of these will be interesting. In this section we will show how only the most relevant association rules are mined using Formal Concept Analysis and Galois closure. We consider a relevant association rule as being non-redundant with a minimal antecedent and a maximal consequent, which is particularly useful for query expansion.

#### **5.1. Association rules**

The discovery of association rules is a widely used technique in data-mining. The general problem was described in [36], in which relations were discovered among pieces of data (called items). An association rule is interesting if it is easily understood by the users, valid for new data, useful, or confirms a hypothesis. The task of association rule mining can be applied to various types of data: any data set containing multiple items.

#### *5.1.1. Definitions*

Let *I* be a set of items, called itemset, and *D* a database of transactions where each transaction *T* (*T D*) is an itemset. An association rule is an implication rule expressed in the form of: *I1*→*I2* where *I1* and *I2* are two itemsets *I1*, *I2 I* so that *I1* ∩ *I2* =. The rule expresses that whenever a transaction *T* contains *I1* then *T* probably also contains *I2*. In other words, the implication rule means that the apparition of the itemset *I1* in a transaction *T*, implies the apparition of the itemset *I2* in the same transaction. However, the reciprocal implication does not have to happen necessarily. *I1* is called antecedent and *I2* is called consequent.

#### *5.1.2. Support*

The support of an association rule represents its utility. This measure corresponds to the proportion of objects which contains at the same time the rule antecedent and consequent. It is possible to calculate the support of an association rule from the support of an itemset. Supp(*Ik*) the support of the itemset *Ik* is defined as the probability of finding *Ik* in a transaction of *T*:

$$\text{Lusp}(I\_k) = \frac{\left| \{ t \in T \mid I\_k \subseteq t \} \right|}{\left| T \right|} \tag{10}$$

The support of the rule *I1*→*I2* written as Supp(*I1*→*I2*) is calculated as follows:

$$\text{Supp}(\mathbf{I}\_1 \rightarrow \mathbf{I}\_1) = \text{Supp}(\mathbf{I}\_1 \cup \mathbf{I}\_2) \tag{11}$$

#### *5.1.3. Confidence*

52 eHealth and Remote Monitoring

**Figure 6.** Determinants of relevance; χ² test.

**5. Knowledge extraction**

**5.1. Association rules**

*5.1.1. Definitions*

*5.1.2. Support*

Variable Full Relevance Partial Relevance

To complete the information retrieval process and to allow interactive query expansion with the health information seeker, we propose in the next section to use "new" knowledge

The knowledge-approach is based upon a data-mining process, called association rules, which can infer "new" relations between medical concepts. A data-mining system may generate several thousands and even several millions frequent association rules, and only some of these will be interesting. In this section we will show how only the most relevant association rules are mined using Formal Concept Analysis and Galois closure. We consider a relevant association rule as being non-redundant with a minimal antecedent and a

The discovery of association rules is a widely used technique in data-mining. The general problem was described in [36], in which relations were discovered among pieces of data (called items). An association rule is interesting if it is easily understood by the users, valid for new data, useful, or confirms a hypothesis. The task of association rule mining can be

Let *I* be a set of items, called itemset, and *D* a database of transactions where each transaction *T* (*T D*) is an itemset. An association rule is an implication rule expressed in the form of: *I1*→*I2* where *I1* and *I2* are two itemsets *I1*, *I2 I* so that *I1* ∩ *I2* =. The rule expresses that whenever a transaction *T* contains *I1* then *T* probably also contains *I2*. In other words, the implication rule means that the apparition of the itemset *I1* in a transaction *T*, implies the apparition of the itemset *I2* in the same transaction. However, the reciprocal implication does not have to happen necessarily. *I1* is called antecedent and *I2* is called consequent.

The support of an association rule represents its utility. This measure corresponds to the proportion of objects which contains at the same time the rule antecedent and consequent. It

Specific Query (M-mt vs mt-mt) p < 10-4 p = 0.3 Indexing Method p = 0.004 p < 10-4 Meta-Term p < 10-4 p < 10-4

represented as association rules extracted by data-mining process.

maximal consequent, which is particularly useful for query expansion.

applied to various types of data: any data set containing multiple items.

The confidence of an association rule represents its precision. This measure corresponds to the proportion of objects that contains the consequent rule among those containing the antecedent. The confidence of the rule *I1*→*I2*, written as Conf(*I1*→*I2*) is calculated as follows:

$$\text{Conf}(I\_1 \to I\_2) = \frac{\text{Supp}(I\_1 \cup I\_2)}{\text{Supp}(I\_1)} \tag{12}$$

Two types of rules are distinguished: exact association rules that have a confidence equal to 100%, *i.e.* verified in all the objects of the database and approximate association rules that confidence<100%.

#### **5.2. Data-mining algorithms**

Several methods are used to extract all of the association rules from a database. The simplest method consists of enumerating all the itemsets from which all the possible association rules could be generated. The total number of itemsets for a database that contains n Boolean attributes is 2n. This naïve method is inapplicable to real-life databases. A more efficient method involves computing itemsets that have a support higher than a given threshold. They are called *frequent itemsets*. The association rules extraction time depends on the frequent itemsets extraction time. Several accesses to the database are necessary to compute the number of database objects in which each frequent itemset candidate is contained. The association rules algorithms by level consider in each iteration a set of itemsets of a particular size, *i.e.* a set of itemsets in a level of the itemsets lattice. The following properties are used by these algorithms to limit the number of the itemsets candidates: all of the super-sets of an infrequent itemset are infrequent, and all the subsets of a frequent itemset are frequents [37]. This method is founded on the two-stepped model that finds all of the rules that satisfy user-specified minimum support and confidence: (i) Generate all large itemsets that satisfy minimum support and (ii) From large itemests generate all association rules that satisfy minimum confidence. Apriori algorithm [37] realizes a number of database accesses equal to the size of the larger frequent itemsets. Many researchers have tried to improve various aspects of Apriori, such as the number of passes and accesses to the data-bases or the time efficiency of those passes. We have chosen to adapt the A-Close algorithm [38] in which new bases for association rules are deduced from the closed frequent itemsets and their generators. These bases consist of non-redundant association rules of minimal antecedents and maximal consequents, *i.e.* the most relevant association rules and are defined by using the closure operator of the Galois connection of a finite binary relation. All frequent itemsets and their support, and therefore all association rules, are deduced efficiently from the frequent closed itemsets without accessing the database.

Supporting E-Health Information Seekers: From Simple Strategies to Knowledge-Based Methods 55

The extracted association rules in the precedent cases are related to the medical domain. To obtain more precise rules we performed experiments on categorized documents according to groups of users: students in medicine, health professionals, and general public to evaluate

In CISMeF, mainly three types of health information seekers are categorized: professionals, students in medicine, patients and lay people. We consider three major resource types: guidelines\*, education\* and patients\*. We also consider two kinds of itemsets: the set of major main headings I={MH\*} and the set of major (main heading/subheading) pairs

Resource type Documents Items Min Max Mean

For all contexts, the minimum support threshold was fixed to minsup=20 and the minimum confidence threshold was fixed to minconf=70% (Table 7). We obtained association rules between major main headings MH\* in the first context where I={MH\*} and between [MH/SH]\* pairs for I={[MH/SH]\*}. For the major resource types patients\* and education\* all association rules (100%) are between two MHs\* and between [MH/SH]\* i.e. one descriptor in the antecedent and one descriptor in the consequent. For the major resource type guidelines\*, 24% of the rules are between more than two descriptors. The characteristics of documents may explain these results: average descriptors were from 1.63 to 2.22 for patients\* and education\* whereas they were from 5.21 to 6.12 for

MH\* 1 64 5.21

MH\* 0 25 1.63

MH\* 0 25 2.22

MH/SH\* 1 70 6.12

MH/SH\* 0 30 1.82

MH/SH\* 0 34 2.73

number of rules is 2,565 (648 exact rules; 1,917 approximate rules).

**5.4. Categorizing documents according to health information seekers**

the influence of categorization on the generation of association rules.

I={[MH/SH]\*)}. The collection is detailed in Table 6.

Guidelines\* 2,727

Patients\* 3,272

Education\* 3,610

**Table 4.** Description of the collections of documents.

guidelines\*.

#### **5.3. Extracting knowledge from e-Health documents**

Our experiments are carried out on the CISMeF database. An extraction context is a triplet *C= (O, I, R)* where *O* is the set of objects, *I* is the set of all the items and *R* is a binary relation between *O* and *I*. Applying this model to our database, the objects are the indexed e-health documents. Each document has a unique identifier and a set of associated descriptors. These descriptors may be MeSH main headings and associations between MeSH main headings and MeSH subheadings. The relation *R* represents the indexing relation between an object and an item, *i.e.* a descriptor that belongs to *I*. We studied different extraction contexts by applying and adapting the A-Close algorithm such as the context of categorized documents, according to the user type and to meta-terms. There is an average of 6.5 descriptors by document in CISMeF with a minimum of 1 and a maximum of 300. This constraint on the number of descriptors *i.e.* the size of the set of items has been considered in the implementation phase of the A-Close algorithm. Indeed, A-Close works on databases with a maximum of 12 items. We have added another requirement to the implementation to avoid long time generation: maximal size of the closed itemsets is fixed to 300 items as it corresponds to the maximum number of descriptors for the documents. As an output, the association rules may be visualized in a file or automatically added to the database to be used in the information retrieval process, mainly by interactive query expansion.

#### *5.3.1. Extracting knowledge from all the database*

The extracted association rules in the precedent cases are related to the medical domain. To obtain more precise rules we performed experiments on categorized documents according to groups of users: students in medicine, health professionals, and general public to evaluate the influence of categorization on the generation of association rules.

## **5.4. Categorizing documents according to health information seekers**

In CISMeF, mainly three types of health information seekers are categorized: professionals, students in medicine, patients and lay people. We consider three major resource types: guidelines\*, education\* and patients\*. We also consider two kinds of itemsets: the set of major main headings I={MH\*} and the set of major (main heading/subheading) pairs I={[MH/SH]\*)}. The collection is detailed in Table 6.

**Table 4.** Description of the collections of documents.

54 eHealth and Remote Monitoring

expansion.

frequent itemsets. Many researchers have tried to improve various aspects of Apriori, such as the number of passes and accesses to the data-bases or the time efficiency of those passes. We have chosen to adapt the A-Close algorithm [38] in which new bases for association rules are deduced from the closed frequent itemsets and their generators. These bases consist of non-redundant association rules of minimal antecedents and maximal consequents, *i.e.* the most relevant association rules and are defined by using the closure operator of the Galois connection of a finite binary relation. All frequent itemsets and their support, and therefore all association rules, are deduced efficiently from the

Our experiments are carried out on the CISMeF database. An extraction context is a triplet *C= (O, I, R)* where *O* is the set of objects, *I* is the set of all the items and *R* is a binary relation between *O* and *I*. Applying this model to our database, the objects are the indexed e-health documents. Each document has a unique identifier and a set of associated descriptors. These descriptors may be MeSH main headings and associations between MeSH main headings and MeSH subheadings. The relation *R* represents the indexing relation between an object and an item, *i.e.* a descriptor that belongs to *I*. We studied different extraction contexts by applying and adapting the A-Close algorithm such as the context of categorized documents, according to the user type and to meta-terms. There is an average of 6.5 descriptors by document in CISMeF with a minimum of 1 and a maximum of 300. This constraint on the number of descriptors *i.e.* the size of the set of items has been considered in the implementation phase of the A-Close algorithm. Indeed, A-Close works on databases with a maximum of 12 items. We have added another requirement to the implementation to avoid long time generation: maximal size of the closed itemsets is fixed to 300 items as it corresponds to the maximum number of descriptors for the documents. As an output, the association rules may be visualized in a file or automatically added to the database to be used in the information retrieval process, mainly by interactive query

exact; 11,738 approximate). The same conclusions are drowned from the case 1 : too

*{SH}*. We obtained 16,976 rules (5,241

manually analyzed by our experts (physicians or medical librarians).

frequent closed itemsets without accessing the database.

**5.3. Extracting knowledge from e-Health documents**

*5.3.1. Extracting knowledge from all the database*

associated with the set of documents O. *I={MH}*

numerous rules to be evaluated manually.

For all contexts, the minimum support threshold was fixed to minsup=20 and the minimum confidence threshold was fixed to minconf=70% (Table 7). We obtained association rules between major main headings MH\* in the first context where I={MH\*} and between [MH/SH]\* pairs for I={[MH/SH]\*}. For the major resource types patients\* and education\* all association rules (100%) are between two MHs\* and between [MH/SH]\* i.e. one descriptor in the antecedent and one descriptor in the consequent. For the major resource type guidelines\*, 24% of the rules are between more than two descriptors. The characteristics of documents may explain these results: average descriptors were from 1.63 to 2.22 for patients\* and education\* whereas they were from 5.21 to 6.12 for guidelines\*.

Supporting E-Health Information Seekers: From Simple Strategies to Knowledge-Based Methods 57

New

FS BR SA FS BR SA

8 66.7%

5 62.5%

2 22.2%

> 2 25%

66.6%

11 87.4%

33.3%

1 12.5%

2 22.2%

1 12.5%

16.7% - 4

7.7%

Our objective is to re-use the numerous association rules that we extracted from the CISMeF database into the information-retrieval process by query expansion. We use Interactive Query Expansion. For example, the association rule *breast cancer* → *mammography* is extracted from the corpus because the keywords *breast cancer* and *mammography* are frequently used together to index the documents. This association rule is as a "new" one because it doesn't exist in the domain knowledge which is, in our case, the MeSH thesaurus. When applying the association rule *breast cancer* → *mammography* on a query containing the term *breast cancer*, an interactive query expansion proposes to the user e-health documents related to *mammography* to complete the search. In medicine and health-related information, [40] have already investigated an efficient algorithm for association rule mining using the MeSH thesaurus. They adopted a MeSH-indexed representation of MEDLINE records, but the evaluation of the interest of the mined associations with respect to the task of PubMed retrieval improvement was not considered by the authors. In [41] many other works on information retrieval and query expansion in the biomedical domain are also presented. Methods to perform query expansion with promising results involve mining user logs [41] and constructing user profiles. And another study on logs in PubMed for searching

Existing knowledge

1 12.5%

55.6%

62.5%

1

7.7% - 1

biomedical and life-science literature online has been performed by [43].

In the literature, a number of methods for performing query expansion have been developed. The solutions given are based mainly on two approaches. The first is the

MH\* - - 4

12.5%

16.7%

**Table 6.** Association rules evaluation according to the MeSH structure

MH\* - 5

MH/SH\* - 5

MH\* 1

MH/SH\* 1

**5.5. Knowledge-based query expansion**

MH/SH\* 1

Exact rules Approximate rules

2 5.3%

3 9.7%

2 18.2%

2 18.2%

2 11.8%

2 16.8%

Existing knowledge

7 18.4%

3 9.7%

2 18.2%

2 18.2%

6 35.3%

> 3 25%

10 26.3%

> 9 29%

4 36.3%

3 27.3%

3 17.6%

2 16.8% New

12 31.6%

> 13 42%

3 27.3%

7 36.3%

6 35.3%

5 41.4%

Resource

Guidelines\*

Patients\*

Education\*

types Items

**Table 5.** Number of rules, exact rules (ER), approximate rules (AR), and number of pairs.

#### *5.4.1. Evaluation of the extracted knowledge*

Not all of the association rules extracted were evaluated: according to the context extraction and the itemset I there are more or less association rules. The more the collection is specialized, and the itemset size is reduced, the less we have association rules to evaluate. As defined, an interesting association rule confirms or states a new hypothesis [38].

Here, we proposed to combine background domain knowledge with simple statistical measures used traditionally in association rules mining for evaluation. We considered several cases of interesting association rules according to relations between MeSH headings. As these relations are defined between two main headings and between two subheadings, we considered only the association rules between two elements. Hence, an interesting existing association rule could associate: a (in)direct son and its father (relation FS); two descriptors that belong to the same hierarchy (same (in)direct father) (relation BR); two descriptors with See Also relation (relation SA). These rules are automatically classified thanks to the MeSH structure. The other rules that satisfy the minsup and minconf are then considered as «new» interesting association rules.

Exact association rules, except for collection patients\*, are mostly new interesting rules: from 62.5% to 87.4%. Therefore, existing rules are mainly from the patients\* collection: 77.8% for MH\* and 75% for MH/SH\*. However, approximate rules, are mostly existing rules (Table 8). Subjective interest measures are based on expert knowledge about the data, *i.e.* that of physicians and medical librarians in this context. New interesting rules for the contexts MH\* and [MH/SH]\* pairs are evaluated manually. 93.8% (resp. 84.8%) of the interesting new rules with conf=1 (resp. conf≥0.7) between major descriptors are validated.

**Table 6.** Association rules evaluation according to the MeSH structure

#### **5.5. Knowledge-based query expansion**

56 eHealth and Remote Monitoring

Guidelines\* 50

Patients\* 20

Education\* 23

hypothesis [38].

validated.

*5.4.1. Evaluation of the extracted knowledge*

considered as «new» interesting association rules.

Item=MH\* Item=[MH/SH]\*

39

24% 76% 76% 20.51% 79.49% 76%

19

45% 55% 100% 42.1% 57.9% 100%

25

26.09% 73.91% 100% 52% 48% 100%

8 31 35

8 11 19

13 12 25

Nb rules ER AR Nb pairs Nb rules ER AR Nb pairs

12 38 38

9 11 20

6 17 23

**Table 5.** Number of rules, exact rules (ER), approximate rules (AR), and number of pairs.

Not all of the association rules extracted were evaluated: according to the context extraction and the itemset I there are more or less association rules. The more the collection is specialized, and the itemset size is reduced, the less we have association rules to evaluate. As defined, an interesting association rule confirms or states a new

Here, we proposed to combine background domain knowledge with simple statistical measures used traditionally in association rules mining for evaluation. We considered several cases of interesting association rules according to relations between MeSH headings. As these relations are defined between two main headings and between two subheadings, we considered only the association rules between two elements. Hence, an interesting existing association rule could associate: a (in)direct son and its father (relation FS); two descriptors that belong to the same hierarchy (same (in)direct father) (relation BR); two descriptors with See Also relation (relation SA). These rules are automatically classified thanks to the MeSH structure. The other rules that satisfy the minsup and minconf are then

Exact association rules, except for collection patients\*, are mostly new interesting rules: from 62.5% to 87.4%. Therefore, existing rules are mainly from the patients\* collection: 77.8% for MH\* and 75% for MH/SH\*. However, approximate rules, are mostly existing rules (Table 8). Subjective interest measures are based on expert knowledge about the data, *i.e.* that of physicians and medical librarians in this context. New interesting rules for the contexts MH\* and [MH/SH]\* pairs are evaluated manually. 93.8% (resp. 84.8%) of the interesting new rules with conf=1 (resp. conf≥0.7) between major descriptors are

Resource types

Our objective is to re-use the numerous association rules that we extracted from the CISMeF database into the information-retrieval process by query expansion. We use Interactive Query Expansion. For example, the association rule *breast cancer* → *mammography* is extracted from the corpus because the keywords *breast cancer* and *mammography* are frequently used together to index the documents. This association rule is as a "new" one because it doesn't exist in the domain knowledge which is, in our case, the MeSH thesaurus. When applying the association rule *breast cancer* → *mammography* on a query containing the term *breast cancer*, an interactive query expansion proposes to the user e-health documents related to *mammography* to complete the search. In medicine and health-related information, [40] have already investigated an efficient algorithm for association rule mining using the MeSH thesaurus. They adopted a MeSH-indexed representation of MEDLINE records, but the evaluation of the interest of the mined associations with respect to the task of PubMed retrieval improvement was not considered by the authors. In [41] many other works on information retrieval and query expansion in the biomedical domain are also presented. Methods to perform query expansion with promising results involve mining user logs [41] and constructing user profiles. And another study on logs in PubMed for searching biomedical and life-science literature online has been performed by [43].

In the literature, a number of methods for performing query expansion have been developed. The solutions given are based mainly on two approaches. The first is the augmentation of query terms to improve the retrieval process without user intervention. The second is the suggestion of new terms to the user which can to be added to the original query to guide the search towards a more specific document space. The first case is called automatic query expansion whereas the second case is called semi-automatic queryexpansion. In [44], the authors tried to evaluate and compare the efficiency of the two methods. Despite the fact that their experiments were based on simulations and not on real human users in most of the cases, the results of the experiments showed that the interactive query expansion method gave more control to the searcher who knows her utility better than any automated system. Researchers also turned to methods such as lexical cooccurrence [45]. Lexical co-occurrence is the process of developing relationships between words based upon their co-occurrence in documents. The similarity of the method we have proposed here with lexical co-occurrence is that the source, which provides the candidate terms for expansion, is the set of the retrieved documents as opposed to some knowledge structure as in thesaurus-based approaches. As a consequence, if the user chooses terms that do not yield results from the expected domain, the terms suggested by the query-expansion algorithm are unlikely to be helpful to the user. A solution may be a simple spell-checker.

Supporting E-Health Information Seekers: From Simple Strategies to Knowledge-Based Methods 59

We have presented in this chapter useful methods to help health information seekers to find resources on the Internet which is the most popular way used nowadays. The experiences were carried out on the CISMeF catalogue in French, but are reproducible for other e-health applications in other languages. These methods include simple ones such as heuristics and spell-checking, and more sophisticated ones such as knowledge extraction from e-health

[1] Hou J, Zhang Y. Effectively finding relevant web pages from linkage information. IEEE

[2] Liu, B. Web data mining: exploring hyperlinks, contents and usage data. Springer.

[3] Keselman A, Browne AC, Kaufman DR. Consumer health information seeking as hypothesis testing. Journal of American Medical Informatics Association 2008;

[4] Koch T. Quality-controlled subject gateways: definitions, typologies, empirical

[5] Abad Garcia F. A comparative study of six European databases of medicallyoriented web resources. Journal of the Medical Library Association 2005; 93(4):467–

[6] Douyère M, Soualmia LF, Rogozan A, Dahamna B, Leroy JP, Thirion B, Darmoni SJ. Enhancing the MeSH thesaurus to retrieve French online health resources in a quality-controlled gateway. Health Information Libraries Journal 2004; 21(4):253–

[8] Nelson SJ, Johnson WD, Humphreys BL. Relationships in Medical Subject Heading. In: Relationships in the Organization of Knowledge, 2001, eds. Kluwer Academic

[9] Soualmia LF, Darmoni SJ. Combining Different Standards and Different Approaches for Health Information Retrieval in a Quality-controlled Gateway. International Journal of

[10] McCray AT, Ide NC, Loane RR, Tse T. Strategies for supporting consumer health information seeking*.* Proceedings of the 11th World Congress on Health Informatics,

Transactional Knowledge Data Engineering 2003; 15(4), 940–951.

overview. Online Information Review 2000; 24(1):24–34.

[7] Baker T. A grammar of Dublin Core. D-Lib Magazine 2000; 6(10).

Medical Informatics 2005; N°74; vol (2-4); pp. 141–150.

**6. Conclusions**

documents.

**Author details**

**7. References**

2007

479.

261.

Publishers, pp. 171–184.

Medinfo 2004; pp.1152–1156.

51(4):484–495.

Lina F. Soualmia, Badisse Dahamna and Stéfan J. Darmoni *CISMeF & TIBS-LITIS EA 4108, Rouen University &Hospital, France*

#### **5.6. Evaluating query expansion based on association rules**

Many ways of navigation and information retrieval are possible in the catalogue. The most used is the simple search (free text interface). As stated in the section 2, it is based on the subsumption relationships. A query (a word or an expression) can be matched with an existing concept. In this case, the result of the query is the union of the resources that are indexed by the concept, and the resources that are indexed by the concepts it subsumes, directly or indirectly, in all of the hierarchies it belongs to. The co-occurrence tools developed for information retrieval bring the terms which frequently appear in the same documents closer together. These terms thus have a semantic proximity. This technique was used very early to allow query expansion. By analogy, association rules may be exploited in a search engine by carrying out an interactive query expansion. This helps the user to formulate his query by using the result of a query to reformulate, filter and re-orientate the query by exploiting the terms related to his query terms. In fact, the user can select suggested terms sets to add them to his initial query. It is useful in the case of non-precise information needs. IQE requires user implication. We developed a web-based evaluation tool of the IQE used by a set of 500 users which are subscribers of the weekly letter "What's new" of CISMeF. 20 queries, and for each one a set of medical terms derived from the extracted association rules were proposed. The evaluation was performed thanks to a Likert scale. The results (76% of the users were satisfied by the propositions) demonstrate the usefulness of this approach. An expanded query by association rules contains more related terms. By using the vectorial model, for example, more documents will be located and this treatment increases recall. In addition, association rules are indication on the possible definition of a term or its context.

#### **6. Conclusions**

58 eHealth and Remote Monitoring

simple spell-checker.

**5.6. Evaluating query expansion based on association rules**

possible definition of a term or its context.

augmentation of query terms to improve the retrieval process without user intervention. The second is the suggestion of new terms to the user which can to be added to the original query to guide the search towards a more specific document space. The first case is called automatic query expansion whereas the second case is called semi-automatic queryexpansion. In [44], the authors tried to evaluate and compare the efficiency of the two methods. Despite the fact that their experiments were based on simulations and not on real human users in most of the cases, the results of the experiments showed that the interactive query expansion method gave more control to the searcher who knows her utility better than any automated system. Researchers also turned to methods such as lexical cooccurrence [45]. Lexical co-occurrence is the process of developing relationships between words based upon their co-occurrence in documents. The similarity of the method we have proposed here with lexical co-occurrence is that the source, which provides the candidate terms for expansion, is the set of the retrieved documents as opposed to some knowledge structure as in thesaurus-based approaches. As a consequence, if the user chooses terms that do not yield results from the expected domain, the terms suggested by the query-expansion algorithm are unlikely to be helpful to the user. A solution may be a

Many ways of navigation and information retrieval are possible in the catalogue. The most used is the simple search (free text interface). As stated in the section 2, it is based on the subsumption relationships. A query (a word or an expression) can be matched with an existing concept. In this case, the result of the query is the union of the resources that are indexed by the concept, and the resources that are indexed by the concepts it subsumes, directly or indirectly, in all of the hierarchies it belongs to. The co-occurrence tools developed for information retrieval bring the terms which frequently appear in the same documents closer together. These terms thus have a semantic proximity. This technique was used very early to allow query expansion. By analogy, association rules may be exploited in a search engine by carrying out an interactive query expansion. This helps the user to formulate his query by using the result of a query to reformulate, filter and re-orientate the query by exploiting the terms related to his query terms. In fact, the user can select suggested terms sets to add them to his initial query. It is useful in the case of non-precise information needs. IQE requires user implication. We developed a web-based evaluation tool of the IQE used by a set of 500 users which are subscribers of the weekly letter "What's new" of CISMeF. 20 queries, and for each one a set of medical terms derived from the extracted association rules were proposed. The evaluation was performed thanks to a Likert scale. The results (76% of the users were satisfied by the propositions) demonstrate the usefulness of this approach. An expanded query by association rules contains more related terms. By using the vectorial model, for example, more documents will be located and this treatment increases recall. In addition, association rules are indication on the We have presented in this chapter useful methods to help health information seekers to find resources on the Internet which is the most popular way used nowadays. The experiences were carried out on the CISMeF catalogue in French, but are reproducible for other e-health applications in other languages. These methods include simple ones such as heuristics and spell-checking, and more sophisticated ones such as knowledge extraction from e-health documents.

## **Author details**

Lina F. Soualmia, Badisse Dahamna and Stéfan J. Darmoni *CISMeF & TIBS-LITIS EA 4108, Rouen University &Hospital, France*

## **7. References**

[11] Grannis SJ, Overhag MJ, Mc Donald C: Real world performance of approximate string comparators for use in patient matching. Studies in Health Technology and Informatics 2004; 107:43–47.

Supporting E-Health Information Seekers: From Simple Strategies to Knowledge-Based Methods 61

[29] Moalla Z, Soualmia LF, Prieur-Gaston E, Lecroq T & Darmoni SJ. Spell-checking queries by combining Levenshtein and Stoilos distances. Proceedings of Network Tools and

[30] Wagner MM. An Automatic Indexing Method for Medical Documents. Symposium on

[31] Grosjean J, Merabti T, Dahamna B, Kergourlay I, Thirion B, Soualmia LF & Darmoni SJ*.* Health Multi-Terminology Portal: a semantics added-value for patient safety. Studies in

[32] Soualmia LF, Griffon N, Grosjean J & Darmoni SJ. Improving Information Retrieval by Meta-modelling Medical Terminologies. Proceedings of 13th Conference on Artificial

[33] Gehanno JF, Thirion B, Darmoni SJ. Evaluation of Meta-Concepts for Information Retrieval in a Quality-Controlled Health Gateway. In: Proceedings of the American

[34] Trombert-Paviot B, Rodrigues JM, Rogers JE, Baud R, van der Haring E, Rassinoux AM, Abrial V, Clavel L, Idir H. GALEN: a Third Generation Terminology Tool to Support a Multipurpose National Coding System for Surgical Procedures. In International Journal

[35] Spink A, Wolfram D, Jansen BJ & Saracevic T. Searching the web: the public and their queries. Journal of the American Society Information Science Technology 2002,

[36] Agrawal R, Imielinski T & Swami AN. Mining association rules between sets of items in large databases. In Proceedings of the ACM SIGMOD International Conference on

[37] Agrawal R. & Srikant R. Fast algorithms for mining association rules in large databases.

[38] Pasquier N, Taouil R, Bastide Y, Stumme G & Lakhal L. Generating a condensed representation of association rules. Journal of Intelligent Information Systems 2005,

[39] Fayyad UM, Piatetsky-Shapiro GP, Smyth P & Uthurusamy R. Advances in Knowledge Discovery and Data Mining 1996. American Association of Artificial

[40] Kahng J, Liao WHK & McLeod D. Mining generalized term associations: count propagation algorithm. In Proceedings of the KDD workshop 1997, pp.203–206. [41] Prince V & Roche M. Information Retrieval in Biomedicine: Natural Language

[42] Cui H, Wen JR & Ma WY. Query expansion and classification by mining user logs.

[43] Lu Z. & Wilbur WJ. Improving Accuracy for identifying related PubMed queries by an

integrated approach. Journal of Biomedical Informatics 2009; 42:831–838.

Applications in Biology 2011, online.

Computer Application in Medical Care. 1991:1011–1017.

Health Technology and Informatics 2011, 166:129-138.

Medical Informatics Association symposium 2007; pp. 269–273.

In Proceedings of the VLDB Conference 1994; pp. 478−499.

Processing for Knowledge Integration 2009. IGI Global.

Knowledge and Data Engineering 2003, 15 (4):829–839.

Intelligence in MEdicine 2011, 6747:215–219.

of Medical Informatics 2000; 58-59: 71–85.

Management of Data 2003; 207–216.

52(3):226–234.

24(1):29–60.

Intelligence.

[29] Moalla Z, Soualmia LF, Prieur-Gaston E, Lecroq T & Darmoni SJ. Spell-checking queries by combining Levenshtein and Stoilos distances. Proceedings of Network Tools and Applications in Biology 2011, online.

60 eHealth and Remote Monitoring

2004; 107:43–47.

29(7):633–637.

1992, 24(4):377-439.

Informatics 2001, 64(2–3):253–258.

Informatics Association 2004, 11(3):179–185.

Information retrieval 2006. 9:543–564.

Soviet Physics Dokl 1966; 10:707–710.

[11] Grannis SJ, Overhag MJ, Mc Donald C: Real world performance of approximate string comparators for use in patient matching. Studies in Health Technology and Informatics

[12] Levenshtein V. Binary codes capable of correcting deletions, insertions and reversals.

[13] Yarkoni T, Balota D, Yap M. Moving beyond Coltheart's N: a new measure of

[14] Soualmia LF. Towards intelligent information retrieval with query expansion

[15] Hodge VJ, Austin J. A comparison of a novel neural spell checker and standard spell

[16] Damereau FJ. A technique for computer detection and correction of spelling errors. In

[17] Peterson LJ. A note on undetected typing errors. Communications of ACM 1986.

[18] Kuckich K. Techniques for automatically correcting words in text. ACM Comput Surv

[19] Kernigham M et al. A spelling correction program based on noisy channel model. In

[20] Brill E, Moore RC. An improved error model for noisy channel spelling correction. In

[21] Toutanova K, Moore RC. Pronunciation Modeling for Improved Spelling Correction. In

[22] Boyer C, Baujard V, Griesser V, Scherrer JR. HONselect: a multilingual and intelligent search tool integrating heterogeneous web resources. International Journal of Medical

[23] Crowell J, Long Ngo QNG, Lacroix E. A frequency-based technique to improve the spelling suggestion rank in medical queries. Journal of the American Medical

[24] Peters L, Kapunsik-Uner JE, Nguyen T, Bodenreider O. An approximate matching

[25] Wilbur JW, Kim W, Xie N. Spelling correction in the PubMed search engine.

[26] Stoilos G, Stamou G, Kollias S. A string metric for ontology alignment. In Proceedings

[27] Yujian L, Bo L. A normalized Levenshtein distance metric. IEEE Transactions on Pattern

[28] Winkler W. The state record linkage and current research problems. Technical report:

method for clinical drug name. AMIA Annual Symposium 2011, in press.

Statistics of Income Division, Internal Revenue Service Publication 1999.

of the International Semantic Web Conference, 2005; 624–637.

Analysis and Machine Intelligence 2007, 29(6):1091–1095.

proceedings of conference on COmputational LINGuistics, 1990. vol. 2.

proceedings of the Association for Comput. Linguistics 2000; 286–293.

proceedings of the Association for Comput. Linguistics 2002; 141–151.

orthographic similarity. Psychonomic Bulletin & Review 2008; 15(5):971–979.

knowledge-based methods. PhD thesis 2004; INSA Rouen.

Communication of the ACM 1964; 7(3):171–177.

checking algorithms. Pattern Recognition 2002, 11(35):2571–2580.

[44] Ruthven I. Reexamining the potential effectiveness of interactive query expansion. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval 2003; pp. 213–220.

**Chapter 4**

© 2012 Gagnon et al., licensee InTech. This is an open access chapter distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

© 2012 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution,

**Codified Knowledge and Decisions in**

**the Electronic Health Record in Quebec**

Duncan Sanderson, Marie-Pierre Gagnon and Julie Duplantie

has been reported that \$1.6 billion has been alloted to this effort [3].

generally failed to learn from the evaluations that had been commissioned.

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/50322

**1. Introduction**

**a Major eHealth Project: Efforts to Introduce**

Electronic health records (EHR) are being implemented in countries around the world, as the hope is that they will allow health personnel to have more timely access to information about patients and their medical history, and to improve quality of care. These objectives have translated into major expenditures and effort in order to try to implement national EHRs. The United States, for example, passed the Health Information Technology for Economic and Clinical Health (HITECH) provisions of the American Recovery and Reinvestment Act of 2009 [1]. In England, from 2003 to 2010, the government spent \$20.6 billion on the National Programme for Information Technology (NPfIT) [2]. In Canada, it

Along with expenditures and efforts to implement these systems, there has recently been an interest in the evaluation of national projects to implement an EHR, both in terms of the results that have been achieved and the road that has been taken [2-4]. Although evaluations of the implementation of national EHRs are still relatively scarce, there have been some remarkable efforts. For example, in the case of England, Greenhalgh et al. [2] note that the national EHR project was expensive and well behind its implementation schedule, and indicate that its success was limited. Furthermore, these authors point out that "policymakers appeared to overlook many of their recommendations and persisted with some of the NPfIT's most criticized components and implementation methods" [2](p. 533). To explain the limited uptake of their recommendations, they note that policymakers

For their part, Stroetmann et al. [4] identify good practices and lessons learned in European countries, while noting that progress has not been simple: "... implementing them (eHealth

and reproduction in any medium, provided the original work is properly cited.

[45] Vechtomova O, Robertson S. & Jones S. Query expansion with long-span collocates. Information Retrieval 2003, 6(2), 251–273.

## **Codified Knowledge and Decisions in a Major eHealth Project: Efforts to Introduce the Electronic Health Record in Quebec**

Duncan Sanderson, Marie-Pierre Gagnon and Julie Duplantie

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/50322

## **1. Introduction**

62 eHealth and Remote Monitoring

[44] Ruthven I. Reexamining the potential effectiveness of interactive query expansion. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and

[45] Vechtomova O, Robertson S. & Jones S. Query expansion with long-span collocates.

Development in Information Retrieval 2003; pp. 213–220.

Information Retrieval 2003, 6(2), 251–273.

Electronic health records (EHR) are being implemented in countries around the world, as the hope is that they will allow health personnel to have more timely access to information about patients and their medical history, and to improve quality of care. These objectives have translated into major expenditures and effort in order to try to implement national EHRs. The United States, for example, passed the Health Information Technology for Economic and Clinical Health (HITECH) provisions of the American Recovery and Reinvestment Act of 2009 [1]. In England, from 2003 to 2010, the government spent \$20.6 billion on the National Programme for Information Technology (NPfIT) [2]. In Canada, it has been reported that \$1.6 billion has been alloted to this effort [3].

Along with expenditures and efforts to implement these systems, there has recently been an interest in the evaluation of national projects to implement an EHR, both in terms of the results that have been achieved and the road that has been taken [2-4]. Although evaluations of the implementation of national EHRs are still relatively scarce, there have been some remarkable efforts. For example, in the case of England, Greenhalgh et al. [2] note that the national EHR project was expensive and well behind its implementation schedule, and indicate that its success was limited. Furthermore, these authors point out that "policymakers appeared to overlook many of their recommendations and persisted with some of the NPfIT's most criticized components and implementation methods" [2](p. 533). To explain the limited uptake of their recommendations, they note that policymakers generally failed to learn from the evaluations that had been commissioned.

For their part, Stroetmann et al. [4] identify good practices and lessons learned in European countries, while noting that progress has not been simple: "... implementing them (eHealth

© 2012 Gagnon et al., licensee InTech. This is an open access chapter distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2012 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

strategies) has proven to be much more complex and time-consuming than initially anticipated. In addition, the complexity of eHealth as a management challenge has also been vastly underestimated" [4](p.1349).

Codified Knowledge and Decisions in a Major eHealth Project: Efforts to Introduce the Electronic Health Record in Quebec 65

which the available knowledge influenced decision-makers (or was available and did not have an influence), and we will explore the reasons for which this may or may not have

Our presentation and analysis centre on decisions, projects, and events, along with aspects of the political, technological, and social context in Quebec, Canada. As we have indicated earlier, because of provincial jurisdiction in the area of health in Canada, significant decisions were made in this area by the provincial government, and for this reason, the province is a useful unit of analysis. The Federal government also funded some projects, and these too represented significant decisions. These are the main decisions that will be considered here, although we also recognize that many other decisions took place and likely affected the projects, such as the choice of partners, the creation and composition of steering committees, the project objectives, the technology used, as well as implementation, training,

The analysis is based on a multiple case study approach [5] which examined the evolution of representative EHR projects in the Province of Quebec from the first large scale pilot project at the beginning of the 1990's until just before the development of the current Quebec EHR project in the late 2000s (le Dossier santé du Québec, or DSQ). Although this approach limits the amount of detail that we can provide for any one project or event, we believe that this longer historical view is also useful. Because of the scope of the current project (the DSQ) and ongoing analysis of it (for example, annual reports by the Auditor General of Quebec), we will not include it here, except for an early initiative that was part of it. Further analysis

For the analysis here and because of space limitations, a subset of projects or actions were chosen for this chapter which are either representative of similar projects or which, retrospectively, can be considered to have been a significant event (see Table 1). The choice of projects and activities that are discussed here was reached by consensus among the authors. The projects selected also allow us to present and discuss several factors which

By codified knowledge, we mean knowledge that has been written down in the form of reports or articles. The particular type that we are mostly concerned with here is scientific evidence that has been produced by evaluation studies of projects, or research into the actual benefits of an EHR, or factors that affect the implementation of EHR projects. Other pertinent codified knowledge may have been produced in fields other than scientific research. In the case of the EHR, an important area of knowledge consists of an understanding of legislation that controls the storage or transmission of a patient's clinical information. Rather than to stretch the notion of scientific evidence in order to accommodate this type of knowledge, we prefer the more general expression of codified knowledge.

For each case study, we analyzed public descriptions, evaluations, and reports concerning the set of projects and events. This information was supplemented by semi-structured

may eventually find that many of our conclusions also apply to it.

influenced the uptake of codified knowledge by the EHR projects in Quebec.

occurred.

**2. Methodology**

and recruitment strategies.

In Canada, a similar broad assessment effort has also occurred recently [3]. A short explanation of certain aspects of the Canadian health system, as they relate to the EHR, could be useful at this point. Canada is a federation of provinces and territories, which are primarily responsible for providing health services to the citizens living in their geographic territory. The individual health organizations (private clinics, public hospitals) have a main role in maintaining the patient's record, although each provincial Ministry of Health has oversight over the public hospitals and, to a lesser extent, some influence on private physician's practices. The provincial and federal Ministries of Health set policy and decide if they want to support EHR projects. However, at the turn of the century (2001), the federal government wanted to insure that patient record systems would be compatible so that a citizen who moved from one province to another could still provide a record of treatment to his or her new physician. This resulted in an organization (Infoway) and a large federal fund to facilitate the development of interoperable systems. This initiative will be further described in a particular section below.

If we return now to the recent analysis of this federal initiative (Infoway), Rozenblum et al. [3] observe that "Canada continues to lag behind other Western countries in adopting electronic health records" [3]. These authors report that, across the provinces and for 2009, only 36% of physicians were using an EHR, and that this was much lower than for countries such as Australia, the United Kingdom, New Zealand, and the Netherlands. The authors carried out a set of telephone interviews of key informants from three provinces across Canada (not including Quebec), and these informants indicated a number of reasons for the poor rate of implementation: lack of an e-health policy (federally and in concert with the provinces), inadequate involvement of clinicians, failure to establish a business case for using electronic health records, a focus on national rather than regional interoperability, and inflexibility in approach.

In this chapter we wish to take a different approach, by examining decisions that have been made during the process of implementing a provincial EHR and the influence that codified knowledge may have had on those decisions. Our general goal is to document, and if possible, gain deeper understanding of the reasons for which scientific evidence in particular may be considered or not in EHR projects. We will argue that in Quebec, but probably not unique to it, key decisions regarding EHR implementation have not always been based on available knowledge, and this could help to explain the limited success of EHR projects here and perhaps elsewhere.

Currently, under the label of evidence-based medicine, there is substantial interest in, and effort by, health researchers to examine empirical evidence in order to identify actions and conditions that are most likely to result in positive outcomes for patients. This chapter builds on this strategy. Where possible, we will point out certain research results, expertise, and evaluation conclusions (which we will call the codified knowledge) that were available to decision-makers at the time of their decisions. We will identify specific occasions during which the available knowledge influenced decision-makers (or was available and did not have an influence), and we will explore the reasons for which this may or may not have occurred.

## **2. Methodology**

64 eHealth and Remote Monitoring

vastly underestimated" [4](p.1349).

described in a particular section below.

EHR projects here and perhaps elsewhere.

inflexibility in approach.

strategies) has proven to be much more complex and time-consuming than initially anticipated. In addition, the complexity of eHealth as a management challenge has also been

In Canada, a similar broad assessment effort has also occurred recently [3]. A short explanation of certain aspects of the Canadian health system, as they relate to the EHR, could be useful at this point. Canada is a federation of provinces and territories, which are primarily responsible for providing health services to the citizens living in their geographic territory. The individual health organizations (private clinics, public hospitals) have a main role in maintaining the patient's record, although each provincial Ministry of Health has oversight over the public hospitals and, to a lesser extent, some influence on private physician's practices. The provincial and federal Ministries of Health set policy and decide if they want to support EHR projects. However, at the turn of the century (2001), the federal government wanted to insure that patient record systems would be compatible so that a citizen who moved from one province to another could still provide a record of treatment to his or her new physician. This resulted in an organization (Infoway) and a large federal fund to facilitate the development of interoperable systems. This initiative will be further

If we return now to the recent analysis of this federal initiative (Infoway), Rozenblum et al. [3] observe that "Canada continues to lag behind other Western countries in adopting electronic health records" [3]. These authors report that, across the provinces and for 2009, only 36% of physicians were using an EHR, and that this was much lower than for countries such as Australia, the United Kingdom, New Zealand, and the Netherlands. The authors carried out a set of telephone interviews of key informants from three provinces across Canada (not including Quebec), and these informants indicated a number of reasons for the poor rate of implementation: lack of an e-health policy (federally and in concert with the provinces), inadequate involvement of clinicians, failure to establish a business case for using electronic health records, a focus on national rather than regional interoperability, and

In this chapter we wish to take a different approach, by examining decisions that have been made during the process of implementing a provincial EHR and the influence that codified knowledge may have had on those decisions. Our general goal is to document, and if possible, gain deeper understanding of the reasons for which scientific evidence in particular may be considered or not in EHR projects. We will argue that in Quebec, but probably not unique to it, key decisions regarding EHR implementation have not always been based on available knowledge, and this could help to explain the limited success of

Currently, under the label of evidence-based medicine, there is substantial interest in, and effort by, health researchers to examine empirical evidence in order to identify actions and conditions that are most likely to result in positive outcomes for patients. This chapter builds on this strategy. Where possible, we will point out certain research results, expertise, and evaluation conclusions (which we will call the codified knowledge) that were available to decision-makers at the time of their decisions. We will identify specific occasions during Our presentation and analysis centre on decisions, projects, and events, along with aspects of the political, technological, and social context in Quebec, Canada. As we have indicated earlier, because of provincial jurisdiction in the area of health in Canada, significant decisions were made in this area by the provincial government, and for this reason, the province is a useful unit of analysis. The Federal government also funded some projects, and these too represented significant decisions. These are the main decisions that will be considered here, although we also recognize that many other decisions took place and likely affected the projects, such as the choice of partners, the creation and composition of steering committees, the project objectives, the technology used, as well as implementation, training, and recruitment strategies.

The analysis is based on a multiple case study approach [5] which examined the evolution of representative EHR projects in the Province of Quebec from the first large scale pilot project at the beginning of the 1990's until just before the development of the current Quebec EHR project in the late 2000s (le Dossier santé du Québec, or DSQ). Although this approach limits the amount of detail that we can provide for any one project or event, we believe that this longer historical view is also useful. Because of the scope of the current project (the DSQ) and ongoing analysis of it (for example, annual reports by the Auditor General of Quebec), we will not include it here, except for an early initiative that was part of it. Further analysis may eventually find that many of our conclusions also apply to it.

For the analysis here and because of space limitations, a subset of projects or actions were chosen for this chapter which are either representative of similar projects or which, retrospectively, can be considered to have been a significant event (see Table 1). The choice of projects and activities that are discussed here was reached by consensus among the authors. The projects selected also allow us to present and discuss several factors which influenced the uptake of codified knowledge by the EHR projects in Quebec.

By codified knowledge, we mean knowledge that has been written down in the form of reports or articles. The particular type that we are mostly concerned with here is scientific evidence that has been produced by evaluation studies of projects, or research into the actual benefits of an EHR, or factors that affect the implementation of EHR projects. Other pertinent codified knowledge may have been produced in fields other than scientific research. In the case of the EHR, an important area of knowledge consists of an understanding of legislation that controls the storage or transmission of a patient's clinical information. Rather than to stretch the notion of scientific evidence in order to accommodate this type of knowledge, we prefer the more general expression of codified knowledge.

For each case study, we analyzed public descriptions, evaluations, and reports concerning the set of projects and events. This information was supplemented by semi-structured

interviews with decision-makers, which allowed us to explore the context in which decisions were made, how available knowledge was actually used (or not) to make decisions, and factors which influenced its use. A total of 31 interviews were completed with key decision-makers who were selected because of their involvement in EHR projects either as policy-makers, evaluators, or project leaders. For the first case study, one of the authors (JPF) co-directed the evaluation of it. In relation to the final case study, two of the authors (MPG and JD) observed several meetings between December 2007 and March 2009 and conducted informal interviews with members of a consulting team.

Codified Knowledge and Decisions in a Major eHealth Project: Efforts to Introduce the Electronic Health Record in Quebec 67

A group of academic experts was set up in order to plan the implementation of the pilot smart health card project. They consulted the literature (which was scarce at the time), along with a research group that had implemented the technology in France [10]. The group in Quebec included a variety of researchers from various fields (health services, political science, evaluation, information technology), as well as medical experts. Significant consideration was given to clinical processes and inter-professional collaboration. A local physician was also involved in the project team as a strategy in order to increase the acceptance by the medical community. The project team used knowledge gained from European projects in particular, to inform the implementation of this pilot project. The project was envisioned to be a way to produce new knowledge about the benefits of the smart card for the health care system, but also to help develop strategies that would be

A specific health card was used for this project, and had a smart chip on it that could record information. The concept was essentially to create a portable personal medical record. For example, patients' allergies or reactions to medication could be noted on the card. Citizens who wanted to participate had to apply and consent to the project (this consent could be withdrawn later), and an office was created to facilitate this. Health professionals needed a personal identification number (PIN) in order to access information on the card, as well as a card reader and IBM microcomputer. It is useful to remember that, at the time, pharmacists often had experience using a computer, but this was not the case for physicians or nurses. When a patient went to visit a health professional, they could present the card and so the information on the card could be shown to him or her, and if suitably equipped and inclined, the professional could in turn add to it. As well, a computer application was offered to pharmacists so that they could check for potential drug interactions. At the end of the project, there were 7248 users (a portion of the city's population), along with 299 physicians, pharmacists, and nurses (almost all of the health professionals who had been

Overall, 73% of the population who had a card used it. Among these patients, about half used it with a pharmacist and a physician [7]. One of the decision aids that was most used (often by pharmacists) was an application that checked for potential drug interactions. The rate of use increased until July 1994, and then decreased, and this coincided with a decision to not buy more smart cards for the project. As well, there were indications that the use of two different computer systems was an irritant, that use of the card increased the time needed for a consultation, and there were occasional problems updating information on the card. In the summer of 1995, a sample of patients and health professionals were polled, and they generally indicated that the card facilitated the communication of health information, in a secure manner. In spite of the problem related to the need to update information, more than half of the health professionals felt that it provided quicker access to information, allowed them to provide better recommendations, and were satisfied with its quality. As well, many indicated that certain characteristics of the project, in particular the slowness of

Some of the professionals felt that the smart health card provided benefits that out-weighed the irritants and disadvantages, whereas for others, the advantages were not sufficient. Most

useful for a future large scale implementation.

identified as potential users) [7].

the system, did not facilitate its systematic use.

For each project or activity discussed, we will briefly note its objectives, the major decisions that were made, whether certain forces may have influenced the decisions, the types of difficulties the project or activity may have experienced, and the knowledge that was gained from it. The following table identifies the projects and activities that will be presented.

**Table 1.** Major Projects and Activities in Quebec, on the Road to an EHR

## **3. Presentation of the case studies**

## **3.1. The Rimouski smart health card project**

One of the earliest electronic health record projects in Quebec took place in a small city in Quebec, Rimouski, from 1993 to 1995. This was one of the first large-scale experiments of this type in the world, although it built on a similar concept that was active in France at the time (for a later description of a similar concept, see [6]). The Quebec government decided to encourage the project, provided financial assistance to it (4 million dollars Canadian), and also sponsored an evaluation [7]. The RAMQ (the provincial organization which manages health insurance, pays private physicians for their interventions, and issues health cards) was the main project manager and associated several stakeholders such as professional associations. Proponents of the project hoped that it would help to meet the objectives of the dominant ideology of the time, which was a shift to ambulatory care (see, for example [8]), and the creation of a better continuum of services by the set of professionals that a patient could see. A report from an important consultation on the reform of health care in Quebec (known as the *Commission Rochon* [9]) had also recommended that a trial be carried out with the card. Two implicit sub-objectives of the experiment were to decrease drug interactions, and to decrease the duplication of lab tests.

A group of academic experts was set up in order to plan the implementation of the pilot smart health card project. They consulted the literature (which was scarce at the time), along with a research group that had implemented the technology in France [10]. The group in Quebec included a variety of researchers from various fields (health services, political science, evaluation, information technology), as well as medical experts. Significant consideration was given to clinical processes and inter-professional collaboration. A local physician was also involved in the project team as a strategy in order to increase the acceptance by the medical community. The project team used knowledge gained from European projects in particular, to inform the implementation of this pilot project. The project was envisioned to be a way to produce new knowledge about the benefits of the smart card for the health care system, but also to help develop strategies that would be useful for a future large scale implementation.

66 eHealth and Remote Monitoring

interviews with decision-makers, which allowed us to explore the context in which decisions were made, how available knowledge was actually used (or not) to make decisions, and factors which influenced its use. A total of 31 interviews were completed with key decision-makers who were selected because of their involvement in EHR projects either as policy-makers, evaluators, or project leaders. For the first case study, one of the authors (JPF) co-directed the evaluation of it. In relation to the final case study, two of the authors (MPG and JD) observed several meetings between December 2007 and March 2009 and

For each project or activity discussed, we will briefly note its objectives, the major decisions that were made, whether certain forces may have influenced the decisions, the types of difficulties the project or activity may have experienced, and the knowledge that was gained from it. The following table identifies the projects and activities that will be presented.

One of the earliest electronic health record projects in Quebec took place in a small city in Quebec, Rimouski, from 1993 to 1995. This was one of the first large-scale experiments of this type in the world, although it built on a similar concept that was active in France at the time (for a later description of a similar concept, see [6]). The Quebec government decided to encourage the project, provided financial assistance to it (4 million dollars Canadian), and also sponsored an evaluation [7]. The RAMQ (the provincial organization which manages health insurance, pays private physicians for their interventions, and issues health cards) was the main project manager and associated several stakeholders such as professional associations. Proponents of the project hoped that it would help to meet the objectives of the dominant ideology of the time, which was a shift to ambulatory care (see, for example [8]), and the creation of a better continuum of services by the set of professionals that a patient could see. A report from an important consultation on the reform of health care in Quebec (known as the *Commission Rochon* [9]) had also recommended that a trial be carried out with the card. Two implicit sub-objectives of the experiment were to decrease drug interactions,

The Rimouski Smart Health Card Project 1993-1995 The Role of the *Commision d'accès à l'information* (CAI) 1993 - *Canada Health Infostructure Partnerships Program* 2001-2003 Changes in the Provincial Laws 2001, 2005 Canada Health Infoway (inclusion of Quebec) 2004 - The *RSVP*: a change management strategy 2007-2009

Project/Activity Approximate Time Period

conducted informal interviews with members of a consulting team.

**Table 1.** Major Projects and Activities in Quebec, on the Road to an EHR

**3. Presentation of the case studies**

and to decrease the duplication of lab tests.

**3.1. The Rimouski smart health card project**

A specific health card was used for this project, and had a smart chip on it that could record information. The concept was essentially to create a portable personal medical record. For example, patients' allergies or reactions to medication could be noted on the card. Citizens who wanted to participate had to apply and consent to the project (this consent could be withdrawn later), and an office was created to facilitate this. Health professionals needed a personal identification number (PIN) in order to access information on the card, as well as a card reader and IBM microcomputer. It is useful to remember that, at the time, pharmacists often had experience using a computer, but this was not the case for physicians or nurses. When a patient went to visit a health professional, they could present the card and so the information on the card could be shown to him or her, and if suitably equipped and inclined, the professional could in turn add to it. As well, a computer application was offered to pharmacists so that they could check for potential drug interactions. At the end of the project, there were 7248 users (a portion of the city's population), along with 299 physicians, pharmacists, and nurses (almost all of the health professionals who had been identified as potential users) [7].

Overall, 73% of the population who had a card used it. Among these patients, about half used it with a pharmacist and a physician [7]. One of the decision aids that was most used (often by pharmacists) was an application that checked for potential drug interactions. The rate of use increased until July 1994, and then decreased, and this coincided with a decision to not buy more smart cards for the project. As well, there were indications that the use of two different computer systems was an irritant, that use of the card increased the time needed for a consultation, and there were occasional problems updating information on the card. In the summer of 1995, a sample of patients and health professionals were polled, and they generally indicated that the card facilitated the communication of health information, in a secure manner. In spite of the problem related to the need to update information, more than half of the health professionals felt that it provided quicker access to information, allowed them to provide better recommendations, and were satisfied with its quality. As well, many indicated that certain characteristics of the project, in particular the slowness of the system, did not facilitate its systematic use.

Some of the professionals felt that the smart health card provided benefits that out-weighed the irritants and disadvantages, whereas for others, the advantages were not sufficient. Most

found it relatively easy to use, although the system was found to be too slow. In general, the physicians and pharmacists indicated that the system contributed to a decrease in drug interactions [7].

Codified Knowledge and Decisions in a Major eHealth Project: Efforts to Introduce the Electronic Health Record in Quebec 69

(http://www.cai.gouv.qc.ca/index-en.html). Health and social service institutions, which created databases that contained health information, fell under the mandate of this organization. Furthermore, the CAI was authorized to comment on draft bills in this area

Because of this mandate, the CAI decided to analyze various EHR pilot projects that took place in Quebec from 1990 to approximately 2005, as well as on proposed provincial legislation. To be clear, it was a decision-maker in this way, since it opted to comment on the EHR projects and hoped to steer them in a way that would allow them to be compliant with various laws, but it did not have a direct role in the selection of projects which were carried out. As an example of one activity, it wrote a report on the Smart Card and its trial in Rimouski [11]. Although the CAI considered the approaches taken in this project to be useful, problems with the implementation of the card were also noted, such as allowing nurses and pharmacists to share a card, and the fact that it consisted of a partial patient

A later, more general study carried out by the CAI in 2001 summarizes their examination of several pilot projects and outlines their general concern: "The transfer of clinical information and the right to privacy are not contradictory. However, their conciliation is complex and fragile, especially when they are implemented" [12](our translation, p. 2). A particular concern was also presented in this report, in that the authors indicate that the provincial health insurance organization, the RAMQ, should not be the organization that handles a centralized health record, and it discussed the reasons that motivated it to take this position. This 2001 report also summarized the state of legislation in this area: "In relation to the laws that relate to the protection of personal health information, these are numerous, haphazard, sometimes contradictory, and often insufficient in relation to these new health technologies" [12](p. 35). An analysis of the state of the current legislation (in 2001) was also carried out. At the end of this analysis, several questions were raised about the legislative foundation for certain practices concerning the communication of information, such as who would be responsible for the information that is collected, and the regulations that would control this. The obvious call for further legislation which was contained in this report eventually lead to

some proposed laws, and to subsequent analysis by the CAI of these proposed laws.

database that would document the medications used by citizens.

In general, one could conclude that the work of the CAI had an influence on the legislation that framed the creation of the current provincial EHR project. For example, in 2004, a law was proposed that would have modified the main law in Quebec that governs health and social services, and the CAI provided an analysis of this [13]. One of the recommendations in it was that staff in a health organization should not be able to provide patient information to staff in other health organizations without the consent of the patient. As well, the authors recommended that the provincial health insurance organization (RAMQ) should not create a

The fact that the CAI examined several projects and proposed laws over an extended period of time indicates that senior managers in the CAI decided that this was an area that needed to be examined and monitored. It was also clear that this organization was sufficiently

and to assess pilot projects.

record.

The evaluation report included a number of recommendations. For example, such a system needed to be compatible with a computer system in a physician's office or a pharmacy, so as to minimize work for the professional. As noted previously, the information provided by a system should also be up-to-date, and consideration should be given to ways to encourage patients to adopt the behavior that project planners hoped they would have, such as presenting the card (this affected whether the information was indeed up-to-date). Local support for the users was also indicated to be a critical factor, as well as the implication of a local leader. As well, the confidentiality of the health information had to be insured.

In a later analysis of the project, Aubert and Bernard [11] add that the training requirements of the health professionals was a significant aspect of the project. As well, they note that physicians' offices were generally not equipped with computers. In addition, according to these authors, reflection about changes in clinical practices that could maximize the use of such technology seemed to be absent (although it was clear that the project hoped to decrease drug interactions, and reduce the number of lab exams).

Although the evaluation included the identification of actions which could contribute to the generalization of the project elsewhere, there was no such immediate follow-up. A later project in the health region of Laval (north of Montreal), from 1999 to 2001, used another smart card, although patient health information was stored in a database managed by the RAMQ. The Quebec government considered the smart card to be a viable approach, and continued to consider its potential use as late as 2002. In particular, the RAMQ announced in 2001 that within three years it would provide the card to 80% of the population [11].

#### **3.2. The role of the Commision d'Accès à l'Information (CAI)**

In Quebec, the CAI (Commission of Access to Information; our translation) was created in 1982, and had a significant influence on events and practices in relation to the EHR. In this section we will describe this organization and some of its activities in relation to electronic health records, and briefly comment on its influence in relation to the development of the EHR in Quebec. From the outset, we should note that this organization was not associated with a single electronic patient record project, but rather several of them. As well, the information it produced was not scientific evidence in the sense that we are generally using it here, but could be considered to have produced pertinent codified knowledge. As we hope to show, the organization produced information of a particular kind, but nevertheless, information that project proponents and the provincial government had to consider.

In 1982, a law was passed in Quebec, which had an objective of instituting better management and protection of information concerning individuals, which was held by the government. An organization was created at the same time to administer this law (the CAI) and it was "responsible for overseeing compliance with the obligations imposed upon public bodies concerning the collection, storage, use and communication of personal information" (http://www.cai.gouv.qc.ca/index-en.html). Health and social service institutions, which created databases that contained health information, fell under the mandate of this organization. Furthermore, the CAI was authorized to comment on draft bills in this area and to assess pilot projects.

68 eHealth and Remote Monitoring

interactions [7].

found it relatively easy to use, although the system was found to be too slow. In general, the physicians and pharmacists indicated that the system contributed to a decrease in drug

The evaluation report included a number of recommendations. For example, such a system needed to be compatible with a computer system in a physician's office or a pharmacy, so as to minimize work for the professional. As noted previously, the information provided by a system should also be up-to-date, and consideration should be given to ways to encourage patients to adopt the behavior that project planners hoped they would have, such as presenting the card (this affected whether the information was indeed up-to-date). Local support for the users was also indicated to be a critical factor, as well as the implication of a

In a later analysis of the project, Aubert and Bernard [11] add that the training requirements of the health professionals was a significant aspect of the project. As well, they note that physicians' offices were generally not equipped with computers. In addition, according to these authors, reflection about changes in clinical practices that could maximize the use of such technology seemed to be absent (although it was clear that the project hoped to

Although the evaluation included the identification of actions which could contribute to the generalization of the project elsewhere, there was no such immediate follow-up. A later project in the health region of Laval (north of Montreal), from 1999 to 2001, used another smart card, although patient health information was stored in a database managed by the RAMQ. The Quebec government considered the smart card to be a viable approach, and continued to consider its potential use as late as 2002. In particular, the RAMQ announced in

In Quebec, the CAI (Commission of Access to Information; our translation) was created in 1982, and had a significant influence on events and practices in relation to the EHR. In this section we will describe this organization and some of its activities in relation to electronic health records, and briefly comment on its influence in relation to the development of the EHR in Quebec. From the outset, we should note that this organization was not associated with a single electronic patient record project, but rather several of them. As well, the information it produced was not scientific evidence in the sense that we are generally using it here, but could be considered to have produced pertinent codified knowledge. As we hope to show, the organization produced information of a particular kind, but nevertheless,

2001 that within three years it would provide the card to 80% of the population [11].

information that project proponents and the provincial government had to consider.

In 1982, a law was passed in Quebec, which had an objective of instituting better management and protection of information concerning individuals, which was held by the government. An organization was created at the same time to administer this law (the CAI) and it was "responsible for overseeing compliance with the obligations imposed upon public bodies concerning the collection, storage, use and communication of personal information"

local leader. As well, the confidentiality of the health information had to be insured.

decrease drug interactions, and reduce the number of lab exams).

**3.2. The role of the Commision d'Accès à l'Information (CAI)**

Because of this mandate, the CAI decided to analyze various EHR pilot projects that took place in Quebec from 1990 to approximately 2005, as well as on proposed provincial legislation. To be clear, it was a decision-maker in this way, since it opted to comment on the EHR projects and hoped to steer them in a way that would allow them to be compliant with various laws, but it did not have a direct role in the selection of projects which were carried out. As an example of one activity, it wrote a report on the Smart Card and its trial in Rimouski [11]. Although the CAI considered the approaches taken in this project to be useful, problems with the implementation of the card were also noted, such as allowing nurses and pharmacists to share a card, and the fact that it consisted of a partial patient record.

A later, more general study carried out by the CAI in 2001 summarizes their examination of several pilot projects and outlines their general concern: "The transfer of clinical information and the right to privacy are not contradictory. However, their conciliation is complex and fragile, especially when they are implemented" [12](our translation, p. 2). A particular concern was also presented in this report, in that the authors indicate that the provincial health insurance organization, the RAMQ, should not be the organization that handles a centralized health record, and it discussed the reasons that motivated it to take this position.

This 2001 report also summarized the state of legislation in this area: "In relation to the laws that relate to the protection of personal health information, these are numerous, haphazard, sometimes contradictory, and often insufficient in relation to these new health technologies" [12](p. 35). An analysis of the state of the current legislation (in 2001) was also carried out. At the end of this analysis, several questions were raised about the legislative foundation for certain practices concerning the communication of information, such as who would be responsible for the information that is collected, and the regulations that would control this. The obvious call for further legislation which was contained in this report eventually lead to some proposed laws, and to subsequent analysis by the CAI of these proposed laws.

In general, one could conclude that the work of the CAI had an influence on the legislation that framed the creation of the current provincial EHR project. For example, in 2004, a law was proposed that would have modified the main law in Quebec that governs health and social services, and the CAI provided an analysis of this [13]. One of the recommendations in it was that staff in a health organization should not be able to provide patient information to staff in other health organizations without the consent of the patient. As well, the authors recommended that the provincial health insurance organization (RAMQ) should not create a database that would document the medications used by citizens.

The fact that the CAI examined several projects and proposed laws over an extended period of time indicates that senior managers in the CAI decided that this was an area that needed to be examined and monitored. It was also clear that this organization was sufficiently

independent from the elected government in order to critically examine the activities of the government. Since many managers of pilot projects informed the CAI of their projects and actively solicited its opinion about a project, this suggests that these project managers were concerned that their projects could potentially contravene existing legislation. At the time, managers felt that the rules were ambiguous about what a project needed to do in order to conform to existing legislation, and many meeting hours were spent discussing what to do about this problem.

Codified Knowledge and Decisions in a Major eHealth Project: Efforts to Introduce the Electronic Health Record in Quebec 71

During the course of the project, a number of factors were observed which influenced its development [16]. For example, the Minister of Health changed repeatedly during this period, and it was stated that these changes increased the level of uncertainty. As well, health professionals were not necessarily available to participate in the project, so consulting

One of the outcomes was a set of standardized clinical forms which were agreed to by health professionals, such as a form to assess patient autonomy. Another was that project proponents realized that it was difficult to comply completely with certain outdated laws, and they communicated this problem to provincial authorities. Still another outcome was that participants noted that significant resources were required in order to manage the change process. Observations were gathered about impacts on the work processes, and a recommendation was made to the effect that these had to be considered. Quite a few documents were produced by the project, which can be taken to be an indicator of a high

amount of consultation, discussion, and analysis which took place during the project.

It would appear that the SI-RIL project was one of the most advanced primary care EHR projects in Canada at the time. All other networked EHR projects in Quebec involved a small group of hospitals (there were a couple of networked hospital projects at the time). An evaluation of part of the SI-RIL project was also carried out by a research group [17]. For example, the authors highlighted the fact that these were very complex projects, and that any one of three types of problems (technological, organizational, clinical) could lead to the end of the project [17]. In addition, also noted in the report, was the desire of project participants to find a way to avoid illegal transmission or storage of patient information.

In the case of the RIGIC and SI-RIL projects, the regional health authority was closely integrated in the project planning and execution. We mention this since the regional health authority has regular communication with the provincial Ministry of Health. This highlights the fact that the provincial government was fully informed of the difficulties met by the proponents of the three projects, along with the outcomes, at least by 2002, when the project

Another significant dimension of these projects was that all of them were based on networked health information systems. Patient information was being made available to professionals in different organizations, without patient intervention. This fact contrasts

In December 2001, the provincial government proposed a law that would have created a provincial smart card and replaced the existing health card. It should be pointed out that the Laval region had also been involved in a second electronic health card project, which began in 1999, and this project was again sponsored by the RAMQ [12]. For those who proposed the law, this project provided an example of the type of technology that would be used. One of the technological differences with the Rimouski project was that a patient's clinical

with a concomitant provincial plan to develop an electronic patient card.

information was centralized in a databank managed by the RAMQ.

them was not an easy matter.

funding ended.

**3.4. Changes in the provincial laws**

#### **3.3. Canada Health Infostructure Partnerships Program**

At the turn of the millennium (2000-2002), the *Canada Health Infostructure Partnerships Program* (CHIPP) was a two-year, \$80 million, 50-50 shared-cost program with an objective of supporting collaboration, innovation, and renewal in health care delivery through the use of information and communication technologies. One of the specific areas that was funded was the creation of EHRs. Project applicants were required to complete a literature review, and to include an evaluation.

In Quebec, three EHR projects were funded through the CHIPP program. One of these, called MOXXI, or the Medical Office of the XXI century, was a research project to examine the potential benefits of an electronic prescription and drug management system for primary care physicians [14]. Two of the project objectives were to streamline the prescription process and to improve patient safety. Another aspect examined was the time requirements and other ergonomic issues related to the use of the system. One of the interesting questions implicit in this project was whether such a computerized system would be a help or a hindrance for the physician. This sort of question had generally not been asked before, and is still very rare in Canada. As one of the outcomes of the project, the authors concluded [15]: "However, there are considerable barriers to developing the interfaces necessary with community-based pharmacies to permit exchange of information between physicians and pharmacists" [15].

A second CHIPP project called RIGIC was essentially a project to create a networked EHR system which would be shared between two hospitals, for oncology patients. One of the outcomes of the project was that shared protocols (between the hospitals) had to be established in terms of access and confidentiality procedures. New policies and procedures also had to be developed in relation to this, including the development of a consent management system. As well, the participants became more aware of the importance of change management, and the need for recurrent funding in order to transition the research project into an ongoing operation.

A third project, the SI-RIL project, consisted of the development of an integrated network system in a health region (the health region of Laval, north of Montreal), for ambulatory patients. This system was designed to enable the sharing of physician orders and patient health information among health workers in the region, with a view to regional coordination of care. The participants were: the local hospital, four community health centres (CLSCs), and more than one hundred general physicians.

During the course of the project, a number of factors were observed which influenced its development [16]. For example, the Minister of Health changed repeatedly during this period, and it was stated that these changes increased the level of uncertainty. As well, health professionals were not necessarily available to participate in the project, so consulting them was not an easy matter.

One of the outcomes was a set of standardized clinical forms which were agreed to by health professionals, such as a form to assess patient autonomy. Another was that project proponents realized that it was difficult to comply completely with certain outdated laws, and they communicated this problem to provincial authorities. Still another outcome was that participants noted that significant resources were required in order to manage the change process. Observations were gathered about impacts on the work processes, and a recommendation was made to the effect that these had to be considered. Quite a few documents were produced by the project, which can be taken to be an indicator of a high amount of consultation, discussion, and analysis which took place during the project.

It would appear that the SI-RIL project was one of the most advanced primary care EHR projects in Canada at the time. All other networked EHR projects in Quebec involved a small group of hospitals (there were a couple of networked hospital projects at the time). An evaluation of part of the SI-RIL project was also carried out by a research group [17]. For example, the authors highlighted the fact that these were very complex projects, and that any one of three types of problems (technological, organizational, clinical) could lead to the end of the project [17]. In addition, also noted in the report, was the desire of project participants to find a way to avoid illegal transmission or storage of patient information.

In the case of the RIGIC and SI-RIL projects, the regional health authority was closely integrated in the project planning and execution. We mention this since the regional health authority has regular communication with the provincial Ministry of Health. This highlights the fact that the provincial government was fully informed of the difficulties met by the proponents of the three projects, along with the outcomes, at least by 2002, when the project funding ended.

Another significant dimension of these projects was that all of them were based on networked health information systems. Patient information was being made available to professionals in different organizations, without patient intervention. This fact contrasts with a concomitant provincial plan to develop an electronic patient card.

#### **3.4. Changes in the provincial laws**

70 eHealth and Remote Monitoring

about this problem.

and to include an evaluation.

between physicians and pharmacists" [15].

project into an ongoing operation.

(CLSCs), and more than one hundred general physicians.

independent from the elected government in order to critically examine the activities of the government. Since many managers of pilot projects informed the CAI of their projects and actively solicited its opinion about a project, this suggests that these project managers were concerned that their projects could potentially contravene existing legislation. At the time, managers felt that the rules were ambiguous about what a project needed to do in order to conform to existing legislation, and many meeting hours were spent discussing what to do

At the turn of the millennium (2000-2002), the *Canada Health Infostructure Partnerships Program* (CHIPP) was a two-year, \$80 million, 50-50 shared-cost program with an objective of supporting collaboration, innovation, and renewal in health care delivery through the use of information and communication technologies. One of the specific areas that was funded was the creation of EHRs. Project applicants were required to complete a literature review,

In Quebec, three EHR projects were funded through the CHIPP program. One of these, called MOXXI, or the Medical Office of the XXI century, was a research project to examine the potential benefits of an electronic prescription and drug management system for primary care physicians [14]. Two of the project objectives were to streamline the prescription process and to improve patient safety. Another aspect examined was the time requirements and other ergonomic issues related to the use of the system. One of the interesting questions implicit in this project was whether such a computerized system would be a help or a hindrance for the physician. This sort of question had generally not been asked before, and is still very rare in Canada. As one of the outcomes of the project, the authors concluded [15]: "However, there are considerable barriers to developing the interfaces necessary with community-based pharmacies to permit exchange of information

A second CHIPP project called RIGIC was essentially a project to create a networked EHR system which would be shared between two hospitals, for oncology patients. One of the outcomes of the project was that shared protocols (between the hospitals) had to be established in terms of access and confidentiality procedures. New policies and procedures also had to be developed in relation to this, including the development of a consent management system. As well, the participants became more aware of the importance of change management, and the need for recurrent funding in order to transition the research

A third project, the SI-RIL project, consisted of the development of an integrated network system in a health region (the health region of Laval, north of Montreal), for ambulatory patients. This system was designed to enable the sharing of physician orders and patient health information among health workers in the region, with a view to regional coordination of care. The participants were: the local hospital, four community health centres

**3.3. Canada Health Infostructure Partnerships Program**

In December 2001, the provincial government proposed a law that would have created a provincial smart card and replaced the existing health card. It should be pointed out that the Laval region had also been involved in a second electronic health card project, which began in 1999, and this project was again sponsored by the RAMQ [12]. For those who proposed the law, this project provided an example of the type of technology that would be used. One of the technological differences with the Rimouski project was that a patient's clinical information was centralized in a databank managed by the RAMQ.

The purpose of the proposed law was to allow the development of a provincial electronic patient record, enabled by the patient's use of a smart card, although the patient record would have only been a summary. A law was also necessary in order to allow easier communication of health information between health organizations, which were subject to restrictions because of laws that were active at the time. With the proposed law, the RAMQ would have become the government's manager of the smart card, and the card would have included both administrative information (the name, address, and date of birth of the patient), and health information. In contrast with the Rimouski project, this information was to be centralized in a database. As part of the legislation development process, the government included a public consultation on the law.

Codified Knowledge and Decisions in a Major eHealth Project: Efforts to Introduce the Electronic Health Record in Quebec 73

tabled reports about the proposed law, there was relatively little public opposition to it, and it was adopted almost a year later. It was clear, however, that it had taken many years for the government to respond to the request from the CAI to enact such a law. With this law, clinical information would now be managed by regional health organizations (there are 18 health regions in Quebec), and this regional organization, even though it was part of the

In 2001, Canada Health Infoway (Infoway), was created in order to accelerate the adoption of compatible electronic health records across the country. This organization was funded by the federal government, but accountable also to the provincial and territories' health ministers, all of whom were represented on a Board of Directors. Projects in the provinces were generally funded equally by Infoway and provincial governments. Over ten years, \$1.6

One of the main priorities of Infoway was to incite provinces to adopt technologies which would allow patient records in one province to be available to health providers in other provinces. Another rationale was that it would be possible to save money and time by

In a report provided in 2005, consultants to Infoway proposed a three-phase approach to health technologies. The first phase would consolidate patient data for viewing by health professionals, the second phase would allow the documentation of care and provide basic decision support, and the third phase would allow physician orders and decision support.

In was not until 2004 that the Quebec government decided to join Infoway. Informants indicated that Quebec did not participate in Infoway from 2001 to 2004, and at the end of this period, the situation was such that it was the only province which had not yet joined the project. One senior official indicated that the discussions during this period were "very, very political." The decision to join Infoway was made in 2004, and was taken by a newly elected Liberal provincial government (the previous ruling party was the Parti Québécois, a sovereignist party). This decision implied that Quebec would need to harmonize its EHR plans with those of Infoway and have its projects and deliverables approved by Infoway in order to receive the funding contribution by the federal government. In particular, local companies which had been supplying technology to Quebec health institutions, would have to be compliant with the standards defined by Infoway (for an analysis of the development

One of the influences that seemed to affect the Quebec government's decision to join Infoway, seemed to be the possibility of obtaining financial advantages. As previously noted, the federal government provided a significant grant to Quebec (and other provinces) as a financial incentive. A form of group purchase of certain technologies also became possible. As a government official commented: "In the end, Quebec chose the same solution

government, was separate from the RAMQ.

billion Canadian was allocated to this project [3].

having common technologies deployed across provinces.

The estimated cost at that time of reaching the 3rd phase was \$10 billion.

**3.5. Canada Health Infoway**

of standards in Canada, see [21]).

More than 50 groups and individuals responded with written comments about the proposed law [18]. There was a public debate about the law in the media, which gave rise to what one commentator considered to be a sterile debate often based on misconceptions [18]. Professional organizations, such as the provincial Nurses Association, Association of Pharmacists, and an association of physicians, also joined this debate. One of the specific criticisms of the physicians was that a centralized database should not be created, and that a government organization should not manage health summaries (a comment directed at the RAMQ) [19]. The CAI also questioned the inclusion of administrative and clinical information together in one database, and the management of this database by the RAMQ. They also reminded the government of the ongoing need (implicitly beyond this law), to widen the legislation that would frame the management and access to health information. It highlighted the recent creation of family medicine groups, and suggested that such groups could be the focus of efforts to computerize health information, since most health services were provided by primary care services. Finally, it issued a clear warning about the project:

... the law that the government has tabled entails solutions that could jeopardize the confidentiality of health information. The text has a number of grey zones which do not allow us to know with precision the orientations that will be followed. [20] (p. 28, our translation)

In one of our interviews, a government official had this to say about the public debate at the time:

We could say that the proposed law was a major error, the law that was tabled became the object of several controversies; several philosophers, the citizens thought that we wanted to create a 'big brother', that it was a way for the RAMQ to gain power over the health system; the physicians did not want to be controlled and were concerned about the way it was developing. (government official)

The result of the controversy and the questions that were raised was that the government withdrew the proposed legislation. In theory, it could have revised the proposed law, but perhaps because of the extent and range of the opposition, it decided to abandon it completely.

Two and a half years later, and under a new government, another law was proposed in December 2004 (proposed law number 83). Although the CAI and other organizations again tabled reports about the proposed law, there was relatively little public opposition to it, and it was adopted almost a year later. It was clear, however, that it had taken many years for the government to respond to the request from the CAI to enact such a law. With this law, clinical information would now be managed by regional health organizations (there are 18 health regions in Quebec), and this regional organization, even though it was part of the government, was separate from the RAMQ.

#### **3.5. Canada Health Infoway**

72 eHealth and Remote Monitoring

translation)

time:

completely.

government included a public consultation on the law.

the way it was developing. (government official)

The purpose of the proposed law was to allow the development of a provincial electronic patient record, enabled by the patient's use of a smart card, although the patient record would have only been a summary. A law was also necessary in order to allow easier communication of health information between health organizations, which were subject to restrictions because of laws that were active at the time. With the proposed law, the RAMQ would have become the government's manager of the smart card, and the card would have included both administrative information (the name, address, and date of birth of the patient), and health information. In contrast with the Rimouski project, this information was to be centralized in a database. As part of the legislation development process, the

More than 50 groups and individuals responded with written comments about the proposed law [18]. There was a public debate about the law in the media, which gave rise to what one commentator considered to be a sterile debate often based on misconceptions [18]. Professional organizations, such as the provincial Nurses Association, Association of Pharmacists, and an association of physicians, also joined this debate. One of the specific criticisms of the physicians was that a centralized database should not be created, and that a government organization should not manage health summaries (a comment directed at the RAMQ) [19]. The CAI also questioned the inclusion of administrative and clinical information together in one database, and the management of this database by the RAMQ. They also reminded the government of the ongoing need (implicitly beyond this law), to widen the legislation that would frame the management and access to health information. It highlighted the recent creation of family medicine groups, and suggested that such groups could be the focus of efforts to computerize health information, since most health services were provided by primary care services. Finally, it issued a clear warning about the project:

... the law that the government has tabled entails solutions that could jeopardize the confidentiality of health information. The text has a number of grey zones which do not allow us to know with precision the orientations that will be followed. [20] (p. 28, our

In one of our interviews, a government official had this to say about the public debate at the

The result of the controversy and the questions that were raised was that the government withdrew the proposed legislation. In theory, it could have revised the proposed law, but perhaps because of the extent and range of the opposition, it decided to abandon it

Two and a half years later, and under a new government, another law was proposed in December 2004 (proposed law number 83). Although the CAI and other organizations again

We could say that the proposed law was a major error, the law that was tabled became the object of several controversies; several philosophers, the citizens thought that we wanted to create a 'big brother', that it was a way for the RAMQ to gain power over the health system; the physicians did not want to be controlled and were concerned about In 2001, Canada Health Infoway (Infoway), was created in order to accelerate the adoption of compatible electronic health records across the country. This organization was funded by the federal government, but accountable also to the provincial and territories' health ministers, all of whom were represented on a Board of Directors. Projects in the provinces were generally funded equally by Infoway and provincial governments. Over ten years, \$1.6 billion Canadian was allocated to this project [3].

One of the main priorities of Infoway was to incite provinces to adopt technologies which would allow patient records in one province to be available to health providers in other provinces. Another rationale was that it would be possible to save money and time by having common technologies deployed across provinces.

In a report provided in 2005, consultants to Infoway proposed a three-phase approach to health technologies. The first phase would consolidate patient data for viewing by health professionals, the second phase would allow the documentation of care and provide basic decision support, and the third phase would allow physician orders and decision support. The estimated cost at that time of reaching the 3rd phase was \$10 billion.

In was not until 2004 that the Quebec government decided to join Infoway. Informants indicated that Quebec did not participate in Infoway from 2001 to 2004, and at the end of this period, the situation was such that it was the only province which had not yet joined the project. One senior official indicated that the discussions during this period were "very, very political." The decision to join Infoway was made in 2004, and was taken by a newly elected Liberal provincial government (the previous ruling party was the Parti Québécois, a sovereignist party). This decision implied that Quebec would need to harmonize its EHR plans with those of Infoway and have its projects and deliverables approved by Infoway in order to receive the funding contribution by the federal government. In particular, local companies which had been supplying technology to Quebec health institutions, would have to be compliant with the standards defined by Infoway (for an analysis of the development of standards in Canada, see [21]).

One of the influences that seemed to affect the Quebec government's decision to join Infoway, seemed to be the possibility of obtaining financial advantages. As previously noted, the federal government provided a significant grant to Quebec (and other provinces) as a financial incentive. A form of group purchase of certain technologies also became possible. As a government official commented: "In the end, Quebec chose the same solution (PACS technology) which saved us about 16 million, because they (the technology company) gave us a discount".

Codified Knowledge and Decisions in a Major eHealth Project: Efforts to Introduce the Electronic Health Record in Quebec 75