since quality has an abstract concept , presenting a practical method for determining it is not easy .
however , this is the conceptual quality that can be felt , perceived , and judged and when a product or good does not have quality , lack of quality can easily be sensed . therefore , to improve the reception and use of a product or service , the way quality is determined should be defined , and its evaluation and control procedure should be understood ( 1 ) , so that during the evaluation process , it is ensured that the delegated tasks and responsibilities are done well ( 2 ) .
one of the services that has had a considerable development in recent years is the world wide web , providing the world with a phenomenon called websites .
websites involve connection points and communication of users with electronic information and considering nowadays all institutes try to present their services to different classes , the world wide web has been successful in meeting this need in generating information and making them accessible with an ever - increasing speed ( 3 ) . in the health sector , many hospitals have designed individual websites for themselves presenting various services . given the sensitivity of hospitals , the services they are seeking to introduce , in response to the increasing development of hospital websites to allow the access of various people to these websites , the issue of website quality has arisen .
this is because the richness of websites and the number of referrals to them , contribute to improving the status of the hospital in the specialized area and in offering services to patients .
if the performance of a hospital website is weaker than the expected level , redesigning the website or substantial enhancement of the volume and quality of electronic journal may be required ( 3 ) . to evaluate the quality of websites , various models and criteria
the first one is the iso-9126 model whose criteria are usability , reliability , efficiency , and functionality .
the mile model , focusing on the usability of the website , measures the quality of websites based on content , services , navigation , cognitive features of the interface , aesthetics , and technology . and finally , the minerva model , proposed to evaluate the quality of cultural websites ( 1 , 4 , 5 ) .
unfortunately , despite the various models presented for evaluating websites , determination of proper models and criteria for evaluating the quality of hospital websites has been understudied .
thus , this research tried to identify the criteria required for a hospital website by investigating the evaluation models of the quality of websites .
it is hoped that the results of this research are found useful in presenting a model for evaluation of the quality of hospital websites to promote their quality .
the studies either in english or persian , that have been related to evaluation of website quality and whose title and abstract included keywords such as evaluation , website quality , and website model , were included in this research .
the international database including science direct , google scholar , pubmed , proquest , ovid , elsevier , springer , ebsco , and regional databases including magiran , scientific information database and persian journal citation report ( pjcr ) , were searched using the keywords of website , evaluation , and website quality , according to table 1 .
all papers , regardless of the methodology including systematic review , experimental , and cross - sectional studies that were related to the research subject and were accessible in their full text were selected .
the models were then reviewed and out of the selected papers , the criteria and sub criteria of website quality were extracted and then classified .
the papers that were in line with the subject whose full text version was accessible , were included , while the papers that had no conceptual relationship with the subject or their full text was not available , were excluded from the study .
the results extracted and investigated in this research were the criteria required for evaluating the quality of hospital websites .
following specialist discussion and opinion exchange , 22 papers were selected ( table 2 ) .
the international database including science direct , google scholar , pubmed , proquest , ovid , elsevier , springer , ebsco , and regional databases including magiran , scientific information database and persian journal citation report ( pjcr ) , were searched using the keywords of website , evaluation , and website quality , according to table 1 .
all papers , regardless of the methodology including systematic review , experimental , and cross - sectional studies that were related to the research subject and were accessible in their full text were selected .
the models were then reviewed and out of the selected papers , the criteria and sub criteria of website quality were extracted and then classified .
the papers that were in line with the subject whose full text version was accessible , were included , while the papers that had no conceptual relationship with the subject or their full text was not available , were excluded from the study .
the results extracted and investigated in this research were the criteria required for evaluating the quality of hospital websites .
to evaluate the papers qualitatively , the extracted articles were studied by two relevant experts ( figure 1 ) . following specialist discussion and opinion exchange ,
to evaluate the quality of hospital websites , various models had been presented , the most famous of which were : the quality criteria in this model had been developed based on iso-9126 model , including usability , reliability , efficiency , and functionality .
the specific feature of this model was determination of a certain domain of evaluation and conductance of the evaluation step by step ( 1 , 6 ) .
this model focuses on usability of the website and measures the quality by combining the inspection by both an evaluator and users ( 1 ) .
the specified criteria in this model are : content : content means the quality of the information inside the website , and its communicative level .
services : services are all the functionalities that the website allows its users to do .
navigation : this criterion encompasses 1 ) various means of access by users to a certain part of the information in the website and 2 ) the logical structure of information to move across one part of information to another .
cognitive features of the interface : it takes the perception and feeling of the user into consideration and deals with how the users memorize the structure of the website .
aesthetics : it addresses the graphic design and template of the website , the type of font , color , size , images , and other graphic features in a website .
technology : it represents the adaption of the website for proper function in a variety of explorers , the security level of the host server , and the interaction between the website and remote database ( 1 , 5 ) .
this model was suggested for evaluation of the quality of cultural websites ( seminaries , archives , libraries , and other cultural institutions ) ( 1 ) . in this model ,
the aim of quality indices in this model is to present indices for evaluation of the quality of websites and support the design and development of the cultural websites ( 5 ) .
this model involves the following axes : clarity : the clarity of objectives , mission , and the identity of the website for users .
effectiveness : the content of the website is related to the validity of the website content and presentation of sound information to users .
maintenance : it focuses on the technical and content support , fresh content , and improvement of the website s technical functions . accessibility : it represents easy access of all users to the website,(for example , access for blind users , those with relative vision , or users with hearing problems ) , the possibility of using a variety of technologies to present information to users , and the functionality with a variety of explorers , operating systems , and devices .
user - centeredness : this means that the website should meet the needs of users .
further , the users should regard the website as useful , attractive , and easy to use .
responsibility : it refers to the ability of the website owners in responding to the questions of users , helping users to participate in generating content and answering the questions posed in a forum .
if a website has been based on standard technologies , techniques , and models , interaction and adaption with other websites and institutions would be easy .
managed : it denotes the legal issues related to protecting copyright and privacy ( 1 , 6 ) .
this model involves entity ( who ) , content ( what ) , services ( why ) , location ( where ) , maintenance ( when ) , usability ( how ) , and possibility ( by what means ) which are defined as follow : entity : includes the sub features of identification of characteristics .
facilities : encompasses resources , information and communication technology ( 5 ) this instrument has 12 dimensions including information suited to users , interaction , reliability , responsibility time , appearance , design , guesswork , innovativeness , stream , coherent connections , business process , and the ability for substitution ( 5 ) ( table 3 ) .
some of them focused on the structure , the homepage content , page content , and subject content ( other links ) of websites for evaluation purposes ( 7 , 11 , 13 , 15 , 1921 , 2325 ) .
other studies mentioned characteristics of audience , website objective , loading , and structural stability among websites as important factors in the success of a website ( 8 , 12 , 14 , 26 ) . also quality , size , language , background , and inclusiveness as the determinants of the final success of a website were referred ( 9 , 12 ) .
another study found that attention to the update rate of pages ( the duration between the last update and the date the website is used ) is essential ( 10 ) .
a technical framework presented seven criteria including : content quality , design quality , organization quality , user - friendliness , functional quality , service quality , and website quality from a technical point of view ( 11 ) .
moreover the evaluation criteria as the accessibility and updating were mentioned ( 10 , 12 , 1518 , 22 , 27 ) .
furthermore , entertainment and fun , and website speed was presented as evaluation criteria for websites ( 1315 ) .
another study stated the security , and confidentiality of personal information as the most important factors affecting the willingness of customers to purchase from a website ( 2325 ) as well as multilinguality ( 27 ) ( table 4 ) .
to evaluate the quality of hospital websites , various models had been presented , the most famous of which were : the quality criteria in this model had been developed based on iso-9126 model , including usability , reliability , efficiency , and functionality .
the specific feature of this model was determination of a certain domain of evaluation and conductance of the evaluation step by step ( 1 , 6 ) .
this model focuses on usability of the website and measures the quality by combining the inspection by both an evaluator and users ( 1 ) .
the specified criteria in this model are : content : content means the quality of the information inside the website , and its communicative level .
services : services are all the functionalities that the website allows its users to do .
navigation : this criterion encompasses 1 ) various means of access by users to a certain part of the information in the website and 2 ) the logical structure of information to move across one part of information to another .
cognitive features of the interface : it takes the perception and feeling of the user into consideration and deals with how the users memorize the structure of the website .
aesthetics : it addresses the graphic design and template of the website , the type of font , color , size , images , and other graphic features in a website .
technology : it represents the adaption of the website for proper function in a variety of explorers , the security level of the host server , and the interaction between the website and remote database ( 1 , 5 ) .
this model was suggested for evaluation of the quality of cultural websites ( seminaries , archives , libraries , and other cultural institutions ) ( 1 ) . in this model ,
the aim of quality indices in this model is to present indices for evaluation of the quality of websites and support the design and development of the cultural websites ( 5 ) .
this model involves the following axes : clarity : the clarity of objectives , mission , and the identity of the website for users .
effectiveness : the content of the website is related to the validity of the website content and presentation of sound information to users .
maintenance : it focuses on the technical and content support , fresh content , and improvement of the website s technical functions . accessibility : it represents easy access of all users to the website,(for example , access for blind users , those with relative vision , or users with hearing problems ) , the possibility of using a variety of technologies to present information to users , and the functionality with a variety of explorers , operating systems , and devices .
user - centeredness : this means that the website should meet the needs of users .
further , the users should regard the website as useful , attractive , and easy to use .
responsibility : it refers to the ability of the website owners in responding to the questions of users , helping users to participate in generating content and answering the questions posed in a forum .
if a website has been based on standard technologies , techniques , and models , interaction and adaption with other websites and institutions would be easy .
managed : it denotes the legal issues related to protecting copyright and privacy ( 1 , 6 ) .
this model involves entity ( who ) , content ( what ) , services ( why ) , location ( where ) , maintenance ( when ) , usability ( how ) , and possibility ( by what means ) which are defined as follow : entity : includes the sub features of identification of characteristics .
facilities : encompasses resources , information and communication technology ( 5 ) this instrument has 12 dimensions including information suited to users , interaction , reliability , responsibility time , appearance , design , guesswork , innovativeness , stream , coherent connections , business process , and the ability for substitution ( 5 ) ( table 3 ) .
the quality criteria in this model had been developed based on iso-9126 model , including usability , reliability , efficiency , and functionality .
the specific feature of this model was determination of a certain domain of evaluation and conductance of the evaluation step by step ( 1 , 6 ) .
this model focuses on usability of the website and measures the quality by combining the inspection by both an evaluator and users ( 1 ) .
the specified criteria in this model are : content : content means the quality of the information inside the website , and its communicative level .
services : services are all the functionalities that the website allows its users to do .
navigation : this criterion encompasses 1 ) various means of access by users to a certain part of the information in the website and 2 ) the logical structure of information to move across one part of information to another .
cognitive features of the interface : it takes the perception and feeling of the user into consideration and deals with how the users memorize the structure of the website .
aesthetics : it addresses the graphic design and template of the website , the type of font , color , size , images , and other graphic features in a website .
technology : it represents the adaption of the website for proper function in a variety of explorers , the security level of the host server , and the interaction between the website and remote database ( 1 , 5 ) .
this model was suggested for evaluation of the quality of cultural websites ( seminaries , archives , libraries , and other cultural institutions ) ( 1 ) . in this model ,
the aim of quality indices in this model is to present indices for evaluation of the quality of websites and support the design and development of the cultural websites ( 5 ) .
this model involves the following axes : clarity : the clarity of objectives , mission , and the identity of the website for users .
effectiveness : the content of the website is related to the validity of the website content and presentation of sound information to users .
maintenance : it focuses on the technical and content support , fresh content , and improvement of the website s technical functions . accessibility : it represents easy access of all users to the website,(for example , access for blind users , those with relative vision , or users with hearing problems ) , the possibility of using a variety of technologies to present information to users , and the functionality with a variety of explorers , operating systems , and devices .
user - centeredness : this means that the website should meet the needs of users .
further , the users should regard the website as useful , attractive , and easy to use .
responsibility : it refers to the ability of the website owners in responding to the questions of users , helping users to participate in generating content and answering the questions posed in a forum .
if a website has been based on standard technologies , techniques , and models , interaction and adaption with other websites and institutions would be easy .
managed : it denotes the legal issues related to protecting copyright and privacy ( 1 , 6 ) .
this model involves entity ( who ) , content ( what ) , services ( why ) , location ( where ) , maintenance ( when ) , usability ( how ) , and possibility ( by what means ) which are defined as follow : entity : includes the sub features of identification of characteristics .
this instrument has 12 dimensions including information suited to users , interaction , reliability , responsibility time , appearance , design , guesswork , innovativeness , stream , coherent connections , business process , and the ability for substitution ( 5 ) ( table 3 ) .
some of them focused on the structure , the homepage content , page content , and subject content ( other links ) of websites for evaluation purposes ( 7 , 11 , 13 , 15 , 1921 , 2325 ) .
other studies mentioned characteristics of audience , website objective , loading , and structural stability among websites as important factors in the success of a website ( 8 , 12 , 14 , 26 ) . also quality , size , language , background , and inclusiveness as the determinants of the final success of a website were referred ( 9 , 12 ) .
another study found that attention to the update rate of pages ( the duration between the last update and the date the website is used ) is essential ( 10 ) .
a technical framework presented seven criteria including : content quality , design quality , organization quality , user - friendliness , functional quality , service quality , and website quality from a technical point of view ( 11 ) .
moreover the evaluation criteria as the accessibility and updating were mentioned ( 10 , 12 , 1518 , 22 , 27 ) .
furthermore , entertainment and fun , and website speed was presented as evaluation criteria for websites ( 1315 ) .
another study stated the security , and confidentiality of personal information as the most important factors affecting the willingness of customers to purchase from a website ( 2325 ) as well as multilinguality ( 27 ) ( table 4 ) .
one of the most useful applications of the web is the web portals or website .
the aim of this study was to identify the models for evaluating the quality of hospital websites .
the results indicated that a great variety and diversity exists among the criteria presented for evaluating the quality of websites . on the other hand , many of these criteria can be considered as the sub criteria of a more general criterion .
a study investigated the factors associated with the success of websites in the area of electronic business in the us .
the study introduced four factors of web success in electronic business as information , the quality of services , use of systems , entertainment and fun , and the quality of system design ( 19 ) .
another study experimentally examined the relationship between the quality of websites and integration of investment of website customers in kuwait .
the researcher categorized the quality of services in four aspects of technical , general and specific content , and the apparent quality .
he believes that these dimensions influence the attitude of website users , where the positive attitude of users in turn affects their tendency to buy through the internet ( 22 ) . a paper called  the government in the digital era : concept , practice , and development in thailand , stated 10 characteristics of an electronic government , arguing that an electronic government should be comprehensive , interactive , inclusive , convenient , accessible , secure , customizable , multifunctional , and flexible ( 16 ) . in spain
experimentally and quantitatively investigated websites of electronic banking with the aim of finding an index for web evaluation , reported that the evaluation index for analyzing the website of commercial , educational , and nonprofit organizations is speed , navigation , and content ( 21 ) .
a paper called  the qualitative framework of evaluation of hospital / medical websites presented a qualitative framework consisting of seven criteria : the quality of content , design , organization , user - friendliness , performance , services , and technical quality ( 11 ) .
a research entitled  electronic government in cyprus proposed a framework for evaluating the development of electronic government .
this framework comprised content , design , and typical features of electronic government websites ( 20 ) .
results of a research introduced six important factors for success of websites as design , content , consumer education , security , customer support , online communities , and the market situation ( 24 ) . a model for the quality of websites with the criteria of correctness , presentation , navigation , content , and
the final success of websites , associated with factors such as quality , size , language , background , inclusion , etc .
and it was stated that one can not mention one or two factors as the only reasons for the success of a website ( 9 ) .
the result of investigation of the selected studies revealed that the criteria of accessibility , content , and apparent features of websites from user perspective such as design , the graphics used in the website , and attractiveness of the page have been mentioned in the majority of studies .
the results also showed that the important point in determining the criteria for evaluating the quality of websites is attention to the major differences in the special features of any website ; features considered during the design and selection of content for groups with various aims , so that a website becomes successful in responding to an objective for which it has been designed .
these features are especially important during the evaluation of websites , since neglecting these features causes many websites to fail to reach the aims for which they have been designed , thus requiring revision .
the results also indicated that the features of the information content , website design , security , and confidentiality of personal information are among the most important factors affecting the tendency of clients to use a website , thus paying attention to it in the design of any website and any evaluation model as a common criterion is essential .
minimum quality criteria for a website are usability , efficiency , ease of use , user friendliness , service , reliability and interaction .
the criteria of accessibility , content , design , security , and confidentially of personal information , as the essential criteria , should be taken into account when evaluating a website .
it is suggested that the criteria such as ease of use , playing a role in bringing about satisfaction of users , the graphics used in the website , attractiveness of the website , and other apparent characteristics of websites are considered as sub criteria for the criterion of user - friendliness .
it is also proposed that the criteria of speed and accessibility of websites , which indicate the efficiency of a website , are included in all models of website evaluation as the sub criterion of efficiency .
 introductionhospital websites are important tools in establishing communication and exchanging information between patients and staff , and thus should enjoy an acceptable level of quality . 
 the aim of this study was to identify proper models and criteria to evaluate the quality of hospital websites.methodsthis research was a systematic review study . 
 the international databases such as science direct , google scholar , pubmed , proquest , ovid , elsevier , springer , and ebsco together with regional database such as magiran , scientific information database , persian journal citation report ( pjcr ) and iranmedex were searched . suitable keywords including website , evaluation , and quality of website were used . 
 full text papers related to the research were included . the criteria and sub criteria of the evaluation of website quality 
 were extracted and classified.resultsto evaluate the quality of the websites , various models and criteria were presented . 
 the web - q - im , mile , minerva , seruni luci , and web - qual models were the designed models . 
 the criteria of accessibility , content and apparent features of the websites , the design procedure , the graphics applied in the website , and the page s attractions have been mentioned in the majority of studies.conclusionthe criteria of accessibility , content , design method , security , and confidentiality of personal information are the essential criteria in the evaluation of all websites . 
 it is suggested that the ease of use , graphics , attractiveness and other apparent properties of websites are considered as the user - friendliness sub criteria . 
 further , the criteria of speed and accessibility of the website should be considered as sub criterion of efficiency . when determining the evaluation criteria of the quality of websites , attention to major differences in the specific features of any website is essential . 
1. Introduction
2. Material and Methods
2.1. The search strategy
2.2. The inclusion and exclusion criteria
2.3. Quality assessment
3. Results
3.1. Results resulted from the identified models
3.1.1. Web-Q-I-M Model
3.1.2. Mile Model
3.1.3. Minerva Model
3.1.4. Seruni Lucci Model
3.1.5. Web-Qual Model
3.2. The results obtained from the reviewed studies
4. Discussion
5. Conclusions