RESTRICTEDWORLD TRADE
ORGANIZATIONWT/COMTD/W/58/Rev.1
26 March 1999
(99-1229)
Committee on Trade and Development
EVALUATING WTO'S TECHNICAL COOPERATION
Note by the Secretariat
Revision
I. INTRODUCTION
1. At the informal meeting of the Committee on Trade and Development (23 November 1998)
and at its formal meeting (7 December 1998), Members agreed that the approach to evaluation of
WTO technical cooperation should be simple, objective, transparent, and practical.  It can be
qualitative, quantitative, or a combination of both.  It can also comprise internal and/or external
audits. In order to develop evaluation modalities, it was agreed to hold informal consultations withmembers of the Committee, who had expressed an interest in the subject, the WTO Secretariat and
with the participation of ITC.  These consultations were held on 17 December 1998. The Secretariat
had prepared an informal note containing two questionnaires on how to obtain information on
technical cooperation activities, which could provide the basis for and input into a broader evaluation
of WTO technical cooperation.
2. In light of the discussions held at that meeting and from comments/suggestions made by
Members and the representative of ITC to improve the proposals, the Secretariat has now revised and
refined the two questionnaires. The revised versions (see attachments) are submitted to the Committee
for its consideration. These two questionnaires would be used simultaneously for each WTO activity
and are designed to provide information regarding the activity held in beneficiary countries, including
the possible impact of the assistance provided and the expected follow-up. Such assessment would
provide an important source of information in the feed-back process and thus be beneficial for theorganization of further activities. Also, this information would be critical in assessing the
effectiveness of various modes of delivery in attaining the main objectives of technical assistance, as
defined in the guidelines for WTO Technical Cooperation, adopted in October 1996. The current
proposals, if adopted, would also generate a feed-back mechanism that allows the lessons of the past
and present to be integrated into future technical cooperation and assistance activities.
3. The present note does not pretend to provide a solution to all issues that have come up in
discussing evaluation modalities, but should be seen as a first, but critical, step in the evaluation
process, i.e. data collection. Some issues raised would need to be considered further in light of the
data obtained including the establishment of an appropriate methodology for evaluation, e.g.
quantitative versus qualitative assessments, internal and external evaluation, the criteria forevaluation, such as the impact on recipients, cost efficiency, degree of success in enhancing
institutional and human trade policy capacity in developing countries.
II. BASIC OBJECTIVES OF TECHNICAL  ASSISTANCE
4. The guidelines for WTO Technical Cooperation (WT/COMTD/8) adopted by the Committee on
Trade and Development on 15 October 1996 identified inter alia , two key objectives:WT/COMTD/W/58/Rev.1
Page 2
(i) assist in the full integration of benefici aries into the multilateral trading system and
contribute to the expansion of their trade;
(ii) strengthen and enhance institutional a nd human capacities in the public sector for an
appropriate participation in the multilatera l trading system; whenever possible and in
consultation with the government concer ned, capacity building activities could include
representatives of the private sector.
5. In order to meet these objectives, the WTO Secretariat has several instruments at its disposal to
deliver technical cooperation, which are outlined in the Implementation Modalities (WT/COMTD/W/29
Rev.1) of 17 October 1997.  While some instruments may seem more appropriate for attaining one or
another objective, all technical cooperation inst ruments are geared towards achieving both goals.
6. The Committee adopted language on the Monito ring, Evaluation and Follow-up of technical
cooperation, as reflected in para. 5 of the Implementation Modalities:
(i) "To ensure the effectiveness of technical cooperation activities administered by the
Secretariat and the optimum use of resources, the Committee on Trade and
Development shall undertake annual and if necessary more frequent reviews, on the
basis of reports provided by the WTO Secret ariat.  In this context, the Committee on
Trade and Development shall consider, at the appropriate time (say every three years),
the possibility of commissioning an exte rnal evaluation of the WTO Secretariat's
technical cooperation activities, based on the Guidelines for Technical Cooperation.
Such an evaluation should follow inte rnationally recognized methodologies;
(ii) appropriate follow-up activities may be established with a view to ensuring the
effectiveness of technical cooperation and training activities.  These can include
training of trainers programmes, the use of internet and interactive technology (CD-
ROM) for self-training, follow up courses, seminars, workshops (national/regional),
making available local technical expertise and advice."
III. SOME FEATURES OF WTO TECHNICAL COOPERATION
7. While the WTO is not a technical cooperation organization per se , the increased complexity
of the multilateral trading system, combined with  WTO's enlarged membership (134 members), have
meant that technical cooperation now occupies an increasingly important place in WTO's overall
activities, involving most Divisions and staff members, who provide expertise in their area of
competence. In 1998, the WTO carried out and/or participated in over 300 technical cooperation
activities, which were budgeted at some CHF 5mio, over 80 per cent of which were financed through
extra-budgetary resources. This compares with some 79 activities carried out in 1995, with a budget
of some CHF 700.000. The average national technical cooperation activity lasts between 3-5 days,
and costs less than CHF 20.000, while a regional seminar averages 10 working days and costs aroundCHF 150.000.
8. WTO technical cooperation contains several features that make it different from technical
cooperation carried out by other international organizations, which often have a 'built-in' evaluation
mechanism, mostly based on UNDP criteria. It would seem, however, that these criteria are not
directly applicable for WTO's technical coopera tion programmes, as the UNDP criteria are geared
towards 'projects', often self-contained and involving large sums of money, e.g. over CHF 1mio each.
Most projects carried out by other agencies (ITC, UNCTAD, UNDP) are of longer duration than
WTO's activities. In addition, these often lead to more tangible, measurable and quantifiable results.
This is not necessarily the case in WTO technical cooperation, with many activities geared towards
human and institutional capacity building. The effects of the assistance could at best be expected to beWT/COMTD/W/58/Rev.1
Page 3
measurable over time, but even then, analytical problems would remain on how to measure the impact
and how to establish a direct relation between WTO's input and the outcome.
IV. DATA GATHERING THROUGH QUESTIONNAIRES
9. For the purpose of systematic information a nd data collection on WTO technical cooperation
activities, two evaluation forms (questionnaires, see attachment) have been prepared in consultation
with all WTO Divisions that are directly or indirectly involved in providing technical assistance and
in coordination with ITC. They would be used si multaneously by all WTO officials involved in the
delivery of technical assistance.  Each questionnaire serves a specific purpose.
10. The first questionnaire, entitled 'Country  Request and Evaluation Form' , to be filled out in
part by the Secretariat and in part by the beneficiary country, essentially captures all technical
information regarding the event to be held in th e beneficiary country. One important element in the
form will be to define the objective(s) of the activity. This needs to be done by the beneficiary country
before the activity is carried out.  The WTO Secretariat can assist, if required to ensure that the
objective can realistically be met by its modes of delivery. It is against a clearly defined objective thatthe attainment or non-attainment of the objective can be measured.
11. Furthermore, at the conclusion of the event (on-site), the representatives of beneficiary
country would be requested to assess the activity and identify a follow-up, if so required. The WTO
official(s) would also be requested to provide an assessment on the activities. The officials would be
expected to provide information on the attendance, the level of participation, whether the target groupwas 'right', the degree of technicality, the skills acquired, whether the programme was completed,
expected follow-up, or any other aspect that it considers critical for attaining the objective of the
mission.
12. As it would appear that for certain activities it takes time to bear fruit, e.g. notification
requirements, introducing and/or adapting legislation, it is proposed to contact the beneficiary countryagain after three to six months. It will be asked again to assess the impact of the activity having taken
place; to provide information on a follow-up to the activity and any other relevant information. This
may include explanations on whether and why it has difficulties in complying with the notification
requirements, implementing agreements and/or following-up on the technical assistance that was
provided, if such was required/expected.
13. Through this questionnaire, a dialogue (interaction) would be established between the
beneficiary country and the WTO Secretariat in a way that hitherto did not exist.
14. The second questionnaire, entitled 'Participant Evaluation / Feed-back Form'  is intended
to improve the quality of WTO's technical cooperation through feed-back by the participants. It is
self-explanatory. It would be handed out to participants in all technical cooperation activities, forwhich the WTO has the lead. If properly filled out by participants, it would provide information on
the objective of the participants in attending the WTO activity, whether expectations were met and on
various aspects of the activity, including the substance, areas covered, presentations, documentation,
logistics, etc. Similar forms are used by ITC. The feed-back of missions would provide a main source
of information on how to further improve the delivery of technical cooperation.
15. These two questionnaires can be refined, expanded and modified in due course, with
experience gained.  To assess the effectiveness of technical assistance, the Committee might thenwish to analyze available information in different ways:
(i) by topic covered, which would reveal detailed information on WTO areas (subjects)
covered by the relevant WTO Divisions and how the information was assimilated by
the beneficiary countries;WT/COMTD/W/58/Rev.1
Page 4
(ii) by country, which would show the efforts undertaken by the WTO Secretariat with
regard to a particular beneficiary country and the country's response;
(iii) by region or sub-region, which would thus cover technical cooperation activities for a
given group of countries;
(iv) by mode of delivery, as defined in the guidelines and reflected in the manual on
technical cooperation, which provide information on the efficiency of a particular
mode of delivery in achieving a certain objective.
16. Moreover, the information could be analyzed in different ways and provide insights from two
angles, i.e. from the point of view of the WTO Secretariat, as a service provider, and from the
receiving end, i.e. the beneficiary country. Thus, the global picture could be rather complete.
17. As to the evaluation of training through WTO's regular trade policy courses, evaluation is
conducted on an on-going basis by the WTO's Traini ng Division. This process may be further refined.
V. EVALUATION OF WTO ACTIVITIES CARRIED OUT JOINTLY WITH OTHER
ORGANIZATIONS
18. The WTO is also involved in several technica l assistance activities carried out jointly with
other organizations, including those performed as part of the Integrated Framework for Trade-Related
Technical Assistance (follow-up to the High Le vel Meeting) involving six major international
organizations and the Joint ITC/UNCTAD/WTO Integrated Technical Assistance Programme in
Selected Least Developed and Other African Countries (JITAP). In these cases, WTO's contribution isgenerally well-defined, and sometimes carried out as 'stand-alone' activity, but it nevertheless remains
part of a set of activities.
19. A proper and full assessment of the project would require more than an assessment of the
WTO component, as the evaluation of WTO's contribution would thus go hand in hand with an
overall evaluation as stipulated in these joint activities and based on the criteria developed for thatpurpose.
20. In several cases, the WTO is called upon to provide expertise in activities and programmes
organized by or jointly held with other organizations.  This can include the World Bank, the World
Customs Organization (WCO), the World Intellectual Property Organization (WIPO), the
International Standards Organization (ISO). In addition, the WTO frequently attends conferences andworkshops, national or regional seminars, organized under the responsibility of other bodies and
where the WTO is called upon to make an input. While all of these activities are often very valuable
in disseminating specific or general information, or in training officials on a specific subject matter
directly related to the WTO Agreements, the impact of WTO's participation cannot be evaluated in
isolation.
21. It would thus seem necessary, for the purpose of evaluating technical cooperation activities, to
distinguish between activities organized by the WTO, with direct responsibility for the activity, andthose where the WTO participates. The evaluation should ideally be confined to activities for which
the WTO has direct responsibility.
VI. EVALUATION OF ACTIVITIES MANDATED UNDER WTO'S TRUST FUNDS
22. As most of WTO's technical cooperation activities are directly financed from member's
contributions to the WTO's  Trust Fund, the Secretariat in this instance is directly accountable to thedonor for the use of the funds.  Moreover, some Members have specified explicitly in their
Memorandum of Understanding (or the exchange of letters accompanying the donation) with theWT/COMTD/W/58/Rev.1
Page 5
Secretariat the elements that the (semi-)annual report is to contain, which can include an evaluation of
how the funds were used and their expected impact on the beneficiary country. The Secretariat willcontinue to meet these reporting obligations.
VII. THE EVALUATION PROCESS IN THE WTO
23. The process of evaluation of WTO's technical cooperation activities would be monitored and
centralised in the Technical Cooperation Divi sion, where all information gathered would
systematically be kept, organized and analyzed to facilitate WTO's internal evaluation. Coordinationwith the relevant WTO Divisions would be ensured on a continuous basis, as well as any follow-up,
where such is required to obtain the specific information from beneficiary countries.
24. It is proposed to start the process by 1 July, so that a first evaluation report can be produced
by early Spring of the year 2000.WT/COMTD/W/58/Rev.1
Page 6
WORLD TRADE ORGANIZATION
ORGANISATION MONDIALE DU COMMERCE
ORGANIZACIÓN MUNDIAL DEL COMERCIO
Country Request and
Evaluation Form
MISSION CODE COMTD Reference WTO Focal Point 
Country Focal Point: lead
ministry/agency
Activity dates Duration (days)
Mode of delivery Initial request/
reference
Activities to be
undertaken/Subject(s)covered
Participating
Countries
WTO/other
lead agency
Specific
objective(s)of themission
Success indicators/
performance targets
Estimated cost of the activityWT/COMTD/W/58/Rev.1
Page 7
Funding
Division(s) Nb of staff
involved travelling
Activities undertaken:
assessment by WTO
Immediate
assessment bybeneficiary (leadagency)
Assessment of
mission after 3-6months bybeneficiary (leadagency)
Nature of follow-up
expected/requested(if any)WT/COMTD/W/58Rev.1
Page 8
WORLD TRADE ORGANIZATION
ORGANISATION MONDIALE DU COMMERCE
ORGANIZACIÓN MUNDIAL DEL COMERCIO
Centre William Rappard  Rue de Lausanne 154  Case postale  CH – 1211  Genève 21
Téléphone:  (+41 22) 739 51 11  Fax:  (+41 22)  731 4206
Internet:  http://www.wto.orgParticipants Evaluation /Feed-Back
Form
Filling out this questionnaire should not take more than 5 minutes. It will
help us to further improve and adapt our forthcoming technicalcooperation activities.  Thank you  in advance for your collaboration.
1.Title of event: __________________________________________________
2. Mode of delivery:
National seminar
Technical mission
Regional seminar Conference/Symposium
Workshop Short training course
3. Country/location where event took place: __________________________
4. Date(s):_____________________
5. Please indicate your area of activity:
 Government  Business
 Other:_______________________________________WT/COMTD/W/58/Rev.1
Page 9
6(a) What were your main objectives for participating in this event?
 Obtain general information a bout the multilateral trading system
 Obtain specific information about one /several WTO Agreements --which ones:
_________________________________________________________
_________________________________________________________
 Address implementation i ssues of WTO Agreements
 Obtain information on 'new' policy issues and 'built-in' agenda
 Use the information to train other persons
 Improve working efficiency in specific areas
 Other (please specify) ______________________________
   _____________________________________________________________
6(b) Were they achieved?     Yes, largely          No, not entirely
Please give a short explanation:
_____________________________________________________________________
__________________________________________________________________________________________________________________________________________
_____________________________________________________________________
7. How would you rate the ev ent you have just attended?
Content (substance)
 Excellent  Good  Satisfactory  Poor
Structure (topics)  Excellent  Good  Satisfactory  Poor
Presentations (style)  Excellent  Good  Satisfactory  Poor
Documentation.Provided
 Excellent  Good  Satisfactory  Poor
Timeliness of  Excellent  Good  Satisfactory  Poor
documentation made availableWT/COMTD/W/58/Rev.1
Page 10
Logistics (travel arrangements, accommodation,
meeting room)  Excellent  Good  Satisfactory  Poor
Duration  Excellent  Good  Satisfactory  Poor
Time per topic  Excellent  Good  Satisfactory  Poor
Other (specify)_____  Excellent  Good  Satisfactory  Poor
Please give a short explanation:______________________________________________________________________________________________________________________________________________________________________________________________________
___________________________________________________________________________
8. Which part(s) of the programme and/or  topic(s) did you find most useful?
a.________________________ b. _______________________________
c.________________________ d. _______________________________
Please give a short explanation:__________________________________________________________________________________________________________________________________________
9. Which areas or aspects, in your view, were not adequately addressed?
a._________________________ b. ______________________________
c. ________________________ d. ______________________________What was lacking?_____________________________________________________________________
_____________________________________________________________________WT/COMTD/W/58/Rev.1
Page 11
10. Which documentation or handouts were the most interesting for your work?
a. ________________________ b. ______________________________
c. ________________________ d.. _____________________________Please explain why?__________________________________________________________________________________________________________________________________________
11. How will you use the informatio n provided during these sessions?
 Disseminate information to:  Colleagues
 Employees
 Business people
 Other
 Provide input into refinement of my  country's policy position on WTO issues.
________________________________________________________________________________________________________________________________
 Improve own working performance. How?
________________________________________________________________________________________________________________________________
 Other ________________________________________________________
12. Does this event require a follow-up        Yes   No 
If yes, please explain:
_____________________________________________________________________________________________________________________________________________________________________________________________
__________