RESTRICTEDWORLD TRADE
ORGANIZATIONWT/COMTD/W/74
21 June 2000
(00-2493)
Committee on Trade and Development
Twenty-Ninth Session
MONITORING AND EVALUATION OF THE WTO'S
TECHNICAL COOPERATION ACTIVITIES
MANUAL FOR TWO'S  SECRETARIAT STAFF DEALING WITH
TECHNICAL COOPERATION ACTIVITIES
Note by the Secretariat
Table of Contents
Page
I. INTRODUCT ION ................................................................................................................3
II. TERMINOLOGY AND CONCEPTS REGARDING MONITORING AND
EVALUAT ION ..................................................................................................................... 3
1. Definition of monitoring and evaluation.............................................................................3
2. Objectives of  evaluation ..................................................................................................... ..4
3. Types of evaluation .......................................................................................................... .....5
4. Monitoring, evaluation and management cycle .................................................................5
5. Dimensions and criteria of monitoring and evaluation .....................................................7
6. Logic of the monitoring and evaluation processes (objectives, logical intervention,
indicators and hypotheses)...................................................................................................8
7. Conclusions and recommendations derived from monitoring and evaluation..............10
8. Reports...................................................................................................................... ...........10
9. Lessons to be drawn.......................................................................................................... ..10
10. Responsibility for monitoring and evaluation..................................................................12
III. CONDUCT OF AN EVALUATIO N.................................................................................12
1. Planning an evaluation ....................................................................................................... 12
2. Defining and prioritising objectives ..................................................................................13
3. Selecting indicators ......................................................................................................... ....13
4. Tools for the collection and analysis of information........................................................14
5. Rating system ................................................................................................................ ......14WT/COMTD/W/74
Page 2
IV. EVALUATION AND  THE WTO'S TECHNICAL COOPERATION OBJECTIVES15
1. Technical cooperation objectives from an evaluation viewpoint....................................15
2. Indicators for ve rifying results ..........................................................................................16
V. THE WTO'S ROLE IN THE AREA OF TE CHNICAL COOPERATION..................21
1. Objectives and content ....................................................................................................... 21
2. Coordination................................................................................................................. .......23
VI. GLO SSARY .................................................................................................................... ....25WT/COMTD/W/74
Page 3
I. INTRODUCTION
The guidelines (WT/COMTD/8) and the implementation modalities
(WT/COMTD/W/29/Rev.1) adopted by the WTO's Committee on Trade and Development (CTD)
provide for the monitoring of each programme and the evaluation of its effectiveness.  The CTD
carries out these tasks by evaluating the results of technical assistance activities so as to ensure thebest possible use of resources in accordance with appropriate evaluation criteria.
In addition, the implementation modalities (WT/COMTD/W/29/Rev.1) allow for the
possibility of an external evaluation approximately every three years using internationally recognized
methodologies.
To underpin this process, the Secretariat has prepared for the CTD a draft framework for the
monitoring and evaluation of technical cooperation activities comprising parameters, criteria and a
methodology to be followed.  Using this framework, the Secretariat prepares an internal evaluation
report annually for submission to the CTD for consideration and action.
The Secretariat has also prepared this Manual for those who manage technical cooperation
activities and WTO staff dealing with such activities so that they may familiarize themselves with the
monitoring and evaluation mechanisms and follow a mo re integrated approach to their work that takes
into account the whole management cycle (planning , programming, implementation, monitoring and
evaluation) in order to achieve better results.
The Manual has been prepared on the basis of the practices followed by other international
organizations with long experience in monitoring and evaluation and with technical cooperation
activities that are similar to those of the WTO in the sense that they aim to develop human and
institutional capacities and concern policy matters.
II. TERMINOLOGY AND CONCEPTS REGARDING MONITORING AND
EVALUATION
1. Definition of monitoring and evaluation
Monitoring is an internal process carried out on an ongoing basis in order to assess and
appraise the implementation of an activity, project or programme.  Its primary purpose is to verifywhether, in relation to the objectives fixed, the resources were available and utilized at the proper time
in the desired quantity and quality.  Monitoring also seeks to ensure that the expected results have
been achieved by verifying that the outputs derived from the services furnished are being used by the
direct beneficiaries.
The objective of monitoring is essentially to help in improving decision-making so as to
enhance performance and the management of activities, identifying the indicators of progress and theshortcomings in time so that those responsible for managing the activities can take the remedial
measures needed.
There is a distinction between  monitoring actual achievements and monitoring the methods
of carrying out a particular activity, programme or project.  The aim of the former is to assess theprogress made in carrying out the activities in terms of time limits, costs and practical results
(inputs/outputs).  The latter relates to the implementation process and comprises an analysis of the
processes used and whether they respond to the sustainability criteria for results.WT/COMTD/W/74
Page 4
If the information obtained during monitoring is analysed, disseminated and stored, it can
provide a useful database when carrying out mid-term and retrospective evaluations of technical
cooperation activities.
Evaluation is the last stage in the management cycle.  It has a  limited duration and its aim is
to provide a systematic and objective appraisal of the relevance, performance and success of technicalcooperation activities being carried out or just completed.  It consists of analysing the results and the
impact of the activities with the aim of introducing any adjustments required and putting forward
recommendations to guide similar activities and help to improve future approaches.  The subsequent
phase of implementation will thus depend on the conclusions and recommendations reached in the
preceding phases.
Unlike monitoring, evaluation is selective and, for practical and budgetary reasons, does not
encompass all the activities.
2. Objectives of evaluation
In general, the objective of evaluation is to appraise the performance of activities in relation
to the following three areas:
(a) The needs and expectations of the beneficiaries and direct users of the services
(relevance, concept and formulation);
(b) satisfactory and consistent achievement of the objectives set for these activities
(effectiveness, efficiency and timeliness);
(c) assessment of the impact of the activities on the beneficiaries (success:  impact,
sustainability and capacity building).
In the OECD approach, evaluation consists of a process of observing, analysing and
appraising activities in order to draw general and specific lessons concerning technical cooperation
policies and the achievement of their objectives.  The aim is not to draw up a list of shortcomings, but
to describe the activities and assess their results so that decision-makers can improve the quality of
assistance.  In assessing quality, the extent to which the activity has reached its objectives, theeffectiveness of the procedures, the concept, the formulation and the impact are all examined.
For the UNDP, evaluation helps to provide lasting results which benefit the target groups.  It
helps decision-making, enhances responsibility, strengthens the learning process and builds capacity.
Experience shows that the main objective of mid-term evaluation is to improve the
implementation and procedures whereas a retrospective evaluation focuses more on assessing thesuccess of the activities (the impact on the beneficiaries and the sustainability of their results).
If monitoring and evaluation are to be effective, they must be designed with a view to
improving technical assistance activities and not to seeking out lacunae or expressing criticism.
In the WTO context, the aim of monitoring and evaluation is to guarantee the effectiveness of
technical cooperation activities and to ensure optimum utilization of resources, as prescribed by the
Guidelines for WTO Technical Cooperation (WT/COMTD/8) and its Implementation Modalities
(WT/COMTD/W/29/Rev.1).WT/COMTD/W/74
Page 5
3. Types of evaluation
There are several types of evaluation according to the classification criterion selected:
a) Internal or external
Depending on the person in charge, evaluation may be internal or external.  Internal
evaluation is undertaken by the persons who themselves designed, formulated and managed theimplementation of the activities evaluated.  It can also be called self-evaluation.  External or
independent evaluation is carried out by persons or institutions which did not take part in the
formulation and implementation of the activities.
b) Mid-term or a posteriori (ex-post or retrospective)
Depending on the time at which it was carried out, an evaluation may be mid-term or ex-post.
It is mid-term if it is undertaken while the activities are being implemented.  This type of evaluationusually focuses on management and implementation.  Subsequent or ex-post evaluation takes place
after the activities have been completed and focuses primarily on general policy and impact study
aspects.
c) Sectoral, thematic, project, programme, approach and/or policy
Depending on the terms of reference and the scope it has been given, an evaluation may be a
sectoral, thematic, project, programme, approach and/or policy evaluation.
In the WTO context, the texts adopted by the CTD provide for an annual evaluation that may
be described as mid-term and internal.  In addition to this annual self-evaluation, the Committeeallows the possibility of carrying out an external (independent) evaluation if needed, which may be
retrospective if it focuses on activities that have already been completed.
4. Monitoring, evaluation and management cycle
Monitoring and evaluation are two distinct but complementary and interactive functions.
Their purpose is to improve the implementation of activities, ensure better results and guarantee their
quality.
Monitoring and evaluation form an integral part of the whole management cycle for technical
cooperation activities.  They are management processes in just the same way as other elements in thecycle such as the examination of requests for technical assistance, the preparation of replies, planning,
programming and the implementation of activities.
On the one hand, monitoring and evaluation help to enhance the quality of activities as a
result of the influence they exert on the decision-making process and the lessons they provide on an
ongoing basis.  On the other, the design and formul ation of activities and programmes have an effect
on the quality of monitoring and evaluation through clearly defined objectives and performance andachievement indicators.
It is extremely important to include monitoring and evaluation components in all the
programming stages, first of all by incorporating the monitoring and evaluation plan in the planning
and programming of activities and then by integrating the recommendations and lessons learned fromthe experience in the formulation and implementation stages.  The different stages in the interaction
between monitoring and evaluation and the other management processes can be briefly summarized as
follows:WT/COMTD/W/74
Page 6
• Programme pre-formulation stage, when the lessons learned should be identified;
• programme formulation stage, when it should be verified that the recommendations
and lessons learned from the experience have in fact been incorporated into the new
activities;
• monitoring and evaluation stage, when the specific objectives of the activities should
be clearly identified and the relevant performance indicators defined.
The WTO's technical cooperation activities are programmed on a three-year/country basis and
are included in the Three-Year Plan adjusted annually and approved by the CTD.
The beneficiary countries are closely associated with the design, preparation and
implementation of activities under the programme.  Although the provision of technical cooperation is
guided by demand – "demand-driven approach"- the WTO intervenes at all these stages and follows a
proactive policy when providing information on technical assistance opportunities available, theidentification and expression of needs–objectives of the beneficiaries, the exploration of areas of
coordination and development of synergy between its activities and those of national, subregional,
regional and international institutions active in trade-and development-related areas.
Technical cooperation activities and programmes are designed and planned in the light of the
WTO's mandate and responsibilities, taking into account the national, subregional, regional and
international situation of the beneficiary country.
After technical assistance requests have been made, they are jointly examined and developed
by the WTO and the beneficiary country so that the output will be well-designed and organized and
meet standards of economy and efficiency.
Examination of technical assistance requests, preparation of activities, formulation,
implementation, monitoring and evaluation constitute the programme cycle.
Programme cycle for the WTO's technical cooperation activities
Receipt of technical
assistance requests        /g116
         Three-year plan and
annual adjustment              /g232
Implementation
/g115
MonitoringWT/COMTD/W/74
Page 7
5. Dimensions and criteria of monitoring and evaluation
In order to assess the quality of activities, programmes or projects, three areas have to be
appraised.  These are the Relevance, the Performance and the Success of the activities.
a) Relevance of an activity or programme
This is measured by the level of validity and pertinence of the objectives in relation to the
internal and external contexts.  The following are the criteria used for this purpose:
• National, regional and international contexts (is the service provided consistent with
the national and regional economic and trade situations of the beneficiary country and
with its overall policy?);
• direct beneficiaries (do the direct beneficiaries have a real need of the activities
undertaken?);
• mandate of the organization responsible for cooperation (are the services provided
part of the providing organization's mandate and functions?);
• comparative advantage in comparison with other organizations (is the providing
organization the best placed to furnish this technical assistance or are there otherrelevant organizations that have achieved a certain degree of excellence in the same
area as the activity?).
b) Performance
This concerns the progress made in relation to the objectives.  It covers the performance of
the inputs, outputs and immediate and long-term results produced by the activities.  The performancecriteria relate to the following:
• Effectiveness, which must reflect the extent to which the objectives have been
reached;
• efficiency, which must allow the best pos sible transformation of the inputs into
outputs, in other words choosing solutions that give priority to using the least-costly
resources to achieve the same objectives;
• timeliness, which requires the production of inputs and results within reasonable
time-limits and at the right time.
c) Success
This can be measured using criteria relating to impact, sustainability and contribution towards
capacity building.  The impact comprises the change s that occur in a given situation following the
implementation of an activity.  Sustainability means that the results last even after the programme has
ended. The impact is considered to be static when the same advantages continue for the same target
groups.  It can be dynamic when the original target groups or other groups use the results of a
programme in a different context or an environment that is changing and adapt them to these changes.
Capacity building helps target groups to become independent and gain experience in dealing with
policy issues and problems.WT/COMTD/W/74
Page 8
6. Logic of the monitoring and evaluation processes (objectives, logical intervention,
indicators and hypotheses)
Monitoring and evaluation are based on an analytical method that allows a close examination
of the results of activities, logically focusing on the objectives and the cause-and-effect relationship
among them.  This method also makes it possible, with the aid of indicators, to verify whether theobjectives have been reached, taking into account the hypotheses which describe factors that may
have affected the success of the activities and are outside the Organization's control.  This analysis is
carried out with the assistance of a logical framework, in the form of a matrix that combines the
elements that are logically linked.
The logical framework is based on an analysis of problems/needs that allows the objectives of
activities to be defined.  The problems are analysed by defining the cause-and-effect relationship ofthe negative aspects of a particular situation and identifying the bottlenecks on which the priorities
must focus.
The first column of the logical framework show s the strategy followed in order to reach the
goals and objectives of the activities.  This is the logical intervention column.  The second and thirdcolumns show respectively the success indicators used to measure the level of achievement and the
sources used to verify these.  The fourth column contains the hypotheses which reflect the conditions
needed to achieve the objectives.  These elements are constantly changing and require ongoing
consultations with the various parties involved.
Logical framework
Intervention logic Indicators Sources Assumptions
Overall objectives
Specific
objectives
Concept anddesign
Activities
Administrative,managerial, and
financial aspects
coordination
a) Objectives
Objectives are defined and clarified in the light of the problems and needs.  The future
situation that will occur once the problems have been resolved must be outlined.  At this level,
prioritising the objectives is extremely important  in order to determine the means to be used to
achieve the objectives.  It should be noted that specific objectives taken together help to achieve
overall objectives, whereas activities, each at their own level, contribute towards the achievement of
the specific objectives.WT/COMTD/W/74
Page 9
b) Logical intervention
Logical intervention is the stage at which the objectives are incorporated into the
corresponding level in the logical framework and the possibilities for transforming them into results
or activities are assessed.  It is the sum of the action undertaken to achieve the objectives fixed.
c) Indicators
Indicators describe the overall objectives, the specific objectives and the results in operational
terms (quantity, quality, target groups, period and place).  They thus give the objectives of the
activities a quantitative and/or qualitative form and show the situation once the objectives have been
reached.  They must provide a measurable picture of the situation.  Operational descriptions make it
possible to verify the relevance and viability of the specific objectives, the performance and the
success of the results obtained, and ensure the monitoring of achievements.
Indicators used to verify the achievement of overall objectives are usually qualitative,
whereas indicators applied to specific objectives and results are usually measurable in quantitative
terms.  Approximation indicators signal potential developments and are used when precise
quantitative and qualitative indicators are difficult to identify.
Sustainability indicators are used when the results of a project or programme are intended to
be permanent, in other words the results and the ac tivities may continue after the external assistance
has ended (for example, training of trainers or instructors;  level of development of administrative and
institutional capacities, role of the WTO Reference Centres, etc.).
d) Sources of verification
Verification sources are needed in order to verify indicators.  Some verification sources can
be found within a project or programme (the activities) while others are outside the programme, i.e. at
the level of national institutions and/or within the framework of the organization providing the
assistance (for example, the WTO).  Credible and accessible data are essential for a good verificationsystem and subsequently for monitoring and evaluation.  Verification sources may be defined during
the preparation of the activities and developed in the course of implementation.
Each performance-measuring system calls for a procedure under which the stakeholders can
agree on the objectives and indicators.  The logical framework is the most appropriate method in thisrespect.  It is one of the "good practices" and used by a large number of development agents.  It may
appear mechanical and simplist, but it must be used in a participatory context and given a critical
examination throughout the cycle of the project or programme.
There is always a link between the logical framework and the basic format of the monitoring,
implementation and evaluation reports to be prepared.  This ensures that there is a certain consistency
in the approach to monitoring and evaluation on the one hand and, on the other, provides a link amongthe programming, implementation, monitoring and evaluation stages, in other words, an interactive
relationship among the various processes of the whole management cycle.
e) Hypotheses
Hypotheses are defined as situations that it is hoped to achieve in order to reach the expected
results and over which there is no control.  In other words, these situations are deemed hypotheses
when they cannot be placed among the "logical interventions".  The formulation of hypotheses is thus
a necessary precondition for achieving the objectives of technical cooperation activities (for example,WT/COMTD/W/74
Page 10
the existence of the political will to make a Reference Centre into an operational instrument, keeping
trained staff in appropriate responsible posts, etc.).
7. Conclusions and recommendations derived from monitoring and evaluation
Conclusions are the evaluator's findings after analysing a situation.  They constitute the
response to his/her questions and are the basis for his/her recommendations.
After each evaluation, the evaluator makes recommendations on proposals for modification,
cancellation or strengthening.  They are put forward with the aim of improving the content, quality,
relevance, impact and sustainability of the results of technical cooperation activities.
Recommendations also help to improve implementa tion procedures and the cost-effectiveness aspect
of the activities evaluated.
8. Reports
Monitoring and evaluation reports are valuable tools and sources of information that can be
used as a basis for decision-making and the learning process for the implementation and future
direction of technical cooperation activities.  They help to supplement the technical cooperation
database by providing evaluation elements, notably information on the relevance, performance andsuccess of these activities and on the lessons learned.
(a) Monitoring reports are drawn up in the course of implementation in order to detect relevance
and performance aspects and the first signs of any potential problems or successes.  Where necessary,
such reports should contain practical recommendations so that the required remedial or reinforcementmeasures can be taken during implementation.  Monitoring reports are prepared on the basis of an
analysis of the elements noted in the course of observation, bilateral contacts, the study of documents
and material, and through the questionnaires filled in by the direct beneficiaries.
(b) Evaluation reports focus both on matters of relevance and performance and on an assessment
of how successful the activities have been at all stages of design and implementation.  Evaluation
reports must contain an analysis of the situation, an overall assessment, findings, conclusions,
recommendations and the lessons to be drawn.  They are prepared on the basis of the elements
contained in the monitoring reports and involve an overall analysis and assessment of the technical
cooperation activities.
9. Lessons to be drawn
It is necessary and even essential that the information provided during the monitoring and
evaluation process should be formulated appropriately, circulated to potential users and used by them.
The main users are those responsible for implementing activities and the beneficiaries, as well as
regional or subregional institutions taking part in the WTO's technical cooperation activities.  The
mechanisms that allow the information obtained from monitoring and evaluation to be utilised can besummarized as follows:
(a) Organization of feedback
Monitoring allows those responsible for managing technical cooperation activities to identify
the following two categories of inform ation during the implementation stage:
• Positive elements that need to be strengthened;
• problems that require remedial action.WT/COMTD/W/74
Page 11
Evaluation allows observations, conclusions and recommendations and provides lessons on
what works and what does not.  The lessons drawn reinforce the learning process and help to give a
better picture of ways in which the quality of activities can be improved.  They focus on the
following:
• Design and formulation, in particular, the objectives, participants' needs and the
selection of success indicators;
• implementation and performance;
• monitoring and evaluation.
The feedback obtained from monitoring and evaluation should be general or specific and have
a broad scope so that it can be used in more than one country or region (transferable).  The approach
to evaluation and the gathering of information could therefore be either thematic or by region.
Feedback should be targeted at the needs of potential users, simple and designed so as to
encourage its follow-up.
(b) Dissemination of feedback
This information could be disseminated to users, the Technical Cooperation Division and
other Secretariat Divisions through the following support mechanisms:
• Database on technical cooperation activities:
- Regular entry into the database of the relevant information obtained from
monitoring and evaluation by creating a "monitoring and evaluation" section;
- ensure that all the Secretariat staff have access to this database or, if not,
ensure that the "monitoring and evaluation" section is fully accessible (bylevel of access);
- give users of this information an opportunity to react and establish a
discussion forum so that the process is interactive and the database is
dynamic;
-  human resources are usually provided to cover these tasks.
• Regular publication of the results of the monitoring and evaluation (bulletin);
• meetings of the Technical Cooperation Division with the focal points of other
Divisions.  It is desirable to hold meetings of the TCD as a whole with the focalpoints of other divisions;
• annual reports to the CTD.
(c) Utilisation of relevant information
The relevant information obtained from monitoring and evaluation should be incorporated
into future activities.  Before implementing new activities, those in charge should ensure that the
feedback has been examined and that the relevant aspects have been taken into account when planning
the activities.  During implementation, they should ensure that the relevant lessons learned from past
experience have been properly included in the services supplied to beneficiaries.
Recommendations concerning the overall direction and the WTO's technical cooperation
policies must be approved by the CTD before they are circulated or used (annual evaluation report toWT/COMTD/W/74
Page 12
the CTD).  Other recommendations and experience could be disseminated and used after they have
been approved within the CTD.
Examples of good practices already used could be collected by the Secretariat staff, even on
an informal basis, and disseminated through the same mechanisms.  Good practice can sometimes be
quite simple, even "straightforward", but have an important impact on the implementation of activities(for example, circulation of training handbooks at an early stage in the form of an easy-to-use file
could help to strengthen participation by beneficiaries in training courses;  the simultaneous use of
academic teaching methods and the WTO's technical methods could be a valuable contribution to a
better understanding of the themes tackled).
10. Responsibility for monitoring and evaluation
The beneficiary countries, donors and the Secretariat are all concerned by the results of the
WTO's technical cooperation activities.  The monitoring and evaluation processes assist this effort by
further improving the design and implementation of the activities and identifying more clearly the
objectives to be attained.  This is why monitoring and evaluation are a responsibilities shared among
the three parties concerned, each at its own level.
Beneficiary countries should be closely involved at all the management stages of the technical
cooperation they receive, particularly the programming, implementation, monitoring, and evaluation
stages.  Their views on the objectives, the quality, the implementation and the impact of technical
cooperation are of vital importance for monitoring and evaluation.  They can make their views known,
inter alia , through questionnaires, interviews and reports to various WTO bodies, particularly the
CTD.
The CTD is the intergovernmental structure which oversees the monitoring and evaluation of
the WTO's technical cooperation activities.  It considers the reports on implementation of technical
cooperation, draws up the framework for monitoring and evaluation, decides to undertake theevaluations needed and reviews the conclusions and recommendations contained in the evaluations.
The Secretariat supports the CTD's work by putting forward appropriate proposals, preparing
implementation and monitoring reports and an annual internal evaluation report.  The Secretariat
organizes and supervises the various stages of any external evaluation decided by the CTD.
III. CONDUCT OF AN EVALUATION
11. Planning an evaluation
The following are the various stages in planning an evaluation:
• Decision by the CTD;
• definition of the objectives of the evaluation;
• development of a methodology for data collection and analysis;
• clear definition of the specific objectives;
• identification of the performance and success indicators and discussing them with
stakeholders;
• data collection and analysis;
• preparation of reports.WT/COMTD/W/74
Page 13
12. Defining and prioritising objectives
The first step is to analyse the problems that have served to determine the objectives of the
activities.  This involves defining the cause-effect relationship of the negative aspects of a particular
situation and identifying the bottlenecks on which the priorities should focus.
In the WTO context, the following are some of the problems which require intervention:
• Limited understanding of the multilateral trading system (MTS) and its Agreements
on the part of personnel in both State bodies and the private sector;
• in some countries, there is a lack of expertise and limited understanding of the
implications of the Agreements, particularly as regards decision-making at thenational level;
• problems in adapting the national, legal and regulatory framework and implementing
certain Agreements such as those on Technical Barriers to Trade, Customs Valuation,
Anti-Dumping Procedures, Trade-Related As pects of Intellectual Property Rights,
etc;
• weakness of the infrastructure when dealing with specific subjects and the lack of an
integrated approach to trade matters;
• considerable difficulty in complying with notification obligations;
• inability of the institutional framework to meet the challenges of a global economy;
• limited research capacity in order to meet the challenges arising from a more open
economy and problems of access to trade information;
• lack of coordination at the national level, particularly as regards amending legislation
and implementing Agreements.
The objectives then have to be defined.  This  means describing the future situation once the
problems have been resolved.  On the basis of the problems-needs identified above, the specific
objectives can be circumscribed, an important step in carrying out monitoring and evaluation under
proper conditions.  It is also necessary to verify the priorities of the objectives and determine themeans to be used to achieve the results.  The choice of performance and success indicators, which are
so necessary for monitoring and evaluation, requires well-defined and prioritised objectives.
Section III on "Evaluation and the WTO's technical cooperation objectives" provides further
information on prioritising objectives.
13. Selecting indicators
As explained above, indicators are usually signals which show changes in a situation or
specific results.  They furnish the proof that the objectives fixed have been reached in the course of
the activities.
They are operational descriptions which should show the relevance and viability of the
specific objectives and results and ensure the monitoring of the achievements.
They should answer the following questions:
• What is the quality and/or quantity of the inputs and outputs?
• How are they presented?
• Who benefits from the services?
• For how long will they be available and how are they progressing?WT/COMTD/W/74
Page 14
The qualitative indicators are used to see whether the general objectives have been reached
(for example, noting active participation in the various WTO processes), whereas quantitative
indicators are generally used to verify the attainment of specific objectives (for example, number of
notifications made over a given period).
Sustainability indicators are used to measure the ongoing results of a project or programme
after the activities have been carried out.  In other words, they measure the sustainability of the impact
of the results even after the assistance has ended (for example, training of trainers or instructors,
creation of WTO Reference Centres, etc.).  Sustainability indicators may relate to the following
elements:
• Inputs continue to be distributed after the activities have been completed;
• the government continues to assist the target group;
• training of trainers or instructors has a cumulative effect over the long term;
• the training budget continues to be available after implementation.
14. Tools for the collection and analysis of information
There are several ways of collecting and arranging information to be used as a basis for
monitoring and evaluation:
• Observations when the evaluator takes part in the activity;
• means of communication such as the telephone and fax;
• questionnaires on feedback to be filled in by the direct beneficiaries of the services
supplied.  These questionnaires should be simple and contain targeted questionsrelating to the relevance, performance and success of the activities.  They should also
contain precise measurement parameters and a corresponding rating system so as to
facilitate the task of those doing the rating (beneficiaries) on the one hand and the
analysis of the information collected with a view to evaluation on the other;
• interviews based on the subjects and scope of the evaluation;
• group discussions targeting beneficiaries with similar objectives;
• analysis of documents, publications and background material by the evaluator and/or
a specified group of readers;
• a survey usually targeted on a particular theme or sector, which calls for a major
effort to collect data to be used for the planned evaluation;
• media (articles in the national or international press on one or other of the activities or
the activities as a whole).
15. Rating system
A rating system is essential for evaluation because it helps to formulate and validate
assessments of the relevance, performance and success of technical cooperation activities using a
scale of numerical, alphabetical or descriptive code s.  It should be used for questionnaires aimed at
collecting information on feedback from beneficiaries (see the questionnaires used in the context of
monitoring and evaluation of the WTO's cooperation activities).
It is recommended that the following be done when designing a rating system:
• Define the parameters of the values to be  assessed according to whether the focus is
on relevance (very relevant, relevant or not relevant)/highly appropriate, appropriate
or inappropriate);  performance (excellent, average or weak);  or success (very
satisfactory, satisfactory or unsatisfactory).WT/COMTD/W/74
Page 15
• Use numerical classification codes together with descriptions in order to facilitate a
statistical analysis of the questionnaires.  In a rating system, there should be an even
number of parameters so as not to influence choices towards the middle mark.
• Study the questions to be asked carefully so that they properly reflect the specific
criteria of relevance, performance and success (for example among the subjects dealt
with during the seminar, which were the most appropriate and the most useful for
your official tasks?  Among the subjects dealt with during the seminar, which were
those that were satisfactorily covered, not satisfactorily covered or not covered atall?).
IV. EVALUATION AND  THE WTO'S TECHNICAL COOPERATION OBJECTIVES
16. Technical cooperation objectives from an evaluation viewpoint
First of all, the overall (or general) objectives must be distinguished from the specific
objectives and a constructive relationship and hierarchy must be established between these two
categories.  It must be clear that specific objectives are stages that have to be completed before
arriving at the overall objectives.
In the WTO, the following are specific objectives that should enable the overall objectives
fixed in the Guidelines for WTO Technica l Cooperation (W/COMTD/8) to be reached:
• Dissemination of an overall picture of the WTO and its Agreements;
• upgrading human resources so as to build capacity to participate in the WTO and its
decision-making process;
• helping to develop institutional and human capacity for effective participation in the
MTS and full and effective utilisation of its provisions and mechanisms;
• gaining the expert knowledge and building the institutional capacity needed for better
implementation of the WTO Agreements.
In design terms, the objectives are prioritised at various levels, each of which necessitates a
particular performance and consequently special criteria and verification indicators.
Inputs are used to undertake activities aimed at supplying outputs
.  As these are services
furnished, the outputs have a quantitative and qualitative scope related to the extent and degree to
which the direct beneficiaries have been reached.  Up to this level of the process, verification ofperformance remains an internal management exercise concerned with economy, efficiency,
distribution of resources, expertise, quality of inputs and excellence of outputs.
To verify the performance of the results and the impact of the activities, it is necessary to go
beyond management aspects and focus on the way in which the direct beneficiaries react to the
outputs provided, their degree of satisfaction and the immediate change in their situation.
Nevertheless, certain problems have to be overcome in order to reach an appropriate
assessment of the performance of the results and the impact, whose precise identification appears
more complex.  The problem stems from the fact that "the relationship between the means and theend" is not always incorporated in the simple pr ioritisation of the objectives.  Let us take as an
example an activity that aims to build capacity in the area of notifications and let us see the sequence
and logical chain between the supply of services and the achievement of the goals and objectives
fixed:WT/COMTD/W/74
Page 16
Activity:  provision of technical advice in the area of notifications
• The Secretariat prepares and provides advice;
• the beneficiaries participate in working meetings;
• the beneficiaries consider that the services meet their needs;
• the beneficiaries gain new knowledge and their proficiency improves;
• the beneficiaries change their pattern of behaviour and modify their notification
practices;
• enhanced proficiency and changes in the pattern of behaviour result in better
implementation;
• improved implementation leads to better participation in the WTO and the MTS.
The level of the sequence at which the outputs, the results  and the goals  have to be specified
must be defined.  Is the provision of technical advice the output?  If this is the case, is the
management personnel responsible for the relevance of activities?
What is the outcome of the activities?  Is it new knowledge and enhanced proficiency or is it a
change in the beneficiaries' pattern of behaviour?  Is it better implementation of the Agreements or
more active participation in the MTS?
As one moves from one level to another in this sequence, the response increasingly eludes the
control of the person responsible for management and becomes subject to the influence of externalfactors and risks.  Consequently, it is not fair to hold the persons in charge of managing the activities
responsible for changes in the situation of the beneficiaries, but neither is it fair to absolve them of all
responsibility.
This is why it is recommended that indicators be identified for specific use at each stage and
each level in the classification of the objectives fixed.  It is more correct to think of an ongoing
process and to assess performance using specific criteria for each level of objectives, keeping a
balance between the verification indicators for the performance of outputs, which can easily be
measured, and those used for the results, whose performance is less easy to control.
In this regard, the use of a logical framework is  strongly recommended in order to resolve the
problem of prioritising objectives and to establish a structured relationship between these on the one
hand and the performance verification indicators on the other.
17. Indicators for verifying results
In the WTO's technical cooperation activities, indicators are pre-set signals that are defined
beforehand during the programming and planning stage and at the time of assessing the needs of thebeneficiaries and fixing specific objectives.  When doing this, the objectives of the activities, the
target beneficiaries and their needs, the expected changes following the implementation of the
activities, the extent and degree of the effectiveness with which the activities reach their objectives,
and the criteria to be fixed for assessing the success of the activities (impact, sustainability and
contribution to capacity-building) must all be considered.
There are quantitative, qualitative and approximation indicators.  The former are used to
measure the results of activities in terms of volume and according to a cost-effectiveness approach
(for example, the number of Reference Centres established over a given period and the cost-
effectiveness ratio between the outputs used and the results achieved; the number of notificationsmade by a country, group of countries or region, and its trend during a given period; the number of
hits on the WTO website by Reference Centres and their trend during a given period).WT/COMTD/W/74
Page 17
Qualitative indicators are used to assess the pattern and attitudes so as to evaluate the impact
and sustainability of the results (for example, the Reference Centres become operational; there is a
better flow of information between the WTO Secretariat and national officials;  active participation in
the WTO;  effective utilisation of the WTO dispute settlement mechanism; a satisfactory level of
basic knowledge for better implementation of commitments, etc.).  The Trade Policy ReviewMechanism could be used as one of the sources of identification and verification of qualitative
indicators.
Approximation indicators are used when the data ar e difficult to find.  They help to show the
trend in performance and the signs of success (for example, as a result of one or more technical
cooperation activities, the beneficiaries seek more detailed information;  special requests by Reference
Centres;  good reaction to new and emerging issues, etc.).WT/COMTD/W/74
Page 18
Examples of objectives and performance indicators
(This list is not exhaustive and the elements therein are purely indicative)
Specific objectives Indicators
1. Disseminate general information on the
WTO, its Agreements, its institutional aspects
and its mechanisms
→Familiarize beneficiaries with the rules and
fundamental principles of the MultilateralTrading System (MTS).
→Contribution towards a better understanding of
the implications and issues at stake in the MTS.*  Comprehensive basic knowledge
(behaviour)
*  Better understanding of the implications and
issues at stake (behaviour)
*  Guaranteed exchange of information between
government officials and representatives of the
private sector (behaviour, mechanisms andpolicy)
2. Improve access to information on the MTSand develop ongoing contacts between the WTO
and beneficiaries
→Reference Centres*  Reference Centre is functioning (institutional
development)
*  Number of Centres becoming operational over
a given period (policy and behaviour)
*  Information flow between the WTO and the
beneficiary countries (behaviour)
*  Number of Internet hits by the Reference
Centres over a given period (behaviour)
3. Build human capacity for more activeparticipation in the WTO and its decision-making
process*  Active participation in the WTO bodies and
processes
*  Capacity to react to new and emerging issues
developed
4. Strengthen capacity for better utilisation ofthe WTO mechanisms*  Better utilisation of the DSU
*  Better utilisation of the TPRM*  Better utilisation of S and D
*  Proper utilisation of the new opportunities
given by the Agreements
5.  Strengthen capacity for better implementation
of the Agreements
Market Access
• To help to establish legislation/regulations
on customs duties in conformity with the
obligations in the WTO
• To help to transpose customs nomenclatures
dating from before the conclusion of the
Uruguay Round into the Harmonized System*  Notification within the time-limits
*  Application of tariff concessions*  Notification within the time-limitsWT/COMTD/W/74
Page 19
Customs Valuation
• To help in establishing implementation
measures (legislation, regulations and
administrative procedures)
• Appropriate structures, including an appeals
authority (help establish these)
• Access to information on customs valuation
methodology
Import Licensing
Adopt laws, regulations and administrative
procedures
Institutions, including focal points and an
appeals mechanism
Preshipment Inspection
• Legislation, regulations and administrative
procedures
• Body responsible for preshipment inspection
(PSI)
Rules of Origin
Legislation, regulations and administrative
procedures
• Appeals mechanism
Agriculture
• Adopt clearly defined criteria for the
"green box" measures
TBT and SPS (measures and support mechanisms
for TBT and SPS provisions)
• Creation or strengthening of regulatory
bodies responsible for standardization
• Creation of focal points for the TBT and the
SPS
• Establishment of appeal mechanisms
Remedial Measures
• Legislation, regulations and administrative
procedures
• Appeals mechanisms
• Investigation mechanism (anti-dumping)*  Notification within the time-limits
*  Appeals mechanisms operational
*  Customs valuation in place
*  Number of valuations over a given period
*  Notification within the time-limits
*  Import licences issued within a reasonable
period*  Regular (annual) replies to questionnaires on
import licensing
*  Notification within the time-limits
*  PSI operating smoothly
*  Body responsible for PSI set-up
*  Notification within the time-limits
*  Origin of products determined
*  Appeals mechanism operational
*  Notification within the time-limits
* Respect for commitments on domestic and
export subsidies
*  Notification within the time-limits
*  Recognition abroad strengthened
* Number of focal points created over a given
period
*  Notification within the time-limits
*  Semi-annual report on anti-dumping actions
*  Abolition of certain subsidies
*  Semi-annual report on countervailingmeasuresWT/COMTD/W/74
Page 20
TRIMs
• Legislation on the withdrawal of
prohibited TRIMs
Trade in Services
• Legislation, regulations and
administrative procedures
• Focal point and appeals mechanisms
• Evaluation of the implications arising
from the Agreement's rights and
obligations
• Identification and assessment of new
market access opportunities
TRIPS
• Legislation and regulations
• Establish intellectual property rights
(patents, copyright, etc.)
• Focal point
• Ensure that the judicial authorities are
able to respect the provisions of the
Agreement*  Notification within the time-limits
* Prohibited TRIMs withdrawn within the time-
limits fixed
*  Notification within the time-limits
*  Initial commitments put into effect
*  Contact points established by the industrialized
countries and utilized by the developing countries
*  Capacity for seizing new opportunities
developed*  Increase in the share of the country or region in
international trade in services
*  Notification within the time-limits
*  IP rights established
*  Focal point created
*  Protection recognized as required by the
AgreementWT/COMTD/W/74
Page 21
V. THE WTO'S ROLE IN THE AREA OF TECHNICAL COOPERATION
18. Objectives and content
The WTO's technical cooperation activities are carried out with the aim of helping the
beneficiaries to understand and implement the agreed international trade rules, ensure their full
participation in the international trading system a nd strengthen their structures on a permanent basis
so as to allow them to derive full benefit from the new trade environment.
In general,  WTO's Technical Cooperation seeks to inform developing country about their
rights and obligations under the WTO Agreements and to help build their capacity to apply the
agreements, to participate in WTO processes and to use its mechanisms. These activities constitute
one of the instruments of special and differential treatment extended to developing countries in the
WTO Agreements and Decisions ( e.g. Decision on Notification procedures, the Trade Policy ReviewMechanism, the Understanding on Balance-of-Payments Provisions, the Agreement on the
Application of Sanitary and Phytosanitary Measures, the Agreement on Technical Barriers to Trade,
the Agreement on Customs Valuation, the Agreem ent on Preshipment Inspection, the GATS, the
TRIPS and the Dispute Settlement Understanding. In addition, the WTO also provides technical
assistance to help applicant counties to comply with the requirements for accession to the WTO.
In accordance with the CTD approved texts and the basic principles of technical assistance
delivery-programme, Technical cooperation is demand-driven. The WTO exercises flexibility to best
tailor the technical cooperation activities to the needs and priorities of individual countries, groups of
countries or regions within demand-driven approach which is the best procedure in reflecting realneeds and priorities of beneficiaries.
However, the WTO Secretariat would like to apply a more proactive technical cooperation
policy by informing potential beneficiaries on the existence, objectives and modes of delivery of its
technical cooperation activities. Proactive policy can be ensured also by involving the beneficiaries indesigning programmes and activities which encompasses necessarily the identification of
problems/difficulties/real needs, specific objectives, success indicators, outputs and activities to be
delivered and carried out under set objectives. The WTO technical cooperation policy could be also
enhanced by identifying possibilities and areas of cooperation and collaboration with relevant
international and regional institutions.
Within the framework of its objectives, the WTO provides technical assistance within its field
of competence and carries out a wide range of activities in the following areas:
a) Institutional matters and the rules and fundamental principles of the MTS
The Secretariat organizes seminars for the purpose of general dissemination of information on
the institutional aspects of the WTO, its Agreements, its monitoring, negotiation and decision-makingmechanisms.  The aim of these activities is to give an overall picture of the WTO so that participants
can become familiar with the rules and fundamental principles of the MTS and can be helped to
understand better its implications and present and future issues.
To improve access to information on the MTS and develop ongoing contacts between the
WTO and the beneficiary countries, the Secretariat sets up Reference Centres in the latter with
documentation and computer equipment.  The staff working in these Centres are given training.WT/COMTD/W/74
Page 22
b) Agreements on Agriculture and Sanitary and Phytosanitary Measures
For activities related to the Agreement on Agriculture, the Secretariat helps beneficiaries
understand better their rights and obligations under the Agreement and respect the commitments they
have undertaken in relation to market access, domestic subsidies, export subsidies, and sanitary and
phytosanitary measures.  With the opening of negotiations on agriculture, technical assistance is
focusing more on negotiation issues.
c) Market access issues
As regards customs duties, the beneficiary countries are helped to transpose the customs
nomenclatures dating from before the conclusion of the Uruguay Round into the Harmonized System.
Technical cooperation on customs valuation focuses on training personnel, drawing up
implementation measures, access to sources of information on customs valuation methodology andadvice on applying the Agreement's provisions.
Other technical assistance needs concern areas such as import licensing, preshipment
inspection, rules of origin, the supply of information for entry into the WTO Integrated Data Base,
etc.)
d) Technical barriers to trade
The main focus of technical assistance on technical barriers to trade concerns implementation
and obligations relating to the adaptation of the nati onal legislative and institutional framework.  The
Secretariat helps beneficiaries to develop international standards and introduce them at the nationallevel.  It also helps to identify potential institutional improvements that would enhance the quality of
certification laboratories and consequently make recognition abroad easier.
e) Rules
The technical cooperation programme in this area includes assistance to Member countries
and countries seeking accession in drafting contingent trade remedy legislation, with a detailed
examination of and written comments on laws and regulations.  It also includes assistance for the
installation or improvement of the national trade remedies system and its conformity with the WTO
provisions.  Technical assistance also consists of holding consultations with delegations regarding the
interpretation and application of the relevant Agreements.
The technical cooperation programme on rules also includes the following elements:
• Information on rule implementation practices in various countries  Preparation of
model legislation for countries setting up a contingent trade remedies investigation
scheme.
• Training in conducting investigations.  Contingent trade remedies database.
Assistance to developing countries utilizing the transition period on subsidies
disciplines, identifying the subsidies programmes that might be prohibited when thetransition period expires.
• Helping to prepare notifications concerning trade remedies and rules.WT/COMTD/W/74
Page 23
f)  GATS Agreement
In the area of trade in services, the Secretariat helps beneficiaries through its information
exchange programme, overall and sectoral assessments of trade in services, and the establishment of
guidelines and negotiating procedures.
g) TRIPS Agreement
• Facilitating implementation;
• Drawing up laws and regulations;
• Creation or strengthening of national intellectual property organizations, including
training of personnel.
h) Accession
The Secretariat gives countries seeking accession advice of a general nature and explains the
advantages of the MTS.  It provides them with practical and technical information on the accessionprocedure and technical assistance in specific areas that are the subject of negotiations, both systemic
and market access-related.  The Secretariat is also called on to help countries seeking accession to
prepare memorandums and schedules of concessions.
i) Notifications
Several developing countries find it particularly difficult to meet their notification obligations
under the majority of the WTO Agreements.   The Secretariat facilitates understanding of the systemand notification provisions through seminars, workshops and technical missions.
A handbook has been drawn up to assist Member countries to carry out their notification
obligations under the Agreements, focussing on the relevant procedures.  This handbook can be
accessed electronically.  Within the Secretariat, there are enquiry points that can be contacted bye-mail to respond inter alia  to any questions that may be raised by the Member countries concerned.
j) Trade policy review
Developing countries are given technical assistance in preparing their trade policy review
reports.
k) Special assistance relating to the settlement of disputes
Legal assistance to countries involved in disputes under the WTO's dispute settlement
mechanism are provided.
Training courses on the rules and procedures governing the settlement of disputes within the
WTO are also provided.
19. Coordination
Coordination with international and regional organizations dealing with trade-related matters
in order to ensure a consistent approach and:
- Identify the respective and complementary fields of competence;
- promote and develop synergy between the WTO's technical cooperation activities and
those of other relevant regional and international organizations;WT/COMTD/W/74
Page 24
- define and implement common projects;
- avoid overlapping;
- establish a central directory of programmes jointly.
Coordination with the governments of donor and beneficiary countries in order to exchange
information on programmes dealing trade-related matters.WT/COMTD/W/74
Page 25
VI. GLOSSARY
Appraisal
Form of evaluation of activities, action,
proposals or reports.  It is usually carried outat the programming stage and serves as the
basis for approving a programme or project.
Appraisal is also carried out at the end of an
activity in order to arrive at a conclusion.
Approach
There are several approaches to providing
technical assistance.  The "project" approach
consists of a process in which synchronized
and complementary services are provided in
order to achieve consistent objectives within agiven period and in accordance with a pre-
designed programme of work.  The
"programme" approach is a process that
consists of implementing a series of projects
and/or activities that have a common purposeand affect several sectors.
Basic data
Data which describe the situation in which the
technical cooperation activities will be carried
out and used as basis to appraise and measure
the performance of these activities.
Benchmark
Reference point or standard used to compare
progress and achievements (for example,
achievements in the past or by other
organizations;  what could reasonably be
achieved;  what has been budgeted or
targeted).
Beneficiaries
The beneficiaries of technical cooperation are
mainly government officials at various levelswhose functions are related to the
implementation of the WTO Agreements.
Representatives of the private sector may be
associated with the WTO's technical
cooperation activities because this can help toimprove their knowledge of the multilateraltrading system and allow them to identify
opportunities arising from the Agreements.
Capacity building
Process by which individuals, groups,
institutions and societies develop their human
and institutional capacity with the aim of
carrying out their tasks, resolving problems,
fixing and reaching objectives.  In the WTO,
one example is the building of capacity so asto be able to play an active role in the
multilateral trading system.
Concept
The design and formulation of technical
cooperation activities reflect the way in which
these activities are conceived in relation to the
real needs of the target beneficiaries, the
national, regional and international situationsand the sustainability criteria of the results
achieved.  The validity of a concept is also
assessed in relation to the mandate of the
providing organization and its comparative
advantage.
Conclusion
An assessment based on a number of
observations or data corresponding to aspecific situation.
Cost effectiveness
An approach which allows the economic
aspects of an activity to be analysed by
comparing various alternatives for achievingthe same objective in economic terms.  Its aim
is to identify the solution that will help to
achieve the objective at the least cost.
Cycle
All the processes employed in carrying out the
various actions undertaken in the context of
technical cooperation activities.  It comprises
operational elements in the form of aconstructive and synchronized sequence.  The
programme cycle of the WTO's technicalWT/COMTD/W/74
Page 26
cooperation activities can be summarized as
follows:  three-year plan (adjusted annually);
programming;  implementation;  monitoring;
evaluation.
Database
The means used to store target information and
arrange it electronically so that it can be
disseminated and used by those responsible for
management, monitoring and evaluation.
Effectiveness
Extent to which the activities attain the
objectives fixed and the results expected.
Effects
Results of technical cooperation activities in
relation to the objectives fixed and the groups
or institutions for which the activities have
been undertaken.
Efficiency
Best possible achievement of the objectives,
i.e. the least costly resources are used to attainthe objectives.
Evaluation
Time-bound process to assess the relevance,
performance and success of activities.  It may
take place during the activities (mid-term
evaluation), after they have been completed
(ex post or a posteriori evaluation).  It may be
internal (self-evaluation) or external
(independent).
Feedback
Information obtained from the monitoring and
evaluation.  It consists of arranging,
disseminating and using relevant informationobtained from monitoring and evaluation and
provides a basis for decision-making and
learning.Goal
The reason for an activity, project or
programme.  It can also be simply termed the
objective.  It is usually the basic reason for
which the results are sought.  It is identified onthe basis of the problems and needs of
potential beneficiaries.
Hypothesis
Factor which affects the achievement of the
objective and is outside the scope of theactivities, the project or the programme and
does not depend on those in charge of
management.  The attainment of the objectives
fixed is to a large extent determined by the
formulation of hypotheses.
Impact
The extent to which the activities have had
results for the beneficiaries in relation to theobjectives.  It represents a change in the
situation of the beneficiaries following the
provision of technical cooperation.  The
impact is usually linked to long-term
objectives, for example, capacity building.
Indicator
Sign which shows that the objective fixed has
in fact been attained.  It describes the situationonce the objective has been reached.
Indicators show changes of a qualitative or
quantitative nature or in the timetable.
Inputs
Means used to carry out technical cooperation
activities (human, financial or material
resources).
Lessons to be drawn
Learning based on the experience applicable to
a generic situation.  The lessons to be drawn
are practical ones and transferable, that is to
say that they can apply to more than one
country.WT/COMTD/W/74
Page 27
Logical framework
A management and evaluation tool in the form
of a matrix containing a series of elements that
are logically and constructively linked.  It
allows a structural analysis of the cause-and-effect relationship among the objectives,
performance indicators and the hypotheses
identified in the context of a series of
activities, a project or a programme.
Logical intervention
The stage of transposing the objectives to the
corresponding levels in the logical framework,
identifying the possibilities for translating
them into activities or results.
Management
All the functions which combine the action to
be taken to attain objectives.  These relate to
the formulation, organization, implementationand monitoring of activities, as well as to the
evaluation of the performance of such
activities.
Monitoring
Ongoing process throughout the activities in
order to provide early indicators of progress or
malfunctioning.  Monitoring of achievements
verifies the progress made in implementation(time-limits, costs and physical realization).
Monitoring of methods of conduct relates to
the implementation process (processes used).
Objectives
Purpose or goal showing the outcome which
the technical cooperation activities seek to
achieve.  There may be overall or specific
objectives.  The latter contribute towards the
achievement of the former.  Objectives are
prioritised by placing them at different levelsin order to circumscribe the responsibility of
the various stakeholders and clarify their
relation to the performance indicators.Observation
The fact noted on the basis of the data
collected in the monitoring and evaluation
process.
Output
Tangible result of an activity, project or
programme in comparison with the objectives
fixed.
Performance
Extent to which the activities have reached
their objectives as regards effectiveness,
efficiency and timeliness.
Programme
See "Approach".
Project
See "Approach"
Providing organization
The organization which manages the
implementation of technical cooperation
activities for beneficiaries in the light of its
own mandate and according to the specific
objectives fixed.
Rating system
Tool that allows the relevance, performance
and success of technical cooperation activities
to be assessed using measurement parameters
(in the form of numerical, alphabetical ordescriptive codes).  It is generally used in
questionnaires to collect feedback from
beneficiaries.
Recommendations
Proposals for the adoption of measures in a
specific situation and identifying the parties
responsible for their implementation.WT/COMTD/W/74
Page 28
Relevance
Extent to which the activities and their
objectives are valid and meet the real needs of
the beneficiaries taking into account the
national and regional contexts and the externalenvironment of the beneficiaries.
Reports
Reports on implementation describe the actual
implementation of the activities without
providing any conclusions or assessments.They describe the implementation of the
activities in relation to the plan adopted.
Reports on follow up identify the early signs
of progress or shortcomings and, in the mid-term, propose remedial measures, the objective
being restricted to implementation.
Reports on evaluation analyse the situation,
provide an overall evaluation, set out findingsand propose recommendations and lessons to
be drawn for the attention of those responsible
for management and the beneficiaries.  The
objective goes beyond implementation and
proposes elements that can be used for futureguidance and technical cooperation policy.
Result
Output, effect and impact.Stakeholders
Groups or institutions that play a role or have
an interest in the objectives and the
implementation of activities, projects or
programmes.  They may be target groups(direct beneficiaries or direct users of
services), competent national and/or regional
institutions, persons responsible for ensuring
that the results come up to expectations and
persons who provide the resources for the
implementation of activities.
Success
Progress, achievements and performance
observed after the technical cooperation
activities have ended.  One way it can bemeasured is through the impact and
sustainability of the results achieved in
comparison with the objectives fixed (capacity
building).
Sustainability
Positive results continue after the activities
end.  The benefits resulting from the activities
continue for the direct beneficiaries or forother groups and in other situations.
Target group
Group of persons taking part in an activity,
project or programme and who will benefit
from the expected results of the action.
Timeliness
Provide the inputs and achieve the results
within the appropriate time-period.
__________