Toolkit Links
What is collective benchmarking?
5 Nov 2008
Source: HRSA Toolkit
Benchmarking is based on the notion that information from a range of countries will provide a better understanding of shared problems; development of new solutions to those problems; or new mechanisms for developing and implementing policy that will improve performance as well as cooperation across countries. Existing international classifications (OECD's Frascati Manual
), indicators (EC Research Directorate General-Eurostat collective benchmarks), and data (Ibero-American Network on Science and Technology Indicators) lack measures covering a broad range of health activities. Many existing indicators or data are too general covering all science and technology activities and capacities; too narrow focusing only on biomedical research activities and capacities; or do not track indicators of policy relevance to low- and middle-income countries.
Methods
First, a review of existing case studies on national health research systems was undertaken, which highlighted the need for quantitative indicators as well as existing qualitative descriptions. Second, explicitly linked to the conceptual framework, HRSA network members proposed 14 core indicators to describe attributes of a well-functioning health research system, related to each functional area. These are presented in the following table:
|
Core indicators by health research system function |
|
|
Function 1: Stewardship |
|
|
Indicator 1 |
Degree to which stewardship function is fulfilled, in the following four areas: |
|
vision |
|
|
priorities |
|
|
ethics |
|
|
monitoring & evaluation |
|
|
Indicator 2 |
Research and development expenditure on explicit national priorities for health research financed by public funds, in relation to total health research and development expenditure financed by public funds, in international $ |
|
Indicator 3 |
Health research proposals submitted for ethical review, in relation to total health research proposals requiring ethical review (e.g., human subjects, genetics, stem cell, animal, etc.) |
|
Function 2: Financing |
|
|
Indicator 4 |
Total funds allocated to health research, in relation to gross domestic product (GDP), in international $ |
|
Indicator 5 |
Public funds allocated to health research, in relation to total health expenditures, in international $ |
|
Indicator 6 |
Share of total funds allocated to health research in each of the following areas: |
|
Non-oriented, fundamental research |
|
|
Health conditions, diseases or injuries |
|
|
Exposures, risk factors that impact on health |
|
|
Health systems research |
|
|
Research capacity building |
|
|
Anything not covered by above categories |
|
|
Function 3: Creating & Sustaining Resources |
|
|
Indicator 7 |
Active health researchers (full time equivalents) in relation to total workforce |
|
Indicator 8 |
Supportive context of research environment |
|
Range and breadth of health researcher networks |
|
|
Transparency of the funding process |
|
|
Quality of the work space and facilities |
|
|
Encouragement of collaboration with others |
|
|
Opportunities to present, discuss and publish results |
|
|
Relevance of health research activities to health problems |
|
|
Education and continuous training |
|
|
Wage of health researchers |
|
|
Nurturing of careers |
|
|
Indicator 9 |
Average wage, by sex, in international $, for: |
|
Newly graduated PhD/doctorate entering health research system with full time position |
|
|
Senior researcher with PhD/doctorate with at least 20 years of post-doctorate experience |
|
|
Senior researcher with PhD/doctorate who is a director of a research institute/large research group |
|
|
Indicator 10 |
Trend in total public funds allocated to health research, in relation to total health expenditures expressed as annual change |
|
Indicator 11 |
Institutions within the health research system with access to at least 20 national and international health journals (full text print or electronic versions) in relation to total institutions within the health research system |
|
Function 4: Producing & Utilizing Research |
|
|
Indicator 12 |
Health research articles published in ISI, Medline and regional reference databases including peer reviewed journals in relation to total active health researchers |
|
Indicator 13 |
Newspaper articles citing health research systems activities in major newspapers in relation to: |
|
All newspaper articles addressing health issues in major newspapers, during a defined time period |
|
|
All news articles in major newspapers during a defined time period, in column centimeters |
|
|
Indicator 14 |
Patents registered resulting from health research in relation to active health researchers |
|
Internationally registered |
|
|
Nationally registered |
|
In addition to these 14 core indicators considered useful in cross-national discussions, 42 descriptive variables were identified that support context-specific national discussions.
For a full list of indicators and variables, click here .
These indicators were identified through a consultative, bottom up, policy oriented process with partners in national governments, ministries of health and science and technology, institutions conducting research on health topics, those that provide training on research methods addressing health topics and international organizations. Third, all national teams (i.e. in Brazil, Cameroon, Costa Rica, Indonesia, Islamic Republic of Iran, Kazakhstan, Lao PDR, Malaysia, Pakistan, Russian Federation, Tanzania, Thailand, and Senegal) conducted a desk review of what data existed already in their respective countries, the format of the data, time series, etc. Almost all core indicators were unavailable in each country. Fourth, HRSA network members, informed by technical advice from WHO and other scientific collaborators, examined each potential indicator and what approach could be used to obtain nationally representative data. Fifth, with WHO technical staff and expert consultants, the HRSA network collectively developed and tested a suite of tools that would collect data on all proposed core indicators and descriptive variables. The resulting toolkit enables measurement and discussion of these indicators and descriptive variables.
For more information on the main headings of the tools and the national sample sizes used in the development phase, click
here .
There are no comments about this article: Please login if you want to submit a comment.