Interciencia
versión impresa ISSN 0378-1844
INCI v.34 n.11 Caracas nov. 2009
TO REALLY EVALUATE OUR SCIENCE
How to evaluate science and scientists is a long standing problem with few solutions for certain fields, particularly in the so-called "developing" countries. The difficulties for an adequate evaluation stem from how appropriate are the indicators utilized to do it, which may vary according to the goals persued.
Probably, the only clear aspect about this matter is that for developed countries with a predominance of the English language as the means of communication among scientists, the evaluation performed with the impact factor implemented by the ISI, resulting from the received citations, represents an adequate first and partial approximation to the scientific quality of researchers, journals and institutions. It is a first and partial approximation, as the indicator distorts its own validity due to the high scores resulting from citations received by papers on methodologies of generalized usage and by thematic reviews in fashionable areas. For journals, it is awkward due to the weight of the large predominant publications. The indicator is also distorted by the low values assigned to papers in some disciplines with particular characteristics that follow a publication pattern that can be considered "atypical". More so by the publication in languages different from English and, therefore, less accessible, or by the lack of interest by mainstream science for certain topics which are of interest, and sometimes of great interest indeed, in countries such as Latin American ones.
The paper by Ricker et al. in this issue of Interciencia develops some of these points and presents a proposal for a better evaluation of scientific production.
The distortions that worry scientometrists in the first world have prompted the development of indicators that are more appropriate to certain ends. One of them is the h index, which measures the relative quality of a scientist or group of scientists through the number of publications and the number of citations they receive. Others are the eigenfactor, which classifies the influence of journals according to their usage, applying network theory, or the SJR of SCImago, which positions journals and countries according to data from the Scopus data base.
The problems posed by the competition generated by an exaggerated race to publish a larger number of papers, or the preference for more extended and dense publications versus dividing the work in multiple reports, has a lot to do with the results of the diverse ways used to evaluate, although they obviously are not an outcome from the latter.
Another aspect of the matter is that in the realm of science and technique, the arguments that pose more value on the vernacular than on the universal cease to make sense in an interrelated world with a globalized economy. Such arguments, nevertheless, are of full value in matters of the culture and the expression of nations. There is diversity in the scientific communities, and their contexts and interests have particular aspects that deserve being taken into consideration.
For Interciencia, as a multidisciplinary journal, the evaluation of this condition is particularly important. Large journals such as Nature, Science, PNAS, for instance, have very high indexes, and it is also more likely to reach a significantly larger value when a journal concentrates on a scientific audience limited to a given specialty. In 2008, Interciencia was displaced in the WoS valuation based on the eigenfactor, from the thematic category of "Multidisciplinary" to that of "Ecology", to which it certainly does not belong. After being historically placed, during its 32 years in the SCI, close to the middle of a group of some 50 journals in its category, in 2008 it falls to the last quartile in a group of some 125 specialized journals.
Notwithstanding, the most harmful aspect of the situation for our countries is the cultural distortion of our scientists, who prefer quoting mainstream papers rather than referring the efforts of their colleagues in institutions of developing countries, as well as the tendency of some to underestimate any evaluation system based on foreign patterns.
Miguel Laufer, Editor











uBio 
