Jacso, Peter.

Comparison of Journal Impact Rankings  in the SCImago Journal & Country Rank and the Journal Citation Reports databases (preprint)

Published version available in  Online Information Review, 2010, Vol 34, No. 4, pp. 642-657.    

 

The open access SCImago Journal & Country Rank (SJR) database offers essential scientometric information for more than 17,000 scholarly and professional journals based on data licensed from the Scopus database of Elsevier. These metrics include traditional as well as novel indicators for measuring publication productivity and prestige at the journal level for the past 13 years. They  offer very informative  new insights to those that have been provided by the subscription-based Journal Citation Reports (JCR) for more than three decades by the Institute for Scientific Information (ISI) and its successor, the Thomson (later Thomson-Reuters) company.  Especially valuable are Scimago’s  features of a) weighting the citations received based on  the prestige of the citing journals, b) the (partial) exclusion of journal self-citations, and c) the  broader base of source journals. They provide new opportunities to analyze and understand their effects on the ranking of journals.

 

Introduction

The open access SJR database was launched in 2007, and has been regularly updated twice a year with new data and new features. It is likely to foster the pursuit for exploring further metrics, and the relationship among them as suggested and practiced by the most competent, highly active  and innovative scientometricians (Egghe 2006a and 2006b,  Leydesdorff, 2009 , Martin, 1996, Moed, 1999 and 2010, Rousseau, 2009) – to name a few, realizing that the best way to characterize the productivity and prestige of journals is through the combination of various measures and indicators simultaneously (Bollen et al., 2006).

This is much facilitated by the fact that the entire SJR data set can be downloaded in tab delimited format for direct import into any spreadsheet and database management software. (This generous option could be improved by providing the same export option for the additional data elements available on the Data  page of the journals’ records that are currently not included in the download format - especially the number of cited and uncited papers per year per journal.)

 

Figure 1.  Data page for  the journal Scientometrics in SJR

The SJR database has already generated considerable interest (Falagas et al., 2008, Siebelt et al., 2010, Torres-Salinas, 2010, Jacso, 2009b). The high ratio of Spanish authors from different universities and research centers indicate the intense involvement of the increasing number of scientometricians in the country who take part in the development and testing of SJR. Many have significant experience and in-depth knowledge of Scopus (De Moya-Anegon, et al. 2007) which contributes to the substantial analysis of SJR. The most comprehensive, very current analysis of SJR is already available as a preprint (Gonzalez-Pereira et al, 2010). On the other hand, the lack of such experience shows in conference presentations, blogs  and even in a peer reviewed journal’s paper (Guz and Rushchitsky, 2009) that in spite of its title discusses primarily the SJR database, but attributes many of its features to Scopus, and repeatedly claims that Scopus is a free service.

The excellent paper on the rankings of information and library science journals by JIF and by h-type indices (Bar-Ilan, 2010) is the closest to this research paper, but it was published too late to make comparisons between her findings about impact factors and h-index variants for the journals in the ILS category of JCR and those produced by the Scimago Group.  The closest product/service to SJR is the also open access Eigenfactor database/service that was launched in 2008 and was evaluated earlier (Jacso, 2010a)   

As of this writing, SJR covers the period from 1999 to 2008. Because most of the impact metrics of SJR uses a 3-year target window, the coverage of SJR can be considered to be of 13 years, even though the year-by year breakdown are available only for the past 10 years. A new edition extending the coverage to 2009 is expected by mid-2010.

SJR is based on data compiled by Elsevier for its subscription-based Scopus database featuring state of the art software options, especially in the sophisticated presentation of the list of cited references (Jacso, 2009g), and through the built-in functions of the Journal Analyzer module. Earlier this year Scopus added the SJR score developed by the Scimago Group to the Journal Analyzer module along with  the SNIP score  (Moed, 2010), another novel scientometric indicator that allows the comparison of the impact of journals across disciplinary areas with very different citation patterns.  

Ever since its debut, Scopus  represented a challenge (and a motivation to expand) for the Web of Science and the JCR databases. While Scopus has a much broader source base than Web of Science, it has two disadvantages. One is that records have been enhanced by cited references only since 1996, the other is that many journals still have significant gaps in their coverage. These shortcomings are  important to realize because they have an effect on calculating long term impact, especially in case of journals which are often measured by several decades rather than just by a  few years time span. Of course, these shortcomings also have an effect on the source bases used in SJR, Scopus, Web of  Science, the Eigenfactor database and consequently on many of the special scientometric indicators, including  the most popular one, the  h-index as was discussed in a series of papers earlier (Jacso, 2008a, 2008b, 2008c).

Because of the above two limitations, the comparisons of JCR and SJR in terms of the number of journals covered can be misleading, without knowing the real extent of coverage in three dimensions: length, width and depth (Jacso, 2009a). This is nothing new, the marketing policy has remained a recurring problem for decades in correctly interpreting the publicity statements of database producers about the real content of databases (Jacso, 1997 and 1998).

Stating that Scimago covers more than twice  as many journals as JCR would not reflect the essential difference that for JCR the source base refers to a particular census year, whereas for SJR it is the total number of journals for which data were imported from Scopus, rather than the number of journals that  have been active or least were cited in the census year and the preceding two years.

In addition, JCR is created only from a subset of all the journals that have been covered by Web of Science, whereas SJR is built from the entire Scopus database (although there are some enigmatic omissions even in the LIS category). It is impressive to read that the most current SJR edition (as of Spring 2010) includes information for 17,124 journals as opposed to the JCR 2008 edition with 6,620 journals in the Science edition, and 1,985 journals in the Social Sciences edition. The total number of unique journals in JCR is less than 8,505 because nearly 10% of  journals are included in both sets (which still need to be searched separately).

It is much more telling that in JCR 2008 there are 170 journals with no publication in 2008. In SJR the number of such journals is more than seven times as high: 1,277. Even more tellingly, there are no journals in this most current edition of JCR that would have not been cited at all in 2008. In SJR, there are more than 2,000 journals that did not get any citations in 2008 for papers published in the past 3 years. There are dozens of journals in the LIS category that belong to this group.

It appears enigmatic why journals with the same SJR-score are assigned to different quartiles. One possible reason is that many journals are assigned to different subject categories where they are in different company of journals and their SJR score qualifies them to one quartile higher or lower  than in another subject category. The journal Scientometrics with an SJR of 0.066, for example, is in the first quartile in the LIS subject category, but in the Computational Theory and Mathematics subject category list  it is “only” in the second quartile. In its summary listing it appears with Q1 label.  

    Figure 2. Excerpt from the list of LIS journals that did not receive any citations for papers published in the previous three years

The closest open access product/service to SJR is the Eigenfactor database (Bergstrom, 2008, and West et al, 2010). It was created from the JCR database suites, and it also uses the Eigenvector centrality measure (Bonacich, 2001) as does SJR to calculate the prestige coefficient  of journals through a sophisticated iteration process, then applies it for weighting the references from the citing journals accordingly.  

It is also an excellent open access resource and Thomson-Reuters started to incorporate  two novel metrics of Eigenfacor.org into JCR since its 2007 edition. The database goes back to the 1995 edition of JCR  and currently it includes data up to the JCR 2008 editions. It must be emphasized that –understandably- neither SJR, nor the Eigenfactor database show the underlying analytical data, the master records and its references, but “only” the aggregate results for the selected census year(s). This is only a limitation when the full source data would be needed to verify some of the results.     

 

Methodology

Originally, I wanted to use the same 52 Information and Library Science (ILS) journals that were tested for WoS [Jacso, 2010b] and the Eigenfactor database [Jacso, 2010a], but two of those journals had no records in SJR, so only 50 journals were used. SJR does have a Library and Information Sciences (LIS) category (as Scopus named this category)  with information about 116 journals, including many highly relevant ones that are not covered by WoS, let alone by the more selective JCR, such as Cybermetrics, International Journal on Digital Libraries, and especially D-Lib.  However, it did not include all the journals in the ILS  category evaluated earlier for the JCR 2007 and EigenFactor 2007 databases.

 

Figure 3. Subject categories related to  library and information science

Figure 4. Excerpt from the result list of a search in the LIS category

Checking the Information Systems category brought up -among the 121 assigned to this category- several additional journals from the list of target journals, but the 33 journals assigned to the Information Systems and Management subject category yielded no additional records for the group of target journals. Searching by title of the journals that were not found in the above categories could make the combination of searches to produce a total of 50 matches for the original list of 52.

The two journals that had to be omitted from the control group are EContent and Online (searched also by ISSN and as two words, as well as a hyphenated word). These omissions are strange because Scopus covers both journals, and covers them well. Both of them are more productive than many of the other journals in the LIS category of Scimago. Online also has higher impact factor than several of the academic journals in the said category. This happens  in spite of the fact that in the world of  professional journals the citation culture is not nearly as prevalent as among academic journals. From the perspective of principle, they would also qualify, because both Scopus (and thus SJR) and WoS (and thus the Eigenfactor database)   –appropriately- include many professional journals in their ILS and LIS categories, respectively, as well as in many other categories.

This somewhat tedious pick and hunt  process made it clear that it would be of great help if users could create, save and modify a set of journals by their own criteria. This would allow to refine, narrow or broaden  a result set and have SJR to reproduce the metrics for the new subset or superset. Without this the results form each search step had to be saved and imported into a spreadsheet, then at the end combine these sets into one unified set by removing those duplicates and triplicates that are assigned to multiple subject categories. In spite of this inconvenience, it is a huge advantage in SJR that it allows searching both by the broad subject areas, and the narrower subject categories. Scopus itself allows direct search only at the 27 broad subject areas, but not on the more than 250 narrower subject categories.

Following this consolidation process, the 2007 subset of the SJR records were matched up with their counterparts in the JCR. This is also a tedious step because SJR uses the fully spelled out title of the journals (e.g. Information Society, Journal of Information Science) while JCR uses  abbreviated titles (e.g. Inform Soc and J Inf Sci). Automatically pairing the records by ISSN does not work either, because SJR uses the ISSN of the online edition of the journals, and JCR uses the ISSN of the print edition.    

As the metrics use different scales it was essential to create a ranked list, so rank positions were calculated for all the metrics to be used in this and subsequent analyses. In case of ties in the metrics, the same rank order position was assigned to all items in tie.

Figure 5a. Excerpts from the consolidated journal data

Figure 5b. Excerpts from the consolidated journal data

Findings

It is to be understood that this study intentionally examined only a small sample. The designers of the SJR software must consider the entire spectrum of more than 17,000 journals in tuning and re-tuning their promising algorithms. Nevertheless, case studies using small samples can provide some insights into this process.

Measuring rank correlation between rank results created by using different algorithms for calculating rank positions is not a panacea because rank positions do not reflect the real differences (distances) between items that are adjacent or close by according to their ranks. Overall, the Spearman rank order correlation coefficient (rs= 0.88) showed high positive correlation between the SJR score and the 2 year impact factor simulating the traditional JIF-2 algorithm using Scopus data. The correlation coefficient (rs= 0.84) was somewhat lower between the rank positions of the set of 50 journals by SJR score versus the genuine JIF-2 score reported by the 2007 edition of JCR , but it  still reflects a high positive correlation.

In the LIS category, the top ranking journal by SJR score, the Journal of Medical Informatics Association (JAMIA), has an SJR value of 0.310, for the second ranked Journal of Health Communication the SJR value is 0.180, and for the third ranked MIS Quarterly it is 0.111, i.e. the distance is much larger than the rank position would suggest. The opposite is also true, when there is a difference only in the third decimal digit of the SJR scores as is  the extreme case  where the 9 position rank order difference between the International Journal of Information Management (25) and Information Technology and Libraries (34) suggests much larger distance in terms of visibility, impact and/or prestige than the actual SJR scores tell about the two journals:0.041 and 0.038.           

It is quite obvious from the SJR scores of the entire test set that in spite of its 3 decimal point precision the SJR score clusters into one category too many journals, i.e. could not  offer refined distinct ranks. The range between the minimum (0.030) and maximum (0.310) SJR scores seems to be  too narrow for this category. There are seven clusters with ties between two journals, five clusters  with triple ties and one with seven journals sharing the SJR score and rank positions. None of the other metrics show such a dense clustering rate.

The 2-year citations per document metrics CpD-2 -which measures the number of citations received in the selected census year (Y) to papers published in the two previous years (Y1+Y2) -  is the functional equivalent of the traditional impact factor in JCR (referred to here as JIF-2, using data from the Scopus database. It  can make finer distinction than the SJR score. Its range is much broader, with a minimum score of 0.040 and a maximum of 7.740. It has 5 mini clusters with tie between 2 journals, and one small cluster with tie between three journals.      

Somewhat similar is the case with the JIF-2 scores that I collected from the 2007 edition of the JCR. There are only two mini clusters with tie between two journals in the mid-range: Aslib Proceedings and Research Evaluation in one, and Journal of Librarianship and Information Science and Social Science Computer Review in the other. The range is narrower (0.000-3.094) by the JIF-2 scores collected from JCR directly, but it still offers more distinctions among journals than the other two.   

Notwithstanding how appealingly high the correlation may seem between SJR and the two other impact factors, for an academic exercise, one of the main purposes for the Scimago project must have been to show how different the rank position  of the journals can be by using an algorithm that uses not only the traditional counting of the number of citations that a journal received, but also the source of origin of the citations in two regards. One is the issue of self citation rate, the other is the prestige  of the citing sources as calculated by SJR.

This distinction was driving also the development of the Eigenfactor database, and very importantly, the adaptation of two of its key indicators, the Eigenfactor Score and the Article Influence  Score by Thomson Reuters in the JCR database.

Scopus has also readily embraced the concept and its implementation by Scimago as the SJR score was added to the very useful Journal Analyzer feature of Scopus while I was doing this research. The metrics can be displayed in both tabular and chart formats. Seeing the gaps in coverage for entire years, as in the case of the Journal of Scholarly Publishing takes away from the pleasure, but also illustrates one reason what may cause  significant differences in the summative league lists of 50 or 100 journals - no matter how sophisticated is the ranking algorithm for and the visualization of the various productivity and prestige indicators when two different sources are used.

 

Figure 6. Tabular data from Scopus with the option to exclude self citations

Despite the overall correlation of results, there are very significant differences between the league lists of journals when they are ranked by the different measures, even within the same database. This is an important angle, because rank order differences may be also heavily influenced by missing volumes and issues in one database (and its derivative), by the differences in the breadth of coverage of potentially citing sources, and by the practice and consistency of assigning document types by the human indexers. This latter matters much  for the impact factors which are supposed to take into account only the “citable” publications in all these databases, but it is questionable because of the subjectivity and inconsistency of the human indexers in making these decisions (Jacso, 2001)

It is easy to get lost in the sea of numbers on the league tables even when single integers are used as for the h-index  or the rank positions. Often one needs to see both the forest and the trees. It would be interesting to see if the Scimago Group could further enhance its excellent visualization of the precious data by making it possible to  the users to see and feel at a glance what are the rank positions and their correlations when different measures are used for ranking.

I created a couple scatter plots in doing this research  and show here one with the size and monochrome limitations for the print version to illustrate my point. In the digital preprint version at http://www.jacso.info/scimago there will be larger graphs  and color marks for the prototypes. In the following scatter plot,  it can be seen and felt immediately what is the rank position of each of the 50 journals by two ranking criteria, the SJR score and the total number of citations the journal received in 2007 for papers published in 2004, 2005 and 2006,  and how is the overall picture of rank correlation by these criteria, as indicated by the two markers. Occasionally, there is only one marker. It indicates that the rank position of the journals is identical by both criteria. The journals are identified by their ID number used in this project, as listed in Figure 4a and 4b, but at this stage it may not be needed. In a Web version the journal titles could of course pop up when hovering above the numbers.

Cycling through several indicator pair combinations would provide a realistic feel about the rank order differences according to the measures used for ranking. Similarly, the same scatter type of plot can be used to compare the effect of using functionally identical measures reported by two different sources, i.e. SJR and JCR. 

Figure 7. Scatter plot allowing to see the forest and the trees

The plot below shows the JIF-2 measure used in JCR, and its functional equivalent in SJR, the CPD-2 metrics of citations received in the census year for the papers published in the previous two years.

While the overall high level of correlation for the 50 journals are obvious, there are some titles where the change in the rank position is quite drastic. 

Figure 8. Scatterplot showing rank positions produced for ranking by functionally identical measures from two different databases

This is the scenario when another chart type, the bump chart  can be better by virtue of showing the journal name as well (the abbreviated variety for saving precious display space). Minor changes  in either directions are not as critical as the medium level changes, and especially the major and drastic changes. It is anyone’s choice what to qualify as minor and drastic change, but the limitations of rank order numbers should be taken into account. No one can argue that the 27 position drop of Law Library Journal (LLJ) from position 22 by JIF-2 score in JCR to position 49 when ranked by SJR score is a drastic change. It is also quite obvious that the key reason for this drastic drop is the exceptionally high self citation rate of LLJ at the journal level in the citations received in 2007 by papers published in 2006 and 2005. JCR reports this ratio to be of 86% (39 self-citations from 45 papers) in the details page about LLJ, but it does not disregard the self citations in calculating the JIF-2 score.

Figure 9.    Bump chart for showing also  the journal names

On the other hand, SJR does so. My understanding is that SJR uses a compromise that only citations received from the same journal that exceeds 33% of the total citations received are excluded. This is a very reasonable compromise as opposed to the extremes of total exclusion/inclusion as practiced by Eigenfactor and JCR for the 2-year impact factor, respectively. This should be clearly stated in the documentation of SJR as it is a critical component of the ranking process.

The drastic drop of the Journal of Information Technology from its 8th rank position by the JIF-2 score in JCR to rank position 22 in SJR by the functionally equivalent citations/document rate, indicating the number of citations received from the census year of 2007 to papers published in the previous two years may depend on other factor(s), such as the prestige status of the citing journals, which is used in the SJR (and the Eigenfactor) databases but not in JCR which considers all citations to be of equal value, irrespective of the impact or prestige of the citing source itself.

These are complex issues and require much further research, verification of the adequacy of the process of identifying journal level self citation which is complicated even for skilled scientometricians when dealing for example, with Law Library Journal which is full of notes, supra notes, infra notes, ibids, see cross references and other similar types of references.

Publishers, editors, librarians, administrators may not have the interest and/or time to explore all these to make a decision which journals should be canceled from the collection of the library. They need a tool that can help them to produce a rank list of journals and some clear explanations and examples about the essential features and factors to create such league lists and understand ones created by others. This is particularly important now that there are several options that for many reasons (including clearly explained and obscure ones) produce very different journal rank lists.     

Conclusions

SJR offers a very well designed and implemented –except where noted- open access resource, with intuitive software and visually appealing data charts and tables. It allows even for the casual users a variety of convenient analysis of the performance of more than 17,000 scholarly and professional  journals (although 2,000 barely have a pulse) in the sciences, social sciences, and arts & humanities areas. Although the latter  has quite  limited coverage of 330,000 documents compared to the grandiose marketing promises (Jacso, 2009a), and especially to the ten times as large Arts & Humanities component of Web of Science (which has nearly 4 million records for papers published in arts and humanities journals), it is a unique resource as there has never been a JCR for Arts and Humanities from Thomson-Reuters.  

Undoubtedly, in this domain journals are not as primary document genres as in the sciences and social sciences, but they still should be included as citation sources. Librarians, other information professionals and researchers in the A&H subject areas will need to know which are the best journals in a particular area (music, dance, literature, religion etc.).

The Scimago Group, and many other universities and research institutions in Spain, have developed in the past ten years a very impressive collection  of databases of general interest (such as the twin products within the SJR suite: the components about the country ranking and the brand new institutional ranking by scholarly productivity and impact; the Science Atlas and several other databases of particular interest for the Ibero-American countries), adding data elements to traditional and novel union catalogs of various types of publications by researchers in Spain, Portugal, Central and Latin America.

While I have  pointed out my concerns about the possible distortion of metrics by countries and institutions (Jacso, 2009c and 2009e) due to the absence of the country and institutional data elements in a significant part of the records in Scopus and WoS (and consequently in the open access, derivative databases, created from those, such as the SJR and Eigenfactor databases).  They represent a far better alternative for bibliometric, scientometric, and informetric purposes than Google Scholar which is ideal for topical searches for academic publications, primarily by virtue of free full text searching, but is far from ideal for bibliometric purposes because of the mess caused by its  ill-trained crawlers and parsers that misattribute authorship, publication years, journal names, and references in millions of records, producing phantom authors, ghost authors and lost authors at an unacceptable rate (Jacso, 2009f).

The Scimago Group is in a very good position to enhance the SJR database as the nationwide co-operation among Spanish researchers specializing in scientometrics seems to be unusually strong. Making their research products freely available for anyone is particularly appealing. They are certainly qualified to develop additional software features mentioned above, as well as content features such as adding a 5 year impact factor as Thomson-Reuters has done since the 2007 edition (Jacso, 2009d), or allowing the users to set their preferred self-citation rate limits. Extending the target window to 15 years or more would be especially useful for journal ranking. It would be also very useful to offer a simple switch option to include/exclude self-citations as Scopus has been doing  it for years for the h-index calculation, and recently introduced it for some of the other performance indicators in the increasingly promising, built-in Journal Analyzer module.       

References

Bar-Ilan, J. (2010), “Rankings of information and library science journals by JIF and by h-type indices” , Journal of Informetrics, Vol. 4 No 2, pp 141-147.

 

Bergstrom, C.T., (2008), “Eigenfactor: Measuring the value and prestige of scholarly journals”, College & Research Libraries News, Vol. 68 No. 5, pp. 314-316.

 

West, J.D,  Bergstrom, T.C. and C.T Bergstrom (2010), “The Eigenfactor Metrics:  A network approach to assessing scholarly journals”, College & Research Libraries (in press ) – preprint version available at http://www.ala.org/ala/mgrps/divs/acrl/publications/crljournal/preprints/preprints.cfm 

 

Bollen, J., Rodriquez, M.A. and Van de Somplel, H. (2006), “Journal status”, Scientometrics, Vol. 69 No. 3, pp. 669-687.

 

Bonacich, P. and Lloyd, P. (2001), “Eigenvector-like measures of centrality for asymmetric relations”, Social Networks, Vol. 23 No. 3, pp.191-201.

 

De Moya-Anegón, F., Chinchilla-Rodríguez, Z.,  Vargas-Quesada, B., Corera-Álvarez, E., Muñoz-Fernández, F.J., González-Molina, A. and Herrero-Solana, V. ( 2007), “Coverage analysis of Scopus: A journal metric approach”, Scientometrics, Vol. 73 No. 1, pp. 53-78.

 

Egghe, L. (2006a). “Theory and practise of the g-index”, Scientometrics, Vol. 69 No. 1, pp. 131-159.

 

Egghe, L. (2006b), “Expansion of the field of informetrics”, Information Processing & Management, Vol. 42 No. 6, pp. 1405-1407.

 

Falagas, M.E., Kouranos, V.D., Arencibia-Jorge, R. and Karageorgopoulos, D.E. (2008), “Comparison of SCImago journal rank indicator with journal impact factor”, The FASEB Journal, Vol. 22, pp. 2623-2628.

 

González-Pereira, B., Guerrero-Bote, V. and Moya-Anegón, F. (2009), “The SJR indicator: A new indicator of journals’ scientific prestige”, pre-print available at: http://arxiv.org/pdf/0912.4141v1

 

Guz, A.N. and Rushchitsky, J.J. (2009), “Scopus: A System for the Evaluation of Scientific Journals”, International Applied Mechanics, Vol. 45 No. 4, pp. 351-362.

 

Jacsó, P. (1997), “Content Evaluation of Databases”, Annual Review of Information Science and Technology, Vol. 32, Ch. 5., American Society for Information Science, Chicago, IL., pp. 231-267.

 

Jacsó, P. (1998), “Analyzing the Journal Coverage of Abstracting/Indexing Databases at Variable Aggregate and Analytical Levels”, Library & Information Science Research, Vol. 20 No. 2, pp. 133-151.

 

Jacso, P. (2001), “A deficiency in the algorithm for calculating impact factors of scholarly journals”, Cortex, Vol. 34, No. 4, pp. 590-594.

 

Jacsó, P. (2008a), “The Plausibility of Computing the H-index of Scholarly Productivity and Impact Using Reference Enhanced Databases”, Online Information Review, Vol. 32 No. 2, pp. 266-283.

 

Jacsó, P. (2008b), “The Pros and Cons of Computing the H-index Using Scopus”, Online Information Review, Vol. 32 No. 4, pp. 524-535.

 

Jacsó, P. (2008c), “The Pros and Cons of Computing the H-index Using Web of Science”, Online Information Review, Vol. 32 No. 5, pp. 673-688.

 

Jacsó, P. (2008d), “ The Pros and Cons of Computing the H-index Using Google Scholar”, Online Information Review, Vol. 32 No. 3, pp. 437-452.

 

Jacsó, P. (2009a), “Database Source Coverage: Hypes, Vital Signs and Reality Checks”, Online Information Review, Vol. 33 No. 5, pp. 997-1007.

 

Jacsó, P. (2009b), “Scimago Journal Rank”, Online, Vol. 33 No. 3, pp. 50-54.

 

Jacsó, P. (2009c), “The H-index for Countries in Web of Science and Scopus”, Online Information Review, Vol. 33 No. 4, pp. 831-837.

 

Jacsó, P. (2009d), Five-year Impact Factor Data in the Journal Citation Reports. Online Information Review, Vol. 33 No. 3, pp. 603-614.

 

Jacsó, P. (2009e), “Errors of Omission and their Implication for Computing Scientometric Measures in Evaluating the Publishing Productivity and Impact of Countries”, Online Information Review, Vol. 33 No. 2, pp. 376-385.

 

Jacso, P. (2009f), “Google Scholar's Ghost Authors”, Library Journal, Vol. 134 No. 18, pp. 26-27. Unabridged version available at: http://www.libraryjournal.com/article/CA6698580.html?q=jacso

 

Jacsó, P. (2009g), “Scopus”, Peter’s Digital Reference Shelf available at  http://www.gale.cengage.com/reference/peter/200906/scopus.html

 

Jacsó, P. (2010a), “Differences in the Rank Position of Journals by Eigenfactor Metrics and the Five-year Impact Factor in the Journal Citation Reports and the Eigenfactor Project Web Site” Online Information Review, Vol. 34 No. 3, (in press) Louise can you get the page numbers for this?

 

Jacsó, P. (2010b), “Eigenfactor and Article Influence Score in the Journal Citation Reports”, Online Information Review, Vol. 34 No. 2, pp. 339-348.

Jacsó, P. (1997), “Content Evaluation of Databases”, Annual Review of Information Science and Technology, Vol. 32, Ch. 5., American Society for Information Science, Chicago, IL., pp. 231-267.

 

Leydesdorff, L. (2009), “Journal maps on the basis of Scopus data: A comparison with the Journal Citation Reports of the ISI”, Journal of the American Society for Information Science and Technology, Vol. 61 No. 2, pp. 352-369.

 

Ma, N, Guan, J. and Zhao, Y. (2008), “Bringing PageRank to the citation analysis”, Information Processing & Management, Vol. 44, pp. 800-810.

 

Martin, B.R. (1996), “The use of multiple indicators in the assessment of basic research”, Scientometrics, Vol. 36 No. 3, pp. 343-362.

 

Meho, L.I. and Rogers, Y. (2008), “Citation counting, citation ranking, and h-index of human-computer interaction researchers: A comparison of Scopus and Web of Science Source”, Journal of the American Society for Information Science and Technology, Vol. 59 No. 11, pp. 1711-1726.

 

Moed, H.F. (2010), “Measuring contextual citation impact of scientific journals” Journal of Informetrics (in press), preprint version available at http://arxiv.org/ftp/arxiv/papers/0911/0911.2632.pdf

Moed, H.F., Van Leeuwen, T.N. and Reedijk, J. (1999), “Towards appropriate indicators of journal impact”, Scientometrics, Vol. 46 No. 3, pp. 575-589.

 

Rousseau, R. and the STIMULATE 8 Group (2009), “On the relation between the WoS impact factor, the Eigenfactor, the SCImago Journal  Rank, the Article Influence Score and the journal h-index”, available at: E-LIS archive, ID: 16448; http://eprints.rclis.org/16448/

 

Siebelt, M., Siebelt, T., Pilot, P., Bloem, R.M., Bhandari, M. and Poolman, R.W. (2010), “Citation analysis of orthopaedic literature; 18 major orthopaedic journals compared for Impact Factor and SCImago” BMC Musculoskeletal Disorders, Vol. 11, pp. 4-10.

Torres-Salinas, D. and Jiménez-Contreras, E. (2010), “Introducción y estudio comparativo de los nuevos indicadores de citación sobre revistas cientificas en Journal Citation Reports y Scopus”, El Professional de la Informacion, Vol. 19 No. 2, pp. 201-207.

Walters, W.H., Wilder, E.I. (2003), "Bibliographic index coverage of a multidisciplinary field", Journal of the American Society for Information Science and Technology, Vol. 54 No.14, pp.1305-7.

 

West, J.D., Bergstrom, T.C. and Bergstrom, C.T. (in press), “The Eigenfactor MetricsTM: A network approach to assessing scholarly journals”, College & Research Libraries, (pre-print) available at:

http://0-www.ala.org.sapl.sat.lib.tx.us/ala/mgrps/divs/acrl/publications/crljournal/preprints/West-Bergstrom-Bergs.pdf