Does a Long Reference List Guarantee More Citations? Analysis of Malaysian Highly Cited and Review Papers

Earlier publications have shown that the number of references as well as the number of received citations are field-dependent. Consequently, a long reference list may lead to more citations. The purpose of this article is to study the concrete relationship between number of references and citation counts. This article tries to find an answer for the concrete case of Malaysian highly cited papers and Malaysian review papers. Malaysian paper is a paper with at least one Malaysian affilation. A total of 2466 papers consisting of two sets, namely 1966 review papers and 500 highly-cited articles, are studied. The statistical analysis shows that an increase in the number of references leads to a slight increase in the number of citations. Yet, this increase is not statistically significant. Therefore, a researcher should not try to increase the number of received citations by artificially increasing the number of references.


Introduction
Researchers seeking citation tracking to find the most influential articles for a particular topic and to see how often their own published papers are cited (Bakkalbasi et al. 2006). On the other hand universities are looking for citations because of its influence in the university ranking , Ioannidis 2010, Bornmann, Leydesdorff, and Wang 2014. A citation count is the number of times a research work such as a journal article is cited by other works. The citation per paper meaningfully influence a number of metrics, including total citation counts, citation speed, the ratio of external to internal cites, diffusion scores and h-index (Carley, Porter, and Youtie 2013).
Citation counts still commonly use for the measure of research papers quality and reputation (Abt and Garfield 2002). The number of citations that an article receives measured its impact on a specific field (Lai, Darius, and Lerut 2012). Citation analysis is one of the most important tools to evaluate research performance (Bornmann et al. 2012). Citation indicator is important for scientists and universities in all over the world (Farhadi, Salehi, Yunus, et al. 2013). In the early stage, the relationship between the number of references and the number of the paper citation was investigated in the 1965 (UZUN 2006, de Solla Price 1965. A long reference list at the end of a research paper may be the key to ensuring that it is well cited (Corbyn 2010, Ball 2008. Hence, citation counts are correlated with reference frequencies (Abt and Garfield 2002). Webster, Jonason, and Schember (2009) raised the question "Does the number of references an article contains predict its citation count?" and found that reference counts explained 19% of the variance in the citation counts. Lancho-Barrantes, Guerrero-Bote, and Moya-Anegón (2010) found that not only the number, but also the citation impact of the cited references correlated with the citation counts for a paper. The higher the impact of the cited references, the higher the later impact of the citing paper (Bornmann et al. 2012). Review articles are usually highly cited compare to other types of papers (Meho 2007).
Review papers represent the existing knowledge in a given field and more likely to be cited (Alimohammadi and Sajjadi 2009). Several bibliometric studies highlighted that citation counts are a function of many factors besides the scientific quality (Bornmann et al. 2012 (Beel, Gipp, and Wilde 2010), add the name of study in the title of all publications (Sarli and Holmes 2011), publishing in a journal with higher impact factor (Vanclay 2013), internet usage (Farhadi, Salehi, Embi, et al. 2013), gross domestic product (GDP) (Gholizadeh et al. 2014

2.Materials and methods
All data were obtained through Web of Science online academic database provided by Thomson Scientific. This database included the necessary information to examine the relationship between reference and citation counts for every review and highly cited papers published in Malaysia since 1980 to October 2013. Science Citation Index Expanded, Social Sciences Citation Index and Arts & Humanities Citation Index, were searched for reviews and highly cited papers. For each paper, all Bibliometrics data, especially the number of references and the number of times the paper has been cited during the interval between the year of publication and the year 2013, have been collected.Two samples set were selected: 1-The sample number one consisted of 1966 review papers in all disciplines from Malaysia, according to the Web of Knowledge's classification system. Citation statistics produced by shorter than three years' time frame may not be sufficiently stable (Adams 2005, UZUN 2006 were removed. In order to select the highly cited paper a threshold 10 times cited per year is considered. The association between the number of references (independent variable) and time cited per year (dependent variable) of highly cited review papers investigated with linear and nonlinear models. 2-The sample number two comprises 500 highly cited publications from Malaysia.
According to the Web Of Science classification, the results are obtained based on the article type and exclude the review articles, editorial material, conference papers and book review.

Results and discussion
Two sets of data 1-1966 review papers and 2-500 high cited papers, were investigated separately.
The results and discussions are coming as follows.

Outliers for sample one (1966 review papers)
Due to the effect of the age of an article, the number of citations cannot be a reference of highly cited paper. Therefore, the citation per year selected as a reference for highly cited paper. Papers with 10 times cited per year is considered as highly cited paper. Figure 3-1 shows the number of times cited per year for 660 review papers. A threshold was visually determined on 50 times cited per year. Papers with more than 50 times cited yearly is called "extremely high cited paper" and detected as outliers. Papers with more than 300 listed references also detected as outliers (3-2). Outlier detection for sample two (500 highly cited papers) Papers with 10 times cited per year is considered as highly cited paper. Papers that cited more than 100 times per year is considered as extremely high cited paper and detected as an outlier. Figure 5 and Figure 6 are showing raw data and filtered data respectively.    Table 3-2 The result of correlation analysis of 500 highly cited papers The association between variables is graphically illustrated with scatter plots. The trend of these associations is shown by the solid lines.

Conclusion
This study shows that since the trend between the citation count and the number of references is not statistically significant, we cannot conclude that there is a significant association between the citation count of Malaysia review papers between the given period and number of references contained in the paper. The correlation coefficient is not statistically significant. However, r = 0.152 based on the population of 721 articles. Malaysian review papers get more citations than other types of papers. The number of references in the article has the lowest impact on the citation compares with review paper. As this study looked only Malaysia review papers and 500 highly-cited article, it would be necessary to conduct a similar study in the otherworld and types of papers. It would be important to examine whether in other types of papers the relationship investigated here have significant correlated or not. The research considered the general definition of citations.
Therefore, future studies may make a diffrentianain between "perfunctory citations" and "organic citations" citations as Tang and Safer (2008) defined "perfunctory citations" is occurred only once and in the introduction, "organic citations" as references cited for "conceptual ideas" and "methodology and data" reasons.