Provision of supplementary materials in Library and Information Science scholarly journals

Purpose – This study seeks to analyse the policies of library and information science (LIS) journals regarding the publication of supplementary materials, the number of journals and articles that include this feature, the kind of supplementary materials published with regard to their function in the article, the formats employed and the access provided to readers. Design/methodology/approach – The study analysed the instructions for authors of LIS journals indexed in the ISI Journal Citation Reports, as well as the supplementary materials attached to the articles published in their 2011 online volumes. Findings – Large publishers are more likely to have a policy regarding the publication of supplementary materials, and policies are usually homogeneous across all the journals of a given publisher. Most policies state the acceptance of supplementary materials, and even journals without a policy also publish supplementary materials. The majority of supplementary materials provided in LIS articles are extende...


Introduction
Primary data are the basis of scientific research.Over the last few years the volume of and speed at which data are obtained and shared have increased enormously due to improved measurement instruments and the use by researchers of the internet for communication.As a result, scholarly communication is going beyond the publication of research reports in the form of journal articles, conference proceedings, monographs, etc. to include the capture, description and reuse of data.
The OECD (2007) defines research data as "factual records (numerical scores, textual records, images and sounds) used as primary sources for scientific research, and that are commonly accepted in the scientific community as necessary to validate research findings".Reflecting the diversity of scientific disciplines, research data can take many forms and may come from many different sources, such as laboratory data, observational data, or public and private records, to name but a few.
In an extensive review of the topic, Borgman (2012) identifies four rationales for sharing research data: 1) to reproduce or verify research (this being the most problematic aim since rarely is enough information provided to reproduce the results, and observations are often associated with specific times and places); 2) to make results of publicly funded research available to the public (which is driving funding agencies to require data release to varying degrees); 3) to enable others to ask new questions in relation to extant data (thereby encouraging meta-analysis, i.e. the combination of data from multiples sources, times and places); and 4) to advance the state of research and innovation (so that researchers can more readily draw upon each other's data).In addition to these rationales, another incentive that could prompt researchers to share their data is a possible increase in the number of citations received (Piwowar, Day & Fridsma, 2007), an argument that reminds us of the 'citation advantage' commonly attributed to open access articles.
In some sciences, data are managed by disciplinary or national infrastructures that are responsible for collecting, storing, preserving and providing data to researchers.In fields such as astronomy or physics there are standard data practices and sharing research data is unavoidable.However, in 'small science' disciplines, data are collected in order to provide an answer to a specific research question.In these fields, researchers themselves collect the evidence that supports the research process, and the sharing of research data remains a limited activity (Cragin et al., 2010).
Researchers willing to release their data may choose among three different options: they may deposit data in public repositories, post datasets on public websites, or provide data in journals in the form of supplementary material to articles.The present study focuses on a 'small science' field, namely Library and Information Science (LIS), the aim being to analyse the policies of disciplinary journals regarding the provision of supplementary materials and the characteristics of the materials released.
A growing number of scholarly journals now provide supplementary materials with their articles.According to a survey conducted among publishers (PARSE.Insight, 2009), 71% of large publishers (i.e.those publishing more than 50 journals) and 58% of small publishers (those publishing fewer than 50 journals) allow authors to submit underlying research data with their manuscripts, mainly accepting Office documents, images and plain text.In a note announcing the introduction of a supplementary data feature, the editor of the Journal of Geotechnical and Geoenvironmental Engineering (Stewart, 2010) explained the main rationale for this decision: it would help to address the problem of restrictions in the length of manuscripts and would enable the publication of materials better suited to electronic media than a paper format, such as movies, audios, animations, colour figures, etc.According to this note, these supplementary data would be subject to the same peer review criteria as the manuscript.
The supplementary materials feature seems to be widely employed in scholarly journals.A longitudinal study based on a sample of 28 medical journals (Schriger, 2011) showed that the percentage of journals offering supplementary material in their web version had increased from 32% in 2003 to 64% in 2009, whereas the percentage of articles including this information had increased from 7% to 25%.However, the increase in the amount of supplementary material in journals has not been welcomed by all.Marcus, in an editorial for Cell (2009), enumerated the drawbacks of supplementary materials for authors (who feel impelled to submit increasingly large amounts of supplementary materials), reviewers (who feel responsible for assessing them with the same attention and standards as the main body of the article) and readers (who find it difficult to navigate through large supplements and are unsure about how carefully they have been evaluated).As a result, Cell decided to establish new guidelines in which each item of supplemental data would be specifically associated with a figure or table in the article.In 2010, after seven years of accepting supplementary data, the Journal of Neuroscience decided to stop doing so within journal articles.The reason adduced by the editor (Maunsell, 2010) was, once again, the exponential growth of supplementary materials, which was undermining the peer review process.A year later, it was the turn of the Journal of Experimental Medicine (Borowski, 2011) to announce that, due to the overabundance of supplementary materials, they would limit them "to essential supporting information".Nature Neuroscience has also recently announced changes to its supplementary information guidelines, moderating supplementary materials while expanding the space allowed for methods in the body of the article (Moderating…, 2012).
The changes in the policies of these journals highlight the lack of standards regarding the management of these materials in the publication process: whether they should be edited or reviewed, how to cite them, or how to preserve them (Carpenter, 2010).As a response to these challenges, the National Information Standards Organization (NISO) and the National Federation of Advanced Information Services (NFAIS) are working to develop recommended practice guidelines for supplemental materials (NISO/NFAIS, 2012).
The value of research data and other supplementary materials varies from discipline to discipline.In fact, it is not clear whether supplementary materials have any use at all, especially in disciplines where data are obtained ad hoc to answer specific research questions and have limited use for other purposes.This variability calls for individual analysis of the characteristics of supplementary material in different fields so as to assess its worth.In this regard, the present article focuses on the analysis of supplementary materials in a set of a 'small science' disciplinary journals.The results will be useful to inform both LIS journal policies regarding the provision of supplemental material and, more broadly, the development of data sharing initiatives, thereby helping to establish standards for the organization and reuse of data.Specifically, the study is underpinned by the following research questions: 1.How many LIS journals have a public policy on the provision of supplementary materials?2. What are the characteristics of these policies in terms of the types of materials accepted and their review?3. How many LIS journals actually offer supplementary materials?How many articles include this feature?4. In terms of the kind of supplementary materials published, what is their function in the article? 5. What kinds of formats are used when publishing supplementary materials?6.What kind of access is provided to supplementary materials?Is access to supplementary materials restricted to journal subscribers or are they freely available to any reader?

Methodology
The first step was to obtain a list of the 77 journals indexed by the ISI Journal Citation Reports in 2010 in the category 'Information Science & Library Science'.Five of these journals were excluded from the sample: Econtent, Library Journal and The Scientist are trade magazines that mainly include news, opinion pieces and description of tools, etc.; Informacios Tarsadalom was excluded from the sample since the present authors do not read Hungarian; and the Annual Review of Information Science and Technology was excluded because it ceased publication after the 2011 volume.
The website of each of the remaining 72 journals (ten of them open access) was visited and the instructions for authors obtained.These instructions for authors were then examined in order to identify any mention of the publication of supplementary materials.If this was present, the This is a postprint (final draft post-refereeing) of an article accepted for publication in Aslib Proceedings 65 (5), 503-514, 2013 policy was analysed in order to determine whether or not the journal accepts these materials, and if so, what type of materials does it publish and how are they reviewed.
The next step involved examining the 2011 online volumes of each journal and listing the articles that provided supplementary materials.The characteristics of the supplementary materials were then recorded in terms of their function in the article (additional information on the topic under investigation, methodological details, additional results or additional references), their format (textual, numerical, audio-visual, etc.), the file formats used, and the availability of the materials to non-subscribers of the journal.
For the purposes of this study, supplementary materials were defined as those considered as such by the journal.In the case of Emerald, six of their journals included 'appendixes' after the references.These appendixes were considered as supplementary materials.

LIS journal policies regarding the provision of supplementary materials
Twenty-six of the 72 journals (36%) indexed in the 'Information Science & Library Science' category of the 2010 edition of the ISI Journal Citation Reports included in their author instructions some kind of information about their policy regarding the publication of supplementary materials.
As can be observed in Table 1, large publishers are more likely to have a policy, and broadly speaking this will be homogeneous across all their journals.For example, Elsevier, which publishes 11 of the journals considered, includes an identical 'supplementary data' section in the guidelines for authors of all its journals.This states that Elsevier journals accept supporting applications, high-resolution images, background datasets and sound clips, etc. Authors are requested to submit the materials in recommended file formats.Sage publishes five LIS journals indexed in the ISI Journal Citation Reports.In this case the policy is categorical: four of the five journals state that they "not currently accept supplementary files".The remaining journal does not include any information on the issue.
Wiley-Blackwell publishes four of the journals considered.Two of them include a very similar statement indicating that they accept data sets and additional figures or tables to be published in their online editions.If the file cannot be accommodated on the journal website it can be made available by the author, free of charge, on a permanent website.Supplementary materials must not be altered after the article has been accepted for publication and, since they are considered an integral part of the manuscript, they are reviewed, although this does not apply if the data are on the author's website to preserve anonymity.Although not stated in the instructions for authors, supplementary materials include a note stating that Wiley-Blackwell is not responsible for the contents or functionality of any supporting information and any queries should be directed to the authors.
The three journals edited by Springer accept multimedia and supplementary files.Most of the information provided by the publisher refers to the file formats required for each kind of material.These materials are published as received from the author without any conversion, editing or reformatting.
Taylor and Francis journals do not only accept supplementary materials (movie clips, mathematical appendixes, laws) but the publisher gives examples of their use in several of its journals in order to inspire authors.Again, most of the information provided by the publisher refers to the recommended file formats.The publisher also states that supplementary material will not be modified -although "it should at least be subject to editor oversight" -and that "more extensive supplementary material (analyses rather than data) ideally should be subject to peer review".
Finally, we have the case of three journals pertaining to different publishers that have a policy regarding supplementary materials.The Journal of the American Medical Informatics Association states that additional tables, figures, data sets and source code can be included for online only publication.The Journal of the Medical Library Association accepts supplementary materials such as extensive data tables, appendixes or survey instruments, the latter being compulsory unless published elsewhere.Finally, Research Evaluation welcomes detailed methods, extended data sets/data analysis or additional figures.
The remaining journals do not have any policy regarding the publication of supplementary materials.Most of them are published by small publishers with three or fewer journals in the field.Emerald, which publishes seven LIS journals indexed in the ISI Journal Citation Reports, is the only large publisher with no common or individual policy for any of its journals.Except for the 'appendixes' of Emerald articles, which are embedded in the HTML and PDF files of the articles, most supplementary materials are contained within separate files to that of the article.Five of the 18 toll-access journals offering supplementary materials provide free access to these files to any reader.

Function of supplementary materials in LIS journals
If we analyse the type of supplementary materials provided with regard to their function in the article we find that most materials are detailed methodological explanations (66%) and additional results (62%).To a lesser extent they include additional references (22%) and Finally, there were a couple of examples of supplementary material that do not refer to any specific part of the article.This is the case of an article that included, as supplementary material, a translation of the article into another language or supplementary material in the form of an errata note.

Formats of supplementary materials in LIS journals
Regarding formats, most articles (78%) included supplementary materials in the form of textsummaries of literature, definitions of terms and variables, instructions to participants, etc. Around one third of the articles (35%) presented supplementary materials in the form of numerical tables.Almost all articles that included references as supplementary materials (22%) employed them to support additional explanations.Around one-tenth (11%) of the supplementary materials were in the form of illustrations -diagrams, images, screenshots, etc. -related to the issues discussed in the articles, while another tenth were additional figures based on the data compiled by the researchers.Seven per cent of the articles included formulae among their supplementary materials, and, finally, three articles featured videos.In all cases, supplementary materials were in a separate file to that of the article, except for the 'appendixes' of Emerald journals that were embedded in the same HTML and PDF files as the article.Thus, the results shown in Table 5 are based on 180 articles, after excluding the 34 articles with supplementary materials published by Emerald journals.Although most supplementary materials were contained in a single file, some articles provided supplementary materials in several files, and in three articles the supplementary materials were in two different file formats.PDF was by far the preferred format for presenting supplementary materials, with nearly two thirds of the articles (64%) making use of this format.Most journals do not seem to edit the supplementary materials, and apparently they are published as submitted by the authors.As a result, their presentation is far from homogeneous.Although most supplementary materials include a reference to their corresponding article, some supplementary material files do not include any information linking the supplementary materials to the main article.

Discussion
Most LIS journals are willing to accept supplementary materials for the articles they publish.Except for four journals from the same publisher which specifically state that they do not accept supplementary files, the remaining journals either have a policy on the issue, stating that they are willing to accept supplementary materials, or, even if they do not have a policy, nonetheless publish supplementary materials with their articles.These results are consistent with those of PARSE.Insight (2009), which stated that publishers of 94% of all the toll and open access journals allow authors to submit research data accompanying their manuscripts.
Despite this willingness to accept supplementary materials, policies -when present -are weak.Journals with a policy are mainly those from large publishers and they simply use a very general statement, common to all the journals from the same publisher.They mainly give examples of what can be considered as supplementary material and offer recommendations on file formats.However, very little is said about the kind of materials accepted, and this seems to be open to the suggestions of authors.Neither do these journals state the amount of additional materials accepted, which could become a problem given the overwhelming amount of data submitted (Maunsell, 2010;Borowski, 2011).Most policies also fail to provide any information on the supervision or review which supplementary materials are subject to.In fact, according to the analysis of the files, it seems that some supplementary materials are not edited at all and are simply published as submitted by the authors, sometimes lacking information as essential as the reference of the article they relate to.Interestingly, some tollaccess journals provide open access to their supplementary materials, suggesting that they give little value to these contents.
Most supplementary materials provided by LIS journals are extended methodological explanations or additional results to those presented in the article.In the case of methodological details, most of the information provided is aimed at increasing the credibility of the research.Although in most cases it would be impossible to reproduce the research in order to verify the results, since data gathering is often linked to specific times and places, the provision of information on sampling strategies, the operationalization of variables, and data collection and analysis techniques, etc. does help to increase the reader's confidence in the reliability of the research and would allow the research to be replicated in other contexts.
The provision of additional results as supplementary material seems to be designed to overcome the space limits of journals and allows authors to introduce large tables and additional figures that could not be encompassed within the usual size of an article.This would also seem to be the rationale for additional introductory materials, such as extensive literature reviews or additional explanations of the concepts discussed in the article.
The types of additional materials employed in LIS result in a high presence of textual information and, to a lesser extent, of numerical tables with very few examples of the provision of audio-visual contents.Interestingly, we have not found any example of raw data supplied as supplementary material so as to enable additional analyses to be performed.This is probably related to the characteristics of LIS research, a field where data are gathered ad hoc by the researcher in order to answer a specific research question, such that they have very little further use.Even in the case of numerical tables provided as supplementary material, these are mostly in PDF files that cannot be manipulated, indicating that readers are not expected to conduct any further analysis.

Conclusion
Despite the increasing number of journals offering supplementary material features, few of them seem to have considered all the implications.Most policies regarding the publication of these materials are vague and focus mainly on file formats.However, further decisions need to be reached in order to avoid future problems regarding the management and preservation of these data.Indeed, in light of its interest to readers and its future usability by other researchers, journals must establish a policy on whether or not they will publish supplemental material.In establishing such a policy they should consider the typology of materials accepted for publication, the amount of supplementary materials accepted and the review or, at least, the editorial process these materials will be subject to.Journal publishers should also consider whether a link to a public website or, ideally, to a repository providing a stable website would be a better solution.
From the point of view of their utility in LIS, additional methodological explanations seem to be one of the most useful additional resources that can be offered to readers.Methodological details not only increase the credibility of results but also allow other researchers to replicate studies in new contexts.

Table 1 .
LIS journals with a policy regarding the provision of supplementary materials

Table 4 .
LIS supplementary material formats

Table 5 .
File formats of the supplementary data