Evaluation of Spanish institutional repositories based on criteria related to technology, procedures, content, marketing and personnel

Purpose: To provide, through a set of indicators, an overview of the way in which Spanish institutional repositories are run and the services they offer their respective institutions and other users. The selected descriptors are based on aspects related to technology, procedures, content, marketing and the personnel responsible for managing repositories. Design, methodology and approach: In order to establish the indicators, a thorough review of the literature was carried out to identify existing indicators that are used to assess repositories. These were divided into five categories (technology, procedures, content, marketing and personnel) with a total of 48 components. An online survey was conducted with the repositories managers of 66 Spanish research institutions in order to verify the degree of fulfilment of the selected indicators. Findings: The survey received forty-six responses, which represented a response rate of 69%. Of these, 44 came from universities and two from research centres. Sixty-five per cent of the repositories have the capacity to import data from and export data to other university systems, mainly CRIS (32%). Most repositories have mechanisms for the large-scale import and export of metadata and digital objects (83%). The use of altmetrics in repositories is widespread (44%). Authors and librarians deposit most frequently (37% and 32%, respectively), in spite of the fact that 44% do not have fulltime staff working in the repository. In more than 80% of the repositories, between 90% and 100% of the deposits are full-text documents. With respect to the tools used to promote the repository within the institution, these are primarily face-to-face training sessions (82%), followed by support materials such as manuals and help pages (65%). The academic authorities encourage open access among researchers in 56% of cases, a significant element in repository marketing. Originality and value: This work proposes a model based on five dimensions and 48 indicators to assess institutional repositories. This approach has been applied to Spanish institutional repositories to provide up-to-date information about their management procedures and promotional methods and the services they offer authors and the university community. This overview of Spanish repositories has


INTRODUCTION
The institutional repository, within the university context, has been defined as "a set of services that a university offers to the members of its community for the management and dissemination of digital materials created by the institution and its community members" (Lynch, 2003).
In the current context of academic communication, the role of institutional repositories is considered vital to ensure that the process is managed by researchers.If repositories also incorporate statistical services, connect to social networks and even include a peer review system, they constitute an alternative to the traditional academic communication system, thereby achieving substantial savings in research funding and ensuring that dissemination and communication are entirely in the hands of the research community (COAR, 2017).Therefore, the quality of institutional repositories is paramount to ensure that they are the best possible tools for achieving this objective.
Thus, several important points related to the management of the repository require analysis: procedures for depositing documents, interoperability with other systems (CRIS -Current Research Information System-, OPAC-Online Public Access Catalog-, etc.), the statistical data provided and how these reach the user, internal marketing, economic resources and the personnel dedicated to managing and maintaining the repository.Early studies on repositories focused more on identifying how to increase content; however, this indicator can be ambiguous, since records are often subject to large-scale automatic deposits.In this respect, Carr & Brody (2007) concluded that the daily deposit rate was more important than the total number of records for ensuring that a repository is actually maintained over time.
One of the first studies regarding evaluation of repositories (Westell, 2006) analysed nine factors in the repositories of English-speaking Canadian universities through a survey in which subjects were asked about institutional policies, funding models, preservation strategies, interoperability and metrics.The analysis revealed that none of the institutions had an open access mandate or a specific budget for the repository, but relied on library resources, and the study concluded that alignment with the institution's objectives is key to the success of a repository.Thibodeau (2007) proposed a theoretical model that included different aspects to be considered when evaluating repositories: service (in other words, identifying the users and their needs); coverage (the documents preserved in the repository); chronological orientation (whether a repository preserves whatever the manager deems valuable or satisfies the needs of users); collaboration with other repositories; and state (the repository's current state of development).Swan (2008) studied the business model of the repositories and proposed a set of performance indicators: content (type of documents); deposit ratio; users' knowledge of open access; repository workflow; and funding.Kim & Kim (2006, 2008) developed an evaluation model applied to the Korean repositories within the dCollection consortium, which is composed of more than 40 universities.The authors proposed 19 items divided into four categories: content (metadata, repository size, characteristics of the document upload procedure and formal characteristics); management and policies (procedural criteria [legal and costs], institutional support for open access and participation of academic staff); system (access mode, integration with other university programs, confidence in the system, interoperability, functionality and cooperation); and, lastly, uses and users opinions (accessibility, usability, user satisfaction, relevance of searches within the repository, fulfilment of deadlines in document uploads and potential users).In Spain, a report on open-access repositories was carried out in 2009 based on data from an online survey conducted among research library directors (Melero et al., 2009).The findings of this study revealed that the factors that contributed to the development of repositories included ease of use, increased visibility of research, integration of systems and quality of searches within the program.Meanwhile, the factors found to inhibit the development of the most commonly mentioned repositories were the absence of funding bodies and institutional policies and mandates, the lack of integration with other research systems and poor coordination of institutional repositories at national level, and the lack of financial support from national funding programmes.In December 2010, the repositories working group of the Spanish network of university libraries (REBIUN) published a guide with 31 evaluation criteria based on existing projects in Europe (DRIVER, DINI).Criteria were divided into seven sections (Barrueco et al., 2010): visibility; policies; legal aspects; metadata; interoperability; visit data and statistics; and security, integrity and authenticity of data.This guide was updated in 2014 and the number of items increased to 53 (Barrueco-Cruz et al., 2014).Maria Cassella (Cassella, 2010) made a distinction between internal and external indicators.Repository managers can use internal indicators to align their strategies with the institution's mission and goals by prioritising those most likely to contribute to their achievement.External indicators, on the other hand, can be used to measure the impact of the repository at national and international level and assess its usefulness as a search tool for end users.Internal indicators include user/depositor perspectives; internal processes (annual deposits, daily deposits, full-text availability); value-added services (option of generating CVs, statistics, author identifiers, RSS); financial factors (cost per deposit, cost per download); and staff learning and growth (dedicated staff, staff training).External evaluation indicators include interoperability (interaction with repository networks); external funding (capacity of the repository to attract funding); and participation in national and international projects.
In a survey conducted by the Primary Research Group on institutional repositories (Primary Research Group, 2012), a set of indicators was defined to study the quality of a group of national repositories, and these were divided into eight categories: marketing; access and statistics; financial aspects; cooperation; impact; cataloguing; rights management; and repository funding and content.
A census of institutional repositories in Germany was conducted in 2012 by the Information Management Department at the Berlin School of Library and Information Science (Vierkant, 2013).Eight aspects were examined: size of the repositories; hosting; language support; value-added services; software; metadata formats; registries; and open access to content.In 2014, a second census examined 50 criteria divided into six groups: general information; usability; value-added services; metadata; interoperability; and community (a factor that refers to a repository's commitment to open access by signing the Berlin Declaration, offering a fund supported by the German Research Foundation or becoming a member of COAR -Conference of Open Access Repositories).The first ranking of institutional repositories in Germany was created based on this census.In 2015, Swiss and Austrian repositories were added and some of the evaluation criteria changed.This ranking is designed to complement the DINI certificate and encourage repository managers to improve their services (Open Access Repository Ranking (OARR), 2015).
In 2013, a survey was carried out among the managers of Spanish university repositories as part of the third REBIUN Strategic Plan, 2020 (Casal et al., 2013).Its conclusions included proposed measures to strengthen the role of repositories and open access in institutions.These measures include the establishment of an advisory service on open access, support for researchers to create their CVs, preserve their work digitally, etc., the unification of statistical criteria in Spanish university repositories and dissemination of visit data and downloads, and development of tools for linking the repository to other institutional applications for managing scientific information (especially CRISs).
The Council of Australian University Librarians carried out a survey in 2014 among repository managers in Australia and New Zealand to obtain an overview of how repositories had evolved since 2012 (CAUL, 2014).The most noteworthy results were that the percentage of universities with a mandate had risen to 37%, that integration between the CRIS and the repository had increased from 10% to 60% since 2012, and that 25% of the institutions were helping to fund their researchers' publications.With respect to the services offered, those that stood out included the option of automatically generating webpages ordered by author, centre, etc., and author homepages in the repository, as well as the option of generating code that authors could embed in other websites.
Attempts have been made to measure the success of institutional repositories by examining their management, services, self-archiving, personnel and resources (Lagzian, Abrizah and Wee, 2015).The purpose of that study was to shed light on the role of repositories; in other words, their actual performance and users' perceived importance.A gap between actual performance and perceived importance was observed in all of the factors studied.The greatest difference was observed in factors related to management and self-archiving.The smallest difference was detected in the area of technology.The results of this study show that, while the respondents were aware of the role played by critical factors in the success of a repository, in practice they were unable to exercise control over all of them.In other words, the challenges of an institutional repository do not lie in its technical implementation, but in bringing about the cultural shift required to ensure that the repository forms an integral part of the institution's research activities.
In recent years (Guédon, 2017), concerns have been raised about publishers appropriating the concept of open access in order to do business (by requesting that authors who wish to disseminate their articles in open access pay fees to the journal, known as article processing charges -APCs).Together with this fact, the process for evaluating researchers depends to a large extent on their publications in journals with a high impact factor (i.e.those indexed in the Web of Science or Scopus databases).A report prepared by COAR (2017) on the future of repositories echoes this situation.The high subscription fees and APCs of journals with a high impact factor make both access and publication difficult for institutions that cannot afford such payments.COAR's report (2017) advocates a sustainable system for sharing research results through a globally distributed network of repositories, free of charge.To achieve this goal, it is essential that all repositories are research-centric.In other words, that they compatible with each other and with the institution's other systems, and include valueadded services such as compatible usage statistics, connection to social networks and even a peer review system (COAR, 2017).It seems fitting to carry out an evaluation of the way in which institutional repositories operate to assess whether or not they fulfil these characteristics.
The objective of this study was to obtain an overview of Spanish institutional repositories in terms of the way they function and the services they offer authors and the institution itself.For this purpose, a set of indicators was established, based on the existing literature and the authors' own experience (Serrano-Vicente, Melero and Abadal, 2014;Serrano-Vicente and Melero, 2015).

METHODOLOGY
Based on previous work and the literature consulted, we created a model with a set of five categories (technology, procedures, content, marketing and personnel), which included 48 indicators (Tables 1-5), to assess institutional repositories.These criteria served to analyse the status of Spanish Institutional repositories regarding their implementation and services created on top of them.This model proposal will constitute our framework with five dimensions and 48 descriptors, to survey institutional repositories in Spain.The limitations of the study are related to the fact that it is oriented to Spanish institutional repositories and in a particular date.That model can also be exported to other countries, adapting whatever could be necessary.

Repository survey
A total of 83 existing institutional repositories were identified at the time of the search (May 2016) listed in BuscaRepositorios directory (http://www.accesoabierto.net/repositorios/) .Repositories that did not contain research documents or did not belong to academic or research institutions were excluded.There were a total of 66 repositories; 61 corresponding to universities and five research institutions.
An online survey was created through the LimeSurvey platform, and an email was sent to the manager of each repository with a text describing the study being carried out and a link to the survey.In addition to the initial email, three reminders were sent on 1 and 10 June and 4 July 2016 to those who had not responded.
The survey was based on a list of repository indicators detailed in a prior study (Serrano-Vicente, Melero and Abadal, 2014).The survey consisted of 52 questions, the first four corresponding to identification data used in the repository and the remainder relating to the five categories mentioned above (technology, procedures, content, marketing and personnel; see Annex 1).

Results and discussion
Forty-six responses were received from a total of 66 emails; most of these were from university libraries (44) and two were from national research centres (the Consejo Superior de Investigaciones Científicas [Spanish National Research Council, CSIC] and the Instituto Español de Oceanografía [Spanish Oceanographic Institute, IEO]).The vast majority of respondents were librarians (93%), but they also included IT staff (4%) and publishers (3%).Ninety-one per cent were managers of the repository, and 9% were managers of part of the repository.
The results are presented below according to the five categories in the online survey: Technology, Procedures, Content, Marketing and Personnel.

Technology
Repository technology is understood as those aspects related to the computer program itself, and any possible improvements that may be incorporated to provide a more efficient service.Some of the results of this section (Q5-Q15 in the survey) are presented in Table 6.
In line with the trend observed in the 2009 report on Spanish repositories, DSpace is still the most widely used software, followed by EPrints (table 6).
In a survey on CRISs and institutional repositories conducted in 2015, which provided results for European universities and research centres only, 56% were found to use DSpace, followed by EPrints with 12% (Ribeiro, Castro and Mennielli, 2016).
Regarding authentication, fifty per cent of users access the repository using a username and password for the program, and the other 50% use LDAP (Lightweight Directory Access Protocol).Fourteen per cent use both systems.
Sixty-five per cent of the repositories have the capacity to import data from and export data to other university systems (table 6); this percentage has increased significantly compared to 2009, when just under 40% of repositories were linked to other university systems (Melero et al., 2009).
Other recent national surveys have addressed connectivity with other systems, and in the case of Australia and New Zealand, 62% of the universities surveyed have integrated their CRIS with their repository (CAUL, 2014).A study conducted in Europe (Ribeiro, Castro and Mennielli, 2016) concluded that CRISs and institutional repositories are complementary, and 65% of the respondents stated that they are interconnected.The repositories are generally linked to the library, integrated search systems and researchers' webpages, and, to a much lesser extent, financial management systems.In 18% of cases, the same system was used for both (Elsevier's Pure or Cineca's IRIS).
Most repositories have mechanisms for the large-scale import and export of metadata and digital objects (83%).Most repositories also have the capacity to export search results (79%) (table 6).
A few years ago, a study of German repositories (Vierkant, 2013) found that 56% of the programs had the capacity to export to bibliographic managers.This percentage had not changed two years later, when the same study was conducted, but with Swiss and Austrian repositories included (Open Access Repository Ranking (OARR), 2015).
In Spain, the percentage of repositories that provide statistics is considerably higher (94%) than in Australia and New Zealand (80%) and in Germany (25%) (table 6).However, author-level statistics are also available in 30% of the repositories in Australia and New Zealand (CAUL, 2014).The census on German repositories (Vierkant, 2013) indicates that the larger the repository, the less likely it is to have statistics.
Altmetrics represent a means of measuring the impact of a digital object through the new media of the semantic web (blogs, Facebook, Twitter, etc.).Such metrics are very agile, because they are updated in real time, and granular, since the impact is measured at article level.Introducing them into institutional repositories encourages academics to deposit their work, because they can access metrics that would otherwise not be available to them.Altmetrics also provide librarians with tools for analysing the repository (Melero, 2015;Roemer and Borchardt, 2015).
The survey revealed that 41% (20 items) of the repositories offer altmetrics.This percentage is lower in Australia (25%) (CAUL, 2014).The type of altmetrics provided by the repositories is mostly that offered by Altmetric.com(90%) (18 repositories).The majority of the repositories surveyed offer the possibility of sharing each document through social networks such as Facebook and Twitter (75%) (33 items), compared to 11% in the Vierkant study (2013) on a number of German repositories, although these data may vary from year to year.

Procedures
The internal management of the repository is crucial to ensure it works properly and to make it more manageable for both users and depositors.The key factors proposed to achieve this goal include compatibility with repository standards, the inclusion of help features for depositors, the implementation of open access in the institution and compliance with intellectual property rights (Q16-Q29 in the survey).
Procedure manuals describe the systems for entering, validating, linking, etc., documents in the repository.The majority of the repositories surveyed replied that these manuals are available on the web (table7), while the remainder either have none or have video tutorials, support tools when documents are deposited or FAQs instead.Style manuals, defined as instructions on how to enter bibliographic data in the institutional repository, are available on the web in fewer cases (table 7).
The practice in Australia has moved towards self-archiving by authors (63%) and administrative staff.In fact, it has been noted that most of the items (more than 60%) come from the CRIS; in other words, they are automatically deposited in the repository once introduced by authors.Downloading metadata from WoS and Scopus represents another form of importing metadata to the repository.All of these results in more work for the repositories managers, who are required to review records (CAUL, 2014).However, an international study found that 54% of the deposits were mediated, especially by repository staff (Dubinsky, 2014).
If these results are compared to those of the study carried out in 2009 (Melero et al., 2009), in which 56% were deposited by librarians and 24% by the authors themselves, it is clear that self-deposits by authors have risen considerably.Authors depositing their research outputs in repositories is more likely to bring about a cultural change that will ensure that an institutional repository and, ultimately, open access, becomes an integral part of an institution's research activities, although such processes depend largely on having infrastructure that allows it (such as interoperability with other institutional systems) (Lagzian, Abrizah and Wee, 2015).
Prior to depositing articles, most depositors are required to check editorial policies regarding permissions for self-archiving.This question relates to the way in which the editorial policy is consulted, since the easier it is to find, the more likely it is that people will actually consult it (Figure 1).There are different approaches in this area: some provide links to SHERPA/RoMEO from the repository, and some reported that they also include/embed reuse licences in the record metadata (CAUL, 2014).Librarians check records before making them public in 83% of cases; twelve per cent of the managers replied that documents are deposited after the embargo and a small percentage (4%) have the option of a document request button if the document is temporarily embargoed (table 7).
The metadata formats provided by repositories vary widely.All of them have adopted the Dublin Core format, the OAI-PMH protocol default format.Of the other formats, the most widely used are as follows (in this order): METS, ESE, MODS, MARC, RDF, ORE, DIDL, QDC, PREMIS, DIM and EDM.
Over the years, one of the aspects that has been studied with most interest is the use of unique author identifiers.Most respondents replied that they do not include an author identifier field in the metadata (Table 7).The survey conducted in Australian libraries revealed that they had not implemented this field, although 21% were considering using ORCID as an author identifier (CAUL, 2014).Most repositories said that they have metadata for the identification of research projects (Table 7).In the case of Australian repositories (CAUL, 2014), only 60% of the repositories surveyed include this information in their metadata.The majority of the repositories surveyed comply with OpenAIRE guidelines (OpenAIRE, 2015)(Table 7).
The last set of questions in this section concerned policies for withdrawing digital objects and reuse licences.Fifty-nine per cent of managers responded that the option of withdrawing records is mentioned.
Most of the respondents include open licences for the reuse of digital objects in the repository (Table 7).In half of cases, these licences are embedded in the files themselves, in 31% of cases they are not included, and in 16% of cases they are included in some documents only.In all cases, they are Creative Commons licences.

Content
The content of the repositories refers to the number and type of documents that are included, an element that has been considered a key factor in other assessments (Q30-Q37 in the survey).Aspects relating to open access and preservation policies, and whether some form of audit has been carried out, have also been studied.
The average number of records entered annually in Spanish repositories varies greatly.It ranges from 18,000 (Digital.CSIC) to 100 (table 8).Dubinsky noted that the average number of items added annually was 1,980 (Dubinsky, 2014).This figure is lower than the average number of documents deposited in the repositories in our study, which stands at 2,120 (excluding the Digital.CSIC repository).
A study by Lagzian et al. (2015) revealed that the number of records entered is not as important for assessing the success of a repository as it used to be.Some studies have claimed that a repository's success is related more to continuous daily deposits than the total number of deposits, since records can be subject to large-scale automatic deposits.A repository should reflect the research results of an institution.Therefore, although the volume of content may be one of the factors used to evaluate a repository, it must be related to the institution's total volume of research and the policies adopted.In terms of the versions of articles that may be deposited according to editorial policies, a high percentage allow preprints and accepted postprints, and all respondents said that they accept versions of record (Table 8).
The percentage of full-text documents has increased significantly with respect to the 2009 study (Table 8).Institutional mandates and recommendations, along with legislation that has been in place since 2011, have resulted in a clear rise in the percentage of full-text documents in repositories: almost half of the respondents replied that 100% of their documents are available as full text, nearly 40% have 90-99% available as full text, and a small percentage have 75-80% or fewer than 75% as full text.
It is difficult to determine how many full-text records there are, because they cannot be distinguished from metadata-only entries (Pinfield et al., 2014).This aspect is also linked to each university's policy, which may require that all publications be deposited, even if open access is not immediate and only the metadata can be viewed.In other cases, only full-text publications or those subject to some kind of embargo can be deposited.
The vast majority of repositories have not been audited (83%) (Table 8).The Digital.CSIC repository has received the Data Seal of Approval certification from the Dutch agency DANS, and the repository of the Universitat Autònoma de Barcelona has also received National Security Scheme certification (awarded by the Spanish government) and obtained ISO 16363 certification.

Marketing
This section examines the extent to which repositories are promoted within and outside the institution.Both aspects are necessary to ensure that, on the one hand, more researchers can get involved by entering content and becoming the face of the institution's research and, on the other hand, the research is disseminated externally (Q38-Q46).
The tools used to promote the repository within the institution are primarily face-toface training sessions, followed by support materials such as manuals and help pages.The academic authorities encourage open access among researchers in more than half of cases, a key element in repository marketing.Other forms of raising awareness include printed materials such as leaflets and posters (Table 9).Subject-specific librarians collaborate with the repository in 40% of the institutions that responded (Table 9).Librarians play a crucial role, and they have become strategic partners of the institutional repository (Scherer, 2016).The other marketing tools mentioned in the survey (20%) were subject-specific guides, seminars, videos, promotion through social networks and screens in buildings.
In addition to these tools, our study revealed the need to convey a message that alludes to open access to research and the benefits it brings for both the institution and the researcher.This message should be disengaged from the library, which serves as a mere intermediary, and be associated instead with the institution's goals.This aspect was not included in the survey, but would be interesting to keep in mind when designing a marketing strategy (Bruns and Inefuku, 2016;Otto, 2016) .
We have already mentioned the importance of promoting open access within the institution (Table 9).Almost a quarter of the institutions have an institutional mandate.Another response indicates that mediated deposit, which is used in 98% of cases, also serves to promote the repository, since authors do not waste time depositing records, one of the problems raised by depositors.In another case (the Spanish Oceanographic Institute), the assessment of researchers productivity is linked to the items deposited in the repository.A small percentage of the institutions (Universidad de Castilla-La Mancha, Universidad de Alicante, Universidad de Cádiz, Universitat de València and Universitat Politècnica de València) responded that they have other incentives, such as financial resources to publish in open access and cover article processing charges (APCs), while a few universities related repository deposits to career advancement (Universidad Internacional de La Rioja, Universidad de Burgos and Universitat Politècnica de Catalunya).Eleven per cent responded that their institutions do not have incentives, or did not give a response.Compared to the situation in 2009 (Melero et al., 2009), these policies have increased considerably, since depositing was mandatory in just 8% of cases eight years ago.Currently, there are open-access recommendations or mandates at 67% of the institutions surveyed, three indicated that they are in the process of establishing an open-access policy (Fundación Universitaria San Pablo CEU, Spanish Oceanographic Institute and Universidad Pontificia de Salamanca) and one institution offers career advancement for deposits (Universidad Internacional de La Rioja).
To identify how much importance is attached to a repository within institutions, the repository managers were asked if any links to the repository are included on corporate websites and, if so, which websites (Figure 2).In most cases, a link to the repository is included on the institution's websites (table 9).A high percentage have a link on the library website (91%).Around a quarter of the institutions provide a link to the repository on their institutional website (26%) or their research service website (22%).
Another form of promotion involves sending statistical reports of downloads and visit data to authors who have articles deposited in the repository.Forty per cent of the managers indicated that no such reports are sent, but that the statistics are public and can be consulted on the repository portal.Thirty per cent offer lists of the most consulted documents, while 18% provide a list of the most consulted authors.Only four repositories send regular reports to authors (Universidad de Granada, Universitat Autònoma de Barcelona, Universidad Carlos III de Madrid and Universitat Politècnica de Catalunya).Statistics are key for managers to illustrate the importance of the repository, since they demonstrate the visibility of research.Moreover, at author level, they represent a marketing tool that can be exploited by sending reports to authors and even awarding prizes and acknowledgements.For example, Purdue University awards prizes to the authors with the most downloads in one year (Scherer, 2016).
The institutions were asked whether marketing actions are also carried out outside the institution.Sixty-five per cent of the respondents said that they promote the repository in media outside the university, including press releases, conferences, blogs, seminars, information leaflets, Twitter and Facebook (Figure 3).

Personnel
The people who work in the repository provide a measure of the importance attached to open access at the institution and have been associated with the value-added services offered (Q47-Q52 in the survey).
Those responsible for managing the repository are usually IT staff and librarians.In some cases, librarians alone assume this function, and in a very few cases IT staff members are responsible, but always in conjunction with another professional (research manager, journal editor, etc.).
As seen in previous studies (Cassella and Morando, 2012;Simons and Richardson, 2012), a majority responded that they do not have full-time staff members in the repository (44%), although 38% of the institutions indicated that someone works in the repository on a full-time basis.In rarer cases, there are teams of two people (9%), three people (5%), or four or five people (2%).A more common scenario is for a group of professionals to work in the repository on a part-time basis.In 52% of cases, between one and five people work in the repository, in 16% of cases between six and 11 people work in the repository, in 16% of cases the figure is between 12 and 25 and, in a further 16%, the figure is between 26 and 60.
In total, the number of library staff members dedicated to working in the repository, on either a full-time or part-time basis, is shown in Figure 4.
A study conducted in Australia indicated that the average number of full-time staff members dedicated to the institutional repository is two (CAUL, 2014).The importance attached to repository management is greater if a member of the library management team coordinates the work.This is the case at most institutions; 76% indicated that a member of the library management team is responsible for the institutional repository.

Yes and No responses from institutional repositories
Some questions in the sections on technology, processes, content and marketing were studied separately, since their responses revealed whether or not the repositories have the services or criteria mentioned in the survey (Yes or No).A high number of positive responses reveal a higher regard for the features and services offered by the repository.Some of these indicators were included in the Open Access Repository Ranking of Germany, Switzerland and Austria (Open Access Repository Ranking (OARR), 2015).
Virtually all repositories have usage and download statistics, and all but three have mechanisms for importing/exporting metadata and digital objects (Figure 5).Repositories that allow documents to be shared through social networks and that support the export of results are also in the majority, albeit to a lesser degree.Altmetrics have not been developed in most repositories, although many institutions plan to use them in the future.
The vast majority of repositories surveyed comply with the OpenAIRE guidelines v 3.0 (2013), have a field for the research project code, and review records prior to their entry in the repository.There are more repositories that offer use licences and have digital object withdrawal policies than do not.The use of ORCID as an author identifier remains an outstanding issue in most repositories.
Two of the most commonly implemented content indicators in repositories are the existence of a public document on governance and the existence of a preservation policy.
Almost all the repositories have a link on a university website (library, research service, main website); they promote the repository through external channels, although not predominantly, and the same proportion carry out content dissemination actions through social networks.Most do not have their own blog, and the majority of institutions offer no incentives for making deposits (figure 5).
According to the scores assigned, mentioned in the methodology, fifteen repositories achieved the highest scores for one or more of the categories (technology, processes, content and marketing).The aim was to complete the study of these particular ones, with the data obtained from the questions whose answers were not yes/no.The results were of interest.
The repository of the Universitat Autònoma de Barcelona achieved the highest scores in most fields (figure 6).This repository stands out for its interoperability with university systems, including its CRIS, publications service software and academic management systems.Searches can be exported to systems such as MARCXML and Dublin Core, in addition to library management systems.With respect to management, it offers the option of consulting editorial policies from the repository and of exporting metadata in seven different formats.In terms of content, it accepts all versions of documents (preprints, postprints and versions of record), 100% of its records are available as full text and it is the only repository that has achieved ISO 16363 certification.It has an institutional mandate, and sends regular statistical reports to authors.It has a team of three people who work full time in the repository (higher than the average of the other repositories) and, in addition, 60% of the library staff contribute in other tasks.
The Universidad de Huelva's repository was established relatively recently; however, it was found to have some of the key characteristics in the categories.For example, it has implemented Altmetrics (technology); offers a field for ORCID (through authority control) and has reuse licences embedded in documents (procedures); 100% of its records are available as full text, and it has a public document with the repository's policy and the preservation policy (content); there is a link to the repository from the institutional website (a rare occurrence) and it has an institutional mandate, which reflects the support of the academic authorities (marketing) (figure 6).
With respect to technology, it is worth noting that all repositories offer the option to export searches, mostly to bibliographic manager formats, and two (Universidad de Las Palmas de Gran Canaria and Universidad de Murcia) via CSV.Sixty-five per cent are linked to the university's CRIS, as well as the publications service or academic management programs.In terms of procedures, the responses were very similar in all cases, except for document embargoes, which are applied automatically in all but one case.Linking the repository to the ORCID identifier is carried out in several ways; perhaps not with a specific metadata field, but by linking from the repository to the catalogue or CRIS, which does contain this information.In any case, most are working towards including the ORCID identifier in their records.All but one of the repositories have a preservation policy.Of the 15 repositories surveyed, 12 have 95-100% open access.Of the remaining three, the University Politechnic of Valencia (UPV) repository has 75% as full text, the CSIC has 62% and DIGITUM (Universidad de Murcia) has just 14% as full text.In terms of marketing, there are two repositories that have links on the institution's main websites (Universidad de Huelva and Universitat Politècnica de Catalunya).Other universities have links on the research service website (Universitat Autònoma de Barcelona, CSIC, Universidad Carlos III de Madrid and Universitat Politècnica de Catalunya) and on the doctoral degrees website (Universidad de Navarra).Funds are available to pay article processing charges at the CSIC and the University of Barcelona, and the Universidad de Alicante and the Universitat Politècnica de València also offer financial resources.The marketing methods used by several institutions include regular reports to authors (Universitat Autònoma de Barcelona, Universidad Carlos III de Madrid, Universitat Politècnica de Catalunya), lists of the most consulted documents (Universitat Politècnica de Catalunya, Universidad de Las Palmas de Gran Canaria, Universidad de Navarra), reports to faculties (Universidad de Navarra) and annual reports (Universidad de Cantabria) (figure 6).

Conclusions
The purpose of this repository assessment model was to verify whether, through key elements of internal management (e.g.procedures, personnel and introduction of content) and external management (e.g.technology and marketing), their effectiveness and level of integration within institutions could be measured.
The report conducted by COAR (COAR, 2017) recommended improving the interoperability of repositories by adopting standard practices to link publications to the corresponding research project, funding entities and institutions.In addition to integrating the repository with other university systems (CRIS, the publications service, etc.), the aspects studied in the technology section of the proposed assessment model cover the objectives of achieving interoperability between repositories, institutional systems and funding entities.In addition, measurements carried out through internal repository elements (statistics) and external repository elements (altmetrics and relationship to social networks), promote a wider dissemination of the institution's research.
Examining the repository's internal procedures relating to the proposed aspects provides for better management of the institution's academic output and allows for monitoring of open access policies at national and international level (OpenAIRE guidelines, research project metadata, etc.).Moreover, by including unique author identifiers (a quarter of the repositories analysed use ORCID, a figure that is rising), a link is provided among other tools that are widely used by authors, such as academic platforms.
Knowing the annual volume of content entered in repositories makes it possible to extract an average per institution.In order to identify the proportion of research that is open access at each institution, a much more detailed study would be necessary.However, these indicators provide an insight into the proportion of documents that are available as full text and, thus, repositories' compliance with institutional mandates or recommendations; this information makes it possible to monitor open access policies at national and international level.
According to the COAR report (COAR, 2017), open access repositories can increase the visibility and citations of published articles, a fact borne out by usage statistics.The case we have studied clearly shows that having the majority of content as open access and offering the option of consulting usage statistics pave the way for disseminating this research more effectively.
The number of people dedicated to the repository has been associated with the number of services available.However, it has transpired that teams are small, partly due to the expertise required to work in this field.The most requested services, however, relate more to the software (permanent URL, for example).Nevertheless, having more staff in the team means that open access to research is given greater importance within the library and the institution.Thus, this represents a useful quality indicator.In summary, the indicators used in this assessment of research repositories are in line with current thinking on best practices (COAR, 2017, OARR, 2015).
They could therefore prove very useful for anyone planning an institutional repository, or for anyone who already has an operational repository and wishes to carry out an assessment.The repository commits to preserving digital objects at specified format

Audit
The repository has been subject to an audit or obtained any certifications Type of audit Fecyt/Rebiun, Drambora, TRAC, etc.

Figure 1 .
Figure 1.Where and how often editorial policies regarding self-archiving in institutional repositories are consulted

Figure 2 .Figure 3 .
Figure 1.Where and how often editorial policies regarding self-archiving in institutional repositories are consulted

Figure 4 .
Figure 4. Personnel involved in the management of repositories and time commitment

Figure 5 .Figure 6 .
Figure 5. Responses to questions about repository policies and the services they offer Definition and/or Domain Promotion within the institution Tools used to promote the repository within the institution: Face-toface training sessions, promotion of open access by the academic authorities, support materials, Leaflets, Posters, Subject-

Table III .
Content based indicators.

Table V .
Personnel based indicators.

Table VI .
Percentage of responding repositories regarding IR technology

Table VII .
Percentage of responding repositories regarding procedures.

Table VIII .
Percentage of responding repositories regarding contents of the repository.Growth, fulltext documents, accepted versions, and IR policies.Results regarding marketing and promotion of the repository within the institution.Percentage of responding repositories