Thematic articles

Institutional Repository Keyword Analysis with Web Crawler

Published:
December 23, 2022
Authors
View
Keywords
License

Copyright (c) 2022 by the authors

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How To Cite
Selected Style: APA
Fujita, M. S. L., Katahira, I., & Tolare, J. B. (2022). Institutional Repository Keyword Analysis with Web Crawler. Central European Journal of Educational Research, 4(2), 54-59. https://doi.org/10.37441/cejer/2022/4/2/11395
Received 2022-08-31
Accepted 2022-11-28
Published 2022-12-23
Abstract

This study aims at investigating procedures of semantic and linguistic extraction of keywords from metadata of documents indexed in the Institutional Repository Unesp. For that purpose, a web crawler was developed, that collected 325.181 keywords from authors, in all fields of knowledge, from February 28th, 2013 to November 10th, 2021. The preparation of the collection, extraction and analysis environment used the Python programming language, composed of three program libraries: library requests, which allows manipulation of hyperlinks of webpages visited through web crawler; BeautifulSoup library, used to extract HTML data through webpage analysis; and Pandas library, which has an open code (free software) and stands for providing tools for high performance data manipulation and analysis. The final listing consisted of 273,485 keywords, which represents 15.9% of the listing initially collected. Results indicated that the most recurring problem was the duplication of keywords, with 51,696 duplicated keywords, representing indicators of inconsistencies in the search for documents. It is concluded that the refinement of keywords assigned by authors eliminates the incorporation of a set of symbols that do not represent the authors’ keywords with the same spelling, but with upper/lower case variations or lexical variations indexing different documents.