Annals of Emerging Technologies in Computing (AETiC)

 
Paper #5                                                                             

Building Dictionaries for Low Resource Languages: Challenges of Unsupervised Learning

Diellza Nagavci Mati, Mentor Hamiti, Arsim Susuri, Besnik Selimi and Jaumin Ajdari


Abstract: The development of natural language processing resources for Albanian has grown steadily in recent years. This paper presents research conducted on unsupervised learning-the challenges associated with building a dictionary for the Albanian language and creating part-of-speech tagging models. The majority of languages have their own dictionary, but languages with low resources suffer from a lack of resources. It facilitates the sharing of information and services for users and whole communities through natural language processing. The experimentation corpora for the Albanian language includes 250K sentences from different disciplines, with a proposal for a part-of-speech tagging tag set that can adequately represent the underlying linguistic phenomena. Contributing to the development of Albanian is the purpose of this paper. The results of experiments with the Albanian language corpus revealed that its use of articles and pronouns resembles that of more high-resource languages. According to this study, the total expected frequency as a means for correctly tagging words has been proven effective for populating the Albanian language dictionary.


Keywords: Albanian language; corpora; dictionaries; natural language processing; part-of-speech tagging.


 
Full Text

This work is licensed under a Creative Commons Attribution 4.0 International License. Creative Commons License


This browser does not support PDFs. Please download the PDF to view it: Download PDF.

 
 International Association for Educators and Researchers (IAER), registered in England and Wales - Reg #OC418009                         Copyright © IAER 2021