Low-resource fine-tuning of llama-3.1 using QLoRA for Nigerian linguistic contexts

Solomon Joseph Udoabba, Bliss Utibe-Abasi Stephen, Oluseun Damilola Oyeleke, Philip Asuquo, Sadiq Thomas

Abstract


Large language models (LLMs) often underperform in low-resource linguistic environments due to under-representation in pre-training data. This work presents NaijaLLaMA-8B, a parameter-efficient adaptation of Meta’s LLaMA-3.1 8B model for Nigerian English and Nigerian Pidgin using the QLoRA fine-tuning
approach. A subset of 7,000 samples was extracted from the NaijaWeb corpus and used to perform supervised fine-tuning on a single Tesla T4 GPU under strict resource constraints. Performance evaluation shows consistent improvements over the base model, with training loss decreasing from 2.04 to 1.98 and perplexity reducing from 8.20 to 7.38. Small but measurable gains were also observed in BLEU, ROUGE-L, and BERTScore-F1 metrics. Although absolute improvements remain modest, the results validate the technical
feasibility of adapting large language models to Nigerian linguistic contexts using limited compute and dataset size. This study establishes a reproducible baseline for Nigerian-focused language model adaptation and demonstrates the practical viability of parameter-efficient fine-tuning under constrained computational environments

 

Received 31 March 2026

Accepted 20 May 2026

Published 23 June 2026


Keywords


Parameter-efficient fine-tuning, natural language processing, low resource languages, Nigerian Pidgin

Full Text:

PDF

References


L. Qin et al., “Large Language Models Meet NLP: A Survey,” arXiv.org, 2024. https://arxiv.org/abs/2405.12819

A. Dubey et al., “The Llama 3 Herd of Models,” arXiv.org, 2024. https://arxiv.org/abs/2407.21783

I. Inuwa-Dutse, “NaijaNLP: A Survey of Nigerian Low-Resource Languages,” arXiv.org, 2025. https://arxiv.org/abs/2502.19784

J. Ojo et al., “AfroBench: How Good are Large Language Models on African Languages?,” Findings of the Association for Computational Linguistics: ACL 2022, pp. 19048–19095, Jan. 2025, doi: https://doi.org/10.18653/v1/2025.findings-acl.976.

E. J. Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models,” arXiv:2106.09685 [cs], Oct. 2021, Available: https://arxiv.org/abs/2106.09685

T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “QLoRA: Efficient Finetuning of Quantized LLMs,” arXiv.org, May 23, 2023. https://arxiv.org/abs/2305.14314

N. Bui et al., “Fine-tuning Large Language Models for Improved Health Communication in Low-Resource Languages,” Computer Methods and Programs in Biomedicine, vol. 263, pp. 108655–108655, Feb. 2025, doi: https://doi.org/10.1016/j.cmpb.2025.108655.

P. W. Khoboko, V. Marivate, and J. Sefara, “Optimizing translation for low-resource languages: Efficient fine-tuning with custom prompt engineering in large language models,” Machine Learning with Applications, vol. 20, p. 100649, Jun. 2025, doi: https://doi.org/10.1016/j.mlwa.2025.100649.

S. A. Ayanniyi, “Naijaweb: A Web Scraped Nigerian Context Dataset,” Huggingface.co, 2021. https://huggingface.co/datasets/saheedniyi/naijaweb




DOI: https://dx.doi.org/10.21622/ACE.2026.06.1.1933

Refbacks

  • There are currently no refbacks.


Copyright (c) 2026 Solomon Joseph Udoabba, Bliss Utibe-Abasi Stephen, Oluseun Damilola Oyeleke, Philip Asuquo, Sadiq Thomas

Advances in Computing and Engineering

E-ISSN: 2735-5985

P-ISSN: 2735-5977

 

Published by:

Academy Publishing Center (APC)

Arab Academy for Science, Technology and Maritime Transport (AASTMT)

Alexandria, Egypt

ace@aast.edu