Llama and Gemma vs IndoBERT and SBERT: Zero-Shot and Few-Shot Sentiment Analysis
Abstract
Keywords
Full Text:
PDFReferences
F. Koto, N. Aisyah, H. Li, and T. Baldwin, "Large Language Models only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU," in Proc. 2023 Conf. Empirical Methods in Natural Language Processing (EMNLP), 2023, pp. 12359–12374, DOI: 10.18653/v1/2023.emnlp-main.760.
Q. Dong et al., "A Survey on in-Context Learning," in Proc. 2024 Conf. Empirical Methods in Natural Language Processing (EMNLP), 2024, pp. 1107–1128, DOI: 10.18653/v1/2024.emnlp-main.64.
H. Steck, C. Ekanadham, and N. Kallus, "Is Cosine-Similarity of Embeddings Really about Similarity?," in Companion Proc. ACM Web Conference 2024, 2024, DOI: 10.1145/3589335.3651526.
K. Zhou, K. Ethayarajh, D. Card, and D. Jurafsky, "Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words," in Proc. 60th Annual Meeting of the Association for Computational Linguistics (ACL), Vol. 2, 2022, pp. 401–423, DOI: 10.18653/v1/2022.acl-short.45.
T. Gao, X. Yao, and D. Chen, "SimCSE: Simple Contrastive Learning of Sentence Embeddings," in Proc. 2021 Conf. Empirical Methods in Natural Language Processing (EMNLP), 2021, pp. 6894–6910, DOI: 10.18653/v1/2021.emnlp-main.552.
A. Buscemi and D. Proverbio, "ChatGPT vs Gemini vs LLaMA on Multilingual Sentiment Analysis," arXiv preprint arXiv:2402.01715, 2024, DOI: 10.48550/arXiv.2402.01715.
M. A. Hasan et al., "Zero- and Few-Shot Prompting with LLMs: A Comparative Study with Fine-Tuned Models for Bangla Sentiment Analysis," in Proc. 2024 Joint Int. Conf. Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), Torino, Italy, 2024, pp. 17808–17818.
D. Cielen, K. W. De Bock, and L. Florès, "Exploring Zero-Shot SLM Ensembles as an Alternative to LLMs for Sentiment Analysis," Information Fusion, Vol. 126, pt. B, art. No. 103666, 2026, DOI: 10.1016/j.inffus.2025.103666.
D. Di Palma, A. De Bellis, G. Servedio, V. W. Anelli, F. Narducci, and T. Di Noia, "LLaMAs have Feelings Too: Unveiling Sentiment and Emotion Representations in LLaMA Models through Probing," in Proc. 63rd Annual Meeting of the Association for Computational Linguistics (ACL), Vol. 1, 2025, pp. 6124–6142, DOI: 10.18653/v1/2025.acl-long.306.
M. Cherakhloo, A. Abbasi, M. S. Sarafraz, and B. V. Vahdat, "Benchmarking Open-Source Large Language Models for Persian in Zero-Shot and Few-Shot Learning," arXiv preprint arXiv:2510.12807, 2025, DOI: 10.48550/arXiv.2510.12807.
G. I. Winata et al., "NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages," in Proc. 17th Conf. European Chapter of the Association for Computational Linguistics (EACL), 2023, pp. 815–834, DOI: 10.18653/v1/2023.eacl-main.57.
M. Ikonić Nešić, S. Petalinkar, M. Škorić, R. Stanković, and B. Rujević, "Advancing Sentiment Analysis in Serbian Literature: A Zero and Few-Shot Learning Approach using the Mistral Model," in Proc. 6th Int. Conf. Computational Linguistics in Bulgaria (CLIB 2024), 2024, pp. 58–70.
S. Münker, K. Kugler, and A. Rettinger, "Zero-Shot Prompt-based Classification: Topic Labeling in Times of Foundation Models in German Tweets," in Proc. 63rd Annual Meeting of the Association for Computational Linguistics (ACL), Student Research Workshop, 2025, pp. 53–63, DOI: 10.18653/v1/2025.acl-srw.4.
S. Schulhoff et al., "The Prompt Report: A Systematic Survey of Prompt Engineering Techniques," arXiv preprint arXiv:2406.06608, 2024, DOI: 10.48550/arXiv.2406.06608.
Y. Tang, D. Tuncel, C. Koerner, and T. Runkler, "The Few-Shot Dilemma: Over-Prompting Large Language Models," in Proc. 3rd Int. Conf. Foundation and Large Language Models (FLLM), 2025, pp. 134–141, DOI: 10.1109/FLLM67465.2025.11391015.
B. P. Allen, F. Polat, and P. Groth, "SHROOM-INDElab at SemEval-2024 Task 6: Zero- and Few-Shot LLM-based Classification for Hallucination Detection," in Proc. 18th Int. Workshop on Semantic Evaluation (SemEval-2024), 2024, pp. 839–844, DOI: 10.18653/v1/2024.semeval-1.120.
L. Geni, E. Yulianti, and D. I. Sensuse, "Sentiment Analysis of Tweets before the 2024 Elections in Indonesia using BERT Language Models," Jurnal Ilmiah Teknik Elektro Komputer dan Informatika, Vol. 9, No. 3, pp. 746–757, 2023, DOI: 10.26555/jiteki.v9i3.26490.
D. G. Mandhasiya, H. Murfi, and A. Bustamam, "The Hybrid of BERT and Deep Learning Models for Indonesian Sentiment Analysis," Indonesian Journal of Electrical Engineering and Computer Science, Vol. 33, No. 1, pp. 591–602, 2024, DOI: 10.11591/ijeecs.v33.i1.pp591-602.
T. D. Purnomo and J. Sutopo, "Comparison of Pre-Trained BERT-based Transformer Models for Regional Language Text Sentiment Analysis in Indonesia," International Journal Science and Technology, Vol. 3, No. 3, pp. 11–21, 2024, DOI: 10.56127/ijst.v3i3.1739.
F. N. Khaqqi, L. Alfat, and I. Nurhaida, "Sentiment Analysis of US-China Tariffs using IndoBERT and Economic Impact on Indonesia," Journal of Applied Informatics and Computing, Vol. 9, No. 6, pp. 3000–3011, 2025, DOI: 10.30871/jaic.v9i6.11544.
A. Bello, S.-C. Ng, and M.-F. Leung, "A BERT Framework to Sentiment Analysis of Tweets," Sensors, Vol. 23, No. 1, art. No. 506, 2023, DOI: 10.3390/s23010506.
B. Wilie et al., "IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding," in Proc. 1st Conf. Asia-Pacific Chapter of the Association for Computational Linguistics and 10th Int. Joint Conf. on Natural Language Processing (AACL-IJCNLP), 2020, pp. 843–857, DOI: 10.18653/v1/2020.aacl-main.85.
V. R. Joseph, "Optimal Ratio for Data Splitting," Statistical Analysis and Data Mining: The ASA Data Science Journal, Vol. 15, No. 4, pp. 531–538, 2022, DOI: 10.1002/sam.11583.
J. Snell, K. Swersky, and R. Zemel, "Prototypical Networks for Few-Shot Learning," in Advances in Neural Information Processing Systems (NeurIPS), Vol. 30, 2017.
L. Pacchiardi, L. G. Cheke, and J. Hernández-Orallo, "100 Instances is All You Need: Predicting the Success of a New LLM on Unseen Data by Testing on a Few Instances," arXiv preprint arXiv:2409.03563, 2024, DOI: 10.48550/arXiv.2409.03563.
A. Grattafiori et al., "The Llama 3 Herd of Models," arXiv preprint arXiv:2407.21783, 2024, doi: 10.48550/arXiv.2407.21783.
Gemma Team, "Gemma 2: Improving Open Language Models at a Practical Size," arXiv preprint arXiv:2408.00118, 2024, DOI: 10.48550/arXiv.2408.00118.
N. Reimers and I. Gurevych, "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks," in Proc. 2019 Conf. Empirical Methods in Natural Language Processing and 9th Int. Joint Conf. on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 3982–3992, DOI: 10.18653/v1/D19-1410.
N. Reimers and I. Gurevych, "Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation," in Proc. 2020 Conf. Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 4512–4525, DOI: 10.18653/v1/2020.emnlp-main.365.
T. G. Dietterich, "Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms," Neural Computation, Vol. 10, No. 7, pp. 1895–1923, 1998, DOI: 10.1162/089976698300017197.
S. Holm, "A Simple Sequentially Rejective Multiple Test Procedure," Scandinavian Journal of Statistics, Vol. 6, No. 2, pp. 65–70, 1979.
B. Efron and R. J. Tibshirani, An Introduction to the Bootstrap. New York, NY, USA: Chapman & Hall, 1993.
DOI: https://doi.org/10.32520/stmsi.v15i11.7018
Article Metrics
Abstract view : 0 timesPDF - 0 times
Refbacks
- There are currently no refbacks.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.






