PENERAPAN RETRIEVAL AUGEMENTED GENERATION MENGGUNAKAN LANGCHAIN DALAM PENGEMBANGAN SISTEM TANYA JAWAB HADIS BERBASIS WEB
Keywords:
Hadith, Langchain, Large Language Model, Question Answering System, Retrieval Augmented Generation
Abstract
Hadis ajaran kedua setelah al-Qur'an yang menjadi panduan bagi umat Islam. Pencarian hadis saat ini kurang interaktif dalam menjawaban pertanyaan, dimana hanya menampilkan dokumen relevan. Penelitian ini bertujuan untuk mengembangkan sistem tanya jawab hadis berbasis web dengan menerapkan Retrieval Augmented Generation menggunakan framework LangChain yang diintegrasikan dengan Large Language Model GPT-4-1106-preview dari OpenAI. Sistem ini dirancang untuk membantu pengguna dalam mencari jawaban yang sesuai dengan 9 kitab hadis. Hasil penelitian menunjukkan bahwa model dapat bekerja sesuai dengan instruksi dan data dengan menyertakan sumber dari hadis terkait. Pengujian dilakukan dengan menguji 10 pertanyaan seputar hadis dengan framework BERTScore dan uji Evaluasi kualitas jawaban dengan mahasiwa ushulludin. Pada pengujian BERTScore rata-rata f1 score sebesar 0,7962, yang menunjukkan kemiripan antara jawaban sistem dengan referensi, pengujian pada Evaluasi kualitas jawaban mencapai persentase akurasi 89,4% yang menunjukkan bahwa responden ”Sangat Setuju” terhadap jawaban yang dihasilkan oleh sistem.
Downloads
Download data is not yet available.
References
[1] M. A. Çalgan, “The Problems in Ḥadīth Usage in Kur’an Yolu Tafsīr within the Context of Qurʾān-Sunnah Unity,” Cumhuriyet Ilahiyat Dergisi, vol. 25, no. 3, pp. 1277–1298, 2021, doi:10.18505/cuid.962041
[2] A. Supian and A. Farhan, “Pemahaman Hadis dan Implikasinya pada Praktek Keagamaan Jamaah Tabligh (Kajian Living Hadis di Kota Bengkulu),” AL QUDS : Jurnal Studi Alquran dan Hadis, vol. 5, no. 2, p. 537, Oct. 2021, doi:10.29240/alquds.v5i2.2501
[3] R. Rahmatullah, “Popularitas Moderasi Beragama: Sebuah Kajian terhadap Tren Penelusuran Warganet Indonesia,” NALAR: Jurnal Peradaban dan Pemikiran Islam, vol. 5, no. 1, pp. 62–77, Jun. 2021, doi:10.23971/njppi.v5i1.2419
[4] R. C. Widayaningsih and M. I. Helmy, “The Fiqh Al-Hadith Of Digital Media: The Method Of Hadith Understanding Of The Website Bincangsyariah.com And Its Contribution To The Moderate Islam Discourse,” Jurnal Ushuluddin, vol. 29, no. 2, p. 163, Dec. 2021, doi:10.24014/jush.v29i2.13954
[5] O. Topsakal and T. C. Akinci, “Creating Large Language Model Applications Utilizing LangChain: A Primer on Developing LLM Apps Fast,” All Sciences Proceedings, 2023.
[6] I. Siragusa and R. Pirrone, “Conditioning Chat-GPT for information retrieval: the Unipa-GPT case study,” Seventh Workshop on Natural Language for Artificial Intelligence, 2023.
[7] Z. Levonian et al., “Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preference,” arXiv preprint, Oct. 2023, arXiv:2310.03184v1
[8] A. Abdi, S. Hasan, M. Arshi, S. M. Shamsuddin, and N. Idris, “A question answering system in hadith using linguistic knowledge,” Comput Speech Lang, vol. 60, Mar. 2020, doi: 10.1016/j.csl.2019.101023.
[9] A. Zubir Rosdi, S. Najihuddin Syed Hassan, N. Asiah Fasehah Muhamad, N. Izzatul Huda Mohamad Zainuzi, M. Shiham Mahfuz, and F. Pengajian Quran dan Sunnah, “PANDUAN ASAS KAEDAH KENAL PASTI STATUS HADIS: KAJIAN DISKRIPTIF PENGGUNAAN ENSIKLOPEDIA HADIS 9 IMAM (Basic Methods in Identifying the Status of Hadith: A Descriptive Overview on the Use of Encyclopedia of Hadith of the Nine Imams),” JOURNAL OF QUR’AN AND HADITH STUDIES, vol. 8, no. 1, pp. 2550–1488, 2023, doi:10.33102/johs.v8i1.225
[10] A. Arora and M. Dell, “LinkTransformer: A Unified Package for Record Linkage with Transformer Language Models,” arXIv preprint, Sep. 2023, arXiv:2309.00789
[11] A. Kean Gao, “Vec2Vec: A Compact Neural Network Approach for Transforming Text Embeddings with High Fidelity,” ArXiv preprint, 2023, arXiv:2306.12689
[12] T. B. Brown et al., “Language Models are Few-Shot Learners,” Adv Neural Inf Process Syst, vol. 2020-December, May 2020, arXiv:2005.14165v4
[13] D. Najafali, J. M. Camacho, L. G. Galbraith, E. Reiche, A. H. Dorafshar, and S. D. Morrison, “Ask and You Shall Receive: OpenAI ChatGPT Writes Us an Editorial on Using Chatbots in Gender Affirmation Surgery and Strategies to Increase Widespread Adoption,” Aesthetic Surgery Journal, vol. 43, no. 9. Oxford University Press, pp. NP715–NP717, Sep. 01, 2023. doi:10.1093/asj/sjad119
[14] S. Y. Yoo and O. R. Jeong, “EP-Bot: Empathetic Chatbot Using Auto-Growing Knowledge Graph,” Computers, Materials and Continua, vol. 67, no. 3, pp. 2807–2817, Mar. 2021, doi:10.32604/cmc.2021.015634
[15] D. Nunes, R. Primi, R. Pires, R. Lotufo, and R. Nogueira, “Evaluating GPT-3.5 and GPT-4 Models on Brazilian University Admission Exams,” ArXiv preprint, Mar. 2023, arXiv:2303.17003
[16] A. Torfi, R. A. Shirvani, Y. Keneshloo, N. Tavaf, and E. A. Fox, “Natural Language Processing Advancements By Deep Learning: A Survey,” arXiv preprint, Mar. 2020, arXiv:2003.01200
[17] M. Abbaszade, V. Salari, S. S. Mousavi, M. Zomorodi, and X. Zhou, “Application of Quantum Natural Language Processing for Language Translation,” IEEE Access, vol. 9, pp. 130434–130448, 2021, doi:10.1109/ACCESS.2021.3108768
[18] Arjun Pesaru, Taranveer Singh Gill, and Archit Reddy Tangella, “AI assistant for document management Using Lang Chain and Pinecone,” International Research Journal of Modernization in Engineering Technology and Science, Jun. 2023, doi:10.56726/irjmets42630
[19] Tejaswini NR, Vidya, and Dr. T Vijaya Kumar, “LangChain-Powered Virtual Assistant for PDF Communication,” International Research Journal of Modernization in Engineering Technology and Science, Jul. 2023, doi:10.56726/irjmets43587
[20] Y. H. Ke et al., “Development and Testing of Retrieval Augmented Generation in Large Language Models-A Case Study Report,” arXiv preprint, 2024, arXiv:2402.01733.
[21] H. Touvron et al., “LLaMA: Open and Efficient Foundation Language Models,” arXiv preprint, Feb. 2023, arXiv:2302.13971v1
[22] J. Hoffmann et al., “Training Compute-Optimal Large Language Models,” Adv Neural Inf Process Syst, vol. 35, Mar. 2022, arXiv:2203.15556v1
[23] J. Yang et al., “Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond,” arXiv preprint, Apr. 2023, arXiv:2304.13712
[24] R. Pedro, D. Castro, P. Carreira, and N. Santos, “From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?,” arXiv preprint, Aug. 2023, arXiv:2308.01990
[25] Y. Xie, C. Yu, T. Zhu, J. Bai, Z. Gong, and H. Soh, “Translating Natural Language to Planning Goals with Large-Language Models,” arxiv preprint, Feb. 2023, arXiv:2302.05128
[26] J. Devlin, M.-W. Chang, K. Lee, K. T. Google, and A. I. Language, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Proceedings of the 2019 Conference of the North, pp. 4171–4186, 2019, doi: 10.18653/V1/N19-1423.
[27] V. H. Pranatawijaya, W. Widiatry, R. Priskila, and P. B. A. A. Putra, “Penerapan Skala Likert dan Skala Dikotomi Pada Kuesioner Online,” Jurnal Sains dan Informatika, vol. 5, no. 2, pp. 128–137, Dec. 2019, doi: 10.34128/jsi.v5i2.185.
[2] A. Supian and A. Farhan, “Pemahaman Hadis dan Implikasinya pada Praktek Keagamaan Jamaah Tabligh (Kajian Living Hadis di Kota Bengkulu),” AL QUDS : Jurnal Studi Alquran dan Hadis, vol. 5, no. 2, p. 537, Oct. 2021, doi:10.29240/alquds.v5i2.2501
[3] R. Rahmatullah, “Popularitas Moderasi Beragama: Sebuah Kajian terhadap Tren Penelusuran Warganet Indonesia,” NALAR: Jurnal Peradaban dan Pemikiran Islam, vol. 5, no. 1, pp. 62–77, Jun. 2021, doi:10.23971/njppi.v5i1.2419
[4] R. C. Widayaningsih and M. I. Helmy, “The Fiqh Al-Hadith Of Digital Media: The Method Of Hadith Understanding Of The Website Bincangsyariah.com And Its Contribution To The Moderate Islam Discourse,” Jurnal Ushuluddin, vol. 29, no. 2, p. 163, Dec. 2021, doi:10.24014/jush.v29i2.13954
[5] O. Topsakal and T. C. Akinci, “Creating Large Language Model Applications Utilizing LangChain: A Primer on Developing LLM Apps Fast,” All Sciences Proceedings, 2023.
[6] I. Siragusa and R. Pirrone, “Conditioning Chat-GPT for information retrieval: the Unipa-GPT case study,” Seventh Workshop on Natural Language for Artificial Intelligence, 2023.
[7] Z. Levonian et al., “Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preference,” arXiv preprint, Oct. 2023, arXiv:2310.03184v1
[8] A. Abdi, S. Hasan, M. Arshi, S. M. Shamsuddin, and N. Idris, “A question answering system in hadith using linguistic knowledge,” Comput Speech Lang, vol. 60, Mar. 2020, doi: 10.1016/j.csl.2019.101023.
[9] A. Zubir Rosdi, S. Najihuddin Syed Hassan, N. Asiah Fasehah Muhamad, N. Izzatul Huda Mohamad Zainuzi, M. Shiham Mahfuz, and F. Pengajian Quran dan Sunnah, “PANDUAN ASAS KAEDAH KENAL PASTI STATUS HADIS: KAJIAN DISKRIPTIF PENGGUNAAN ENSIKLOPEDIA HADIS 9 IMAM (Basic Methods in Identifying the Status of Hadith: A Descriptive Overview on the Use of Encyclopedia of Hadith of the Nine Imams),” JOURNAL OF QUR’AN AND HADITH STUDIES, vol. 8, no. 1, pp. 2550–1488, 2023, doi:10.33102/johs.v8i1.225
[10] A. Arora and M. Dell, “LinkTransformer: A Unified Package for Record Linkage with Transformer Language Models,” arXIv preprint, Sep. 2023, arXiv:2309.00789
[11] A. Kean Gao, “Vec2Vec: A Compact Neural Network Approach for Transforming Text Embeddings with High Fidelity,” ArXiv preprint, 2023, arXiv:2306.12689
[12] T. B. Brown et al., “Language Models are Few-Shot Learners,” Adv Neural Inf Process Syst, vol. 2020-December, May 2020, arXiv:2005.14165v4
[13] D. Najafali, J. M. Camacho, L. G. Galbraith, E. Reiche, A. H. Dorafshar, and S. D. Morrison, “Ask and You Shall Receive: OpenAI ChatGPT Writes Us an Editorial on Using Chatbots in Gender Affirmation Surgery and Strategies to Increase Widespread Adoption,” Aesthetic Surgery Journal, vol. 43, no. 9. Oxford University Press, pp. NP715–NP717, Sep. 01, 2023. doi:10.1093/asj/sjad119
[14] S. Y. Yoo and O. R. Jeong, “EP-Bot: Empathetic Chatbot Using Auto-Growing Knowledge Graph,” Computers, Materials and Continua, vol. 67, no. 3, pp. 2807–2817, Mar. 2021, doi:10.32604/cmc.2021.015634
[15] D. Nunes, R. Primi, R. Pires, R. Lotufo, and R. Nogueira, “Evaluating GPT-3.5 and GPT-4 Models on Brazilian University Admission Exams,” ArXiv preprint, Mar. 2023, arXiv:2303.17003
[16] A. Torfi, R. A. Shirvani, Y. Keneshloo, N. Tavaf, and E. A. Fox, “Natural Language Processing Advancements By Deep Learning: A Survey,” arXiv preprint, Mar. 2020, arXiv:2003.01200
[17] M. Abbaszade, V. Salari, S. S. Mousavi, M. Zomorodi, and X. Zhou, “Application of Quantum Natural Language Processing for Language Translation,” IEEE Access, vol. 9, pp. 130434–130448, 2021, doi:10.1109/ACCESS.2021.3108768
[18] Arjun Pesaru, Taranveer Singh Gill, and Archit Reddy Tangella, “AI assistant for document management Using Lang Chain and Pinecone,” International Research Journal of Modernization in Engineering Technology and Science, Jun. 2023, doi:10.56726/irjmets42630
[19] Tejaswini NR, Vidya, and Dr. T Vijaya Kumar, “LangChain-Powered Virtual Assistant for PDF Communication,” International Research Journal of Modernization in Engineering Technology and Science, Jul. 2023, doi:10.56726/irjmets43587
[20] Y. H. Ke et al., “Development and Testing of Retrieval Augmented Generation in Large Language Models-A Case Study Report,” arXiv preprint, 2024, arXiv:2402.01733.
[21] H. Touvron et al., “LLaMA: Open and Efficient Foundation Language Models,” arXiv preprint, Feb. 2023, arXiv:2302.13971v1
[22] J. Hoffmann et al., “Training Compute-Optimal Large Language Models,” Adv Neural Inf Process Syst, vol. 35, Mar. 2022, arXiv:2203.15556v1
[23] J. Yang et al., “Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond,” arXiv preprint, Apr. 2023, arXiv:2304.13712
[24] R. Pedro, D. Castro, P. Carreira, and N. Santos, “From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?,” arXiv preprint, Aug. 2023, arXiv:2308.01990
[25] Y. Xie, C. Yu, T. Zhu, J. Bai, Z. Gong, and H. Soh, “Translating Natural Language to Planning Goals with Large-Language Models,” arxiv preprint, Feb. 2023, arXiv:2302.05128
[26] J. Devlin, M.-W. Chang, K. Lee, K. T. Google, and A. I. Language, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Proceedings of the 2019 Conference of the North, pp. 4171–4186, 2019, doi: 10.18653/V1/N19-1423.
[27] V. H. Pranatawijaya, W. Widiatry, R. Priskila, and P. B. A. A. Putra, “Penerapan Skala Likert dan Skala Dikotomi Pada Kuesioner Online,” Jurnal Sains dan Informatika, vol. 5, no. 2, pp. 128–137, Dec. 2019, doi: 10.34128/jsi.v5i2.185.
Published
2024-05-23
How to Cite
Muhammad Irfan Syah, Nazruddin Safaat Harahap, Novriyanto, & Suwanto Sanjaya. (2024). PENERAPAN RETRIEVAL AUGEMENTED GENERATION MENGGUNAKAN LANGCHAIN DALAM PENGEMBANGAN SISTEM TANYA JAWAB HADIS BERBASIS WEB. ZONAsi: Jurnal Sistem Informasi, 6(2), 370 - 379. https://doi.org/10.31849/zn.v6i2.19940
Section
Articles
Copyright (c) 2024 ZONAsi: Jurnal Sistem Informasi
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Bahasa Indonesia
In English
