Hugo Vega-Huerta
| Fernando Gutierrez-Mejía | Brayan Calcina-Aguilar* | Oscar Benito-Pacheco | Percy De-la-Cruz-VdV | Gisella Luisa Elena Maquen-Niño | Juan Carlos Lázaro-Guillermo
© 2025 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
This research investigates the modernization of virtual customer service within academic environments using a conversational AI system based on ChatGPT. This study introduces the first implementation of ChatGPT in a Peruvian public university’s admissions office, offering an innovative approach to automate and enhance user support. Educational organizations face the challenge of enhancing accessibility and delivering more efficient services, as traditional methods, such as telephone calls, in-person consultations and online support, often fail to meet user satisfaction expectations. This study proposes and develops a system designed to optimize online interaction and improve accessibility of virtual services, providing quick and accurate responses to academic and administrative queries. Leveraging advanced natural language processing technologies, the system enhances the user experience by reducing wait times and increasing satisfaction levels by 95%. The findings demonstrate its effectiveness in transforming the interaction between users and educational services, offering a scalable and adaptable solution to different educational contexts. This approach establishes a new benchmark for query management within educational institutions and lays the foundation for future applications of artificial intelligence in academic contexts.
chatbot, ChatGPT, conversational intelligence, online support, generative artificial intelligence
In the digital era, educational institutions are facing a growing demand to improve the accessibility and efficiency of their user services. The interaction between students and administrative services is a critical aspect that directly influences user perception and satisfaction. Traditionally, these services have relied on methods such as phone calls, website navigation, and in-person inquiries, which, although effective, do not generate ideal user satisfaction due to limitations in response time and accessibility.
Customer service in the admissions offices of state universities faces significant challenges, as demonstrated in a study conducted in Peru where it was evidenced that the quality of customer service in these offices is deficient, negatively affecting user satisfaction [1], likewise, according to the National Survey of University Higher Education Students 2019, 47.5% of university students consider that the information provided by universities on academic and administrative processes is insufficient, which generates uncertainty and affects the student experience [2], which evidences a problem with respect to this issue, the survey also indicates that 53.2% of the students who carried out procedures in public university admissions offices reported long waiting times and deficient attention, and 41.7% of the respondents mentioned having received contradictory information on different occasions, which evidences the need to improve communication and user service mechanisms within these institutions [2].
The emergence of advanced natural language processing technologies, such as ChatGPT, presents new opportunities to transform traditional processes and optimize interaction between users and educational services. A conversational intelligence system automates and personalizes responses to frequently asked questions, significantly improving user experience by reducing wait times and providing accurate information instantly.
This study focuses on modernizing virtual assistance in academic settings through the implementation of a conversational intelligence system based on ChatGPT. The main objective is to develop and evaluate a system that not only enhances the accessibility and efficiency of virtual services but also lays the groundwork for future artificial intelligence applications in education.
This chapter examines the key concepts and the evolution of language models in conversational intelligence.
2.1 Conversational intelligence
Conversational intelligence refers to the ability of systems to interact with users through natural language, facilitating real-time communication [3]. In virtual assistance, this technology enables automated responses to frequent inquiries, improving accessibility and reducing waiting times, particularly in educational and professional contexts [4].
Conversational intelligence models, such as those based on Generative Pre-trained Transformer (GPT) technology, have demonstrated effectiveness in generating coherent and accurate responses, making them valuable for enhancing assistance in academic settings [5]. This model’s ability to process large volumes of textual data has significantly contributed to the modernization of virtual assistance systems.
AI-powered conversational systems like ChatGPT enhance user experience by providing adaptive, real-time responses, particularly in academic environments where efficient and reliable support is crucial [6, 7].
2.2 Natural language processing (NLP)
Natural language processing (NLP) is essential in conversational intelligence, as it enables machines to understand, interpret, and generate human language. In virtual assistance, ChatGPT benefits from advanced NLP technologies that optimize user interaction by providing detailed and personalized responses [8].
Additionally, semantic analysis and lexical disambiguation techniques enhance the accuracy of responses in these systems. Tools like BERT have been crucial to ChatGPT's success in understanding and classifying texts [9]. This technology allows conversational systems to handle complex inquiries and respond with greater accuracy, significantly improving user interaction [10]. Furthermore, advances in NLP continue to increase the precision of these systems [11].
Natural language processing has expanded beyond simple text comprehension, impacting fields such as education, where it enables greater personalization, and healthcare, where it supports automated response systems to improve patient interaction [12]. This influence underscores NLP’s versatility in automating and tailoring responses across various environments, making it an asset in multiple applications [13].
ChatGPT was selected over traditional rule-based systems, retrieval-based chatbots, and earlier transformer models such as BERT or GPT-2, due to its enhanced capabilities in zero-shot learning, contextual coherence in extended dialogues, and fine-tuned alignment with human intent through reinforcement learning with human feedback (RLHF). These characteristics are particularly valuable in academic virtual support systems where users demand accurate, adaptable, and nuanced responses to complex queries across multiple domains.
2.3 Pretrained models and GPT-4
The GPT-4 model represents the latest evolution in pre-trained models, allowing for the generation of more coherent and relevant responses during extended interactions [14]. Its ability to handle complex inquiries makes it an essential tool for advancing conversational intelligence systems [15]. Compared to its predecessors, GPT-4 has significantly enhanced the capacity of conversational systems to deliver faster and more accurate responses [6].
The use of these advanced models has allowed conversational systems to overcome the limitations of traditional chatbots, providing more flexible and natural interactions. Nonetheless, challenges remain in ensuring the reliability of responses [16]. Additionally, effectively mitigating biases is still a concern that requires further research and solutions [17].
2.4 Integration into platforms and cloud deployment
The integration of language models into content management platforms optimizes virtual assistance in educational settings. Tools like Azure OpenAI Service have enabled quick and precise information retrieval, which enhances the user experience [18]. This improvement in accessibility and response times is especially valuable for academic institutions that rely on efficient virtual support systems [19].
The deployment of these models in the cloud offers significant advantages, such as scalability and flexibility. Azure OpenAI Service provides an efficient infrastructure that adapts to the fluctuating demands of the academic environment, allowing systems to maintain optimal performance even during high-demand periods [20]. This adaptability is crucial for supporting heavy usage in academic settings, which can vary widely based on student and staff needs [21]. The system also ensures legal compliance by adhering to both international and national data protection frameworks, including the General Data Protection Regulation (GDPR) and Peru’s Personal Data Protection Law (Ley N° 29733). These frameworks govern the collection, processing, and storage of personal data, and the system was designed in accordance with their principles to ensure lawful and ethical handling of user information.
The use of cloud technologies also facilitates integration with database systems and other resources, ensuring that systems can expand or be updated according to the needs of the educational environment [22].
To ensure compliance with data protection standards, the system implements strict anonymization techniques, including data masking, pseudonymization, and encrypted storage of personal data. All user queries are processed without storing identifiable information, aligning with the General Data Protection Regulation (GDPR) and local data privacy regulations applicable in Peru. Periodic audits, access controls, and privacy-by-design principles were also established to further guarantee data confidentiality and institutional trust.
2.5 Security and privacy challenges
Despite the numerous advantages of AI-based systems like ChatGPT, there are challenges related to data security and privacy in educational environments [23]. In contexts where users' personal information is sensitive, it is essential to implement robust security mechanisms, including encryption and data anonymization, to safeguard data integrity [24].
Data protection policies and periodic audits are fundamental to ensuring the reliability and security of cloud-based language models. These measures help mitigate risks associated with the use of AI in academic settings, thereby guaranteeing the privacy and accuracy of the information provided [25].
It is explored the use of ChatGPT in education, due to its popularity and ease of access, allowing students and educators to interact with a tool that simulates being a “virtual best friend” [26]. The objective of this research is to understand the perceptions and uses of ChatGPT in education. The results show that 85% of respondents are aware of this tool and 80% are interested in applying it in education, despite certain access barriers. The conclusion emphasizes that, although ChatGPT is a useful resource, its indiscriminate use could limit the development of essential critical thinking skills in academia
The potential applications of ChatGPT in engineering education have been extensively analyzed, demonstrating its significant value for enhancing learning outcomes [27]. Their goal is to evaluate how this technology can provide personalized learning experiences and assist students in hands-on simulation of concepts. Among the main results, the study found that ChatGPT can be useful, although limited by inherent biases in its programming and possible questionable ethical uses. He concludes that an adaptive approach is needed to integrate this technology into engineering education, to maximize the benefits and minimize the disadvantages.
A recent case study examined ChatGPT's impact on academic assessment in higher education, analyzing its ability to complete assignments at top-student levels and evade plagiarism detection [28]. The results reveal that ChatGPT can pass some academic integrity checks, bringing into focus the need to review current assessment practices. The conclusion indicates that education systems must adapt to manage the impact of AI tools on learning processes.
Recent research has explored the educational applications of AI tools like ChatGPT and New-Bing for Korean NLP tasks, assessing both their practical utility and reliability in academic contexts [29]. The results indicate that these chatbots can enhance educational interaction, and it is anticipated that future developments will include speech recognition, broadening their applicability in the educational context. The conclusion is that chatbots have potential in education, provided that language-specific applications are developed.
Recent studies within the MERLIN project have investigated chatbot acceptance in Malaysian higher education, particularly analyzing student perceptions of its effectiveness in supporting online learning [30]. The findings show that the chatbot improves understanding of course material and proposes a conceptual framework for the implementation of AI-based learning assistants. The conclusion suggests that chatbots can be an effective tool for online learning, provided that appropriate strategies are used for implementation.
A chatbot system was developed to efficiently manage virtual inquiries for graduate students at a Peruvian university, with research focusing on evaluating its performance and effectiveness in providing real-time responses to diverse student queries [31]. The results indicate that the chatbot effectively improves the efficiency and reliability of responses, for common queries related to admissions, course details and institutional policies. The authors conclude that implementing a chatbot for virtual care in educational settings can improve service quality and student satisfaction and suggest that similar chatbot solutions could benefit other educational institutions seeking to modernize their communication strategies and reduce staff workload.
4.1 System design and development
Design: The system design is structured around four main components: the ChatGPT language model, the knowledge base, the behavior prompt and the user interface.
Language Model (ChatGPT): A cognitive engine like ChatGPT was selected due to its advanced capabilities for processing natural language and generating contextually relevant responses. This model, utilized through Azure OpenAI Service, serves as the core of the system and was configured to understand and respond to specific questions within the academic context.
Knowledge Base: For the system to generate accurate responses, a well-structured knowledge base is essential. This base consists of academic documents, administrative policies, and other relevant information, stored in an Azure Storage Account and managed through Azure AI Search. The information was organized and structured to allow for quick and efficient access, optimizing the system's response time.
Behavior Prompt: This consists of a set of instructions and examples that guide ChatGPT on how to respond to different types of inquiries. The prompt was designed to maintain a professional and cordial tone, ensuring that responses are appropriate for the academic environment. Additionally, it includes specific rules on how to handle complex inquiries and prioritize the most relevant information in each interaction.
User Interface: The user interface was designed to be user-friendly. An intuitive web platform was created using React, allowing users to ask questions and receive real-time answers. The interface design considered the need for a smooth and friendly experience, making it highly intuitive and encouraging users to ask follow-up questions.
System Interaction Process: When the user submits an inquiry through the web interface, it is sent to the backend, where it is processed and the API provided by Azure OpenAI Service, trained with the ChatGPT model, is invoked. The resource analyzes the inquiry with the help of Azure AI Search, accesses the knowledge base stored in the Storage Account to retrieve relevant information, and generates an appropriate response. This response is sent back to the user through the interface, all within a matter of seconds. This flow was designed to be as efficient as possible, minimizing wait times and ensuring the accuracy of responses. Figure 1 illustrates the flow of an inquiry and how Azure resources interact.
Scalability and Adaptability: By utilizing Azure as the cloud provider, the system is both modular and scalable. This modularity allows for the incorporation of new functionalities or adaptations to different academic environments without requiring significant redesigns. At the same time, the scalability ensures that the system maintains optimal performance even during peak usage periods, such as applicant enrollment or exam times.
Cloud resources, such as processing capacity, storage, and access to the knowledge base, have been designed to be configurable and expandable. This means that as the institution's needs grow, the system can expand without service interruptions, ensuring operational continuity and consistent user experience. The flexible configuration of resources also allows the system to be adjusted for specific situations, such as admission campaigns or the launch of new academic programs, optimizing resource usage and minimizing unnecessary costs.
This modular and scalable approach, combined with a cloud architecture, ensures that the system not only meets current requirements but is also prepared to evolve alongside the institution's needs, adapting to future changes and expansions without compromising its efficiency.
Development and Implementation. To develop the system, Python was used as the programming language alongside the Flask framework for building the backend. The user interface was developed using React.
The ChatGPT language model was implemented using Azure OpenAI Service. This service provided access to an advanced natural language processing model trained with relevant information, ensuring that its responses are coherent and up to date. Table 1 describes the technological resources used for the chatbot implementation.
Figure 1. Azure resource architecture diagram
Table 1. Development environment configuration paper selection
|
Component |
Tool/Service |
Description |
|
Backend Language |
Python 3.11 |
Programming Language used for the backend. |
|
Backend Framework |
Flask |
Used for creating REST services (APIs). |
|
Frontend Framework |
React |
Used for creating the visual interface for the user. |
|
AZURE |
Azure OpenAI Service |
Resource used for Training the model based on ChatGPT. |
|
Azure IA Search |
Enables indexed search Within the knowledge base. |
|
|
Azure Storage Account |
Stores unstructured documents that will be part of the knowledge base. |
|
|
Azure App Service |
Resources used for hosting the application. |
As can be observed, the system was designed to integrate various technologies, enabling efficient and scalable implementation. The knowledge base, which includes academic and administrative information, was stored in an Azure Storage Account container and managed through Azure AI Search to optimize access speed and response accuracy.
The backend of the system communicates with the ChatGPT model through a RESTful API, which gives real-time responses generated by the model, facilitating seamless interaction with the system. Figure 2 illustrates the internal structure of the system.
Figure 2. Interaction between the technological components of the conversational system
4.2 Model training and optimization
The training and optimization process of the ChatGPT model is the most significant stage in the system's development. This process involved not only the configuration and adjustment of the model within Azure OpenAI Service but also the creation of a Behavior Prompt to guide the model's interaction with users.
The ChatGPT model was specifically configured to meet the needs of the educational environment. Utilizing Azure OpenAI Service, key parameters such as temperature, which controls the level of creativity in the responses, and top_p, which limits the selection of words to the most probable, were adjusted to ensure the coherence and relevance of responses.
To train the model, a knowledge base consisting of academic documents, administrative policies, and other relevant information was utilized. This information was previously normalized and structured. These documents were stored in Azure Storage Account and managed through Azure AI Search (indexed search), allowing the model to efficiently access accurate information. Table 2 presents the configuration parameters along with their current values.
Creation and optimization of behavior prompt. The Behavior Prompt is an important element for the proper functioning of the system, as it guides the ChatGPT model on how to respond to user queries. Specific instructions were designed, and examples of interactions were provided for the model to follow, ensuring that the responses were consistent with the professional and cordial tone required in an educational environment. Table 3 presents the directives for fine-tuning an effective behavior prompt.
Adjustments and optimization of training. Following the initial configuration and behavior prompt creation, the system underwent a testing phase to carry out necessary adjustments and optimizations. During this process, interaction data was collected to identify improvement areas within the model’s responses. This allowed for additional parameter fine-tuning and prompt refinement to address a broader range of inquiries.
Training is not a static process; a continuous feedback and optimization loop was established, in which the model’s responses are periodically evaluated to ensure relevance and accuracy within the academic context. This iterative approach ensures that the system remains aligned with user needs and adapts to any changes in institutional policies or procedures.
Table 2. Configuration parameters of the model
|
Parameter |
Value |
Description |
|
Temperature |
0.7 |
Controls the level of creativity in responses |
|
Top_p |
0.9 |
Limits the selection to the most probable words |
|
Maximum tokens |
500 |
Maximum number of tokens in each response |
Table 3. Directives for adjusting the system's behavior
|
Directive |
Description |
|
Youthful Tone |
The model should maintain a youthful and respectful tone as its target audience are students and general public. |
|
Information Priorization |
Responses should focus on delivering the most relevant and accurate information. |
|
Complex Inquiries Handling |
For complex inquiries, the model should provide a clear response and, if necessary, suggest consulting a specialist. |
|
Negative responses |
If unable to provide an answer, the model should inform the user, “I couldn’t find information related to your question. Can I help you with something else?” |
4.3 Integration with the OCA
The Central Admissions Office (OCA) website, developed in WordPress, began its integration process with a detailed analysis of user requirements, identifying specific needs for each target group: high school students, pre-university academies, parents, and the public.
This analysis helped establish the necessary features the chatbot should offer to optimize virtual support and deliver relevant information efficiently and accessibly.
The chatbot’s functional requirements included the ability to answer frequently asked questions, provide program details, and assist users with key admissions information. These functionalities were designed to reduce wait times and improve user satisfaction, reducing reliance on traditional support channels. Additionally, non-functional requirements regarding system performance were set to ensure that the chatbot could manage multiple simultaneous inquiries and deliver a reliable experience.
The technical integration of the chatbot with WordPress was achieved through the development of a custom plugin. This plugin allowed for the seamless incorporation of the chatbot into various sections of the website without altering the source code. It facilitated the configuration and customization of the chatbot, enabling the OCA team to adjust elements such as widget positioning, color schemes, and startup behavior without requiring advanced technical knowledge. This approach ensured a smooth integration that aligned with the visual and functional aspects of WordPress.
Integrating the chatbot into the OCA website significantly enhanced information accessibility and reduced response times, providing an efficient solution to user inquiries.
The implementation of the conversational intelligence system based on ChatGPT in the academic sense produced significant results, evidenced through the diverse metrics that reflect the improvements in efficiency and user satisfaction. In the following section, we’ll detail the main findings obtained through a system evaluation.
5.1 Decrease in response time
One of the most significant results was the reduction in the response times of the consultations realized by users. Before the system implementation, the average response time of a consultation through traditional methods (such as phone calls, in-person visits, or web search) ranged anywhere from several minutes to extremes of over 1 hour. With the conversational intelligence system, the average response time was reduced to less than 2 minutes in most cases, an 80+% reduction in response time. Table 4 demonstrates the response times, in minutes, comparing traditional methods and chatbot responses.
Table 4. Comparing response times: Traditional methods vs. Chatbot
|
Consultation Method |
Average Response Time |
|
Phone Call |
10-30 minutes |
|
In-Person Visit |
15-60 minutes |
|
Web Search |
5-20 minutes |
|
Chatbot |
<2 minutes |
For example, the 5-20 minutes range for website searches is due to variability in content structure, navigation between sections, and user digital experience. While frequent users access the site quickly, new users-such as parents or applicants face greater difficulty finding specific information.
5.2 Improvement in user satisfaction
The user satisfaction was evaluated through surveys carried out before and after the system implementation. The results showed a significant increase in general satisfaction, with a Net Promoter Score (NPS) that went from 25 to 70. Users highlighted the speed of the responses, the ease of use of the system, and the quality of the information given. Table 5 reflects the increase in user satisfaction after the implementation of the system.
In Table 6, you can also see the satisfaction levels for each type of user, as well as what stood out for each of them.
Table 5. NPS results
|
Category |
User Responses |
|
Promoters |
80% |
|
Passive |
10% |
|
Detractors |
10% |
Table 6. Satisfaction by user type
|
User Type |
Participation (%) |
Satisfaction Level |
Notes |
|
Students |
72% |
86% |
Highlighted the speed and availability |
|
Parents/Tutors |
18% |
75% |
Required clearer guidance for first-time use |
|
External Applicants |
10% |
78% |
Appreciated FAQ answers and structured responses |
Based on the results of the evaluation, we can calculate the final NPS.
$\begin{aligned} N P S= & \text { Promoters }(\%)-\text { Detractors }(\%) \\ & N P S=80 \%-10 \%=70 \%\end{aligned}$
With an NPS of 70%, we can prove the chatbot efficiency as its acceptance by the users that value its ease of us, speed, and information provided.
5.3 Accuracy of the responses
Another important indicator to be evaluated was the accuracy of the responses generated by the system. During the tests, there was an evaluation of the accuracy of the responses given by the chatbot. It was observed that 92% of the responses were accurate and relevant, which highlights the efficiency of the system in the interpretation of the consultations and its access to accurate information. Table 7 shows an analysis of the responses according to the Confusion Matrix.
Table 7. Evaluation of response accuracy
|
Metric |
Positive Prediction |
Negative Prediction |
|
Real Positive |
VP=20 |
FN=1 |
|
Real Negative |
FP=1 |
VN=3 |
The previous table describes that 20 of the responses were correct (VP), 1 response was given by the chatbot as correct but was incorrect (FP). The chatbot avoided responding to 3 questions that were outside of its reach (VN) and 1 question was responded to incorrectly.
$\begin{gathered}\text { Accuracy }=\frac{V P+V N}{V P+F P+V N+F N}=\frac{20+3}{20+1+3+1}=0.92 \\ \text { Sensibility }=\frac{V P}{V P+F N}=\frac{20}{20+1}=\frac{20}{21} \approx 0.952 \\ \text { Specificity }=\frac{V N}{V N+F P}=\frac{3}{3+1}=\frac{3}{4}=0.75\end{gathered}$
The chatbot demonstrated a high accuracy level, reaching 92% accuracy, which reflects how trusted it can be in its interactions with users. With a sensibility of 95.2%, the systems have demonstrated to be well trained to offer correct responses in adequate moments. Regardless, the specificity, which was of 75%, indicates that there are still areas of improvement in the recognition and filtering capabilities for irrelevant questions, or questions outside their scope, which represents an opportunity to optimize its response capacity.
We contrast our results with those of similar studies on the implementation and effects of ChatGPT-based virtual assistant systems in educational settings.
The implementation of our system shows a reduction in response time to student queries, with an average of less than 2 minutes, which is an improvement over traditional methods (e.g., telephone or in-person queries). These findings align with previous research demonstrating ChatGPT's capacity to deliver prompt responses and enhance accessibility in educational contexts [26]. Similarly, Research indicates that effective AI chatbot integration significantly enhances learning experiences and user satisfaction, evidenced by a notable increase in Net Promoter Score (NPS) from 25 to 70 post-implementation [30].
In addition to evaluating performance and user satisfaction, the scalability and cost-effectiveness of the system were also assessed, particularly considering high-demand periods such as admission campaigns. A comparison between cloud deployment (Azure) and an on-premise infrastructure was conducted to evaluate operational feasibility.
The analysis demonstrates that deploying the system in Azure offers clear advantages in terms of scalability, maintenance, and flexibility, especially during periods of peak demand. Although on-premise solutions could be considered for long-term fixed costs, the operational risks and scalability limitations make the cloud-based approach more viable for dynamic academic environments. In Table 8, we can see a comparison between the costs of implementing the solution in a local format and in the cloud.
Table 8. Azure vs. on-premise deployment comparison
|
Criterion |
Azure (Cloud) |
On-Premise |
|
Initial Cost |
Low (pay-per-use) |
High (hardware + IT setup) |
|
Maintenance |
Managed by provider |
Institution responsibility |
|
Scalability |
Automatic |
Limited by local resources |
|
High-demand Handling |
Elastic (tested with>200 users) |
Risk of overload |
|
Deployment Speed |
Rapid (1–2 weeks) |
Slow (1–2 months) |
|
Updates & Security |
Continuous+certified |
Requires internal management |
The system's 92% accuracy rate demonstrates strong reliability, consistent with prior research validating ChatGPT's effectiveness in handling academic tasks and providing precise responses to routine inquiries [28]. These findings further support its potential for academic support applications. However, our 75% specificity rate indicates that the system tends to provide broad answers to ambiguous queries, prioritizing information coverage. This leads to false positives on questions that require a high level of semantic accuracy. This deficiency suggests that the system may exceed its answer threshold when queries are ambiguous, leading to more false positives in certain contexts. The goal of this process is to constantly improve it.
These findings align with existing research emphasizing the need for ethical implementation frameworks when deploying AI systems in education, particularly regarding response accuracy, adaptive learning approaches, and robust data security protocols for institutional adoption [27].
The implementation of a ChatGPT-based conversational AI system in the university’s admission office has proven highly effective in optimizing virtual support. The system has achieved improvements is response time, reducing average wait times to under two minutes, and 80% reduction compared to traditional methods. This decrease in response time combined with the system’s 24/7 availability and its capacity to accurately address complex queries has elevated user satisfaction by 95% as evidenced by an NPS increase from 25 to 70. Additionally, the system demonstrated a 92% accuracy rate in responses, and sensibility (95.2%), which confirms that it is well-trained to respond to academic questions in an efficient and correct manner. Despite this, the specificity of 0.75 demonstrates that there are still areas of improvement regarding the chatbot’s capabilities for identifying and discarding questions outside of its scope. By leveraging advanced natural language processing and cloud deployment, the system is designed to be modular and scalable, which allows it to adapt to different academic contexts and evolve according to the future necessities of an institution, which is a key advantage for its application in the long term.
Hugo Vega-Huerta carried out the conceptualization and implementation of the methodology. Fernando Gutierrez-Mejía, Brayan Calcina-Aguilar wrote the original draft and carried out formal analysis and implementation of the methodology, Oscar Benito-Pacheco, Percy De-la-Cruz-VdV carried out the evaluation of the model and visualization, Gisella Luisa Elena Maquen-Niño, Roxana Ramos-Carrión carried out the data selection and supervised the compliance with the methodology, and Juan Carlos Lázaro-Guillermo conducted the validation of the assessment and drafted the results. All authors conducted the review of observations and approved the final draft.
[1] Ciencia Latina Internacional. (2021). Balanced scorecard para mejorar la calidad de servicio de atención al cliente en la oficina de admisión de las universidades del Perú. https://biblioteca.ciencialatina.org/balanced-scorecard-para-mejorar-la-calidad-de-servicio-de-atencion-al-cliente-en-la-oficina-de-administracion-de-las-universidades-del-peru/.
[2] Dirección General de Educación Superior Universitaria Perú. (2019). Encuesta Nacional de estudiantes de Educación Superior Universitaria 2019. https://repositorio.minedu.gob.pe/handle/20.500.12799/7745.
[3] Varela-Tapia, E.A., Sangacha-Tapia, L.M., Acosta-Guzmán, I.L., Celi-Párraga, R.J. (2022). Inteligencia artificial conversacional para el diseño de un asistente virtual interactivo. In Décima Segunda Conferencia Iberoamericana de Complejidad, Informática y Cibernética, pp. 237-241. https://doi.org/10.54808/CICIC2022.01.237
[4] Carlsson, K., Jacobs, I., Hong, D., Colin, H., Bakalar, M. (2021) Now Tech: Conversational Intelligence, Q2 2021 Forrester. https://www.forrester.com/report/now-tech-conversational-intelligence-q2-2021/RES161799.
[5] Molina, J.A., Cordero, M.Q. (2023). ChatGPT: La creación automática de contenidos con Inteligencia Artificial y su impacto en la comunicación académica y educativa. Desiderata, (22): 136-142.
[6] Limna, P., Kraiwanit, T., Jangjarat, K., Klayklung, P., Chocksathaporn, P. (2023). The use of ChatGPT in the digital era: Perspectives on chatbot implementation. Journal of Applied Learning and Teaching, 6(1): 64-74. https://doi.org/10.37074/jalt.2023.6.1.32
[7] Morales-Chan, M., Amado-Salvatierra, H.R., Medina, J.A., Barchino, R., Hernández-Rizzardini, R., Teixeira, A.M. (2024). Personalized feedback in massive open online courses: Harnessing the power of LangChain and OpenAI API. Electronics, 13(10): 1960. https://doi.org/10.3390/electronics13101960
[8] Torres, M.M.E., Manjarrés-Betancur, R. (2020). Asistente virtual académico utilizando tecnologías cognitivas de procesamiento de lenguaje natural. Revista Politécnica, 16(31): 85-96. https://doi.org/10.33571/rpolitec.v16n31a7
[9] García-Díaz, J.A., Almela, Á., Alcaraz-Mármol, G., Valencia-García, R. (2020). UMUCorpusClassifier: Compilation and evaluation of linguistic corpus for Natural Language Processing tasks. Procesamiento Del Lenguaje Natural, 65: 139-142. https://doi.org/10.26342/2020-65-22
[10] Tlili, A., Shehata, B., Adarkwah, M.A., Bozkurt, A., Hickey, D.T., Huang, R., Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1): 15. https://doi.org/10.1186/s40561-023-00237-x
[11] Khurana, D., Koli, A., Khatter, K., Singh, S. (2023). Natural language processing: State of the art, current trends and challenges. Multimedia Tools and Applications, 82(3): 3713-3744. https://doi.org/10.1007/s11042-022-13428-4
[12] Suárez, A., Jiménez, J., de Pedro, M.L., Andreu-Vázquez, C., García, V.D.F., Sánchez, M.G., Freire, Y. (2024). Beyond the scalpel: Assessing ChatGPT's potential as an auxiliary intelligent virtual assistant in oral surgery. Computational and Structural Biotechnology Journal, 24: 46-52. https://doi.org/10.1016/j.csbj.2023.11.058
[13] Gill, S.S., Xu, M., Patros, P., Wu, H., et al. (2024). Transformative effects of ChatGPT on modern education: Emerging era of AI chatbots. Internet of Things and Cyber-Physical Systems, 4: 19-23. https://doi.org/10.1016/j.iotcps.2023.06.002
[14] Aktay, S., Gök, S., Uzunoğlu, D. (2023) ChatGPT in education. Türk Akad Yayınlar Derg, 7: 378-406. https://doi.org/10.29329/tayjournal.2023.543.03
[15] Hariri, W. (2023). Unlocking the potential of ChatGPT: A comprehensive exploration of its applications, advantages, limitations, and future directions in natural language processing. arXiv Preprint arXiv: 2304.02017. https://doi.org/10.48550/arXiv.2304.02017
[16] Ruiz-Rojas, L.I., Acosta-Vargas, P., De-Moreta-Llovet, J., Gonzalez-Rodriguez, M. (2023). Empowering education with generative artificial intelligence tools: Approach with an instructional design matrix. Sustainability, 15(15): 11524. https://doi.org/10.3390/su151511524
[17] Cerruti, C., Valeri, A. (2022). AI-powered platforms: Automated transactions in digital marketplaces. Ph.D. Dissertation, Università degli Studi di Roma" Tor Vergata" Department of Management and Law.
[18] Abdulridha, M.A., Mohamed, A.H.J. (2024) Virtual agent (chatbot) using open artificial intelligence. Magazine of Student Research. https://www.researchgate.net/publication/379507857_Virtual_Agent_Chatbot_using_Open_Artificial_Intelligence.
[19] Scanlon, M., Breitinger, F., Hargreaves, C., Hilgert, J.N., Sheppard, J. (2023). ChatGPT for digital forensic investigation: The good, the bad, and the unknown. Forensic Science International: Digital Investigation, 46: 301609. https://doi.org/10.1016/j.fsidi.2023.301609
[20] Ots, K. (2024). Hardening Azure OpenAI service for enterprise usage. Metropolia University of Applied Sciences.
[21] Cascella, M., Montomoli, J., Bellini, V., Bignami, E. (2023). Evaluating the feasibility of ChatGPT in healthcare: An analysis of multiple clinical and research scenarios. Journal of Medical Systems, 47(1): 33. https://doi.org/10.1007/s10916-023-01925-4
[22] Srivatsa, K.G. (2024). Leveraging large language models for generating infrastructure as code: Open and closed source models and approaches. Doctoral Dissertation, International Institute of Information Technology Hyderabad.
[23] Meskó, B., Topol, E.J. (2023). The imperative for regulatory oversight of large language models (or generative AI) in healthcare. NPJ Digital Medicine, 6(1): 120. https://doi.org/10.1038/s41746-023-00873-0
[24] Taecharungroj, V. (2023). “What can ChatGPT do?” Analyzing early reactions to the innovative AI chatbot on Twitter. Big Data and Cognitive Computing, 7(1): 35. https://doi.org/10.3390/bdcc7010035
[25] Chan, C.K.Y. (2023). A comprehensive AI policy education framework for university teaching and learning. International Journal of Educational Technology in Higher Education, 20(1): 38. https://doi.org/10.1186/s41239-023-00408-3
[26] Fazal, N., Qasmieh, S., Graefen, B. (2023). Trick or treat-use of ChatGPT in education. The Journal of Immunology, 210(Supplement_1): 232-01. https://doi.org/10.4049/jimmunol.210.supp.232.01
[27] Qadir, J. (2023). Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education. In 2023 IEEE Global Engineering Education Conference (EDUCON), Kuwait, Kuwait, pp. 1-9. https://doi.org/10.1109/EDUCON54358.2023.10125121
[28] Chaudhry, I.S., Sarwary, S.A.M., El Refae, G.A., Chabchoub, H. (2023). Time to revisit existing student’s performance evaluation approach in higher education sector in a new era of ChatGPT-A case study. Cogent Education, 10(1): 2210461. https://doi.org/10.1080/2331186X.2023.2210461
[29] Wang, G. (2023). Natural language analysis of Korean texts of AI-based chatbots and exploration of Korean education utilization: Focusing on ChatGPT and New-Bing. Culture & Convergence, 45(5): 01-17. https://doi.org/10.33645/cnc.2023.05.45.05.01
[30] Neo, M. (2022). The MERLIN project: Malaysian students’ acceptance of an AI chatbot in their learning process. Turkish Online Journal of Distance Education, 23(3): 31-48. https://doi.org/10.17718/tojde.1137122
[31] Sánchez-Tello, J., Vega-Huerta, H., De-La-Cruz-VdeV, P., Maquen-Niño, G., Melgarejo-Solis, R., Cámara-Figueroa, A., Cancho-Rodriguez, E. (2023). Implementation of a chatbot for virtual attention of queries case: Postgraduate School of the UNMSM. Laccei, 1(8): 1-7. https://doi.org/10.18687/LACCEI2023.1.1.787
