Decision Support System on Faculty Profiling Using Full-Text Search Algorithm: A Tool for Evaluating Faculty Performances

Information technology plays a vital role in various organizations. It lowers costs, improves service quality, and requires less time and effort. Management information system and decision support system are types of information which are considered the backbone of most of the organizations’ technologies [1]. Many organizations, including academic organizations, are converting their manual transactions into computerized working system in terms of record-keeping, cost reductions, and real-time data access to expedite and improve work efficiency, increasing employee productivity and decreasing manpower. In connection, one of the responsibilities of the Human Resource Department Office (HRDO) in the academic setting is to track faculty members’ relevant records, such as academic, trainings and development, and status for the re-hiring purposes. Additionally, providing real time feedback to the administrators from the HRDO is very important in the decision-making process.

Cavite State University is the province of Cavite's major university in the Philippines. The university occupies 72 hectares (180-acres) which is located in the municipality of Indang. It has 10 colleges and 12 satellite campuses. With around 1,200 faculty and staff, it provides products and services to more than 25,000 university-wide students. As the number of students increases over time, there is a strong need to hire new faculty members and re-evaluate the performances of the existing faculty members to support the university’s mission and goals. Evidently, if the individual performance is strong, there is a higher chance that the organization will perform outstandingly [2]. In connection, determining the qualification of the faculty members is vital [3]. Being the organization’s actors, it is a must that they possess necessary skills and expertise in attaining organizational goals and objectives. Hence, all these will be achieved if an effort to contribute is observed from these organization's actors.

The current practice of the university involves a manual collection, analysis, and organization of faculty profile from various colleges and campuses, more particularly for the purpose of re-hiring, tenureship, and promotion. The process starts by collecting personal data through the submission of three copies of Personal Data Sheet and Position Description Form. These forms provide personal data of the faculty, including personal information, educational background, eligibility, field of specialization, work experience, training programs attended, and status of appointment before the start of the semester. The collection process is tedious and time consuming. To assist the central HRDO in the collection process, human resource coordinators are designated in every campus. The next process is the collection of evaluation of the faculty performance, which includes class observation, student evaluation for the teachers (SET), and the Individual Performance Commitment and Review results. These data will be consolidated and analyzed manually by the authorized persons for the process of tenure-track.

For years, this traditional approach of faculty profiling worked for the central HRDO and in each campus. However, there is a lack of monitoring and tracking for each faculty profile, such as career progression, real time feedback and faculty performance analysis, skill set and experiences, appointments, promotions, and tenureship. Without centralized system in managing this process, the monitoring of faculty records and performances and providing efficient records and results will be compromised. As per Liu [4], a decentralized management pattern may result in information that is unclear and inconsistent. There is also no real time feedback in faculty performance analysis. Without organizational wide plan, organization may encounter problem, such as data duplication, data redundancy, poor data availability, poor data security, and error prone data. Hence, a faculty profiling system that provides a centralized repository, offers real time feedback, and delivers fast and accurate information is needed, supporting the management [5].

Standardized resource allocation, centralized information exchange, and quick decision-making and command are only a few benefits of the centralized management pattern [6]. Moreover, the foundation of technology-induced human resource management practice lies on the idea that employing information technology would impact the organizational hierarchy [7]. Therefore, it is essential to develop a system that manages all the faculty members’ data and monitors their performance in assisting management on efficient profiling and decision-making. With a good integration of a strong human resource management and information technology system, not only the functionality of the university will improve but the organizational culture as well.

Provided the established need, this research aims to design and implement an information system entitled “Decision Support System on Faculty Profiling” using full-text search algorithm which provides a centralized repository for the faculty profiling and monitors and assesses faculty records and performance for real time feedback, assisting the management in decision-making. This research also focuses on the function of faculty profiling which entails the process of maintaining, storing, and retrieving confidential information about faculty members within an Enterprise Resource Planning (ERP). The system further provides an information management system that allows the human resource coordinators to collect all the data of the faculty members online. In particular, the system has a portal which will be used by the faculty members to upload their necessary documents, such as PDS and PDF. Utilizing the portal, the faculty can update their profile for monitoring. Moreover, the system can generate printed and textual reports, including educational attainment, specialization, length of service, academic rank, seminars and training attended, and list of research involvements. It has a feature that consolidates, evaluates, and interprets faculty performance based on the standards and instruments of the university. Also, the system enables the human resource coordinators to upload evaluation results of faculty members for real-time feedback. Unlike the custom of one drive tool, the developed system provides decision support mechanism regarding faculty performance through the analysis of faculty performance for each program, measuring teaching effectiveness, peer, student, and supervisory evaluation. Given that all data from colleges and campuses can be viewed by the central HRDO, the system sets proper security control for accessing the data of each faculty member, ensuring that all data will be accessed only by an authorized user.

All in all, the “Decision Support System on Faculty Profiling” using full-text search algorithm is a system that will reduce the amount of time in collecting data and will condense the amount of time faculty members will spend on submitting data. As the system allows the administration to collect the data in a common format, tedious monitoring and evaluating process of faculty members’ records will be improved, further assisting the administrator in decision making regarding faculty management. These technologies have the potential to save time and resources, while improving the accuracy and efficiency of profiling.

Considering the systematic study of planning, developing, and assessing the developed system, this research is classified as developmental research. Relatively, input-process-output (IPO) is the conceptual model of the research and served as a guide in the development of the system shown in Figure 1.

Fundamentally, this model is divided into three major phases. The model started with the input phase wherein all necessary requirements, such as knowledge, software, and hardware, were gathered before the development of the system. Primarily, the knowledge requirements include the manual concepts, ideas, and process of faculty profiling in the entire university. These also include the collection, organization, and analysis of all faculty data for profiling process. Following this component is the software requirements which include programming languages. Particularly, these include Java Script and jQuery, for the scripting language; Bootstrap 3 and Cascading Style Sheet (CSS), for the sheet language; XAMPP, for the local web server; My Structured Query Language (MySQL), for the database; and Adobe Photoshop CS6, for the graphic editor. Lastly, the hardware requirement consists of the computer server.

1.png

Figure 1. Conceptual model

2.png

Figure 2. Waterfall model

Moreover, the development of the system is grounded in several key factors. It is a structured project, it has a clear stable requirement from the start and it is time bounded. Given these factors, the researchers used a waterfall model depicted in Figure 2. When a project must follow to rigorous guidelines, the waterfall technique is preferred due to its emphasis on deliverables at each step and its effectiveness when project goals are clearly understood from the beginning [20].

Moreover, it focuses on a systematic approach that progresses linearly through distinct, easily comprehensible, and explicable stages. Gathering and analyzing requirements, designing, testing, and implementing are the four stages of this process.

The project commenced with requirement gathering and analysis. This phase is considered the crucial part of the methodology. It has been observed that project managers continue to face significant obstacles due to the high failure rate of projects. However, this issue can only be resolved if they take into account appropriate risk management, effective interpersonal skills, and thorough project planning [21].

Upon identifying the process of profiling and role of the users, the client and the researchers discussed and determined the scope and constraint of the system until they come up with the agreement on how the system will be established. The researchers prepared a detailed document for the design phase that outlines the software architecture which includes the client/server-tier model, user interface, and system components. The researchers conducted interviews from the users, including the human resource coordinators, chairpersons, faculty members, and management information system officer of the university to identify the business process of faculty profiling. Observations regarding the manual process of the faculty profiling, such as providing recommendation for re-hiring, tenureship, and promotion process, were also conducted.

In connection, a context diagram of the business process of the manual faculty profiling presented in Figure 3 was also used. As displayed, the faculty will submit all the required documents to the human resource coordinator of each campus of which the collection, review, and verification take one to three days. If there will be corrections, the human resource coordinators will inform the concerned faculty to resubmit the documents. Once approved, all documents will be reproduced in three copies and it will then be submitted to the central HRDO for reviewing. If there will be corrections and clarification, the central HRDO will notify the campus human resource coordinator to inform the faculty for action, covering about one to three days of processing. Further, once approved, they will collect two copies of each document, and will then be stored in the filing cabinet per campus. One copy will be sent back to the respective campus. All approved documents will be used for faculty profiling, including evaluation and analysis, for future retention, re-hiring, and promotion.

3.png

Figure 3. Context diagram the manual process of faculty profiling

Moreover, the researchers prepared a use-case diagram shown in Figure 4, displaying different users and their roles and responsibilities to the system.

This approach allows the researchers to identify the access level of the different users for security purposes. Guided by the use-case diagram, the researchers designed first a normalized database and it was further used by the system. It was then followed by the faculty module, human resource and chairperson module, and the administrator module.

4.png

Figure 4. Use case diagram

In addition, the researchers used full-text search algorithm shown in Figure 5 to query extensive text data set stored electronically and to return summarized results which contain recommendations regarding individual performances of the faculty [22]. The flowchart has 9 stages, the first stage is the Start this where search process is initialized. The second stage is the Faculty Data, in this stage all information of the faculty members based on their PDS will be gathered to be used for the next stage which is the Preprocess Data. This stage prepares the collected data for further analysis including tokenizing, normalizing of the texts and removing irrelevant characters based on the search query. For efficient and faster retrieval, Build Inverted Index is the next stage it maps each specific words to a list of documents that contains words without scanning to every documents. When a query is received from the user the next stage which is Process Search Query will be responsible in analyzing the query to proceed to the next stage which is the Retrieve Matching. This stage will find documents based on the user’s query terms. The next stage is the Rank results, this stage will rank the documents based on the previous stage and based on the relevance of the documents which the user needed. Display Result is the stage which shows the ranked documents to the user and provides insights into faculty performances.

The process served as the guideline for development of the system. The data were analyzed to understand the scope and objectives to complete the development of the study including the successful implementation of the software. Furthermore, prior studies were benchmarked to support the development of the entire system, including theoretical concepts and architectural designs [23-31].

The testing phase was conducted once the development was completed. Using block box testing shown in Table 1, functionality testing was conducted. The researchers created a list of modules based on the use case diagram and tested the functionality of the modules to ensure that each component of the software is working properly in accordance with the set objectives.

In addition, the researchers utilized the F1-score, which captures both precision and recall, to evaluate the system performance. Table 2 shows evaluation metrics with four exhibits, each with two tests. Test no. 1 is an evaluation of one query, while test no. 2 is an evaluation of multiple queries. In addition, to get the precision, recall, and F1 score of each exhibit, the data sets were extracted from the system per query with 5 variables, such as Retrieved Documents (RTD), Relative Documents (RLD), Relative Retrieved Documents (True Positive) (TP), False Positive (FP), and False Negative (FN).

5.png

Figure 5. Flowchart of implementing full-text search algorithm in the development

Table 1. Block box testing

Table 2. Evaluation metrics

Table 3. Precision, recall and F1 results

Table 4. General interpretation of F1

In connection, Table 3 shows the score of precision, recall, and F1 of each exhibit per test based on the data sets. The interpretation was guided by the score range from 0 to 1, with 1.0 being the highest and 0.0 being the lowest shown in Table 4.

Furthermore, both tests in exhibits A and C got a score within the range of 0.7-0.8, while only test number 1 in exhibits B and D got a score within the range of 0.7-0.8, which implies that the system performance is good, balancing precision and recall effectively. In contrast, both test number 2 in exhibits B and D got a scores of 0.63 and 0.65, which implies that the system performance is fair and may need improvement depending on the case. Based on findings, the overall performance of the system is good, which means that it is acceptable for many use cases.

To support the findings, the researchers also utilized Google Chrome Devtools to measure the performance of the system shown in Figure 6. Largest Contentful Paint (LCP) measures the loading speed of the main content, while Interaction to Next Paint (INP) is the overall responsiveness during the interaction of the user with the system. LPC got 1.65 seconds, while the INP got 24 milliseconds; both scores indicate a good performance.

6.png

Figure 6. Google Chrome Devtools metric screen shot

In addition, the developed system was implemented and evaluated based on the ISO 25010. This tool is internationally recognized in verifying if a system is conforming to standards. Once completed, the system was deployed and implemented in one of the campuses of the university as a pilot area.

3.1 Research locale

The study was conducted at Cavite State University located in the province of Cavite. For the development of the system, unit heads, chairpersons, and faculty members of the various universities, and software development experts from private sectors were amongst the participants.

3.2 Research participants

Purposive sampling was employed to identify and choose the cases with the highest information content in order to maximize the utilization of the available resources. Finding people or groups of people who are informed and experienced about a particular issue is therefore necessary [32]. In addition, Stratton [33] stated that this sampling technique is a population sampling technique where researchers select participants based on their experiences, traits, or interest in a specific group, offering a cost-effective and time-efficient approach. In connection to the nature of the development of the system.

A total of 100 participants took part in the study, including 28 academic personnel, one human resource coordinator, one department chairperson in three universities in Cavite, and ten software engineers in private sectors with a minimum of 5 years in their field of specialization to gather their perspective as software specialist. The selection of participants was based on the shared experiences relative to the process of faculty profiling. Given their shared expertise and comprehension of the procedure, they were able to significantly contribute to the conversation, producing high caliber results.

3.3 Research instrument

In order to validate and evaluate the functionality of the developed system, the researchers use a standardized survey instrument from ISO 25010, which has the following criteria: functional suitability, performance efficiency, compatibility, usability, reliability, security, maintainability, and portability [34]. Functional suitability refers to the functions of the developed software based on the identified system objectives. Under specific settings, performance efficiency refers to how well a developed system performs in terms of the amount of data it uses. As the system is online-based, compatibility is considered to ensure that all users can utilize the system anywhere and anytime. Usability, on the other hand, refers to which developed software can be used successfully, efficiently, and acceptably by the users to attain specific goals in a specific context. Reliability is the extent to which a system operates as intended over a predetermined period of time under predetermined circumstances. Security refers to how well a product or system protects data and has appropriate access level for their types and levels of authorization. Maintainability is the ease with which a system or product can be updated to enhance, rectify, or adjust to changes in the environment or requirements. Lastly, a system, product, or component's portability refers to how well it can be moved from other environment to another.

3.4 Data gathering procedure

The purpose of this study was to gather quantitative data regarding the overall functionality of the system based on the set objectives. In collecting necessary data, a survey instrument was prepared. It was divided into three parts: (1) the introduction, which includes a short summary of the survey’s purpose; (2) the personal data of the participants, which includes their years in service and field of specialization; and (3) the statements for each criterion, which are based on ISO 25010. The participants were selected based on the similarities that they had about the process of faculty profiling and their technical expertise.

The gathered quantitative data of the study were analyzed in a descriptive approach. Frequency count, which is often a data representation that displays the percentage of observations for each data point or group of data points, was employed by the researchers. Moreover, the researchers interpreted the overall ratings of the system per criterion using mean averages shown in Table 5.

Table 5. Mean interpretation table

3.5 Ethical considerations

For ethical considerations, a letter of consent was provided which discussed the purpose of the research and assurance that all information will be kept confidential. Their identities and affiliation were not revealed as this research was guided by the Data Privacy Act.

The central HRDO and the human resource coordinators of the campuses are using the conventional way of faculty profiling. This approach of faculty profiling worked in the university system for years. However, there is a lack of monitoring and tracking for each faculty profile, especially in terms of career progression, real-time feedback, performance analysis, and faculty skill set and experiences. Findings from the evaluation were used to obtain the conclusion. In general, there is a declining management on faculty profiling within the organization. Some identified issues were decentralization, data redundancy, poor data availability, poor data security, and no real-time feedback and faculty performance analysis. Hence, a decision support system for faculty profiling is needed in this regard. Provided the study’s outcomes, the following conclusions relative to the developed system for each identified challenge are listed below:

Decentralization. Every semester, the requirements of the faculty members (e.g., PDS, PDF, SET, PES, PEER, and supervisory evaluations) should be submitted and updated in three copies: two copies will be filed in the central HRDO and the other copy will be filed in each campus or college where they will be designated. Data were duplicated and stored in several locations, leading to inconsistencies of data. King [36] pointed out that when IT systems and services are centralized, organizational policies and procedures are simpler to enforce, and the costs of duplicating overhead and facilities observed in decentralized IT systems and services may be avoided. Hence, the developed system provides a centralized repository for organizing all faculty profile in different departments, colleges, and campuses. It has been suggested that centralized IT services and systems have better ability in creating and managing company policies and processes. From the standpoint of project management, uniform client experiences and predictable service results are produced by consistent processes and activities [37].

Data Redundancy. As the university has various colleges and campuses, decentralization is evident of which duplication and redundancy or inconsistencies of data may occur in connection with the submission of faculty members. Generally, data redundancy refers to the practice of keeping data in two or more places and occurs due to decentralization. Updating of files and profile is the primary concern of this module. The system provides an information system that allows the human resource coordinator in collecting all the data of the faculty members online. It will also have a portal which will be used by the faculty members in uploading their records and updating their profile online, eliminating data redundancy while accurately updating the faculty profile.

Poor Data Availability. The system has a module that can generate graphical and textual reports. This module can consolidate data for different colleges and campuses within a short period of time, making relevant data available anytime and anywhere. In addition, graphical and textual reports can be generated based on the specific queries using full-text search algorithm.

Poor Data Security. Given that all the data of the faculty members are not organized and spread in various forms (PDF, PDS, evaluation forms, such as PEER, PES, SET, and supervisory) and locations, such as local and central HRMO, security and confidentiality are compromised. The system has proper security control for accessing the data of each faculty member so that all the data will be accessed only by an authorized user.

No Real Time Feedback and Faculty Performance Analysis. Due to the large amount of data of each faculty member to be monitored and interpreted alongside faculty skill set and career progression monitoring, real-time feedback to all faculty members from different campuses and colleges is jeopardized. The system will provide decision support system that will help the management for decision making. This module will maintain comprehensive faculty information, such as faculty skill set and experience and faculty details for career progression, and monitor research involvement. Likewise, it will help identify and analyze faculty performance to measure teaching effectiveness, peer evaluation, performance evaluation, and supervisory evaluation. The system generates evaluation reports that aid in the identification of gaps and flaws in faculty teaching approaches based on relative performance.

Overall, after extensive research and software development, the implementation of the system based on the results given by the users and experts can lead to the potential improvement of the current approach of faculty profiling. Provided acceptable findings, the developed system allows sharing of information between faculty and administrative staff within the university in real-time without making any data update. The system can be distributed to other campuses, allowing sharing of information and results of decisions over the Internet and further achieving transparency and data-driven decisions. Furthermore, our system supports the attainment of the Sustainable Development Goals—SDG 4 for Quality Education by improving transparency since the system provides information on faculty qualifications in terms of teaching experience based on the field of specialization and research experiences that will help the students to find the right mentors. In addition, the administrator can track the faculty development to identify gaps in skills or research areas to create faculty training and promote lifelong learning among faculty to ensure the quality of teaching and research. The system also supports SDG 9 for Industry, Innovation, and Infrastructure, which showcases faculty profiles specifically in their specializations and research, which can attract industry partners interested in innovation and eventually lead to possible industrial opportunities. It also supports digital infrastructure to manage the university’s resources more efficiently. Moreover, the system supports SDG 17 for Partnerships for the Goals by sharing faculty profiles, which promotes potential collaboration for education, research, and community extension services in various universities, industries, and local government units (LGUs).