Ontology-Driven Semantic Information System for Adaptive Assessment in Kazakh-Language Computer Science Education: A Case Study

A structured scientometric and literature review was conducted to identify recent research on ontology-driven information systems, semantic technologies, adaptive architectures, and low-resource NLP. The search covered Scopus-indexed publications from 2004 to 2025 and followed a PRISMA-based procedure for identification, screening, eligibility assessment, and inclusion.

The PRISMA procedure provided a transparent and reproducible method for selecting literature relevant to the research problem; it did not form part of the system development or experimental methodology. This review identified the research gap and positioned the proposed semantic assessment system within the existing literature. Figure 1 presents the study selection process.

tu_pian_1.png

Figure 1. PRISMA-based flow diagram of literature identification and selection for the related-work analysis

The search yielded 342 publications. After duplicates were removed and titles, abstracts, and full texts were screened for eligibility, 30 studies were included in the scientometric and methodological analyses. The selected studies were analyzed using scientometric and bibliometric visualization methods to identify thematic trends, the development of the field, and relationships among research topics. Figure 2 shows the principal characteristics of the Scopus dataset.

tu_pian_2.png

Figure 2. Main bibliometric characteristics of the Scopus dataset on ontology-driven semantic information systems

The analysis indicated increasing research attention on ontology-driven information systems, adaptive semantic architectures, artificial intelligence, and low-resource NLP. It also identified active work in ontology engineering, semantic similarity, intelligent interaction, and adaptive semantic technologies. These findings establish the relevance of ontology-based semantic processing and provide the basis for the methodological and architectural gaps addressed in this study.

2.1 Ontology-based information systems

Ontology-based information systems support semantic knowledge representation, information integration, intelligent retrieval, and contextual processing in complex digital environments [7-10]. Ontology engineering formally represents domain concepts, semantic relations, hierarchical dependencies, and other types of knowledge.

Established resources, such as the National Cancer Institute Thesaurus and AGROVOC, demonstrate the value of structured semantic vocabularies. Ontology engineering environments, including Computer-aided Ontology Development Architecture (CODA) and Semantic Turkey, support knowledge acquisition and ontology management in diverse domains [11-15].

Previous studies have shown that ontology-driven architectures can improve semantic consistency, contextual interpretation, and interoperability among heterogeneous information resources. Applications include semantic web systems, knowledge-based question answering, educational information management, and adaptive knowledge-processing environments [16-21].

Recent research has emphasized ontology matching and semantic reasoning for intelligent retrieval and automated processing. However, many systems focus on static knowledge representation and organization rather than adaptive interaction or interpretation of user-generated responses. Few studies have examined ontology-driven processing in low-resource languages or multilingual educational systems. Therefore, the integration of low-resource NLP with ontology-based adaptive assessment remains under-explored.

2.2 Semantic similarity and knowledge representation

Semantic similarity analysis supports the contextual interpretation of textual and conceptual information. Keyword matching and lexical similarity can be effective when expressions overlap directly, but they may not recognize conceptually related entities when exact lexical correspondence is absent. Fuzzy semantic analysis and ontology-matching methods allow a more flexible comparison of related information structures [7, 8, 22, 23]. They can also support assessments when lexical correspondence is incomplete.

Ontology- and semantic-graph-based representations explicitly model the conceptual dependencies, hierarchical relationships, and contextual associations. These structures support interpretable semantic retrieval and adaptive information processing. Nevertheless, many semantic evaluation systems provide limited contextual adaptability in multilingual low-resource settings. Ontology-based assessments of Kazakh language responses remain insufficiently studied.

2.3 Natural language processing for low-resource languages

NLP for low-resource languages is an important area of multilingual information systems research. Most advanced models have been developed for high-resource languages, whereas low-resource settings often lack lexical resources, annotated datasets, and language-specific processing tools.

Multilingual methods, cross-lingual transfer learning, and ontology-driven language processing are relevant to low-resource NLP [3-6, 24-26]. However, ontology-based contextual interpretation in agglutinative languages such as Kazakh remains under-explored.

Kazakh presents additional processing challenges owing to its agglutinative morphology, productive suffixation, syntactic variability, and context-dependent interpretation. These properties complicate concept extraction, semantic retrieval, and ontology mapping. Reliable Kazakh-language semantic assessment, therefore, requires morphological analysis, syntactic processing, and domain-based semantic interpretation before ontology mapping.

2.4 Adaptive semantic information systems

Adaptive semantic information systems integrate ontology-based processing, recommendation mechanisms, contextual interaction models, and knowledge management (KM). These components can improve semantic interoperability, feedback generation, and adaptive interactions [17-21, 27-32].

Previous studies have reported improvements in contextual retrieval, semantic consistency, personalized support, and interaction management. Ontology-driven systems also make conceptual dependencies and assessment decisions traceable. However, systems that rely primarily on keyword retrieval or fixed answer rules still provide limited contextual interpretations.

The proposed system integrates functions that prior approaches have often treated separately. Kazakh-language responses were linguistically pre-processed, mapped to concepts and relations in a domain ontology, converted into ontology-based response structures, and compared with reference structures using weighted similarity and fuzzy thresholds. The resulting assessment provided adaptive feedback and recommendations.

Ontology mapping was, therefore, an intermediate operation rather than the final objective. The contribution lies in combining ontology mapping, concept-level comparison, fuzzy classification, and adaptive feedback within a domain-specific assessment workflow that does not replace general-purpose ontology-alignment methods.

4.1 Ontology structure and graph statistics

The developed ontology comprises grade-specific OWL modules for computer science in grades 5–11. It represented domain concepts, semantic relations, hierarchical and prerequisite structures, glossary resources, and multilingual lexical units derived from curriculum materials. For structural evaluation, these modules were transformed into the normalized merged graph defined in Section 3.3.

After class identifiers were normalized and identical classes across grade-specific modules were consolidated, the graph contained 1,280 unique named OWL class nodes and 1,339 unique directed structural edges. Its density was approximately 0.0818%. The mean in-degree and out-degree were each approximately 1.046, giving a mean total degree of approximately 2.092. These values characterize the implemented ontology as a sparse, predominantly hierarchical graph.

This structure provides the domain representation used to interpret learner responses in the proposed intelligent information system.

4.2 Semantic matching performance

The semantic assessment mechanism was evaluated using 300 learner-generated responses and 1,281 manually annotated correspondence pairs. The analysis considered correspondence with the reference ontology, mapping consistency, contextual interpretation, and stability of the assessment output. Table 1 summarises the four correspondence outcomes used to calculate the classification metrics.

Table 1. Ontology correspondence classification results

Table 1 shows 531 TPs, 555 TNs, 94 FPs, and 101 FNs. The resulting accuracy was (531 + 555) / 1,281 = 0.8478, rounded to 0.848. The evaluation metrics were calculated as follows:

Accuracy $=\frac{(T P+T N)}{(T P+F P+F N+T N)}=\frac{(531+555)}{1281} \approx 0.85$

Precision $=T P /(T P+F P)=531 /(531+94)=0.85$

Recall $=T P /(T P+F N)=531 /(531+101)=0.84$

$F 1-\text {score}=2 \times(\text {Precision} \text {×} \text {Recall}) /(\text {Precision}+ \text {Recall})=0.85$

The 94 FPs and 101 FNs indicated limitations at several stages of the pipeline. FPs may occur when individually relevant concepts produce a high aggregated score, even if the complete relational meaning does not match the reference. FNs may occur when paraphrases, lexical variants, or Kazakh-language constructions are not captured during concept extraction or ontological mapping. Manually assigned weights and fixed fuzzy thresholds may also affect borderline cases.

5.png

Figure 5. Distribution of user-generated responses by the number of ontology concept mappings per response

These observations identified concept extraction, ontology coverage, relational representation, and decision thresholds as plausible error sources. Because the original evaluation did not use a predefined error taxonomy or retain category labels for individual errors, their frequencies could not be quantified retrospectively. Figure 5 presents the distribution of the mapped ontology concepts across the 300 responses. Overall, the system identifies contextual correspondences that are not limited to direct lexical overlaps.

4.3 Confidence interval analysis

A 95% confidence interval was calculated for semantic-matching accuracy using the normal approximation (Wald interval) for a binomial proportion.

With 1,086 correctly classified ontology correspondence pairs out of 1,281 observations $\hat{\rho}=0.8478$, the interval was calculated as $\hat{\rho} \pm \frac{1.96 \sqrt{\hat{\rho}}(1-\widehat{\rho})}{n}$, to yield a $95 \%$ confidence interval of approximately $[0.83,0.87]$.

The resulting interval indicates the statistical uncertainty associated with the estimated semantic-matching accuracy within the evaluated dataset.

4.4 Example of ontology-based semantic matching

The following example illustrates an ontology-based assessment procedure: The semantic question was: ‘What is information coding?’

The learner response was: “Coding means transforming information into symbols for storage and transmission.”

After Kazakh-language preprocessing and ontology mapping, the system extracts the concepts of information, coding, transformation, storage, and transmission. The reference structure contains information, coding, representation, storage, and communication.

The system matched ‘transformation’ with ‘representation’ and ‘transmission’ with ‘communication’. The resulting score was S(Oᵣ, Oₖ) = 0.89, and the responses were classified as follows.

1. Semantic assessment label: Correct.
2. Semantic correspondence level: High.

This example demonstrates how ontology relations support correct assessment when a learner response differs lexically from the reference representation.

4.5 Comparative evaluation

The proposed system was compared with keyword-based lexical matching and rule-based assessment based on predefined answer patterns. All methods were evaluated using the same response and correspondence pairs.

Table 2 shows that the ontology-driven system achieved higher accuracy, precision, recall, and F1-score than both evaluated baselines. This comparison was limited to the selected lightweight baselines and did not establish superiority over the embedding-based methods.

Table 2. Comparative evaluation of assessment approaches

The largest improvement occurred for conceptually related responses without direct lexical overlap, which is consistent with the intended function of ontology-based matching.

4.6 Adaptive semantic interaction results

The adaptive module generates feedback and recommendations based on the assessment results and the learner’s interaction history. Recommendations targeted ontology concepts that were absent or weakly represented in the responses.

Three domain experts assessed the recommendation relevance using three criteria: correspondence with the identified weak or missing concept, consistency with the assessment result, and usefulness in directing the learner to the appropriate reference knowledge. A recommendation was labelled relevant when it met at least two criteria. Table 3 summarizes the results.

Table 3. Adaptive recommendation performance

The experts classified 256 of the 300 recommendations as relevant with an accuracy of 0.85. Thus, most generated recommendations were consistent with expert judgements under the stated criteria. These findings indicate that ontology-based assessment can support context-aware recommendations and personalized feedback within the evaluated domain.

4.7 System effectiveness and interpretation

The experimental results suggest that ontology-driven semantic technologies can improve contextual semantic interactions and the interpretation of user-generated responses within intelligent semantic information processing systems. Within the evaluated prototype, the system

1. Performed ontology-based semantic assessment.
2. Identified conceptual relations between learners and reference representations.
3. Generated adaptive feedback and recommendations.
4. Processed Kazakh language learner responses.
5. Provided context-aware support in a low-resource language setting.

The higher performance relative to the selected baselines indicates that explicit ontology relations contributed information beyond the keyword overlap. The following subsections interpret these findings in relation to computational complexity, scalability, ontology maintenance, methodological limitations, and alternative approaches to semantic processing.

4.8 Computational implications of ontology-driven semantic processing

The results indicate that explicit ontology structures can improve correspondence identification when learners and reference responses use different lexical forms. This interpretation is restricted to the evaluated dataset for computer science in Kazakhs.

This architecture separates linguistic pre-processing, ontology mapping, similarity calculation, classification, and feedback generation. This modular structure makes each processing decision traceable to recognized concepts, relationships, weights, and thresholds.

The ontology also explicitly represents hierarchical and prerequisite relations, which supports interpretable feedback and controlled knowledge management. No runtime benchmark was conducted; the findings concern semantic and functional performance rather than computational efficiency.

The case study provides a basis for testing comparable symbolic or hybrid architectures in other low-resource language and educational settings.

4.9 System scalability and architectural considerations

The multilayer architecture separates client interaction, linguistic and semantic processing, recommendation, and ontology-based storage. This modular organization permits individual components to be extended or scaled independently.

Separating preprocessing, ontology mapping, evaluation, and knowledge management also simplifies maintenance and integration with different computing environments.

Because the ontology and knowledge base are independent of the client interface, the architecture can support different devices, external semantic platforms, and cloud-based services.

Nevertheless, scalability depends on the ontology size, relation density, and interaction volume. An increase in these factors may increase the processing latency, retrieval complexity, and graph-traversal overhead.

Semantic indexing, consistency validation, and repeated similarity calculations over evolving ontologies may incur additional computational costs.

Let n be the number of concepts extracted from a response and m the number of candidate reference concepts. Direct pairwise comparison has the worst-case complexity O(n, m), whereas weighted aggregation is linear in the number of evaluated concept pairs. The actual cost depends on indexing, candidate selection, relation density, and reference-set size; this bound does not represent the measured end-to-end performance.

The runtime performance was not benchmarked because the experiment focused on correspondence identification. End-to-end latency, throughput, concurrent user capacity, processor and memory utilization, and network overhead were therefore not measured. The implementation should be regarded as a functional research prototype rather than a production-ready deployment.

Future deployment studies should measure the response time, ontology-mapping latency, matching time, throughput, resource utilization, and performance as ontology size and concurrent loads increase. Graph databases, semantic indexes, ontology caching, and parallel similarity calculations should also be evaluated as optimization strategies.

4.10 Ontology maintenance and knowledge base evolution

Ontology maintenance is a central limitation of ontology-driven systems. Domain models must be updated when curricula, terminology, knowledge, or prerequisite relationships change.

The prototype did not implement ontology versioning or lifecycle management functions. Construction and refinement were performed manually without automated change tracking, version comparison, rollback, or lifecycle states. Therefore, maintenance and controlled evolution are requirements for future deployment rather than demonstrated capabilities.

As ontology grows, preserving semantic integrity and preventing inconsistent mappings will require increasingly systematic validation.

Extending the ontology beyond computer science or the grades 5–11 curriculum would require new concept modelling, relation verification, multilingual adaptation, and domain-specific knowledge engineering.

Incomplete mapping, inconsistent hierarchies, and misaligned dependencies may reduce the correspondence accuracy and quality of adaptive feedback.

Semi-automated refinement and alignment may reduce the burden on domain experts, whereas automated consistency checks can support reliable knowledge base evolution.

Future implementations should record explicit version identifiers and change logs, validate the consistency after each update, and provide controlled comparisons and rollback between ontology versions.

4.11 Limitations of rule-based and fuzzy semantic approaches

Several methodological limitations qualify the reported performance of the rule- and ontology-based assessment procedures.

The evaluation used 300 learner responses and 150 question–response pairs from one educational domain. Although these data produced 1,281 expert-annotated correspondence pairs, the corpus is too limited to establish generalization across subjects, learner populations, age groups, institutions, or languages. Because the method did not learn parameters from the corpus, conventional training–test splitting and k-fold cross-validation were not applied. The findings should therefore be interpreted as case study evidence of feasibility.

Fuzzy assessment depends on manually specified concept weights and predefined thresholds. Therefore, the performance may vary with ontology design, weighting decisions, and calibration procedures.

No sensitivity analysis examined alternative values for the 0.50 and 0.80 thresholds. Therefore, the robustness of the classifications to these boundaries is unknown. Before transferring to another domain, the thresholds should be recalibrated using independent domain-specific validation data.

Symbolic ontology representations provide interpretable relations and transparent decision paths; however, they may represent ambiguous or implicit meanings less effectively. Ontology construction, relation definition, and validation require domain expertise, and the associated maintenance effort increases with system size and complexity.

This method does not use probabilistic representations or neural embeddings. Therefore, its performance depends on the preprocessing accuracy, ontology completeness, and relation coverage. This design favors traceability but may reduce flexibility in dynamic or linguistically ambiguous settings.

Hybrid models that combine ontology engineering with semantic embedding and transformer-based processing should be investigated to improve adaptability while retaining interpretability.

Embedding-based baselines were not included in the quantitative evaluation. Therefore, the results do not establish whether the ontology-driven system outperforms multilingual contextual or subword-based models. Future studies should use the same responses, reference labels, and evaluation metrics.

The manually defined concept weights were not optimized on an independent validation set. Therefore, subjectivity in the weighting scheme cannot be excluded. Sensitivity analysis, structured expert consensus, and data-driven weighting methods should be evaluated.

Inter-annotator agreement was not quantified, and individual expert labels were discarded. Future studies should use a predefined annotation protocol, preserve individual judgements, report agreement coefficients, and document how disagreements are resolved.

4.12 Comparison with knowledge graph, embedding, and large language models-based approaches

This subsection provides a conceptual comparison rather than an additional quantitative benchmark. Embedding-based and large language model (LLM) baselines were excluded from the experiment, so this study makes no claims of empirical superiority over these approaches.

Knowledge graphs, semantic embeddings, transformer architectures, and LLMs are increasingly being used for semantic retrieval, contextual processing, and educational interactions. Recent reviews document both the opportunities and limitations of LLMs in education [33-35].

Each approach differs in terms of requirements for data, computation, interpretability, and domain control. These differences provide the basis for positioning the proposed symbolic method within current semantic information system research.

Compared with embedding-based representations, ontologies express symbolic relations directly. The proposed method records the domain hierarchies, prerequisites, and correspondence decisions explicitly, which supports traceable assessment.

Knowledge graphs provide flexible representations and large-scale retrievals across heterogeneous resources. However, their construction may require extensive linked data, complex integration procedures, and specialized graph-processing infrastructures.

Transformer-based methods can model contextual language variation without an explicit ontology; however, they commonly require substantial pre-trained resources and computational capacity. In contrast, the proposed system applies domain-controlled mapping and thresholds in a resource-constrained setting.

LLMs support contextual interpretation and natural-language generation, but their probabilistic outputs may be difficult to trace to specific domain relationships. Their data and infrastructure requirements may also limit their deployment in low-resource language settings.

The proposed architecture prioritizes explicit domain representation, traceability, and controlled correspondence decisions. These properties suit the assessment tasks in which the basis of a decision must be inspected.

Future work should compare ontology-, embedding-, and LLM-based methods on the same Kazakh-language dataset. Hybrid architectures should also be evaluated to determine whether contextual flexibility can be improved without sacrificing traceability. These comparative statements remain conceptual until the controlled experiments are completed.