Anissa Benlarabi*
| Amal Khtira | Bouchra El Asri
© 2025 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
Nowadays, a product is best when it constructs a memorable experience in the user mind. Therefore, the focus on designing and assessing user experience has grown significantly in recent years. However, in the context of software product lines, user experience analysis is often overlooked. This paper addresses this gap by integrating user experience into the product derivation process. It introduces an automated tool designed to handle various tasks associated with user experience during the software product line derivation. The tool employs questionnaires as a means of measurement. Taking user experience into account in software product lines allows designers to gain deeper insights into the needs of future customers, helping them create more competitive and innovative products.
software product lines, derivation, user experience, user experience questionnaire
Product Line Engineering (PLE) is a field focused on mass production, initially introduced in industry to reduce time-to-market and costs. Due to its successful outcomes, it quickly spread to other areas. Software Product Line (SPL) engineering involves two main processes: domain engineering, where shared assets for a business domain are developed, and application engineering, where customized products are created and delivered to end-users by adapting those common assets [1].
In recent years, domain engineering has garnered significant attention. The researchers considered that the implementation of user requirements at the domain level and the development of common assets were sufficient to achieve the SPL goals. However, the absence of methodological support for application engineering has negatively affected both the effort required and the time to market. The challenges and issues related to product derivation in SPL were explored in references [2-4], where the authors examined these problems through industrial case studies. Therefore, many works appeared to deal with requirements engineering at the derivation level. They tackled functional and non-functional requirements, but they nevertheless ignored totally the hedonic aspects of product design and validation.
The hedonic quality aspects are an integral part of a more generic notion known in software development as user experience (UX). User experience is a holistic concept that highlights the significance of personal evaluations, emotions, and motivational inclinations before, during, and after interacting with a technical product [5].
UX design and evaluation have gained a dominant place in contemporary commercial policy, it is becoming increasingly important to measure the UX to check if a product is better than competition. The most used instrument to measure UX in the literature is the questionnaire [6-9], since it is economical, easy, simple to use and do not require a certain level of technical expertise.
The main contribution of this paper is the incorporation of the UX concept in the process of SPL derivation, starting from the requirement's analysis to the test and the validation. Hence, we propose an improved derivation process for SPL which takes into account the UX in the different steps of the process. We use the questionnaire as an instrument to design and evaluate the UX. For this purpose, we selected factors from the standardized questionnaires in the literature, in our previous work [10] we presented the resulted factors. In this paper we developed a framework untitled UXDBM-SPL that automates the tasks related to UX design and measurement.
The rest of this paper is structured as follows: The next section outlines the background and motivations for our research, detailing the SPL derivation process, its challenges, and related studies addressing nonfunctional properties in SPL. Section 3 presents our literature review on UX within the software domain. Section 4 introduces our proposed approach. In Section 5, we examine the most significant UXQs discussed in the literature. Section 6 presents our framework, and Section 7 concludes the study.
We started our work by identifying the challenges of SPL derivation, then we studied the literature about non-functional requirements analysis and validation for SPL products. Results are presented in this section.
2.1 Software product line derivation
The main purpose of SPL engineering (SPLE) is to improve time-to-market and costs. This can be achieved by directly deriving products from the existing common assets without developing them from scratch. The process responsible for this operation is called the application process or derivation process. Hence, the efficiency of this process seems to be one of the most important challenges of SPL engineering. Without guidance on the main phases, activities and roles in the derivation process, the goals of SPLE remain unachievable.
Almost all the researchers who worked on the SPL derivation stressed that "there is a lack of methodological support for application engineering and, consequently, organizations fail to exploit the full benefits of software product families [11, 12].
Through our literature, we identified the practices and challenges of product derivation. We were interested in the work performed in SPLs companies with a real insight into the activities of this process. We extracted the following set of challenges from the case studies that were performed with the following industrials: Robert Bosch GmbH [2], Thales Nederland B.V. [2], the owner of a product line (SMART), which produces applications for managing doctor’s offices, labs of clinical pathology, and medical exams [3], and finally a company that has developed integrated systems for the management and operational control of complementary social security entities since 1996 in Brazil [3]. The main challenges are:
•The error-proneness: the derivation process relies on expert knowledge in decisions about variant selection and also in controlling dependencies among assets. Implicit dependencies can cause invalid selection, which requires the intervention of experts.
•The complexity: it is caused by the existence of numerous variants. The lack of semantics and behavior in variant interface descriptions is one of the main causes of reimplementation of variants, which results in multiple versions of the same variant and effort waste in implementing existing variants.
•The volume of the documentation: it is caused by over-explicit documentation.
•The Insufficiency: the irrelevance of the documentation: In most cases, investigating all consequences of changes in core assets and updating the documentation accordingly is not a priority. Hence, mistakes keep reoccurring in several projects.
•The time consumption: this occurs due to insufficient scoping during product derivation or the lack of tool support to streamline the derivation process.
We noticed that the functional and technical aspects of the desired products are the primary concern in the derivation process. Non-functional properties were not tackled, despite their importance. This can be attributed to the fact that the need to deliver an operational product is prior to evaluating its quality attributes.
2.2 Non-functional properties evaluation in SPL derivation
A non-functional property (NFP) is described by Robertson and Robertson [13] as "a characteristic or quality that a product must possess, such as its appearance, speed, or accuracy”. Nowadays, the measurement of NFP for products is no longer sufficient; customers want to derive products that respect their non-functional requirements. In our previous work [10], we introduced examples of NFP used for SPL. In this paper we focus on the type of NFP (quantitative or qualitative) and the stage of its application.
Montagud et al. [14] reviewed literature about evaluation of quality of SPL through NFP measurement and found that over 90% of works deal with properties that can be measured quantitatively, and in general, measures are related to variability, reusability, complexity and evolution. On the other hand, Soares et al. [15] focused on approaches that handle runtime NFPs in SPL; he classified them into three categories: Quality prediction approaches that seek to predict runtime NFPs based on historical data or expert knowledge or features configuration analysis. Quality estimation works that observe the runtime behavior of derived products and measures NFPs from code source analysis, and finally, feature selection category whose aim is to find the best configuration regarding a NFPs objective function. In general, the studies do not provide guidelines to follow when practitioners reuse their approaches. We present in what follows some works dealing with NFPs in SPL. Siegmund et al. [16] proposed an approach in which a goal objective function is defined on the basis of user-defined non-functional requirements. The purpose of this approach is to find the optimal product configuration using a NF goal objective function. The approach differentiates between quantifiable and qualitative properties, and focuses on NFP that can be quantifiable with a metric scale like maintainability and performance [17]. Sincero et al. [18] suggests incorporating NFPs into the feature model, where each variant or feature selection in a database will be linked to information about its effect on a set of NFPs. Hence, a customer can be informed about the impact of a configuration on the NFPs of interest for him before deriving the product. This approach classifies NFPs into measurable and non-measurable groups and focuses only on the measurable ones.
Ghezzi et al. [19] proposed a framework designed to evaluate NFPs both during design time and runtime, specifically focusing on those that can be quantitatively expressed using probabilistic metrics. In the design stage, the models of a dynamic SPL are analyzed against expected NFPs. Then, in the run-time stage, the framework monitors data about NFPs and verifies changes happening in NFPs measurement according to context.
In this section, we present our review of UX in the software domain and deal with the main research questions that were tackled in the literature.
3.1 User experience
In modern software development, User Experience (UX) has become an essential part of software non-functional properties. Users expect to interact with a product efficiently and in a very comfortable way. In addition to this, the product must be stimulating, appealing and attractive to the user's interest. Hence, it is no longer possible to separate goal-oriented interaction from hedonic qualities in software design. In the literature, the term "user experience" has been attributed to a broad spectrum of meanings, encompassing not only traditional usability but also elements such as aesthetics, hedonic qualities, emotional responses, and experiential aspects of technology use [20, 21]. I Initially, classic Human-Computer Interaction (HCI) approaches primarily focused on achieving functional goals during product interaction. However, the emergence of early research on User Experience (UX) [22-24] highlighted that interaction extends beyond conventional HCI concerns to encompass emotional and affective dimensions. As stated in our previous work [10], [ISO 9241-11: 2018] defines UX as the “perceptions and responses of users, encompassing their emotions, beliefs, preferences, perceptions, comfort, behaviors, and achievements before, during, and after use”. Forlizzi and Battarbee [20] emphasized that UX is a result of a user’s internal state (e.g., predispositions, expectations, needs, motivation, mood), the characteristics of the designed system (e.g., complexity, purpose, usability, functionality), and the context (or environment) within which the interaction occurs (e.g., organizational or social setting, meaningfulness of the activity, voluntariness of use). The components of UX and their relationships are described in the CUE-Model [22] Components of User Experience Model illustrated in Figure 1.
Figure 1. The UX model
Our literature review in the UX research area tackled four research questions: RQ1: What are the methods used to evaluate UX? RQ2: When are the data about UX in interaction with a product collected? RQ3: What are the metrics used in UX evaluation? RQ4: What are the challenges of UX evaluation using different methods? In the following subsections, we will give a response to each question.
3.2 RQ1: UX evaluation methods
Existing methods of UX evaluation were divided into three categories [25-28]:
•Self-reported measurement methods [29, 30]: in this category, the users report their experience individually without the participation of an evaluator; the instruments used are surveys and questionnaires; most of these instruments are scale-based and allow measuring quantitatively the UX.
•Observational measurement methods [31-34]: this class of methods require the intervention of an expert who observes the user while interacting with the product, interviews, observation or video records are an example of the instruments used by the evaluator.
•Physiological measurement methods [35, 36]: this category measures the physiological response of the user in a controlled environment. The most common techniques are: galvanized skin response (GSR), which measures the stress through skin activities; electroencephalography (EEG) to identify user feelings by observing brain activities; electromyography (EMG) to measure stress through muscular activities; and eye tracking to observe the visual attention [25, 37-39].
The first method is the most used in the literature, the instrument used to collect the data is predominantly the questionnaire.
3.3 RQ2: Time of UX data collection
According to the majority of the UX evaluation studies [27], the best moment to collect data about UX is after or during and after the interaction with a product. Few works deal with the UX evaluation before usage [28], the purpose of such an evaluation is generally to collect user requirements.
In conjunction with the used method [26], for the self-reported methods, the frequent period of UX evaluation is after the interaction, while the period of observational and physiological measurement is generally during the interaction.
3.4 RQ3: Metrics of UX evaluation
A UX metric is considered an indicator that measures quantitatively a user’s feelings and experience when interacting with a product [40]. According to Maia and Furtado [27], the most frequently cited indicators in the literature are "desirable," "usable," "attractive," and "valuable". Some studies [25, 40] categorized UX metrics into the following categories: performance metrics, which are related to the task executed by the user; as an example of this category, we cite: the task success or the time to achieve the task. The second type of metrics is behavior metrics, which are used to measure the user’s body reaction. The third category is the self-reported metrics that estimate the user’s perception and opinion. The last type of metrics is issue-based metrics, whose aim is to gauge the problems that user encounter during his usage of the product.
3.5 RQ4: Challenges of UX evaluation
Through our literature, we summarized all the challenges identified by the research works dealing with the UX evaluation [26-28, 41, 42], they are:
•The difficulty to measure the UX because of the influence of several factors like feelings, culture and communicability.
•The heterogeneous profiles of the users in the case of the products used by a large segment of customers.
•The individual aspect of the UX, this means that every user reports his individual experience in interacting with a product.
•The difficulty to define the criteria to measure the UX.
•The availability of the customer especially for long term evaluation (before, during and after usage).
•The correction of the product's problems with respect to the UX evaluation result: it seems difficult to correct the problems because the evaluation is carried out on the final version of the product. Hence, the modification in case of problems seems costly.
The difficulty of incorporating other techniques like agile practices and machine learning algorithms into the UX design process. Designers had a limited understanding of how these techniques could help predict user satisfaction and make better UX design decisions.
In this section, we start by our review of UX works in SPL domain, then we present the UX lifecycle. Subsequently, we introduce our improved version of the SPL derivation process
4.1 UX in SPL
Traditional criteria such as the respect of time and budget are no longer considered sufficient to assess the success of a system. Success depends also on whether the system meets the desired outcomes or not, Bano and Zowghi [43] worked on user involvement in software product development lifecycle, they carried out a case study at a financial institution within an Australian State Government organization (ASG), focusing on two projects: CRM and Portal. The study involved interviews with both internal and external users of the two systems, collecting data throughout various stages of development: pre-implementation, implementation, post-implementation, and post-installation. Their analysis revealed that user involvement fostered a sense of control over the system development, leading to a perception of ownership of the final product. Satisfaction with the system after installation was attributed to the positive reflections on the involvement experienced during the software development process [43]. Notwithstanding the importance that UX gained in modern software development, it was entirely ignored in software product line engineering, in this field, quality in 99% on cases is measured in quantitative manner [14], through their systematic review showed that most of measures evaluate attributes related to maintainability (92%), attributes related to user experience was less tackled. As stated by Yerram et al. [44], UX design can improve quality of products, the developers can create effective and enjoyable solutions when they understand the objectives, and preferences of users. In reference [45], a cases study was performed in automotive industry who is the leader in the field of product line engineering. They interviewed UX professionals who stressed the need of statistical support based on user interaction data to leverage feature elicitation, and their prioritization. For them, user modeling based on user interaction and user experience data may offer valuable design support.
Very few works tackled the UX in SPL; most of them address user interface design problematic [46, 47], Harutyunyan and Riehle [48], who introduced the notion of UXD, or User Experience Design. Through multiple case studies in an international company, he defined three stages for UXD: definition, implementation and management, and for each stage, he proposed a set of best practices that have to be employed.
4.2 UX lifecycle
The UX lifecycle is composed of three stages [49] as described in Figure 2.
•Discover: The first stage consists of gathering the UX requirements that include requirements from the business and users, this stage requires the participation of UX teams, designers and the final users. The requirements are based on the workflow, business needs, and the user. In the beginning, a profile of the different types of users is built. We must note that it is not just about user interface, but the workflow itself.
•Build: In this stage, the technical teams are engaged to develop the application in accordance with UX requirements defined in the previous stage. It should be stressed that the efficiency and the effectiveness of workflows and user interface are not sufficient to achieve the UX goals; the NFP such as performance are extremely important.
•Measure: Once development is complete, the UX team checks that UX requirements have been implemented correctly. The tests to be performed in this phase must be defined beforehand. If all the tests are OK, the technical team proceeds with the deployment. Thereafter, the UX team analyzes the deployed solution based on user feedback and evaluations. User intervention in the UX evaluation is essential; it allows determining if the initial UX requirements were correctly implemented, and points out where the application does not meet the expectations.
Figure 2. The UX lifecycle
4.3 The enhanced SPL derivation process
Previously we worked on UX evaluation in the software product line derivation process [10], we proposed and enhanced version of the derivation process described by Deelstra et al. [2], our proposition consists of integrating the UX evaluation in the initial validation phase, the derivation process is comprised of two main phases: the initial phase, where two options are available: the first is to construct the product from the common assets by selection, the second one is to select an old or a reference configuration and adapted according to the desired product requirements, after constructing the product and initial validation is done, here we propose to separate this step into two task: the functional and non-functional tests including tests of NFP evaluation, and the task of UX evaluation. The iteration phase starts after the initial phase in order to implement corrections to the derived product. The process is described in Figure 3.
Figure 3. The enhanced derivation process
4.4 The enhanced SPL derivation process: An improved version
In this paper, we try to cover the whole UX lifecycle. Hence, the new process that we propose includes the three stages of the UX lifecycle: discover, build and measure in the product derivation process.
•Discovering UX requirements: This task should be integrated into requirements engineering, where UX requirements are collected. Incorporating this step into the product derivation process allows us to create a reference database containing UX reports for each product. Consequently, during configuration selection, we consider the UX outcomes of the chosen configuration alongside its functional qualities.
•Building UX in the products: In the case of construction, the selection of technical and architectural components must respect the UX requirements specified by the users. The UX team has to work together with the technical team in order to achieve this.
•Measuring UX requirements: after the derivation of the product either by construction or by configuration selection, the validation commences with the test of functional and functional properties, and then continues with the validation of the UX requirements.
The improved version of the process is illustrated in Figure 4.
Figure 4. The enhanced derivation process: An improved version
The improved process covers the two processes of domain and application engineering, as well as the development stages of requirements, implementation and validation.
The process of application engineering starts with functional and UX requirements gathering. The teams choose to reconfigure an existing product based on the similarity between the requirements and the UX results of the existing products and the requirements of the desired one. Then, the product is derived either by construction or selection, and finally, the product is tested based on existing test cases and also evaluated according to UX requirements. In the case of a new product, a set of tests cases and UX factors are selected from the existing referential in order to perform the validation. The process may sometimes require a new iteration. When the final version of the product is validated, the UX evaluation data is saved into a database in order to be used in further reconfigurations.
In this section, we present the standardized UX questionnaires in the literature. Despite their efficiency, we cannot apply one of them or all of them to all the products of a SPL. Hence, we propose a set of generic factors that the UX team of an SPL can use to design a tailored questionnaire, customized to meet the specific needs of users.
5.1 UX questionnaires
The questionnaire instrument has been widely used to measure UX in the last decades since it is economical, easy, simple to use and does not require a certain level of technical expertise. In addition, the questionnaire allows for a quantitative measurement of a product’s UX [50].
There are a great number of questionnaires in the literature covering a whole range of UX aspects. Obviously, none of the existing questionnaires contains all the UX aspects because this would increase the length of the questionnaire, thereby producing an unacceptably high response time [51].
Some questionnaires known as “standardized questionnaires," were the most commonly used in the literature. "Standard" means that they result from a careful construction process that guarantees accurate measuring of the intended UX qualities [50].
Some standardized questionnaires focus exclusively on usability aspects [52-55], while others address the broader scope of user experience (UX) [5-9]. These questionnaires include a collection of UX factors, each representing a specific aspect of user experience. Each factor comprises multiple items, typically presented as statements that users rate on a predefined scale [56].
Hereinafter, we present the most commonly used Standardized Evaluation Questionnaires:
•meCUE [5]: The modular evaluation of key Components of User Experience comprises four validated modules, each addressing different aspects of product perception—instrumental and non-instrumental, user emotions, usage outcomes, and overall attractiveness. Within each module, six to eight items were developed that specifically capture the relevant aspect of user experience. The items consisted of statements combined with a seven-point Likert scale, using response labels ranging from “strongly disagree” to “strongly agree” to gauge agreement levels.
•AttrakDiff [6]: the first questionnaire that was created in 2003. The first version of Attrakdiff consists of 23 items to be marked by the user, where each item is constructed by a 7-point semantic differential. The current version Attrakdiff is composed of 28 items structured in 4 factors: pragmatic aspects and hedonic aspects divided into 3 categories: identification, stimulation and global attractiveness, each item is composed of a pair of contradictory terms for example annoying-captivating, the two antonyms represent the ends of the evaluation scale which is composed of seven points.
•UEQ [7]: The User Experience Questionnaire (UEQ) contains thus the scales Attractiveness (six items), Perspicuity, Dependability, Efficiency, Novelty and Stimulation (four items each). Each item can be rated on a 7-point Likert scale. Answers to an item therefore range from -3 (fully agree with the negative term) to +3 (fully agree with the positive term). Half of the items start with the positive term, the rest with the negative term (in randomized order). A short version was proposed later [57], it contains eight items, grouped into two scales. This short version can be used in scenarios where the user cannot fill out the long version of the questionnaire.
•VISAWI [8]: The Visual Aesthetics of Website Inventory (VisAWI) appears to be a sound measure of the visual aesthetics of websites, comprising facets of both practical and theoretical interest. Items were designed to respect a strict quality level. Participants are requested to express their level of agreement with each item on a 7-point Likert scale ranging from 1 (‘strongly disagree’) to 7 (‘strongly agree’). The questionnaire contains 18 items representing the four subscales of the VisAWI: Simplicity, Diversity, Colorfulness, and Craftsmanship.
•SUPR-Q [9]: Standardized User Experience Percentile Rank Questionnaire, it measures four aspects of the quality of the user experience: usability, trust, appearance, and loyalty, with two items for each factor. Items are ranked using a 5-point scale (1 = strongly disagree and 5 = strongly agree). With the exception of one item, users are asked to respond to a 11-point scale, wherein 0 signifies a lack of likelihood and 10 signifies an extreme likelihood.
Figure 5. Statistics about UXQ uses
According to the review performed by on the UX standardized questionnaire [58], the most used ones are: AttrakDiff, UEQ, and meCUE. Over time, the three questionnaires have seen a steady progression. The graph in Figure 5 shows the total uses of these three questionnaires per year.
5.2 UXQ for SPL
In order to determine which questionnaire fits better as a set of products and which does not, a study was conducted at the University of Applied Sciences Emden/Leer with 61 participants [56]. The participants had to choose a product from seven well-known products. For each factor, the participants were asked to rate the importance of the factor for the chosen product. The results show that some factors were evaluated as important for all the products. However, other factors were not selected by all the users; for instance, the trust/credibility factor is more important for safety-related systems such as online banking than in navigation applications such as Firefox. The indication of importance of the factors depends on the products and the perceived importance and cannot be universally valid.
In order to take the full advantage of the presented questionnaire, we extracted all the factors from the UX questionnaires (AttrakDiff2, UEQ, VISAWI, meCUE, and SUPR-Q). We identified 14 factors, outlined below in Table 1, along with their corresponding UXQ origins. This list of factors is explained in detail in our previous work [10].
Our objective is to create a database of factors and their corresponding items to assist the UX team in designing a tailored questionnaire for each product based on its specific UX requirements. To achieve this, we analyzed items from various questionnaires and compiled a final list of 66 items for inclusion in the database. Each item is associated with a single factor, and in cases of redundancy, the item is assigned to the factor it aligns with most closely.
Table 1. The list of factors
|
Factor |
Origin |
Description |
|
Attractiveness |
[5-9] |
Is the product attractive ? |
|
Usability |
[6, 9] |
Is the product practical? |
|
Identity |
[5, 6] |
Can the product reflect the identity of its user? |
|
Stimulation |
[6, 7] |
Using the product is stimulating? |
|
Simplicity |
[7, 8] |
Is the product easy to understand? |
|
Efficiency |
[5, 7] |
Can users achieve their objectives efficiently and with ease? |
|
Dependability |
[7] |
Does the product respond expectedly and constantly to user interactions? |
|
Loyalty |
[5, 9] |
Does the product influence users to promote it? |
|
Diversity |
[8] |
Does the product impart a sense of aesthetic richness and diversity? |
|
Craftsmanship |
[8] |
Is the product artfully constructed with focus on detail? |
|
Novelty |
[7] |
Is the product innovative, and able to engage and captivate users? |
|
Credibility |
[9] |
Does the product seem reliable and trustable? |
|
Positive or Negative Emotions |
[5] |
Does the product elicit emotional reactions, both positive and negative, from users? |
|
Intention to Use |
[5] |
Does the product Captivate users to the point of losing all sense of time? |
In this section, we present our framework, UXDBM-SPL (UX discovering, building and measuring in the SPL derivation process). This framework provides the necessary tools to carry out the activities related to UX in the process of Figure 4.
6.1 The framework description
Our framework is composed of a web application that helps UX teams conceive the UX questionnaire forms and visualize their results. The application allows users to fill out the questionnaire conceived by the UX teams. Data are saved to a database in order to be used by the technical team, designers, and UX team in future configurations. An additional tool is provided by the framework to achieve comparisons between desired UX requirements and historical UX data, with the aim of retrieving the most similar product. The framework as illustrated in Figure 6 is composed of a web client, a web server and a database to save data.
Figure 6. Architecture of the framework
Our solution offers a set of functionalities for the users of the application. The application allows essentially for:
•Creating a UX questionnaire by selecting a set of factors.
•Saving the responses of users on the UX questionnaire.
•FINDING similar products on the basis of the questionnaire elaborated by the UX team in collaboration with the users.
In Table 2, we present the algorithm that finds similar products on the basis of similarity between factors selections of the new product and the existing ones.
Table 2. Retrieving similar products algorithm
|
In Figure 7, we present the algorithm that finds similar products on the basis of similarity between factors selections of the new product and the existing ones.
The framework can be used by the following factors: the UX team, the final User, the designers, and the technical team. In Figure 7, we describe the use cases of these actors.
Figure 7. Use cases of different actors in the platform
6.2 Examples
In this section, we present two examples. The first concerns the task of creating UXQ by UX team, after validation of the UXQ conceived we introduce the generated UXQ which could be submitted to customer in the second example. Figure 8 illustrates the first example.
In the web page of Figure 9, the UX team has to fill out the project and the customer names, then they create a customized UXQ on the basis of the interviews realized with the final customer. They have to select a list of factors from the 14 factors presented in Section 5.2. After the validation of the selected factors, a new web page appears with the resulted questionnaire, which is composed of the different items of the selected factors and the scale to be filled by the final customer. The generated UXQ is illustrated in web page Figure 9. The UX team has the possibility to submit their questionnaire directly to the final customer or to modify it before submission.
Figure 8. UXQ conception page of the UXDBM-SPL platform
Figure 9. UXQ generation page of the UXDBM-SPL platform
This paper emphasizes the importance of incorporating the concept of User Experience (UX) into Software Product Line (SPL) engineering. To address this, we propose a method for integrating UX design and evaluation into the SPL derivation process. In today's competitive landscape, measuring UX is essential to determine whether a product outperforms its competitors. Ultimately, the most successful product is the one that provides a memorable experience that the user will want to share with others.
We concentrated on incorporating UX into every stage of the derivation process, from gathering UX requirements to testing and validation. Building on our previous work, we propose an enhanced version of the derivation process. Additionally, we introduced several UX-related tasks that can be undertaken by the UX team, end-users, designers, and the technical team.
We use the UXQ as an instrument for UX measurement; our choice is based on a review of the literature about UX measurement instruments. We found that UXQs enable an instant and quantitative assessment of user interaction, and furthermore, they do not require specialized knowledge.
From our analysis of UX questionnaires (UXQs), we identified 14 factors that represent key quality aspects of User Experience. These factors provide a foundation for UX teams to develop tailored questionnaires that align with specific user needs.
The tasks related to UX in the SPL derivation process were automated on our platform, UXDBM-SPL. At the present time, we are working on the validation of our platform with a case study on a SPL of text editors. We intend to include the comparison with the historical data in order to retrieve the most similar products based on UX requirements, which enables us to derive the most profit from the SPL engineering.
[1] Van der Linden, F. (2002). Software product families in Europe: The Esaps & Café projects. IEEE Software, 19(4): 41-49. https://doi.org/10.1109/MS.2002.1020286
[2] Deelstra, S., Sinnema, M., Bosch, J. (2004). Experiences in software product families: Problems and issues during product derivation. In Software Product Lines: Third International Conference, SPLC 2004, Boston, MA, USA, pp. 165-182. https://doi.org/10.1007/978-3-540-28630-1_10
[3] de Souza, L.O., O’Leary, P., de Almeida, E.S., de Lemos Meira, S.R. (2015). Product derivation in practice. Information and Software Technology, 58: 319-337. https://doi.org/10.1016/j.infsof.2014.07.004
[4] Deelstra, S., Sinnema, M., Bosch, J. (2005). Product derivation in software product families: A case study. Journal of Systems and Software, 74(2): 173-194. https://doi.org/10.1016/j.jss.2003.11.012
[5] Minge, M., Thüring, M., Wagner, I., Kuhr, C.V. (2017). The meCUE questionnaire: a modular tool for measuring user experience. In Advances in Ergonomics Modeling, Usability & Special Populations: Proceedings of the AHFE 2016 International Conference on Ergonomics Modeling, Usability & Special Populations, Florida, USA, pp. 115-128. https://doi.org/10.1007/978-3-319-41685-4_11
[6] Hassenzahl, M., Burmester, M., Koller, F. (2003). AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. Mensch & Computer 2003: Interaktion in Bewegung, pp. 187-196. https://doi.org/10.1007/978-3-322-80058-9_19
[7] Laugwitz, B., Held, T., Schrepp, M. (2008). Construction and evaluation of a user experience questionnaire. In HCI and Usability for Education and Work: 4th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society, USAB 2008, Graz, Austria, pp. 63-76. https://doi.org/10.1007/978-3-540-89350-9_6
[8] Moshagen, M., Thielsch, M.T. (2010). Facets of visual aesthetics. International Journal of Human-Computer Studies, 68(10): 689-709. https://doi.org/10.1016/j.ijhcs.2010.05.006
[9] Sauro, J. (2015). SUPR-Q: A comprehensive measure of the quality of the website user experience. Journal of Usability Studies, 10(2): 68-86.
[10] Benlarabi, A., Khtira, A., El Asri, B. (2023). Evaluation of user experience in software product lines derivation process. In 2023 3rd International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET), Mohammedia, Morocco, pp. 1-7. https://doi.org/10.1109/IRASET57153.2023.10152989
[11] Rabiser, R., Grunbacher, P., Dhungana, D. (2007). Supporting product derivation by adapting and augmenting variability models. In 11th International Software Product Line Conference (SPLC 2007), Kyoto, Japan, pp. 141-150. https://doi.org/10.1109/SPLINE.2007.22
[12] O'Leary, P., McCaffery, F., Thiel, S., Richardson, I. (2012). An agile process model for product derivation in software product line engineering. Journal of Software: Evolution and Process, 24(5): 561-571. https://doi.org/10.1002/smr.498
[13] Robertson, S., Robertson, J. (1999). Mastering the Requirements Process. Harlow, UK: Addison Wesley.
[14] Montagud, S., Abrahão, S., Insfran, E. (2012). A systematic review of quality attributes and measures for software product lines.Software Quality Journal, 20: 425-486. https://doi.org/10.1007/s11219-011-9146-7
[15] Soares, L.R., Potena, P., Machado, I.D.C., Crnkovic, I., de Almeida, E.S. (2014). Analysis of non-functional properties in software product lines: A systematic review. In 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications, Verona, Italy, pp. 328-335. https://doi.org/10.1109/SEAA.2014.48
[16] Siegmund, N., Rosenmüller, M., Kuhlemann, M., Kästner, C., Saake, G. (2008). Measuring non-functional properties in software product line for product derivation. In 2008 15th Asia-Pacific Software Engineering Conference, Beijing, China, pp. 187-194. https://doi.org/10.1109/APSEC.2008.45
[17] Siegmund, N., Rosenmüller, M., Kästner, C., Giarrusso, P.G., Apel, S., Kolesnikov, S.S. (2013). Scalable prediction of non-functional properties in software product lines: Footprint and memory consumption. Information and Software Technology, 55(3): 491-507. https://doi.org/10.1016/j.infsof.2012.07.020
[18] Sincero, J., Schröder-Preikschat, W., Spinczyk, O. (2010). Approaching non-functional properties of software product lines: Learning from products. In 2010 Asia Pacific Software Engineering Conference, Sydney, NSW, Australia, pp. 147-155. https://doi.org/10.1109/APSEC.2010.26
[19] Ghezzi, C., Molzam Sharifloo, A. (2013). Dealing with non-functional requirements for adaptive systems via dynamic software product-lines. In Software Engineering for Self-Adaptive Systems II: International Seminar, Dagstuhl Castle, Germany, pp. 191-213. https://doi.org/10.1007/978-3-642-35813-5_8
[20] Forlizzi, J., Battarbee, K. (2004). Understanding experience in interactive systems. In Proceedings of the 5th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, Cambridge, USA, pp. 261-268. https://doi.org/10.1145/1013115.1013152
[21] Hassenzahl, M., Tractinsky, N. (2006). User experience-a research agenda. Behaviour & Information Technology, 25(2): 91-97. https://doi.org/10.1080/01449290500330331
[22] Thüring, M., Mahlke, S. (2007). Usability, aesthetics and emotions in human–technology interaction. International Journal of Psychology, 42(4): 253-264. https://doi.org/10.1080/00207590701396674
[23] Hassenzahl, M. (2001). The effect of perceived hedonic quality on product appealingness. International Journal of Human-Computer Interaction, 13(4): 481-499. https://doi.org/10.1207/S15327590IJHC1304_07
[24] Forlizzi, J., Ford, S. (2000). The building blocks of experience: an early framework for interaction designers. In Proceedings of the 3rd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, New York, USA, pp. 419-423. https://doi.org/10.1145/347642.347800
[25] Hussain, J., Ali Khan, W., Hur, T., Muhammad Bilal, H.S., et al. (2018). A multimodal deep log-based user experience (UX) platform for UX evaluation. Sensors, 18(5): 1622. https://doi.org/10.3390/s18051622
[26] Inan Nur, A., B.Santoso, H., O.Hadi Putra, P. (2021). The method and metric of user experience evaluation: a systematic literature review. In Proceedings of the 2021 10th International Conference on Software and Computer Applications, Lumpur, Malaysia, pp. 307-317. https://doi.org/10.1145/3457784.3457832
[27] Maia, C.L.B., Furtado, E.S. (2016). A systematic review about user experience evaluation. In Design, User Experience, and Usability: Design Thinking and Methods: 5th International Conference, DUXU 2016, Held as Part of HCI International 2016, Toronto, Canada, pp. 445-455. https://doi.org/10.1007/978-3-319-40409-7_42
[28] Rivero, L., Conte, T. (2017, October). A systematic mapping study on research contributions on UX evaluation technologies. In Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems, Joinville, Brazil, pp. 5. https://doi.org/10.1145/3160504.3160512
[29] Keskinen, T., Hakulinen, J., Heimonen, T., Turunen, M., Sharma, S., Miettinen, T., Luhtala, M. (2013). Evaluating the experiential user experience of public display applications in the wild. In Proceedings of the 12th international Conference on Mobile and Ubiquitous Multimedia, Luleå, Sweden, pp. 7. https://doi.org/10.1145/2541831.2541840
[30] Mäkelä, S., Bednarik, R., Tukiainen, M. (2013). Evaluating user experience of autistic children through video observation. In CHI'13 Extended Abstracts on Human Factors in Computing Systems, Paris, France, pp. 463-468. https://doi.org/10.1145/2468356.2468438
[31] Foglia, P., Zanda, M., Trading, I.O.N. (2014). Towards relating physiological signals to usability metrics: a case study with a web avatar. WSEAS Transactions on Computers, 13: 624-634.
[32] Yao, L., Liu, Y., Li, W., Zhou, L., Ge, Y., Chai, J., Sun, X. (2014). Using physiological measures to evaluate user experience of mobile applications. In Engineering Psychology and Cognitive Ergonomics: 11th International Conference, EPCE 2014, Held as Part of HCI International 2014, Heraklion, Crete, Greece, pp. 301-310. https://doi.org/10.1007/978-3-319-07515-0_31
[33] Harrati, N., Bouchrika, I., Tari, A., Ladjailia, A. (2016). Exploring user satisfaction for e-learning systems via usage-based metrics and system usability scale analysis.Computers in Human Behavior, 61: 463-471. https://doi.org/10.1016/j.chb.2016.03.051
[34] O'Brien, H.L., Lebow, M. (2013). Mixed-methods approach to measuring user experience in online news interactions. Journal of the American Society for Information Science and Technology, 64(8): 1543-1556. https://doi.org/10.1002/asi.22871
[35] Wiebe, E.N., Lamb, A., Hardy, M., Sharek, D. (2014). Measuring engagement in video game-based environments: Investigation of the User Engagement Scale. Computers in Human Behavior, 32: 123-132. https://doi.org/10.1016/j.chb.2013.12.001
[36] Read, J.C. (2012, October). Evaluating artefacts with children: age and technology effects in the reporting of expected and experienced fun. In Proceedings of the 14th ACM International Conference on Multimodal Interaction, Santa Monica, USA, pp. 241-248. https://doi.org/10.1145/2388676.2388727
[37] Holman, M., Adebesin, F. (2019). Taking the subjectivity out of UX Evaluation with Emotiv EPOC+. In Proceedings of the South African Institute of Computer Scientists and Information Technologists 2019, Skukuza, South Africa, pp. 30. https://doi.org/10.1145/3351108.3351139
[38] Guo, F., Ding, Y., Liu, W., Liu, C., Zhang, X. (2016). Can eye-tracking data be measured to assess product design? Visual attention mechanism should be considered. International Journal of Industrial Ergonomics, 53, 229-235. https://doi.org/10.1016/j.ergon.2015.12.001
[39] Maia, C.L.B., Furtado, E.S. (2016). A study about psychophysiological measures in user experience monitoring and evaluation. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computing Systems, São Paulo, Brazil, pp. 7. https://doi.org/10.1145/3033701.3033708
[40] Albert, B., Tullis, T. (2022). Measuring the User Experience: Collecting, Analyzing, and Presenting UX Metrics.Morgan Kaufmann.
[41] Abbas, A.M., Ghauth, K.I., Ting, C.Y. (2022). User experience design using machine learning: A systematic review. IEEE Access, 10: 51501-51514. https://doi.org/10.1109/ACCESS.2022.3173289
[42] Hinderks, A., Mayo, F.J.D., Thomaschewski, J., Escalona, M.J. (2022). Approaches to manage the user experience process in Agile software development: A systematic literature review. Information and Software Technology, 150: 106957. https://doi.org/10.1016/j.infsof.2022.106957
[43] Bano, M., Zowghi, D. (2015). A systematic review on the relationship between user involvement and system success. Information and Software Technology, 58: 148-169. https://doi.org/10.1016/j.infsof.2014.06.011
[44] Yerram, S.R., Mallipeddi, S.R., Varghese, A., Sandu, A.K. (2019). Human-centered software development: integrating user experience (UX) design and agile methodologies for enhanced product quality. Asian Journal of Humanity, Art and Literature, 6(2): 203-218. https://doi.org/10.18034/ajhal.v6i2.732
[45] Ebel, P., Brokhausen, F., Vogelsang, A. (2020). The role and potentials of field user interaction data in the automotive UX development lifecycle: An industry perspective. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Virtual Event, USA, pp. 141-150. https://doi.org/10.1145/3409120.3410638
[46] Gabillon, Y., Biri, N., Otjacques, B. (2015). Designing an adaptive user interface according to software product line engineering. In ACHI 2015: The Eighth International Conference on Advances in Computer-Human Interactions, Lisbon, Portugal, pp. 86-91.
[47] Pleuss, A., Hauptmann, B., Dhungana, D., Botterweck, G. (2012, June). User interface engineering for software product lines: the dilemma between automation and usability. In Proceedings of the 4th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, Copenhagen, Denmark, pp. 25-34. https://doi.org/10.1145/2305484.2305491
[48] Harutyunyan, N., Riehle, D. (2019). User experience design in software product lines. In Proceedings of the 52nd Hawaii International Conference on System Sciences, pp. 7503-7512.
[49] Howard, C., Baines, J. (2023). UX Lifecycle: The Business Guide to Implementing Effective Software User Experiences. Mercury Learning and Information.
[50] Schrepp, M., Hinderks, A., Thomaschewski, J. (2017). Construction of a benchmark for the user experience questionnaire (UEQ). International Journal of Interactive Multimedia and Artificial Intelligence, 4(4): 40-44. https://doi.org/10.9781/ijimai.2017.445
[51] Schrepp, M., Thomaschewski, J. (2019). Design and validation of a framework for the creation of user experience questionnaires. International Journal of Interactive Multimedia and Artificial Intelligence, 5(7): 88-95. https://doi.org/10.9781/ijimai.2019.06.006
[52] Lewis, C., Wharton, C. (1997). Cognitive walkthroughs. In Handbook of Human-Computer Interaction, pp. 717-732).
[53] Gediga, G., Hamborg, K.C., Düntsch, I. (1999). The IsoMetrics usability inventory: an operationalization of ISO 9241-10 supporting summative and formative evaluation of software systems. Behaviour & Information Technology, 18(3): 151-164. https://doi.org/10.1080/014492999119057
[54] Brooke, J. (1996). SUS: A quick and dirty usability scale. Usability Evaluation in Industry.
[55] Kirakowski, J., Corbett, M. (1993). SUMI: The Software Usability Measurement Inventory. British Journal of Educational Technology, 24(3): 210-212. https://doi.org/10.1111/j.1467-8535.1993.tb00076.x
[56] Hinderks, A., Winter, D., Schrepp, M., Thomaschewski, J. (2019). Applicability of user experience and usability questionnaires. Journal of Universal Computer Science, 25(13): 1717-1735. https://doi.org/10.3217/jucs-025-13-1717
[57] Schrepp, M., Hinderks, A., Thomaschewski, J. (2017). Design and evaluation of a short version of the user experience questionnaire (UEQ-S). International Journal of Interactive Multimedia and Artificial Intelligence, 4(6): 103-108. https://doi.org/10.9781/ijimai.2017.09.001
[58] Díaz-Oreiro, I., López, G., Quesada, L., Guerrero, L.A. (2019). Standardized questionnaires for user experience evaluation: A systematic literature review. Proceedings, 31(1): 14. https://doi.org/10.3390/proceedings2019031014
