Design of a Balanced Model for Evaluating the Outcomes of ERP Projects

Design of a Balanced Model for Evaluating the Outcomes of ERP Projects

Youssef Zaitar

Department of Computer Science, Laboratory of Mathematics, Computing and Applications-Information Security, Faculty of Sciences, Mohammed V University in Rabat, Rabat 1000, Morocco

Corresponding Author Email: 
y.zaitar@um5r.ac.ma
Page: 
457-462
|
DOI: 
https://doi.org/10.18280/isi.270312
Received: 
16 February 2022
|
Revised: 
5 April 2022
|
Accepted: 
19 April 2022
|
Available online: 
30 June 2022
| Citation

© 2022 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).

OPEN ACCESS

Abstract: 

This paper aims to design a balanced scorecard to measure the contribution of ERP systems to improving the multidimensional performance of companies from different perspectives (internal processes, customer, financial, learning & innovation). Then, the proposal of an ERP impact model (MI-ERP) which illustrates this contribution validated by empirical studies carried out in several large organizations. The analysis of the results shows that the companies studied have mainly benefited from the advantages of ERP in terms of improving overall performance (between 63% and 90%). This improvement is strongly manifested at the “Internal processes” level, more than on the other aspects of performance. The study further describes the situation of ERP in large organizations in Morocco.

Keywords: 

ERP, IT impacts, evaluation model, balanced scorecard, performance

1. Introduction

Currently, several studies show that ERP systems are essential and have a multitude of advantages for companies. According to the new ERP report of Panorama Consulting Group [1], ERP generated several categories of benefits: Operational Efficiency Benefits: Improving productivity and efficiency, reducing operating and/or labor costs, optimizing inventory levels, Removing silos, Standardizing operations. Reporting and Visibility Benefits: Gaining access to real-time data, gaining access to real-time data. Growth and Competition Benefits: Improving the customer experience, improving interactions with suppliers, Building new operating models. Technology Benefits: Reducing IT maintenance costs. However, the initial investment in these projects can present an obstacle at first instance: purchase of the software package license and hardware, integration of the solution, support and training... amounting to hundreds of thousands of dollars, or even millions of dollars. Researchers have been discussing for a long time the difficulty of justifying these investments as indicated by the famous paradox of Solow [2]: “You can see computers everywhere except in productivity statistics”. Indeed, companies are aware of the link between performance and investments in ERP. But remain unable to measure this contribution in a quantitative way. The measurement instruments and performance ratios traditionally used to calculate the profitability of an investment are not suitable for this type of project. Thus, for several years, researchers have been discussing this difficulty of conceptualizing and evaluating the value and impact of ERP systems [3, 4].

It therefore seems relevant to propose in this manuscript, a balanced model to evaluate the contribution of ERP to improving the performance at different dimensions (Financial, customer, internal process and learning & innovation), validated by empirical studies realized in several large organizations.

2. Related Work

2.1 Benefits of ERP systems

Several authors have contributed to identifying the benefits of ERP systems. According to Madapusi and D'Souza [5], Five measures to evaluate performance: information availability, information quality, standardization, inventory management, and on-time delivery. For Ifinedo and Nahar [6], six dimensions to assess the impact of an ERP: Systems Quality (Accuracy of data, easy to learn, integration of data, efficiency), Information Quality (Timely information, important information, relevant information, usable information, available information, Vendor Quality (Adequate technical support, credibility, good relations, experience, good communication), Individual Impact (Improving individual productivity, Benefits for individuals' tasks, high quality of decision making, Time saving, Workgroup impact (Improving employee participation, improving organizational communication, creating a sense of responsibility, improving sub-unit efficiency, solution efficiency), Organizational impact (Competitive advantage, customer satisfaction, facilitating business process change, supporting decision making, better use of organizational data resources). For Chaabouni [7], three dimensions of the benefits of an ERP: economic dimension (financial performance), organizational dimension (information quality, communication), and human dimension (user satisfaction). As for Frang and Lin [8], ERP allows four perspectives of benefits: Financial perspective: Reduce corporate operating cost, Increase revenue growth. Customer perspective: Reduce transaction time, Customer satisfaction. Internal perspective: Integrating working flows subunits, Avoidance of operational bottlenecks. Innovation & Learning Perspective: Enhancing employee productivity, Reliability of software vendor. According to Shang and Seddon [9], ERP can generate five categories of benefits: operational (cost and time reduction, improving productivity), managerial (human resource management, decision making), strategic (decision support and business growth), technological (business and technological flexibility, technological costs reduction), and organizational (supporting and facilitating organizational changes, creating a common vision). For Markus et al. [10], ERP offer two types of benefits: on the one hand, technical such as elimination of data redundancy, reduction of errors, reduction of costs, integration..., and on the other hand, business such as process improvement, reduction of administrative expenses, reduction of response time to customers, standardization of procedures, improvement of decision support.

The benefits of ERP are multiple. Some authors talk about the financial benefits, others discuss the organizational and/or human benefits..., the research should be oriented in the sense of designing a framework that covers all dimensions of ERP contributions.

2.2 Evaluation models

It is difficult to apply investment evaluation techniques to IT projects, due to the inseparability of IT and work systems (Alter [11]; Baumard and Benvenuti [12]; Reix [13]). And also because of the difficulties of identifying and measuring their intangible costs and impacts [14]. Indeed, Ballantine and Stray [15] conducted two surveys in 54 companies to compare the evaluation practices of IT investments with those of other types of investments. The results of this study showed that: In the case of IS/IT projects the simpler financial techniques (payback, cost benefit analysis and return on investment) are used, whereas for the second survey we find greater use of sophisticated financial techniques (net present value and internal rate of return).

In order to evaluate a system, one can choose between three strategies depending on the temporary dimension: the first one is before the implementation (ex-ante evaluation) which aims to evaluate the future impact of the system on the organization, it is a question of measuring the capacity of the company to adopt the project. The second, during implementation, and the third after implementation (ex-post evaluation) which aims to measure the contributions made by the system on performance. To this end, several models have been developed, such as causal models, which aim to establish a direct cause-and-effect relationship between IT investments and the company's performance. Process models that aim to measure the intermediate impacts of IT on the organizational structure and business processes. Contingency models that aim to identify the indirect impacts of IT on performance. After these models, we find the Balanced Scorecard (BSC), created by Kaplan and Noron [16] to overcome the limitations of traditional scorecards that treat each indicator separately from the other. This model (Figure 1) is based on the analysis of four perspectives that take into account both financial and non-financial indicators. The performance indicators of the four perspectives of the BSC are linked together by a chain of causality [17]. To obtain good results in terms of financial indicators and thus satisfy shareholders (financial perspective), the company must have satisfied, loyal and profitable customers (customer perspective), for whom quality processes must be developed (internal process perspective). Hence the need for motivated and competent personnel, reliable and efficient information systems, and good organizational learning management (learning and growth perspective).

Figure 1. Balanced scorecard [16]

3. Proposed Model

The objective of our model called MI-ERP (ERP Impact Model) (Figure 2) is to illustrate the contributions of the ERP system in terms of the four dimensions of performance (internal processes, finance, customer, learning & innovation), by implementing all of its modules and integrating them into the entire value chain (primary and secondary activities).

The main or primary activities, which constitute the reason for the existence of the company: They correspond to the acquisition of basic resources, to manufacture, to distribution, to after-sales service.

Support or secondary activities, which correspond to the infrastructure of the company, procurement and purchasing, human resource management, technological development ensuring the conduct of the main activities.

This relationship can be expressed as follows:

Figure 2. ERP impact model (MI-ERP)

4. Research Methodology

A model must be verified. It is therefore necessary to make assumptions according to the four perspectives of performance (Table 1), which will be confronted with the reality in the companies.

In our study, we focus more on the ex-post evaluation based on the Balanced Scorecard model. The reason for this choice is justified by the fact that this phase is of great interest to managers and users who are looking for the added value of the new system in the management of their activities. Indeed, Ross and Vitale [18] note that the organization experiences a major disruption during the implementation of the system, which negatively influences performance. They note that after a marked drop following the implementation of an ERP, post-implementation performance improves significantly. And for the choice of the BSC model, is justified by the multidimensional nature of ERP with all its modules that require measures of success on different financial and non-financial dimensions. Indeed, several researchers such as Rosemann and Wiese [19]; Markus and Tanis [20]; Chand et al. [21], have indicated that the Balanced Scorecard (BSC) approach can be an appropriate technique to evaluate ERP performance.

To realize our study, we conducted a series of interviews and meetings with the Directors of Information Systems, Financial Directors, Sales Directors, and Human Resources Directors of several large companies located in the North African region, from different sectors of activity such as: mining, chemicals /pharmaceuticals, banks, distribution, industry, etc. We targeted large structures because they are financially strong enough to be able to implement an ERP and these various modules. In addition, it is in this type of company that the various processes can be found.

Half of these organizations have been on the market for more than 50 years and the other half for more than 30 years. The staff of the majority of these companies exceeds 1000 people.

During the interviews, we asked the people we met to rate the contribution of ERP to improving the performance of their organizations. For each question asked, we have associated a response grid which contains four columns corresponding to the following evaluations: “true”, “rather true”, “rather false” and “false”. Based on the LAVINA questionnaire, each response is assigned a weighting coefficient: 1 – 0.7 – 0.3 – 0. The evaluation of the level of performance according to the criteria of each dimension consists in calculating the average of the sum of the points obtained according to the four columns. Thus, the overall performance is the average of the performance levels obtained.

These hypotheses are based on investigation of the literature on the benefits of ERP systems, as well as on our own reflection. They are chosen for their relevance and measurability. Each axis of the survey consists of five questions, the purpose of which is to information that will allow the evaluation of the level of performance in each area. Obviously, these twenty assumptions alone do not claim to cover all the information to be collected for a system/process, but their use offers several advantages. Firstly, it allows the scope of the information to be defined. Secondly, it reduces the time needed to collect information on the system/process.

Table 1. Assumptions of ERP contributions on the four performance dimensions

Assumptions

Internal Processes dimension: Does ERP improve internal business processes?

I 1

Facilitation of data analysis and decision making

I 2

Improvement of cooperation, coordination and communication between departments

I 3

Standardization of processes between the company's subsidiaries

I 4

Facilitation of control tasks and better traceability of operations

I 5

Production of available and updated information in real time

Customer dimension: Does ERP support effectively customer needs?

I 6

Improvement of work satisfaction (employee satisfaction)

I 7

Improvement of interactions and communication with customers and suppliers

I 8

Reduction of delivery time

I 9

Enhancement of the company's brand image with customers and suppliers and maximizing their confidence

I 10

Improvement of reputation of IT department

Finance dimension: Does ERP improve financial performance?

I 11

Reduction of administrative costs (printing, etc.)

I 12

Reduction of production costs

I 13

Reduction of management and maintenance costs of information systems

I 14

Reduction of manpower costs

I 15

Improvement of results and profitability

Learning & Innovation dimension: Does ERP increase learning and innovation?

I 16

Improvement of staff knowledge and skills

I 17

Creation of a sense of responsibility

I 18

Improvement of employee participation, efficiency and productivity

I 19

Improvement of the company's growth

I 20

Improvement of the firm's capacity to deploy new functionalities of the information system and adaptation to technological changes

5. Result and Discussion

5.1 The ERP situation in the studied organizations: Name, age, modules installed

The study indicates that the organizations surveyed are using ERP systems for more than 10 years. SAP ranks first (67%), followed by Oracle Applications and Microsoft Dynamics in second place (22%), and other ERPs in third place (11%).

Regarding the installed modules, the study shows (Figure 3) that the ‘financial management’ module is adopted by all companies, followed by the ‘sales management’, ‘inventory management & purchase’ modules installed in more than 80% of companies, then, the ‘management control’, ‘HR management’, ‘decision support’, ‘production management’ modules installed around half of the companies, then the: ‘maintenance management’ and ‘customer relationship management’ modules are found in 1/3 of the companies.

Figure 3. The utilization rate of ERP modules

5.2 Validation of assumptions

Table 2 measures the impacts of ERP on the criteria of the four performance dimensions of the studied organizations. Reliability Cronbach's alpha was calculated, examined for validating the internal consistency and reliability statistics scales. Indeed, the value of this indicator is: Cronbach’s Alpha=0.883 (value that exceeds the acceptable level: 0.7).

5.2.1 Impacts on the first dimension (Internal processes)

The results show a remarkable improvement in the performance of internal processes following the implementation of ERP (Figure 4). Indeed, all the measurement criteria have an improvement score that exceeds 0.85. It can be seen that ERP has mainly improved cooperation, coordination and communication between departments. It has facilitated the tasks of control and better traceability of operations, and also, it has allowed the production of available and updated information in real time.

Figure 4. Rate of ERP impacts according to internal processes dimension

5.2.2 Impacts on the second dimension (Customer)

The ERP has made a positive contribution to the performance on the customer dimension (Figure 5). Indeed, three indicators exceeded a score of 0.7: the improvement of interactions and communication with customers and suppliers, the reduction of delivery time, and the reputation of IT department. The other two criteria exceed 0,6: Improvement of work satisfaction (employee satisfaction), Enhancement of the company's brand image with customers and suppliers and maximizing their confidence.

Figure 5. Rate of ERP impacts according to customer dimension

5.2.3 Impacts on the third dimension (Finance)

ERP has clearly contributed to increasing results, profitability, and reducing administrative costs. However, we can see that ERP has less stimulated the following criteria (Figure 6): Reduction of production costs, reduction of management and maintenance costs of information systems, reduction of manpower costs. To understand this last point, we can say that one of the strengths of ERP systems is the automation of tasks, which leads to a reduction in the number of employees and thus to a reduction in labor costs. However, in the studied context, it is difficult to reduce the number of jobs or to lay off an employee for purely financial reasons.

Figure 6. Rate of ERP impacts according to Finance dimension

5.2.4 Impacts on the fourth dimension (Learning & Innovation)

ERP has contributed to improving this dimension (Figure 7). In fact, all the criteria have an improvement score that exceeds 0.65: Improvement of staff knowledge and skills, Creation of a sense of responsibility, Improvement of employee participation, efficiency and productivity, Improvement of the company's growth. In particular, the criteria: improvement of the firm's capacity to deploy new functionalities of the information system and adaptation to technological changes has a score of 0.78.

Figure 7. Rate of ERP impacts according to learning & innovation dimension

5.2.5 Impacts on overall performance: The four dimensions of the Balanced Scorecard

The results indicate that the ERP implemented by these companies have increased their overall performance (between 63% and 90%). This improvement is more significant at the "internal processes" level than other aspects of performance.

The responses permitted the evaluation of all the components of our model.

Table 2. Measuring the impact of ERP on the criteria of the four performance dimensions

Impacts

False =0

Rather False =0,3

Rather True =0,7

True =1

Average per impact

Average per dimension

Nb Cit

Freq(%)

Nb Cit

Freq(%)

Nb Cit

Freq(%)

Nb Cit

Freq (%)

Internal Processes dimension: Does ERP improve internal business processes?

I 1

0

0

0

0

8

44,4

10

55,6

0,87

0.90

I 2

0

0

0

0

4

22,2

14

77,8

0,93

I 3

0

0

2

11,1

4

22,2

12

66,7

0,86

I 4

0

0

0

0

4

22,2

14

77,8

0,93

I 5

0

0

2

11,1

0

0,0

16

88,9

0,92

Customer dimension: Does ERP support effectively customer needs?

I6

0

0

3

16.7

15

83.3

0

0

0.63

0.69

I 7

2

11,1

1

5,6

8

44,4

7

38,9

0,72

I 8

2

11,1

1

5,6

8

44,4

7

38,9

0,72

I 9

2

11,1

2

11,1

8

44,4

6

33,3

0,68

I 10

2

11,1

0

0,0

11

61,1

5

27,8

0,71

Finance dimension: Does ERP improve financial performance?

I 11

0

0,0

6

33,3

4

22,2

8

44,4

0,70

0.63

I 12

2

11,1

4

22,2

10

55,6

2

11,1

0,57

I 13

2

11,1

4

22,2

11

61,1

1

5,6

0,55

I 14

2

11,1

6

33,3

6

33,3

4

22,2

0,56

I 15

0

0

0

0,0

12

66,7

6

33,3

0,80

Learning and Innovation dimension: Does ERP increase learning and innovation?

I 16

0

0

3

16,7

12

66,7

3

16,7

0,68

0.73

I 17

0

0

3

16,7

8

44,4

7

38,9

0,75

I 18

0

0

4

22,2

9

50,0

5

27,8

0,69

I 19

0

0

1

5,6

13

72,2

4

22,2

0,74

I 20

0

0

1

5,6

11

61,1

6

33,3

0,78

6. Conclusion and Future Work

In this article, we conceptualized a balanced model to measure the contribution of ERP to improving the overall performance of organizations (at the internal process level, at the customer level, at the financial level, and at the learning & innovation level). This model was validated by empirical studies conducted in several large companies. The analysis of the results clearly shows that the studied companies have mainly benefited from the advantages of ERP in terms of overall performance (between 63% and 90%). This improvement is more significant at the "internal processes" level than other aspects of performance. Generally, the results present a strong convergence of the studied organizations on the positive impact of ERP implementation.

This study further presents the situation of ERP in large organizations in Morocco: the organizations surveyed are using ERP systems for more than 10 years. SAP ranks first (67%), followed by Oracle Applications and Microsoft Dynamics in second place (22%), and other ERPs in third place (11%). Regarding the installed modules, the study shows that the ''financial management'' module is adopted by all companies, followed by the ''sales management'', ''inventory management & purchase'' modules installed in more than 80% of companies, then, the ''management control'', ''HR management'', ''decision support'', ''production management'' modules installed around half of the companies, then the: ''maintenance management'' and ''customer relationship management'' modules are found in 1/3 of the companies.

Performance evaluation faces several challenges. This is due to various reasons including the unavailability of economic information, the relatively high cost of collecting information and the difficulty of isolating the impact of the IT investment. We propose for future research to adopt a longitudinal approach to better identify this impact. Thus, to expand the list of indicators for evaluating the contribution of ERPs to improving performance and the implementation of measurable quantitative indicators.

  References

[1] Panorama Consulting Group. (2021). The 2021 ERP report. An independent, niche consulting firm specializing in business transformation and ERP system implementations for mid to large sized private and public sector organizations worldwide.

[2] Solow, R. (1987). We see the computer age everywhere except for productivity statistics!.in New York Times Book Review.

[3] Kauffman, R.J., Weill, P. (1989). An evaluative framework for research on the performance effects of information technology investment. Proceeding of the Ninth International Conference on Information Systems, Boston, pp. 377-388. https://doi.org/10.1145/75034.75066

[4] Rowe, F. (1994). Des banques et des réseaux: productivité et avantages concurrentiels. FLUX Cahiers scientifiques internationaux Réseaux et Territoires, 50-54.

[5] Madapusi, A., D'Souza, D. (2012). The influence of ERP system implementation on the operational performance of an organization. International Journal of Information Management, 32(1): 24-34. https://doi.org/10.1016/j.ijinfomgt.2011.06.004

[6] Ifinedo, P., Nahar, N. (2007). ERP systems success: an empirical analysis of how two organizational stakeholder groups prioritize and evaluate relevant measures. Enterprise Information Systems, 1(1): 25-48. https://doi.org/10.1080/17517570601088539

[7] Chaabouni, A. (2006). Implantation d’un ERP (Enterprise Resource Planning): Antécédents et Conséquences. In AIMS, XVème Conférence Internationale de Management Stratégique, Annecy/Genève, pp. 13-16.

[8] Fang, M.Y., Lin, F. (2006). Measuring the performance of ERP system-from the balanced scorecard perspectives. Journal of American Academy of Business, 10(1): 256-263.

[9] Shang, S., Seddon, P.B. (2000). A comprehensive framework for classifying the benefits of ERP systems. In Proceedings of the Sixth Americas Conference on Information Systems, Long Beach, CA, pp. 1005-1014.

[10] Markus, M.L., Axline, S., Petrie, D., Tanis, S.C. (2000). Learning from adopters' experiences with ERP: problems encountered and success achieved. Journal of Information Technology, 15(4): 245-265. https://doi.org/10.1080/02683960010008944

[11] Alter, S. (1999). The Siamese twin problem: A central issue ignored by dimensions of information system effectiveness. Communications of AIS, 2(20): 40-55.

[12] Baumard, P., Benvenuti, J.A. (1998). Compétitivité et systèmes d'information. De l'outil d'analyse au management stratégique’. InterÉditions, Paris: Dunod. https://www.abebooks.fr/9782225832529/Comp%C3%A9titivit%C3%A9-syst%C3%A8mes-dinformation-CD-Rom-loutil-2225832528/plp.

[13] Reix, R. (2002). Systèmes d’information et performance de l’entreprise étendue. F. Rowe (éd.), Faire de la recherche en systèmes d'information. Paris, France: Vuibert, pp. 333-349.

[14] Riel, P.F. (1998). Justifying information technology projects. Industrial Management, 40(4): 22-27.

[15] Ballantine, J., Stray, S. (1998). Financial appraisal and the IS/IT investment decision making process. Journal of Information Technology, 13(1): 3-14. https://doi.org/10.1080/026839698344927

[16] Kaplan, R.S., Norton, D.P. (1996). The balanced scorecard: Translating strategy into action. Harvard Business School Press, Boston, MA.

[17] Germain, C., Trebucq, S. (2004). La performance globale de l’entreprise et son pilotage: Quelques réflexions. In Semaine Sociale Lamy, 1186: 35-41. 

[18] Ross, J.W., Vitale, M.R. (2000). The ERP revolution: Surviving vs. thriving. Information Systems Frontiers, 2(2): 233-241. https://doi.org/10.1023/A:1026500224101

[19] Rosemann, M., Wiese, J. (1999). Measuring the performance of ERP software–a balanced scorecard approach. In Proceedings of the 10th Australasian Conference on Information Systems, 8(4): 773-784.

[20] Markus, M.L., Tanis, C. (2000). The enterprise systems experience-from adoption to success. Framing the Domains of IT Research: Glimpsing the Future Through the Past, 173(2000): 207-173.

[21] Chand, D., Hachey, G., Hunton, J., Owhoso, V., Vasudevan, S. (2005). A balanced scorecard based framework for assessing the strategic impacts of ERP systems. Computers in Industry, 56(6): 558-572. https://doi.org/10.1016/j.compind.2005.02.011