Audit of National Projects as a Factor in Achieving Sustainable Development Goals

In 2015, the UN approved a common program to achieve peace and prosperity for all the people of the planet. As part of the program, an “action plan for people, planet, and prosperity” was developed, which includes 17 sustainable development goals (SDGs) and 169 targets aimed at stimulating economic growth, social inclusion, and environmental protection [1]. The implementation of these ambitious tasks rests with states, posing questions about methods to achieve SDGs and how they meet task management in the public sector. Blanc and Montero [2] highlight the need to integrate SDGs into national projects, which are presented as a project management tool in the public sector. The adaptation of SDGs to national projects can fix indicators (strategic goals) determined based on approved methodologies in national development projects. In this regard, the goals laid down in the Agenda for Sustainable Development (hereinafter referred to as the Agenda) are achieved during the implementation of national development projects. In the Russian Federation, to achieve strategic goals, state programs are developed and customized to individual events, as well as national projects and federal projects, which may be part of state projects. The implementation of state projects aims at strategic goals, while projects should help to achieve a unique result.

The INTOSAI Strategic Plan for 2017-2022 emphasizes the role of supreme audit institutions (SAIs) in assessing the achievement of SDGs (Figure 1).

1.png

Figure 1. The contribution of SAIs to the achievement of SDGs audit institutions [3]

Source: Compiled by the authors based on Achieving SDGs during the COVID-19 pandemic: The role of states and supreme

There is no standardized approach and unified methodology for auditing national projects. The voluntary national report compiled by the Russian SAI and presented for the first time in July 2020 at the UN Political Forum states that most of the SDGs are to some extent embedded in strategic and program documents of the Russian Federation and there are positive results (for example, SDG 1 “No poverty”, SDG 4 “Quality education”, and SDG 8 “Decent work and economic growth”). The Accounts Chamber of the Russian Federation claims that there is no sufficient understanding of the mechanism for achieving and managing SDGs [4]. There is no distinction between the specific impact of national projects on the expected effect from their implementation, while most of the country's budget expenditures are based on program-targeted financing (According to the Conclusion of the Accounts Chamber of the Russian Federation on the draft federal law "On the federal budget for 2022 and for the planning period of 2023 and 2024" [5], budget allocations, envisaged by the draft law for the implementation of 47 state programs, make up about 78% of the total planned budget allocations of the federal budget for 2022). There are typical problems in terms of target indicators: a significant number of indicators without planned values for the corresponding year; no actual values for several indicators; an excessive number of indicators that do not characterize the achievement of socio-economic development goals; underestimation of the predicted values of the state project indicators. The relevance of these issues remains, which is confirmed by the expert assessment of the Accounts Chamber of the Russian Federation [5], according to which “35 out of 47 state projects do not correspond to strategic planning documents”. There are many indicators and no actual values for several indicators, which hinders monitoring and making managerial decisions.

The analysis of activities of the Accounts Chamber allows forming an audit matrix for national projects (by types of activities and subject areas of the audit).

The matrix (Figure 2) proves that there is no unified approach to auditing national projects that considers all areas and stages of their implementation, including quantitative and qualitative indicators.

2.png

Figure 2. Matrix for the audit of national projects

Source: Compiled by the authors based on the official website of the Accounts Chamber of the Russian Federation [6]

The study aims at developing evidence-based proposals and improving the methods of auditing national projects used by the control and accounting bodies. As a result of the study, it was found that the current methodology is limited to two areas: the audit of the implementation of measures and the audit of the execution of budget expenditures, which does not allow for forming an objective opinion about the quality of project implementation. We believe that the process of audit and quality control of a national project should be carried out at all stages of its life cycle. This conclusion allowed us to form a fundamentally new approach to the methodology for auditing national projects. The audit of national projects should cover the following areas: the formation of the national project, its implementation, presentation, and reporting. Based on the audit results, national projects can contribute to the achievement of the national development goals of the Russian Federation and the UN SDGs for the period until 2030.

In the course of the study, we analyzed documents for 2020 and 2021, including eight reports of the control and accounting bodies of the Russian Federation on the audit of national projects in accordance with the work plan of the Accounts Chamber as part of monitoring the implementation of national projects [14-22].

These reports mainly consist of the following sections: the grounds for holding an expert-analytical event, the subject, the object, the study period, the timing, intermediate results, conclusions, and suggestions (recommendations). To assess the conclusions (reports) of SAIs of the Russian Federation based on the audit of national projects for their compliance with the project analysis stages based on the project life cycle, the PMBOK approach was used. It considers control as its separate stage. Consequently, the control function should be extended to all stages, and the life cycle of national projects should have the following form (Figure 3).

3.png

Figure 3. The supposed life cycle of a national project

Source: Compiled by the authors based on the Project Management Body of Knowledge (PMBOK) [23]

Based on the life cycle of national projects, we suggest that the audit of national projects should assess compliance in the following steps:

I. The stage of planning the budgetary expenditures of project management, including determining needs (expectations of benefits) when setting targets and indicators in accordance with national goals.

II. The stage of executing budget expenditures of project management, including determining the effectiveness, legitimacy, and target use of public resources, establishing the level of cash execution, analyzing the limits of budget obligations, and accepting budget obligations, as well as achieving the established indicators.

III. The stage of reporting on the results of project management, including reporting data on the progress of projects and their placement in the national project management subsystem of the State Integrated Information System (SIIS) “Electronic Budget”.

The analysis of the stages ensures the complete audit of projects: from development to the legality and efficiency of budgetary funds [24]. This concept contributes to the formation of a unified and end-to-end audit of strategically important projects for the socio-economic development of the budget system of the Russian Federation.

In addition to the complete coverage of the project audit, it is important to pay attention to the consistency of scorecards, criteria, and indicators that assess the project management of budget expenditures of the Russian Federation, compliance with strategic goals and needs, analysis of resource support and activities, the final result (strategic performance) caused by the implementation of the national project. In this regard, to assess the quality of planning and determine the need, a quality matrix was applied based on the criterion of strategic performance (The criterion of strategic performance is the degree of achievement (achievability) of immediate and final results, the logic for achieving immediate and final results, the significance of the final results for target groups due to the creation and functioning of the relevant immediate results. The completeness of immediate and final results, their exhaustive list, their chronology (timeliness), the risks and opportunities for obtaining can be determined.) and the result-oriented approach (A result-oriented approach is the analysis of actual or expected immediate or final results based on the establishment of criteria and deviations from them, and the formulation of recommendations aimed at ensuring that such deviations are eliminated.) used in the strategic audit of “programs, projects/quality of processes” [25]. The scheme for applying the strategic performance criterion is shown in Figure 4.

4.png

Figure 4. The scheme for applying strategic audit criteria [25]

The recommended quality matrix determining the planned achievement of strategic goals was based on the qualitative and quantitative indicators of the summary of national projects, namely: the established national development goals of the Russian Federation on a par with the targets characterizing their achievement; the planned and actual values of key indicators of national projects aimed at the systematic achievement of socially significant results and tasks; the planned and spent financing of national projects within the federal budget. To establish the achievement of the key indicators of national projects, we used the performance indicator coefficient (K_per) and the coefficient of financing performance (K_fin). At the end of the financial year, the quality matrix is presented in Table 1.

Table 1. The general planning quality matrix

Source: Compiled by the authors

where, $K_{per}$ is the achievement of key indicators (performance ratio of indicators);

$K_{fin}$ is the level of budget execution (budget execution ratio).

To calculate such indicators as K_fin and K_per, we used the following formulas:

$K_{per}=\frac{\mathrm{F}_{ind}}{\mathrm{P}_{ind}} * 100 \%$      (1)

where, $F_{i n d}$ is the actual value of a key indicator; $P_{ind}$ is the target value of a key indicator.

$K_{f i n}=\frac{\mathrm{F}_{f i n}}{\mathrm{P}_{f i n}} * 100 \%$     (2)

where, $F_{fin}$ is the actual value of budget execution;

$P_{fin}$  is the target level of funding.

If the achievement of a non-specified key indicator is assessed, but the overall achievement of national project indicators is determined, then K_per is converted into the K_per General indicator ($O K_{per}$):

$\mathrm{O} K_{p e r}=\frac{\sum K_{p e r}}{\sum \Pi * 100 \%}$     (3)

where, $\sum \Pi$ is the number of key indicators of national projects.

The overall quality level is determined by the following formula:

$K_q=O K_{p e r} * \frac{K_{f i n}}{100}$       (4)

where, $K_q$ is the overall level of quality.

The value of $K_q$ should be one.

For the needs of assessment at the end of the financial year, a detailed planning quality matrix was compiled (Table 2).

Based on the criterion of strategic performance, this matrix allows helps better analyze the achievement/non-achievement of indicators and funding and strategic goals.

While considering the reports of the Accounts Chamber of the Russian Federation for 2020-2021, we assessed the compliance of audit objectives for each expert-analytical activity and their achievement (non-achievement). Due to the generalization of analytical materials, a matrix was compiled to achieve the goals and results of the audit of national projects (Table 3) and a matrix of compliance with the information and analytical support for the audit of national projects (Table 4).

Table 2. The detailed planning quality matrix

biao_2-1.png

where,

biao_2-2.png

Source: Compiled by the authors

Table 3. The matrix of achieving goals and results of the audit of national projects

Source: Compiled by the authors based on the official website of the Accounts Chamber of the Russian Federation [6]

Table 4. The compliance matrix of information and analytical support for the audit of national projects

Source: Compiled by the authors based on the official website of the Accounts Chamber of the Russian Federation [6]

During the planning audit, it is necessary to conclude the decision to initiate a national project. This conclusion is consistent with the conclusions of Velasquez and Hester [30], who proposed the use of multi-criteria methods in deciding whether to launch a project.

We cannot but agree with Marnada et al. [31] who emphasized the need for a more flexible, dynamic, and scalable approach to project management. The authors revealed that uncontrolled changes could potentially lead to delays and cost overruns on the project, which is significant for auditing the execution stage of a national project.

Regarding the audit of reporting data on the progress of projects (analysis of the reporting stage), Głodziński [32] believes that when evaluating the effectiveness of a project, it is necessary to use a horizontal (with an emphasis on the scope of analysis) and a vertical approach based on methodological assessment. Both approaches complement each other. The horizontal approach is associated with focusing on project stakeholders, available resources, possible methods, measures, or systems to support the project implementation, as well as the need to analyze both financial and non-financial criteria and factors for the project's success. The latter are most consistent with the goals of national projects. It is also important to assess the achievement of the desired results of the project, which complies with our conclusion about the need to assess the final result (strategic performance) as the effect of implementing the national project. However, the horizontal approach does not allow assessing the project structure that considers direct and indirect benefits and results of the national project (economic, social and environmental, etc.). This shortcoming is compensated by the vertical approach that focuses on the methodological development of assessment and includes three main areas: 1) assessment methods are built into project management methodologies/methods (PMBoK, Prince 2, APM); 2) the development of the project as an element of increasing its efficiency; 3) the project assessment process, including financial and non-financial criteria.

While describing differences in the project-based approach in the public sector, Volden and Welde [33] proposed to assess the effectiveness of a public project according to six criteria, considering the operational, tactical, and strategic levels of project success. At the operational and tactical level, it is proposed to evaluate only the effectiveness in terms of costs, time and quality, regardless of whether the project objectives are achieved. At the strategic level, the authors recommend applying such criteria as other impacts (positive and negative side effects from project implementation), relevance (preserved need for the project), sustainability (maintenance of benefits throughout the project), and benefits-costs (cost-benefit analysis). These proposals are consistent with the study results in terms of the need to conduct strategic, financial, and performance audits, which will cover the entire cycle of using public resources (from project development to obtaining the final result).

Summing up the discussion, we note that the results of our study develop and combine the conclusions of individual authors on evaluating the effectiveness of the implementation of national projects. A distinctive feature of this work is the complexity of methodological approaches to the audit of national projects. We believe that the methodology used by SAIs should include an independent assessment of the reasonable volume and timing of project expenditures (analysis of the planning stage), the results of actions and efficient use of federal and other resources (analysis of the execution stage), and the quality of reporting data on the progress of projects (analysis of the reporting stage).