Challenges and prospects of evaluation of programs in public finance management in Georgia

Размещено на http://www.allbest.ru/

Georgian Technical University

Challenges and prospects of evaluation of programs in public finance management in georgia

D. Oboladze, Dr. of Sci. (Business Administration), Researcher Ph. D, R. Otinashvili, Dr. of Sci. (Economics), Prof.

Georgia

Abstract

The establishment of a program assessment mechanism in the public sector of Georgia is an important element for supporting the ongoing public finance management reform and improving the budgeting process as well as enhancing the budget planning stage and ensuring the efficient, productive and prudent management of budgetary means. The present paper discusses the problem of the absence of a program assessment mechanism in the public sector, reviews alternative solutions to the problem and, by applying a cost-effective method, analyzes the best alternative. This best alternative envisages the establishment of a centralized structural unit at the Ministry of Finance which will be responsible for the assessment of the programs/subprograms of ministries/budgetary organizations. The concluding part of the policy paper offers those measures and reforms in the form of recommendations which must be undertaken for the successful implementation of the best alternative.

Keywords: Public finances, program budgeting, assessment indicators, cost-effectiveness analysis.

Створення програм механізму оцінки в державному секторі Грузії є важливим елементом підтримки поточної реформи управління державними фінансами та вдосконалення процесу складання бюджету, а також підвищення рівня планування бюджету і забезпечення ефективного, продуктивного та розумного управління бюджетними коштами. Обговорено проблему відсутності програми механізму оцінки в державному секторі, розглянуто альтернативні рішення проблеми і шляхом застосування економічно ефективного методу проаналізовано найкращу альтернативу. Найкраще рішення передбачає створення централізованого структурного підрозділу в Мініс-терстві фінансів, яке відповідатиме за оцінку програм / підпрограм міністерств / бюджетних організацій. У заключній частині програ-много документа запропоновано ті заходи і реформи у формі рекомендацій, які необхідно вжити для успішної реалізації найкращого альтернативного рішення.

Ключові слова: державні фінанси, програмне бюджетування, показники оцінки, аналіз економічної ефективності.

Создание программ механизма оценки в государственном секторе Грузии является важным элементом поддержки текущей ре-формы управления государственными финансами и совершенствования процесса составления бюджета, а также повышения уро-вня планирования бюджета и обеспечения эффективного, продуктивного и разумного управления бюджетными средствами. Обсу-ждается проблема отсутствия программы механизма оценки в государственном секторе, рассматриваются альтернативные ре-шения проблемы и, путем применения экономически эффективного метода, анализируется наилучшая альтернатива. Наилучшее решение предусматривает создание централизованного структурного подразделения в Министерстве финансов, которое будет отвечать за оценку программ / подпрограмм министерств / бюджетных организаций. В заключительной части программного доку-мента предлагаются те меры и реформы в форме рекомендаций, которые необходимо предпринять для успешной реализации наи-лучшего альтернативного решения.

Ключевые слова: государственные финансы, программное бюджетирование, показатели оценки, анализ экономической эффек-тивности.

Introduction. The establishment of a program assessment mechanism in the public sector of Georgia is an important element for supporting the ongoing public finance management reform and improving the budgeting process as well as enhancing the budget planning stage and ensuring the efficient, productive and prudent management of budgetary means. The paper discusses the problem of the absence of a program assessment mechanism in the public finance management sector, reviews alternative solutions to the problem and, by applying a cost-effective method, analyzes the best alternative. Taking into account the shortage of specialists within this field in Georgia as well as limited budget resources and a limited period of implementation, we have selected the second alternative as the best one from among those proposed. This best alternative envisages the establishment of a centralized structural unit at the Ministry of Finance which will be responsible for the assessment of the programs/subprograms of ministries/budgetary organizations. The concluding part of the policy paper offers those measures and reforms in the form of recommendations which must be undertaken for the successful implementation of the best alternative. The basic budget policy framework system has already been established in Georgia but it requires further strengthening. Moreover, additional work on reforming is needed in the following spheres: Improvement of program budgeting - on program assessment mechanism, strengthening links between planning, budgeting and policy framework.

Description of the problem. The program budget utilizes programs/subprogram assessment indicators to evaluate how the program achieves the objectives for evaluating and measuring the results achieved within the programs/subprograms and comparing their expected/planned results. However, despite the assessment of the performance of the budget organization / program through the results assessment indicators is the budget-oriented mechanism of the outcome, this does not allow the prior (ex-ante) assessment of the program / sub-program success (positive effectiveness) and the real benefit of the results achieved within the program for the welfare of the society (ex-post). Because of the program budget assessment indicators are oriented to assess and measure the achieved results, they can only be used for the results achieved through the implementation of the program and not for the preliminary assessment.

Evaluation of results and impacts within the programs/sub-programs in Georgia is made only through assessment indicators, which determines an incomplete identification of the program's actual influence/effect. This is related to the fact that the division of the effect of the results achieved within the program/sub-program is related to external and non-controlling factors.

The actuality of the abovementioned problem is determined by the fact that in the spending agencies currently participating in the budgetary process, they are not established as a mechanism / system of appraisal (pre-or later), which the benefit of the programs/sub-programs presented in the draft bill would be measured based on it. In particular, expenses are not based on any statistical or non- statistical analysis, such as cost-benefit, cost-effectiveness, cost-benefit ratio or other similar analysis.

Program Budgeting is a new milestone in the improvement of the budgetary process of Georgia and is focused on the outcomes of the programs and not the amount of financial resources allocated to them. An important element of ensuring efficient, productive and economical management of budget support is to introduce the mechanism of analyzing the results achieved through the evaluation (ex-ante) and programs in the budgetary organizations for supporting of this reform and budget planning phase.

Systemic deficiencies are found on the planning and reporting stages of the program budgeting. Problems related to the planning stage were analyzed in the report of the State Audit's Office: "opinion on the draft state budget law 2017" [10].

Consequently, the work deals with the alternatives to solving the above problem and analyzes the best option among these alternatives by means of efficiency.

Ways of solving problems (alternatives). In order to solve the lack of a systematic mechanism of program evaluation in the management of public finances and to ensure efficient management of budgetary resources, the work reviews several options and is based on cost- effectiveness analysis.

Alternative I - Establishment of a structural unit in each Ministry, which will be responsible for assessing programs / sub-programs of this budget organization.

This alternative provides the establishment of a special service or department for each Ministry responsible for assessing the preliminary, intermediate and final impact/effect of the programs/sub-programs of the spending institution. As the Ministries differ from the field of activity, specificity, goals and tasks, the above structural units will be staffed by the staff who are familiar with the activities of the institute, have great experience and organizational vision. Because the content of the program, the expected results and their impact on the relevant target groups depends on the specialized field, the program's evaluation by the professional will be much more effective, efficient and comprehensive than an unaware person in the field. At the same time, creating a program evaluation service in each ministry provides to assess and analyze a large number of programs / sub-programs that will facilitate relevant and effective decision-making in the process of planning and budgeting.

This alternative is often found in international practice. For example, in Australia, which has several decades of experience in the field of program budgeting and is considered a leading country in the successful creation and implementation of the result-oriented budgeting system, each ministry shall be responsible for assessing the existing programs and taking into account the obtained results for effective use of budget funds within its framework. The mentioned structural units in the ministries are observed Department of Finances subordinate to the Ministry of Finance, which is the central part of the process [4].

It should be noted that in parallel to the positive sides, the first alternative is characterized by several negative aspects. In particular, today the program assessment in Georgia is not at the level of ministries and the specialists of this sector are very few not only in the public sector, but throughout the country as well. Accordingly, in most cases there will be retraining of employees in the ministries and training of program evaluation techniques for them. In addition, creation of such structural units will be associated with large expenditures, moreover, the mentioned alternative considers to create Program Assessment Service in each Ministry (today, 10 Ministries in Georgia). The possibility of the implementation of such initiatives by the state and simultaneous implementation of the abovementioned reform in all ministries is very low. This alternative will be unproductive to solve the problem in a short period of time. It requires a great deal of time and resources, so it is considered for long periods. In addition, it should also be taken into consideration that the objective of evaluation of the program/sub-program from the Ministry may be challenged. In this case, the Ministry will always have an incentive to evaluate the results of its own implemented or implementing program/subprogram positively, moreover, if there is a risk that negative appraisals will have a reduction in assignments allocated to the Ministry at the programs/sub-programs with negative assessment. Consequently, if the first alternative is implemented, it will have great importance for the independence of the structural unit of program evaluation. However, the existence department subordination to the Ministry will significantly restrict its independence and objectivity.

Alternatives II - Creation of a centralized structural unit subordinating to the Ministry of Finance, which will be responsible for assessing the programs / sub-programs of the Ministries / Budgetary Organizations.

In case of implementation of this alternative, unlike previous alternatives, only one central structural unit will be created subordinating to the Ministry of Finance The creation of the unit under the Ministry of Finance is conditioned by the fact that the Ministry of Finance is a mediator between the government and the Parliament, as well as it has already been responsible for the consolidation of reports on, which's direct and direct obligation will be to evaluate budgetary organizations' programs / sub-programs. It should be taken into consideration that one service resource will be impossible to evaluate all program/sub-program. Accordingly, the programs to be evaluated by the structural unit should be selected from their importance, funding volume, priority or in the interests of the target groups. In this case, the Government, Parliament or other interested party may be initiated by the necessity of evaluating certain programs.

This alternative is implemented in practice in different countries. In particular, an independent structural unit is created in India, which's discretion includes assessing programs / sub-programs and determining their efficiency, benefits and success. This structure is an independent government agency that carries out its activities independently from all spending agencies and analyzes the evaluation of a program or sub-program based on the Planning Commission's request [4]. reform financial management fund

Creation of one centralized structural unit is associated with much less expenditures and it can be implemented in a short period of time rather than the establishment of many such services in each Ministry individually. In addition, this alternative enables the public sector to collect the relevant knowledge and experience in the evaluation techniques of the program and in the case of success, create a similar agency in each Ministry. However, it is noteworthy that in case of existence of one centralized structural unit, it will be impossible to evaluate all the programs/sub-programs of the Ministry, due to limited resources.

Alternative III - Hiring of private companies or international organizations by the Ministries (Budgetary Organizations) to assess their programs/sub-programs.

In case of implementation of this alternative, the Ministries will hire independent private companies or international organizations to evaluate their programs/subprograms. According to international practice, the budget organization can hire a private company that specializes in evaluating the programs in the sphere of field activities of the Ministry. However, as there are not private specialized companies in Georgia, this alternative does not provide for the private sector to assess the specialized programs/sub-programs separately.

Mostly, the mentioned alternative in international practice is often used in one phase of one or more of the programs, after implementation the first stage or even the end of the first year. By assessing the private company, the government is obliged to demonstrate usefulness and effectiveness of the program/sub-program, whether it has achieved its goals and results, whether the benefit received from the program was valuable. Finally, this assessment helps the government to decide how valuable and fair it is to continue / maintain this program in the future. For example, the countries are actively using the Operations Evaluation Department to assess the success (or failures) and effectiveness of their programs. Also, the International Food Policy Research Institute is often used to assess programs related to the healthcare sector [4].

Since almost any public agency lacks a systematic mechanism of programmatic evaluation and its creation depends on a long time, resource and effort, the third alternative can be considered as the fastest and easiest solution of the problem. Furthermore, it is noteworthy that the staff of private companies working on the evaluation of the programs are quite competent in this field, have knowledge and experience to evaluate programs, which does not have any representatives of the public agency, in which the structural unit of program evaluation should be set up in the first alternation. Furthermore, since the private company is an independent, external evaluator, it will not have any subjective interest in the achieved results of the program/sub-program and on the target groups in their evaluation process. Consequently, the conclusion presented by the private appraiser will be more objective and reliable than the evaluation of the structural unit subordinating to the budgetary organization. It should also be noted that implementation of the third alternative will contribute to the development of private sector and the number of companies oriented on the creation of new products (program evaluation).

In parallel to the positive sides, the fact is that the knowledge and experience of private companies in the program evaluation technique significantly exceeds public agencies, though they are not specialized in specific areas such as infrastructure projects, healthcare, education and so on. Consequently, the absence of specialized knowledge can create certain barriers to the process of evaluation of programs/subprograms.

Alternatives IV - "status quo", i.e. so called a zero alternative means to keep the current situation unchanged. This alternative provides for inactivity and unchanged position without any interference or policy/reform. Today there is no established, systematized mechanism in the Georgian public sector to evaluate programs/sub-programs within the framework of budgetary organizations. Additionally, there are many problems and shortcomings in the implementation of the program budget. Therefore, it can be said that in case of "status-quo alternatives", the problem in the nearest future will not be solved, but even the basic stage will not be established.

Methodology: Cost-effectiveness analysis. In this work, the cost-effectiveness method is used to select the best option among the presented alternatives. Analysis of cost-effectiveness allows for each alternative to determine the current state of improvement and the best alternatives to solve the problem based on mutual comparison.

Results. This method uses the spending ratio of expenditure for the implementation of each alternate with the average rate of its efficiency (in our case for 5 year period2). As a result, the lowest rate coefficient is the most optimal choice. In the process of analysis, firstly, the cost of each alternatives was presented for the 5-year period, which was subsequently discounted at a discount rate of 6 % (Table 1). A more detailed analysis of expenditures is presented as an annex for each alternative. The highest efficiency indicator (61) has been awarded to II Alternative from the three above discussed alternatives, because the Evaluation Service subordinating to the Ministry of Finance is the most optimal option for the state budget and the existing environmental conditions. III alternative, that effectiveness rate is 59, follows them. The lowest effectiveness rate (26) was awarded to I Alternative, as it is almost impossible to fulfill this stage due to the absence of qualified personnel and sufficient resources.

Finally, the lowest and most cost-effectiveness coefficient (53,634) of the alternatives is awarded to II alternatives (Table 1). As for the cost-effectiveness coefficients of I and III alternatives, the amount was

financial statements and budget execution of budgetary organizations.

2 The work discusses the problem solving in the short term. Consequently, the estimate of alternatives in the cost-effectiveness analysis process is calculated for a 5-year period.

calculated according to the level of 842,119 and 227,039 Finance (II Alternative) is the most optimal alternative to

units. This indicates that the creation of a centralized solve the problem in the work.

structural unit of programs assessment in the Ministry of

Table 1. Expense-effectiveness Analysis

Source: Author's calculation.

Recommendations. Despite the fact that the cost- effectiveness analysis of the second alternative has had the advantage, it does not mean that the state should not take care of the realization of alternatives and practice. The advantage of the second alternative was due to its short-term implementation and relatively less expenditure. At the same time, the creation of a centralized structural unit of program evaluation in the subordination of the Ministry of Finance will contribute to the development of the sphere in the public sector, acquiring experience, enhancement of knowledge and establishment of qualified personnel, which will create the basis for the creation of appropriate program evaluation departments in each Ministry.

As a result, it is recommended to take the following steps for an effective solution to the problem:

¦ In order to ensure effective, targeted and reasonable disposal of budget funds, first of all work should be started on the creation of a centralized structural unit of program evaluation subordinating to the Ministry of Finance;

As there is a lack of staff in the public sector, specialists (experts) should be invited and employed by the private and non-governmental sector from this field;

The above mentioned experts should conduct intensive training and civil servants should be trained for the purpose of enhancing qualifications and experience in the program evaluation;

The relevant authorities should take into consideration experience, best practices or textbooks of other countries in this regard, and start active work on the development of programmatic evaluation methodology.

In the nearest future the experimental assessment of programs/sub-programs should be conducted, as practical activity will facilitate more effective and timely introduction of the process.

Along with increasing staff qualities and accumulating experience, steps should be taken time to time to create relevant program evaluation service in each Ministry.

Table 2. Indicators of Efficiency

Source: Elaborated by the authors.

Movement from the budget system with traditional organizational structure to the program budget is an important reform of Georgia to improve the budgetary process of Georgia and ensure efficient management of the public financing system that facilitates the improvement of decisions about budget allocation and the effective planning process. Furthermore, the program budget increases the incentives of more efficiently allocated appropriations for spending institutions and provides information about participants of the budget process and less effective functioning programs. This allows to avoid ineffective, uneconomic and unreasonable expenditure of budgetary funds, so it is importance that the implementation of the program/sub-program evaluation mechanism presented by the program budget in the public sector. Consequently, through the analysis of cost-effectiveness, the most optimal alternative to solving this problem is creation of a centralized structural unit of program evaluation in the Ministry of Finance. Taking into consideration the existing resources and the short period of implementation, this alternative is the best option for the development of programmatic assessment culture in the public sector, development of this field, accumulating experience, enhancing knowledge and creating qualified staff. It should also be said that implementation of this alternative is only the beginning of the whole process, which will be the basis for the future development of appraisal departments in each Ministry and Budget Organization.

The basic budget policy framework system has already been established in Georgia but it requires further strengthening. Moreover, additional work on reforming is needed in the following spheres: Improvement of program budgeting - on program assessment mechanism, strengthening links between planning, budgeting and policy framework [7].

References

1. Outcome oriented budgeting - better practice manual // State Audit Office of Georgia, 2014.

2. Suvedi M. Conducting Program and Project Evaluations: A Primer For Natural Resource Program Managers in British Columbia / M. Suvedi, S. Morford // Forest Research Extension Partnership, 2003.

3. Treasury H.M. The Green Book: Appraisal and Evaluation in Central Government / H.M. Treasury. - London, 2003.

4. The World Bank. Public Sector Performance - The Critical Role of Evaluation. - Washington, D.C.,1998.

5. On Approval of Program Budget Methodology // Order No. 385 of the Minister of Finance of Georgia, 2011.

6. The Public Policy Paper Series includes policy documents developed within the Capacity-Building of the Georgian Leadership Community (CBGL) project // Georgian Foundation for Strategic and International Studies, 2015.

7. Oboladze D. Strategic Planning as a Necessary Prerequisite to Successful Program Budgeting in Georgia / D. Oboladze // Bulletin of Taras Shevchenko National University of Kyiv. Economics, 2016. - Vol. 6(183). - P. 26-32.

8. Barry H. Potter. Guidelines for Public Expenditure Management / Barry H. Potter and Jack Diamond // International Monetary Fund, 1999.

9. Simson Rebecca. A guide to public financial management literature for practitioners in developing countries / Rebecca Simson, Natasha Sharma & Imran Aziz // Overseas Development Institute, 2011.

10. Report on budget execution // The State Audit Office of Georgia, 2017.

Размещено на Allbest.ru