In spite of it, given research had a lot of pre-determined limitations, and the scope of studied school was small, so this article cannot be considered as reliable as far as it does not have enough scientific evidences. That is, it is not possible to build any conclusion based on this research about the national education. Moreover, her method does not take into account that schools have to be ranked with specific tool according to their success in different fields.

From the point of view of another group of Russian scientists (V. N. Averkin, O. M. Zaichenko, M. V. Alexandrova) [4], the term of efficiency itself in major part is just a common yardstick of how certain educational institution uses its financial resources. At the same time, efficiency interpreted as a ratio between resources spent to the incomes obtained. Their research does not focus on how to evaluate efficiency, because some groups of inputs parameters or resulting variables may have different contributions to the result, and may also be, for example, not only quantitative, but also qualitative. In this case, the probability of their incorrect use in the model for calculating the efficiency grade is very high.

Combining the previously described models, there is merit in refer to an example of what indicators are highlighted to derive the final aggregated assessment of the quality of education that an educational institution can provide, by the Ministry of Education and Science of Russia in its Order [50] from 2012. In total, 50 indicators are used that relate to 5 significant types of university activities:

1. Education

2. Scientific research

3. International

4. Financial

5. Infrastructure

Among these parameters the average score of Unified State Exam (USE, EGE) of various groups of applicants was estimated, the number of studies per employee, the share of foreign students, the available laboratories space per student, the percentage of candidates and doctors of science in the teaching staff and so on. Thus, in general, these indicators are largely representative for the evaluated sphere, however, they still do not take into account the peculiarities of the “strengths” of a particular university.

For instance, technical or natural-science universities have larger laboratory areas, but at the same time, basic research, and especially discoveries, in these fields require a large amount of financial resources, as well as time. In contrast, humanitarian studies practically do not require the provision of laboratories, and they are carried out in a much shorter time. This means that according to some indicators, it will be better to have an educational institution in the field of technical or natural sciences, and on the other - humanitarian. The introduction of uniform minimum requirements for these universities also leads to a shift in the attention of universities from the main activity to the solution of bureaucratic issues of compliance with the formal characteristics of quality.

In addition, universities that do not meet the minimum requirements set by Order № 583 are considered inefficient and cannot be rated as high quality. In turn, this reduces its state funding and reduces the attractiveness of the school among applicants and deprives it of students with high potential, which leads to further deterioration in performance. Thus, despite the primary objectivity of this method of assessing the efficiency of universities, it can be said that it is also not without flaws.

Therefore, it worth to turn to the experience of foreign studies. Often, the assessment of efficiency in the field of healthcare and education is carried out using similar methods or even jointly, that is, within the framework of one study. A special feature of foreign education in many countries of Europe and North America is the fact that, along with state ones, there are also a large number of private educational institutions at all levels of education. In their case, the issue of performance gains the importance of the reputation factor.

In a research by P. O'Brien [37] on the topic of improving the quality of the educational process in France, by developing a system of motivation, he conduct an analysis of the dependence of the efficiency of student learning on not only the characteristics of the institution. The author of the article notes the strong dependence of the quality of the achieved education on the social origin of the student, e.g. an employment status of his parents. The efficiency of education also depends on the motivation of students. As for quantitative criteria of the efficiency it could be taken the proportion of graduates who got the university degree among all enrolled students.

The approach of this author does not consider how deeply and effectively students were able to master the educational program. And, surely, this approach does not consider the quality of how the educational institution functions in order to accomplish its main objective, but it expands the scope of further research, forcing the authors to also include additional characteristics of database elements in a simple sample. This, of course, allows to improve the quality of analysis and reduce the proportion of subjective and erroneous judgments.

Thus, despite the crucial importance of assessing the efficiency of educational institutions, both nationally and domestically, it was found that many of the approaches used have their own weaknesses and often use a number of assumptions that can distort the real situation, as well as to influence the activities of educational institutions in the future. The difficulty of developing an objective and representative approach to the evaluation of an educational institution is explained by the fact that any school or university is characterized by a multitude of incoming and resulting factors, as well as internal properties that form a unique internal environment. Educational institutions are multicriteria systems, so bringing their parameters to a single standardized scale for evaluating performance is impossible.

Through the described research examples, it became possible to determine the crucial issues, which cause the greatest common difficulties in the process of evaluating the efficiency of educational institutions to their ranking:

• Method capacity to operate with both quantitative and qualitative variables;

• Assessment the process regarding the pre-determined conditions of the system instead of the result;

• Subjectivity of evaluation of qualitative factors and comparative analysis performance;

• Selection of the weights of the resulting factors, that is, the assessment of their contributions to performance.

There are exist a lot of approaches to assess multicriteria systems. The most common among them are ignoring several factors, equalizing the weights of criteria in the model, or expertly determining the weight of the contribution of each of the factors to the resulting performance or efficiency. Moreover, in this case full expert assessment can be applied without isolating the quality components of the evaluated system. However, all these methods have obvious shortcomings in objectivity. When evaluating the efficiency using one method, the results can be directly opposite on the other scales. Thus, the evaluation of multicriteria elements does not reach its main goal - it is not reliable enough to build any judgments on its basis.

This circumstance follows logically from the very property of multicriteria systems: some elements are more successful in one dimension, while others succeed in another. The classic examples of multicriteria systems are banks, institutions in the field of education and health care, government agencies, branches of companies, and so on. As a rule, the assessment of the efficiency of such elements become even more complicated by the presence of qualitative criteria.

2.2 Using the DEA method to evaluate the efficiency of educational institutions

There are a numerous study evaluating the efficiency of universities with DEA method as one that allows to eliminate or significantly reduce the listed difficulties in assessment multicriteria systems.

Any independent decision-making unit (DMU) of a multicriteria system has the ability to convert in the course of its activities "inputs", that is, the resources and parameters of the system at the beginning into "outputs", any new parameters and meaningful results. To evaluate the efficiency of this DMU means to evaluate the efficiency of its conversion performance from “inputs” into “outputs”.

The DEA approach bases on the classical understanding of the efficiency, which was laid down in the theory of economics and management in 1906 by U. Pareto [17]. In this case, the efficiency (effectiveness, productivity) of unit's performance interpreted as an achievement of the maximum possible results in the conditions of pre-determined available resources for DMU so that it does not impair any of the resources or results.

From there, with the existing set of system elements and a sufficient amount of data on their properties, it is possible to develop based on the proposed model a list of the most efficient units that, according to the principles of Pareto-efficiency, will lie in the Pareto-optimal area. It does not require setting criteria weights, each of the DMU, having its own characteristics for each of the important factors, is arranged in a certain way in the N-dimensional coordinate system (where N is the number of criteria) and according to its location can be considered efficient (Pareto-optimal) or not. If the studied DMU is not lying on the efficiency frontier, then for that element DEA selects personal set of the most efficient similar units, which could be examples for it to follow, that is, they will become objects for benchmarking with best practices.

The described DEA approach for estimating parameters of multicriteria systems has several obvious advantages when using elements that contain multiple “input” and “output” parameters. As a result, any of the units of the system could be assessed by its performance efficiency within the selected criteria. In addition, it can be developed a list of benchmarking units among the system that could make the activity of the whole system more balanced and efficient. For the financing issue of the elements of the system, the DEA solutions give a clearer understanding of the current state of the system and, with proper interpretation, could in fact be a reliable analytical tool to support the decision-making process.

Due to the high practical applicability of the DEA in the field of education, there is a wide base of representative studies on the evaluation of the efficiency of schools and universities using the DEA method, which is a reliable experience of the relevant application of this method in this field.

The data obtained as a result of the application of the method can be used in several practical spheres. On the one hand, this may form the basis of the rating, since the degree of compliance with the efficiency criterion could be used as the basis for ranking the elements. On the other hand, it is possible to create various pie charts, reflecting the distribution of the main directions of improving the efficiency of the elements. This tool allows to identify the most significant shortcomings in the management of the whole system, as well as to develop ways for their correction. In addition, according to the results of the analysis of the data set, it is possible to distribute the DMU into clusters (groups) in accordance with the specific features owned by a group, and also on their basis develop individual approaches to management and increase efficiency. However, the existing database of applications of the described method, as well as the studies described earlier, is not without flaws.

A. Boussofiane describes [12] how to apply the DEA by the example of the process of selecting “inputs” and “outputs” for analyzing schools from a theoretical point of view. According to the author of the article, the creation of a complete picture of all areas of the school as “inputs” are provided by such indicators as the number of teachers, the quality of applicants entering the school, the social status of parents, as well as the funds allocated for teaching materials. “Outputs” for this model, the author identified the following: the number of students who have passed the GCSE (national exam in the UK, conducted according to the results of secondary school education), the number of students who have passed this exam with “excellent grade, the success of students in sports, music, and also the employment of graduates.

This list of parameters, of course, characterizes several areas of school activity and includes both quantitative and qualitative factors, as well as environmental factors, such as the social status of the student's family. But at the same time, the question of how these indicators should be expressed and characterized remains uncovered.

In turn, D. Sutherland (a member of the Organization for Economic Cooperation and Development) with his colleagues in their article [43] in 2007, devoted to calculating the essential indicators of the success of state activities in the field of financing primary and secondary education, understands “education efficiency” not as an individual quality of work of each individual educational institution, but believes, that the efficiency of education is possible, and even necessary, to be judged precisely from the perspective of how well educational institutions are financed.

The research was built on the basis of a multi-criteria model that considered two types of “input” variables: directly controlled by the educational system and indirect (environmental). The first type of variables included: the ratio of the teaching staff per 100 students, teacher qualifications, class sizes, number of teachers, and so on. With such initial variables, the “outputs” are the attendance of courses, the duration of education, the level of knowledge achieved by students, both in terms of knowledge homogeneity and the average score of the PISA test. The main source of information for the study was the database of the international PISA test (from 2003).

In this case, the assessment of the cost-efficiency of the educational services provided in the country was based on the amount of expenditures on education for the period, deflated by the price change index. These costs, in turn, determine the indicators of technical equipment of students, as well as the motivation of teachers. External factors were also included into the model. The social status of students was identified through such factors as the welfare of the family, the immigrant status and the language spoken at home.

However, the approach used to assess efficiency through the definition of costs does not provide an accuracy in estimation the value of educational services. In this context, it would be more appropriate to use data such as, for example, study hours provided to students and teachers' salaries.

The strength of that work is the assumption of statistical noise. Since this method is very sensitive to measurement errors, while interpreting the results of inefficiency data, a certain part of units, which according to DEA did not lie on the efficiency frontier, but was close enough to it, was also classified as efficient.

The DEA method was applied in 2008 [34] for assessment the efficiency of government activity to improve the quality of education and health services in Chile. For education, the results of the national exam among students in Chile were also used as the main output, and the “inputs” were described as external (or environmental) factors, such as the socio-economic background of students, and financial and technical parameters: the number of teachers, the state expenditures of education per student. To isolate the influence of the technological efficiency factor, a special BCC modification of the DEA method was applied.

As a result, a comparison of Chile's units from different ownership groups: municipal, partially owned by the state and completely private, showed that there was a significant gap in the efficiency of teaching among them, which revealed a heterogeneity in the quality of education in the country.

State policies on educational system are also evaluated using the described method in Belgium [18]. The division of educational institutions into regions, as well as the granted autonomy to them in matters of personnel, funding and teaching methods, distinguish this system from other countries. For example, the DEA method was used on the following data groups: not only characteristics of students were used as “input”, but the quality of management, that is, the list and number of areas of responsibility of chairmen, management independence, number of educational institutions in the district, share of full-time professors, the equivalent of full employment, education of students' parents. That is, as “inputs”, researchers estimate the full range of indicators that are accountable for independent monitoring, as well as characteristics of students.

To study the possibilities of improving the educational activities of the OECD [52] countries, the “inputs” were: the number of teachers per 100 students, the number of computers per 100 students, the social status of students, their language experience and accessibility to educational resources at home. This research provides important additions to the experience gained through the analysis of the efficiency of educational institutions using the DEA method and includes in the list of “inputs” such parameters as the technical equipment of students, expanding the number of sources of getting knowledge, which is an integral part of the modern educational process. "Output" is presented in the form of the average performance of each individual educational institution in the national test in literacy, mathematics and natural science, that is, those scientific areas that are elements of compulsory education.

A wide range of examples of use DEA to assess university efficiency were studied by Russian researchers I. Abankina, F. Aleskerov, V. Belousova, K. Zinkovsky and V. Petrushchenko in 2013 [2]. As an outcome, they showed how this model were used in different countries and, more importantly, from which perspective researches evaluated efficiency. Due to a fact that university has at least two major outcome dimensions as teaching and research activity, it can be concluded that a university efficiency, especially in a case of comparison, usually measured by its capacity to convert given (constant) income to a certain outcome. Thus, most of articles on this topic were built on an output-oriented models. Furthermore, many studies were aimed to indicate the most important factor which impacts an overall university efficiency the most.

For instance, J. Johnes from UK [29] assessed 109 local educational institutions trying to indicate dependence of its efficiency on the status (university or college) expressed through absence or presence of their specialization. Author divided the whole cohort on three groups in order to eliminate an impact of the status on the cohort efficiency and to determine whether there is any significant variation between groups in terms of efficiency. Noteworthy, that she used aggregated variables such as quality of students and quality of graduates. Both variables were basically the number of students or graduates adjusted for their academic performance.

In 2015, using the dynamic DEA method [19], which is called the “black box”, the change over time in 2011-2013 was estimated [45] for the performance of municipal colleges in Vietnam (Tran C.-D. T. T., Villano R. A), which relate to a higher education level. Efficiency in this study was implied in the aspect of both academic and financial activities, which is a logical consequence of the characteristics of the field of education. Obviously, academic efficiency meets the main objectives of higher education institutions to increase the value of students as knowledge holders. At the same time, economic efficiency reflects the quality of the financing policy pursued by the state as one of the main stakeholders of municipal educational institutions. Among other things, based on the available data, the researchers evaluated the efficiency of the colleges located in the city in comparison with the others.

The number of teaching personnel, as well as employees not engaged in teaching activities, but belonging to the financial and educational divisions, the total payment of tuition fees, government funding, operating costs, research revenues, and laboratories, which echoes the vision of the Ministry of Education of the Russian Federation. The “outputs” in this study were the number of current full-time students, as well as graduates. It should be noted that in this context the quality and depth of knowledge on any of the available systems was not assessed in any way. This leads to the fact that the academic component of the activities of universities is not fully disclosed.

In Germany, 73 state universities were evaluated [47] by S. Warning on a basis of their specialization with a purpose of identification of strategic groups. Teaching and research performance quality has also been taken into account. Such formulation of the research problem revealed which areas of educational system are in demand to be improved and corrected. It is remarkably that only costs (mainly labor) were used as an input parameter of the model, while outputs has been changed from one model to another.

The DEA method was used by a group of these Russian scientists in 2013 [1] (F. Aliskerov and his colleagues) in evaluating the performance of Russian universities. In the course of this study, several tools were used, including the described method, which made it possible to identify the degree of efficiency of universities. As criteria, as the practice of analysis of previous researches shows, the following groups of characteristics were singled out:

· Parameters of educational activities

· Parameters of scientific activity

· Institutional parameters

The results of the application of the methodology for the evaluation of multi-criteria DEA systems provide useful information that can be applied in the future in a very wide range of management decisions. First, this method allows managers to identify the most and least efficient units of the system. Second, it reveals the development potential for each DMU, showing the existing gaps in the optimal and real values of the “inputs” and “outputs”. Thus, on the basis of this information, a decision maker can agjust his policy within the system, distributing raw materials, as well as financial, informational and other resources to improve the position of each element, even those that are the most successful.

As noted earlier, for inefficient units, the elements that are the best practices of efficiency can be easily recognized in the model. So, for each inefficient DMU there is a certain list of reference units that are its benchmarks. Based on the reference units, as well as the significance of the contribution of their background to the potential development of an inefficient element, a composite DMU is obtained, one to which this element can aspire as a result and what it can realistically achieve. Thus, the practice of planning the development of elements occurs not only with the help of heuristic approaches and on the basis of the manager's professional judgment, but it could be also numerical decision in the direction of development.

Compilation the methodologies of already existing educational institutions efficiency researches based on DEA method helped to extract five main groups of factors correspond to five main streams of university assessment highlighted by a Russian government.

Table 5

The most frequently used factors to assess educational institutions with DEA by groups

Group	Input	Output
Education	Number of professors per 100 students Share of regular lecturers Quality (degree, experience) of professors Number of teaching hours Teachers' remuneration Knowledge level of entrants Ability to learn	Number of students successfully passed national exam (PISA, GCSE, USE) GPA of graduates Course attendance Graduates to entrant ratio
Scientific research	Classroom size Laboratory size Student's equipment	Number of scientific papers made by professors and students SCI (Science Citation Index)
Extracurricular activities	Number of students in total CAS (Creativity, Activity, Service [57]) activities Administrative autonomy	Students' achievements in sport, music etc. Self-administration
Finance	Public funding in total and per student	Financial efficiency
Infrastructure	Social status of parents Parental educational level Access to educational sources outside university Immigration status	Graduates' employment rate

Summarizing all the above, it should be noted that as a “input” variable, studies regularly used indicators of equipment of educational institutions with teaching staff and technical resources (computers, laboratories). Among the “inputs”, a special place was also given to the qualitative characteristics of students while entering an educational institution, for example, language experience, quality of previous education, social status and education of parents. The sample was also supplemented by such important factors as the availability of educational resources outside the university, the amount of state funding and tuition fees, if applicable.

One of the significant shortcomings inherent in almost all models was the exclusion of extracurricular activities from the range of factors reflecting efficiency. The activity of educational institutions is formed not only from the educational achievements of students, but also due to the contribution of success in other areas, the development of which is also an objective of the educational infrastructure. Examples of these areas are sports, music, participation in debates, case studies, and so on. The actual measure of the quality of education is employment or the possibility of admission to prestigious universities for continuing education, the use of which has also not been identified in practice.

Thus, DEA method is a common tool to assess the efficiency of the elements of multicriteria systems, an example of which is the education system, without neglecting some of the “input” or “output” factors and without using expert weighting methods for the determination of the significance of these factors. The method allows not only to evaluate how efficient universities are, but also by means of inclusion different variables, it allows to study the factors that determine the efficiency of the elements of a system.

Using the example of a retrospective review of current research over the time period from 1976 to 2019, the practical significance of this method for the educational environment was shown.

3. Examination of the dependence of the efficiency of the university on its scale

3.1 Research method and data collection

For the research purpose, the two-stage method was applied, which consisted of two modifications of DEA method and further regression analysis. In the first instance, it was taken CCR and SBM modifications of DEA and tested the coherence between the results. In both cases output-oriented model was used as far as it reflects the main objective of the study and the nature of the university. That is to say, university capacity to convert given inputs to an output was studied, and within the framework of determined concept of the educational system efficiency, the main outcome of university describes through its outputs.

Then, the linear regression between university scale and obtained model results was evaluated in order to investigate an existence of their dependence and its significance. In order to eliminate artificial improvement of the results, DMUs were tested at the CRS level, which means that their scale had no impact on the efficiency results within the DEA model.

A. Charnes, W. Cooper and E. Rhodes [16] developed a DEA method as an approach to solve problems of such complexity in 1978. They first described a method that would allow relatively simple and fairly objective to assess the activities of similar independent elements of a multicriteria system with multiple “inputs” and “outputs” based on large databases, but it is still widely used in order to educational systems. This methodology of this approach was named after its inventor's surnames - CCR.

In 1993, studying the advantages and disadvantages of DEA method, the Danish scientists [3] noted that with small amounts of “inputs” and “outputs” compared with the number of observations, an artificial improvement of elements in the model can occur. So, if there is only one input or output, the DMU may be among the efficient ones, although these factors may not play any significant role in the area under study at all. For a sufficiently reliable study, it is necessary to use such a number of observations that will be equal to the product of multiplication the number of “inputs” to the number of “outputs”. It is also very popular opinion that the number of DMUs must not exceed the squared number of factors in total.

The mathematical model of the method is based on the following postulates:

· n is the number of estimated DMUs;

· m is the number of “inputs”;

· s - the number of "outputs";

· x_ij is the number of “inputs” i that DMU_j uses;

· y_rj is the number of "outputs" r, which as a result of the activity receives DMUj;

· x_ij and y_rj ? 0 ? DMU_j;

· v_i, u_r - weights of the variables “input” and “output”;

· Each DMU has at least one positive “input” and “output”.

In general, the problem solved by the mathematical model is a ratio between a multicriteria output to a multicriteria input. Efficiency is understood, in this case, precisely by each unit's performance in the specified conditions framework. Thus, the mathematical model, with the help of which the efficiency is measured in the DEA method, is:

With the following limitations:

Thus, estimation of an each DMUs efficiency has come down to solving the equation of maximizing the use of current resources and factors for obtaining meaningful results. Comparison of the efficiency of units occurs by solving the presented problem for each DMU separately, and for each of them the weights will be individual. It also has an infinite number of solutions if variables (u*, v*) are optimal, since multiplying them by any non-zero coefficient will also lead to an optimal solution. Therefore, in order to simplify the formula, it is possible to bring it in a linear form so that in the future linear programming tools can be used. In particular, for the pair (u, v) there exists a solution in which the denominator of the mathematical model is 1, and u is transformed into µ, then the equation takes the following form:

With the following limitations:

Such a formulation of the problem leads to the possibility of using a dual problem, which is distinguished by higher productivity in terms of the speed of finding solutions, and is also more reliable, since it always has solutions. The direct dual linear problem is called the Farrell model [20].

But the fact that the system element is the most efficient among those presented does not mean that it uses its “inputs” so optimally that it is completely efficient. In other words, some efficient units within the existing DMU, in essence, as well as inefficient ones, have non-zero reserves for development. According to many economists, the Farrell model does not fully satisfy the conditions of objectivity and ignores the presence of these non-zero slacks of DMUs. That is why that model is also called the weak-performance model.

Then scientists formulate an understanding about existence of s_i⁺ and s_r^- gaps, the absence of which identifies the efficiency of the evaluated elements of multicriteria system. Efficiency in terms of DEA implies that DMU performance can be named fully efficient if and only if и* = 1 and s_r^+* = s_i^-* = 0. Thus, the mathematical model of the dual problem in the CCR model is:

With the following limitations:

Constant л is a value that allows to create an ideal fully efficient unit, similar in “inputs” or “outputs” as an estimated one DMU, but more efficient. It also helps to evaluate the unit's comparative efficiency not in comparison with existing examples, but in comparison with itself. In this case, the indicator и can take values from 0 (complete inefficiency) to 1 (total efficiency) and is an estimate of the desired unit efficiency [44]. This approach allows system elements to retain their unique features, which, perhaps with the right approach from a management perspective, can not only increase the efficiency of this element, but also lead to a synergistic effect of their use, that is, to a significant increase in productivity.

Several researches have also made their own corrections in the model. K. Tone paid more attention to the fact that the significance of the lack of reserves for improvement is much higher for the efficiency of the element, rather than just a comparative superiority over other elements of the system. This model is called SBM (Slack-based Measure). In his article, he conducts an analytical comparison of the results of the CCR and SBM models, by the example of which he shows that in the case when absolute efficiency is especially important for the objectives of the task, his method gives more accurate and detailed results.

Despite the described advantages of the model, it has a significant limitation, since it does not allow analyzing elements with zero values by at least one of the parameters. Thus, to use this modification, it is required to exclude from the sample elements that have zero values for the selected factors.

As a result, the choice was made to use be models: traditional CCR and modified SBM, since they would make it possible to obtain a more objective understanding of the problem.

In addition to the application that was shown earlier, the DEA method can also be used to estimate the return-to-scale effect and pure technical efficiency, if it is supplemented with the following condition:

This modification of the original model has been developed later in 1984 and is called BCC [6]. This model allows to estimate how inefficient the unit is due to its current performance. That is, the model can consider not only the change in the factors themselves, but also implies some assessment of the element relative to its prospects for development and viability, clearing the unit being analyzed from the influence of technical efficiency. To a great extent, this is useful for units of different scale, since current efficiency sometimes cannot withstand increases, but it is also not always necessary to scale up work and resources in any direction.

This modification allows to analyze the scale efficiency of the unit due to the assumption of variable returns to scale (VRS). Traditionally, the model works under the assumption that the return on scale is constant (CRS) and that with an increase in scale, the other components of the unit will proportionally increase. Since in the framework of this work, the dependence of the efficiency of the university on the scale is studied, in order to avoid the artificial improvement of the results of the model, it is essential to use a model with a constant return on scale. Otherwise, the model will initially include an impact of the scale of the university in assessing its efficiency, leading to an increase of the statistical significance of scale variable in the subsequent regression analysis.

In case if a researcher begins to solve a problem from “inputs”, that is, evaluates the optimality of “outputs”, assuming the condition that incoming variables and resources are in some way predetermined constants for which DMUs are not can influence, it means that the model used for the research purpose an output-oriented. However, there is another situation when the goal of evaluating the efficiency of “inputs” is set as the goal of evaluating the efficiency of multicriteria systems. Since the first model is called output-oriented, the second, respectively, got its name in a similar way and is called input-oriented. In general, these models are similar to each other, the main difference between them lies precisely in what group of factors the researcher focuses on and the operation of the mathematical model and analytical auxiliary apparatus. At the same time, some software tools allow to assess non-oriented models, in case there is no certainty or difference in which of the groups of factors is more priority or critical for increasing efficiency. Within the framework of this work, the nature of the objects of the subsequent study implies the use of an output-oriented method.

To perform model calculations, was used the free customization for Microsoft Access named MaxDEA Basic. The main advantage of this program is that it has to limit on the number of DMUs assessed as much as of factors used. It also has almost all DEA modifications, among which, definitely, chosen CCR and SBM output-oriented CRS models.

Hereafter, an add-in for data analysis in Microsoft Excel was used, which allowed to build linear regression models, within the framework of which the statistical significance of the university scale factor was tested to explain its efficiency resulting from DEA model.

The data were taken from the open online source of national Monitoring of the efficiency of higher education system in Russia in 2017 [63] since by the time when research was performed it was the most up to date data. In spite of the source provided the ranking of the performance efficiency made by the source itself, it was decided to use only primary data in order to acquire better objectivity of the model.

The Monitoring source contained data on 1314 educational institutions in Russia, among which were universities, their local branches, colleges, institutes. Furthermore, the sample involved both profit and nonprofit universities as much as state-owned and private educational intuitions. Thus, the sample was heterogeneous and, consequently, it could mislead to an influence on the efficiency rather than just scale.

In order to avoid it, of all the educational institutions were selected only those that meet the following characteristics:

· Universities, as the investigation is made on the basis of assessment of universities' performance

· Made for profit, due to the crucial importance of the financial efficiency in the overall efficiency of universities

· State-owned

· Main, parental unit, because their autonomy is extremely valuable for their possibility to manage their performance

Therefore, the homogeneity of the sample was achieved and from 1314 educational institutions were selected 361 state-owned commercial universities. After selecting the appropriate elements to run the model, each university was randomly assigned with codes as follows: “DMU” and a three-digit sequence number. This requirement was dictated by the program, which subsequently read this encoding. All codes are attached and can be found in the Appendix.

During the data collection process, it was identified that DMU215 and DMU358 has to be excluded due to a lack of data, so that it is impossible to analyze them. Accordingly, the data about 359 Russian universities were collected, among which were also those with zero exits at certain factors. In order to use SBM model, and to the overall possibility for the system to detect inefficient DMUs compare to others, it is critical to remove all zero values, either by removing elements or parameters.

The original dataset had 121 parameters. Among these parameters were also those that characterized the scale of a university, such as the total number of students, as well as the number of students in full-time and part-time courses. These factors were applied in the regression, and therefore for the purposes of the study were excluded from the list of factors affecting the efficiency of a university within DEA. Factors that overlap or are similar in value or meaning were then removed from the model. For example, " Income of the university from educational activities from foreign sources" and "Amount of funds from educational activities received by the educational organization from foreign citizens and foreign legal entities" were excluded. Further, factors that are not significant from the point of view of the study were excluded. Such factors include the "Area of indoor sports facilities". As a result of the selection, 34 factors were selected, among which 20 inputs and 14 outputs were identified. All of them are listed in the Appendix section.

After reducing the list of factors, there were still several outstanding issues, such as:

· According to the requirements of the program and, in particular, SBM modification, the database uploaded for analysis should not contain zero values, while the system had 11% of zero outputs.

· The number of factors squared exceeded the number of observations, which, according to the practical experience of previous researches, could lead to a decrease in the efficiency of the model itself.

As a result, within the framework of the obtained model, it turned out that only 17 elements out of 359 were inefficient, which represented only 4.7% of the scope. Slack-based model, at the same time, could not be started due to program limitations.

A number of tests were carried out to overcome the existing data shortcomings. Simultaneously were performed couple of mathematical tests with the professional judgement of the researcher.

To analyze the significance of the factor, the proportion of zero values for this factor in the total number of universities was calculated. Thus, the factors for which that ratio was relatively high, and universities did not show any results were in a group for subsequent exclusion in order to reduce the number of factors that decrease the quality of the model. In addition, universities have been tested in the similar way. Those that showed the lowest performance were also among those that should have been excluded from the model.

Furthermore, factors variation was investigated for the purpose of distinguishing those that are not significant in terms of their contribution to the evaluation of the efficiency of some elements in comparison with others. The factors with least variation are likely to have the minor impact on whether an element will be efficient or not based on the results of the model, so that they could be omitted from the data without significant quality loss.

In addition, some factors affect each other in one way or another, so a correlation analysis of the factors was performed. Thus, for instance, factors that are highly correlated with the others should be eliminated from the model, as they are explained by other features, and can only lead to a deterioration in the quality of the model through artificial impact. The values of the correlation test are given in the Appendix section. The result of this test was the number of factors correlated with the given one by more than 0.3.

A compilation of all the results of mathematical tests with the author's judgement is shown in Table 6:



Table 6

Results of factors exclusion tests

Factor code	Zero values test, %	Variation test, %	Correlation test	Judgement
N1	68	581	6	Can be omitted
N2	0	93	5	Better not to exclude
N3	0	12	15	Better not to exclude
N4	9	141	2	Can be omitted
N5	3	338	8	Better not to exclude
N6	1	239	14	Better not to exclude
N7	1	91	12	Better not to exclude
N8	1	136	13	Can be omitted
N9	50	722	0	Can be omitted
N10	12	76	0	Better not to exclude
N11	3	115	7	Can be omitted
N12	10	134	0	Better not to exclude
N13	0	65	0	Better not to exclude
N14	40	155	4	Can be omitted
N15	20	152	8	Can be omitted
N16	3	106	1	Better not to exclude
N17	36	194	2	Can be omitted
N18	39	197	10	Can be omitted
N19	0	92	2	Better not to exclude
N20	63	298	11	Can be omitted
N21	14	321	12	Can be omitted
N22	0	49	2	Better not to exclude
N23	0	48	2	Better not to exclude
N24	1	56	0	Better not to exclude
N25	0	57	0	Better not to exclude
N26	2	21	0	Better not to exclude
N27	0	64	6	Better not to exclude
N28	0	70	13	Better not to exclude
N29	0	58	7	Better not to exclude
N30	0	14	5	Can be omitted
N31	0	10	5	Can be omitted
N32	0	27	1	Better not to exclude
N33	0	42	9	Better not to exclude
N34	6	65	8	Can be omitted

Therefore, as a result, 18 the most impactful and meaningful for the model factors were selected. Among them were represented all from five distinguished key dimensions of the university.

Education

· The average national exam score of students admitted to study under the bachelor and specialist programs

· The number of postgraduate students, residents, assistant trainees of an educational organization per 100 students

· The number of research and development staff with the degree of candidate and doctor of sciences, per 100 students

· The proportion of professors under the age of 40 years

· The average salary of teaching staff

Scientific research

· The number of citations of publications published over the past 5 years, indexed in the information and analytical system of scientific citation Scopus per 100 scientific workers

· The total amount of research and development work

· The share of R&D income in the total income of the educational organization

· The total number of organization publications per 100 scientific workers

Extracurricular activities

· The proportion of the number of foreign students obtained bachelor, specialist, master's degrees, in the total number of students

· The share of foreign students in the total number of students studying bachelor, specialty, master's programs

Finance

· Educational organization revenues from funds from income-generating activities per scientific worker

· Educational organization income from all sources based on the number of students

Infrastructure

· The total area of the educational and laboratory facilities per student

· The number of personal computers per student

· The proportion of the cost of machinery and equipment (not older than 5 years) in the total cost of machinery and equipment

· The number of copies of printed educational publications from the total number of library fund storage units that are registered, per student

· Percentage of graduates employed during the calendar year following the year of graduation in the total number of graduates

The selected indicators are able to sufficiently reveal the degree of tools and infrastructure equipment of the educational and research processes for the successful development of both types of activities. In addition, the indicators characterizing the outputs show the degree of success of the university in terms of both the educational and the scientific aspects. The international activities of universities also have a significant impact on the reputation and further development of an educational institution.

Despite the selection, there were still several universities with zero values for critical indicators of their efficiency. For the purpose of this work, it was decided to exclude them from the sample as predeterminedly inefficient and potentially capable of distorting the results of the model.

Thus, the model ready for launch contained 18 factors, 11 of which characterized the inputs, and 7 were the outputs. 327 Russian state-owned commercial universities were examined using two modifications of the DEA model - traditional CCR and modified SBM models, both were output-oriented and launched under conditions of constant returns to scale (CRS) in order to exclude the impact on the university scale.



3.2 Results and discussion

Based on the selection of the research method, data collection and selection, the results were obtained for two models. Of the 327 elements in both cases, 128 were identified as inefficient. The total number of students was used as an indicator of the scale of the university. A similar approach has already been used previously by other researchers [5, 10, 15, 24] who have studied the issue of the influence of the scale of the university on its efficiency.

Both methods have identified 128 inefficient universities. The efficiency ratio of these universities, depending on their numbers, is shown in Exhibit 3, Exhibit 4 and Exhibit 5. A strong positive linear correlation by 78,91% between the results of the model is observed. Despite this, the results of the CCR model are more shifted towards total efficiency, since they show the efficiency of the units relative to the sample, while the SBM model estimates the full efficiency of the elements, including their development potential outside the sample. Therefore, the indicators of the second model are more biased towards 0, and some elements are almost completely inefficient (Exhibit 3).

Exhibit 3 Comparison of inefficient elements of CCR and SBM models

The main difference between graphs (Exhibits 4 and 5) is explained by the main principle underlying the slack-based method (SBM). CCR method only measures the relative efficiency of the units within the sample, while SBM computes the potential efficiency of DMUs, projecting it to a total efficiency frontier.

Exhibit 4 Distribution of the efficiency of universities according to the CCR model depending on their scale

Exhibit 5 Distribution of the efficiency of universities according to the SBM model depending on their scale

There is a noticeable tendency on the graph that the most inefficient universities are the smallest in the number of students they teach. To test the hypothesis about the influence of the scale of the university on its efficiency, a linear regression was constructed. The explained factor in this case was the DEA model results. The number of students was an explanatory variable.

...