The Dunning–Kruger Effect in Knowledge Management Examination of BSc Level Business Students

. The Dunning-Kruger effect (DKE) in higher education evaluation is one of the current research areas of psychology, educational science, and management science (in our case). Its importance is that the less prepared one is, the less accurately one can judge what performance is expected of him. What is more, he will err more and will overestimate himself. The present study aims better to understand the phenomenon with new, small-sample empirical results. The study is part of a research series that has been ongoing at the University of Debrecen since 2015. It not only quantitatively expands the literature but also includes the course of Knowledge Management among those examined. During the research, students were asked both before the examination (N = 63) and after the examination (N = 76) to guess how many points they would achieve on a multiple-choice test. It supports the presence of DKE, both in the case of pre-examination and post-examination self-evaluations. Using four multivariate linear regression models, we examined whether the sign value or absolute value of the errors made during the guesses show a correlation - in addition to the available control variables - with the test score. Our results showed that the more accurate the pre-examination and post-examination estimations were, the higher the students' actual score was, while the less they tended to overestimate their preparation. This supports the presence of DKE, both in the case of pre-exam and post-exam self-evaluation


Introduction
The phenomenon our research is focused on is named after David Dunning and Justin Kruger, two coauthors who won the IgNoble prize in 2000 in the category of psychology (Abrahams, 2022) for their "modes report" (Kruger and Dunning, 1999). The Dunning-Kruger effect (from now on, DKE) has become popular both in science as well as in popular culture: there are an increasing number of studies in international journals (the simple search command "TITLE-ABS-KEY (dunning-kruger)" resulted in 162 records in Scopus on 18/11/2022), it has a Wikipedia page (Wikipedia, 2022), a dedicated song in the 'Incompetence Opera' (Érdi, 2019), and so on. David Dunning defines this phenomenon as follows: "People with substantial deficits in their knowledge or expertise should not be able to recognize those deficits. Despite potentially making error after error, they should think they are doing just fine. In short, those who are incompetent, for lack of a better term, should have little insight into their incompetence -an assertion that has come to be known as the Dunning-Kruger effect" (Dunning, 2011, p. 260). Poor performers face a "double burden": they will make many mistakes, but they will not recognize it (Kruger and Dunning, 1999). Consequently, they are not motivated to make efforts to increase their knowledge; they think they are competent already.
In the last two decades, some literature tried to understand the phenomenon's nature or find alternative explanations for the experiences. The rest of the literature tried to widen the scope of the research. Today, DKE has been documented in several areas of life: logical reasoning (Kruger and Dunning, 1999 Kun et al., 2022), since its essence includes the comparable measurement of preparednessprimarily due to talent management (Dajnoki, Héder and Kőmíves, 2018; Szondi and Gergely, 2021)which leads to ideal data sources. Our study joins the latter trend.
The motivation for studying the appearance of DKE in higher education is primarily the abovementioned "double burden". If the effect is significant, more attention should be paid to the fact that the students performing worse should realize as soon as possible that they need improvement since DKE can hide this from them. The communication directed towards them, the frequency of accountability etc., should be designed in light of these.
DKE also leads to allocation and performance measurement problems. Weaker students will devote too few resources to learning, and good students too much. Although the former seems to be a more severe case, the latter also causes loss since too much learning is necessarily at the expense of another activity (even learning for another course).
It also means that self-evaluation can only be used to a limited extent in assessing student performance or not with a stronger DKE. In addition, incorrect student self-evaluation can also distort the students' assessment of the lecturer's performance (even in the case of a realistic lecturer's assessment, a weaker student believes that they are undervalued, which they may consider a mistake or malice).

Hypotheses
In our study, we investigate the presence of the DKE at a knowledge management examination of BSc level business students at the University of Debrecen, Faculty of Economics and Business. The definition mentioned in the introduction (Dunning, 2011) was adopted as follows for measurability.
Those who perform poorly on the Knowledge Management course's knowledge test (examination) tend to overestimate their performance (H1) and estimate it less accurately (H2) than students who perform better than them.
In the case of both hypotheses, we examined the students' pre-examination prediction of their performance and their post-examination evaluation. We labeled these sub-hypotheses H1a, H1b, and H2a, H2b.

Data and methods
We used partly secondary and partly primary data. The primary data were the answers to separate questions before and after the examination. The secondary data was provided by the examination-test results of the course Knowledge Economy, whose content is that of a Knowledge Management course for the fall semester of 2021. The course was in English, compulsory for third-year BA international students majoring in Business Administration and Management and Hungarian (domestic) students specializing in Shared Services Expert in the same major. The test contained 20 multiple-choice questions (1 out of 4 correct answers, no points deducted in case of a wrong answer). The assessment was written in two consecutive rounds, and the students were randomly assigned to the rounds using MS Excel's quasi-random number function. The test was written in person, on paper. Before and after the examination, we asked the students how many points out of 20 they thought they would achieve and what the average result of the group would be, rounded to a cardinal number. The exact estimate was motivated by giving the students 0.5-0.5 points per hit (a maximum of 2 points in total) in addition to the test score. In this paper, we do not analyze the answers regarding the group average, only those regarding one's performance. The response rate for the pre-test question was 77% (68 out of 88 possible answers) and 95% (84 answers) for the post-examination question. The students were motivated to receive an extra point (above 100%) if they got a correct guess, but there was no deduction for an incorrect or missed estimate. Multivariate linear regression models were used for data analysis. The dependent variables of the model were the individual error indicators: signed error before the examination (H1a), the absolute value of the error before the examination (H2a), signed error after the examination (H1b), the absolute value of the error after the examination (H2b). The available control variables used for the models were the following: the test written during the first, second or third rounds, attended the Hungarian-language or the English-language course (the course to which the assessment was connected was done by both groups together, in English), the group the student wrote the test in (groups A and B were in the first round, C and D in the second round, and they were the same in terms of content, but the order of the questions was different). Students who were not in the first year and/or did not take the examination for the first time were excluded from the analysis, thus in the end, the sample included 63 cases for the pre-test question and 76 cases for the post-test. Unfortunately, we did not have information about the gender and age of the students. The explanatory variable in each model was the actual score. The variables were not standardized because the known background variables were either used to narrow the sample or were included in the model. A detailed description of the variables is given in Table 1.

Variable Description SCORE
The student's actual score ERROR1 (The student's score estimate before the examination) -SCORE ERROR2 (The student's score estimate after the examination) -SCORE A_ERROR1 The absolute value of EHIBA1 A_ERROR2 The absolute value of EHIBA2 TURN2 1 if the student took the examination in the second round, 0 otherwise. HUN 1 if the student participates in the Hungarian-language program, 0, if in the international program. YEAR2 1 if the student is a sophomore, 0 otherwise. GROUP_A 1 if the student wrote the "A" group test, 0, otherwise. GROUP_C 1 if the student wrote the "C" group test, 0 otherwise. GROUP_D 1 if the student wrote the "D" group test, 0 otherwise. Table1. Defining the variables Note: Group 'B' had the highest headcount; thus, it is omitted during the analyses.

Graphic representation of the phenomenon
Suppose we apply the representation method in the eponymous study of the examined phenomenon (Kruger and Dunning, 1999) to the raw data ( Figure 1), a typical image of DKE echoes. Moving from worse to better performance, the degree of student overvaluation decreases, and then in the fourth quarter, we experience undervaluation. According to the score achieved, the self-evaluation before the test is higher for the lower 75%, while the self-evaluation after the test is higher for the upper 25%. It may indicate that the estimates of the better and the worse prepared students have become more accurate: those who initially overestimated themselves adjusted their scores downward after the examination, while those who initially underestimated themselves increased.  Of course, no significance test was performed during the representation, we could not take into account the other available variables, and since we represented both estimates simultaneously, we had to leave out of the analysis those who only provided one value. In the following, we will make up for these shortcomings.

Linear regression analysis
The estimate of one's performance before the examination (Table 2) is significantly related to the achieved score both in sign and absolute value. In other words, those who achieve higher and higher scores overestimate themselves less and less (or possibly underestimate themselves more and more) and get closer and closer (regardless of the direction) to their actual value. The explanatory power of the model of the signed error is significant and roughly double that of the model for the absolute value, and the SCORE coefficients show an even more significant difference. In other words, student performance (in the opposite direction) is associated more strongly with overestimation than with the estimate's accuracy. Since the SCORE variable was entered into the model after the others, we could also conclude that the inclusion of the score significantly improved the proportion explained by the variance of the dependent variable.   Table 2. Self-assessment errors before the examination. Notes: * p < 10%, ** p < 5%, *** p < 1%, **** p < 0,1% The experience was similar even when the students estimated their performance after the examination (Table 3). Here, too, we obtained results corresponding to the DKE: both the sign and absolute value of the errors showed a significant, negative correlation with the score achieved. The higher the score, the less the post-examination revaluation was characterized by an estimate far from the valid score.
Compared to the models before the examination, the explanatory power of the model of the signed error was even higher than that of the model for the absolute value. In the latter, the only significant variable was SCORE.  Table 3. Self-assessment errors after the examination. Notes: * p < 10%, ** p < 5%, *** p < 1%, **** p < 0,1%

Conclusions
Based on the linear regression analysis results, both our hypotheses and all four sub-hypotheses were confirmed, as we showed a significant, negative relationship between the self-assessment error variables and the achieved score variable in all cases, taking into account the available background variables. In other words, during the examined knowledge management examination, the students who achieved a lower performance tended to overestimate their performance to a greater extent both before (H1a) and after (H1b) the examination than their better-performing peers. At the same time, the lower actual score was accompanied by a greater distance between the estimated and the actual score, i.e. lower estimation accuracy, both before (H2a) and after (H2b) the examination.
Therefore, we found evidence for the presence of DKE in the Knowledge Management course, which could also be demonstrated in addition to other available variables (examination round, examination group, national or international training program, student year).
Since it was not only the presence of DKE but also its size (adjusted R 2 ), and it was significant, the "double burden" must also be taken into account and, as much as possible, must be eliminated. It means either providing students with more frequent mandatory feedback (e.g. multiple assessments) or teaching them to evaluate their knowledge better (the latter goes beyond the scope of a professional subject).
Interestingly, those who achieved the best results reduced their underestimation tendency after the examination, while those who performed less well did the opposite (reduced their overestimation). The explanation for this is the improvement of self-assessment accuracy during the examination, which is uniform in the entire sample; that is, the different direction of the decrease in over-and underestimation is necessary due to the increase in accuracy.