Supporting the education of engineering mathematics using the immediate feedback method

. In the literature, several methods are suggested to deal with problems regarding the efficiency of mathematics education including techniques that help integrate new knowledge into long-term memory. We examined how effective the application of the immediate feedback method is in teaching engineering mathematics. The article presents the method used and the results obtained during the study


Introduction
Nowadays, the effectiveness of the learning process is increasingly discussed in higher education, including engineering education.The main reason for this is that bachelor's degree programs (BSc) -in terms of conditions, such as the students' competence, way of thinking, motivation, professional dedication, and expectations -are beginning to resemble secondary school education more than what we classically call university education.Mathematical competencies (algorithmic thinking, problem-solving ability, abstraction ability) are increasingly valued in engineering fields such as automation, digitisation, robot programming, development of smart and autonomous devices, and machine-machine communication.Due to the importance of knowledge, the question regarding the effectiveness of the education of engineering mathematics comes to the fore.With its concepts, theorems, procedures and notation system, mathematics provides the information that can be used to describe the laws determining the operation of engineering facilities (Vígné Lencsés, 2001).
From teachers' practice and from the literature, countless classical and modern methods are known for measuring the effectiveness of mathematics education.By the document titled A Framework for Mathematics Curricula in Engineering Education (SEFI, 2013), an engineer needs the ability to recognize, use, and intelligently apply mathematical concepts, procedures, and laws in a given environment or situation.Adapting to the mindset and characteristics of engineering education, in our research we approach effectiveness from the side of efficiency, claiming that mathematics is an integral part of the system of professional subjects in modern engineering education.Thus, in our approach, efficiency can be measured by the level at which students are able to apply the learned mathematical knowledge in solving technical problems.In our opinion, in order to correctly evaluate the effectiveness, the ability to create models and solve problems should be examined both in the case of technical problems occurring in the mathematical environment and within technical subjects or during engineering work.In addition to the increasing importance of mathematical knowledge that can be used in practice, it can be seen that the ever-increasing obstacle to the understanding of higher-level, abstract knowledge is the decreasing average level of basic mathematical thinking and calculation skills.The results of a national survey conducted in 2009 in mathematics, physics and chemistry among students entering technical and natural science higher education showed that a significant number of students starting their higher education begin their studies with a low level of preliminary mathematical knowledge (Csákány & Pipek, 2010).
The general lack of mathematical preparation is also proved by the fact that many Hungarian universities are introducing and making it mandatory for firstyear students to write criteria papers to measure their level of knowledge of high school mathematics (BME, BGE, DE, ELTE, PTE, SZTE, etc.) and to participate in catch-up courses.At the Faculty of Engineering, University of Debrecen, we assess the knowledge level of incoming students every year; our experience shows that the skill level that all students are guaranteed to have is extremely low.The large discrepancy between the knowledge assumed in the mathematics programs of the university courses and the real knowledge of the high school graduates results in the majority of students only considering meeting the minimum requirements as their goal and their showing a lack of interest in acquiring even the professionally useful knowledge.It is unfortunate to experience that after the 'bad' experiences gained during secondary studies, few students are motivated to independently mathematically model the problems raised and to learn about the solution, even if they are clearly related to the profession they are studying.
The goal of the tool system we use in the teaching of engineering mathematics is to increase the efficiency of learning and to develop a higher level of application skills during engineering studies.Our methodology is based on parallel discussion of analytical and numerical methods; assignment of project tasks in Mathematics and professional courses, project-based learning; and integrating technical problems into classwork at different levels of model making.Among the tools used there is the immediate feedback method, the results of which are presented in this article.

The immediate feedback method
Research in didactics of mathematics can be divided into three large groups: (1) the cognitive psychologic foundations of human cognition; (2) teaching experiences of master teachers -how they introduce the new material, how they check students' understanding, how they help when needed; (3) methods provided for solving complex tasks: thinking aloud, giving ideas, using elaborated examples.
Most brain researchers accept Baddeley's Model of Working Memory: sensory memory, working memory, and long-term memory (Baddeley et al., 2009).Longterm memory is the storehouse of our knowledge.It stores knowledge in schemas.Schemas are mental structures; with their help, we organize and structure our knowledge.Schemas are called up from long-term memory to understand certain situations and problem situations.We create the schemas in the working memory which we integrate into the existing schemas of the long-term memory.Long-term memory has no capacity limits, and no time limit is known, either.The relationship between working memory and long-term memory is criterial in the process of effective knowledge acquisition.Even complex schemas are considered one unit of information, so their retrieval into the working memory does not occupy capacity.Automating the schemas is an essential condition for complex problem solving, as their application does not require extra working memory capacity (Ambrus, 2014).
Epstein compared the immediate feedback assessment technique with a test completed on a traditional (Scantron) form.Acquiring forms using the immediate feedback assessment technique improved memory and performance even for items repeated from previous unit tests.Similarly, students evaluated using the immediate feedback assessment technique were significantly more likely to answer items correctly in the final exam they had previously answered incorrectly.These results held even when all participants used Scantron forms for the final exam (Epstein et al., 2001).
To control the teaching process efficiently, frequent reviews are necessary.The effectiveness of assessment methods usual in university mathematics education (1-2 tests per semester) is low, the results are only for evaluation and cannot be used for substantive correction of the teaching process.The common tools for improving the teaching process the most are: the observation of class work and the end-of-lesson check.Interactivity must be increased during the lessons because without interaction there is no attention.An important element of interactivity is asking for immediate feedback.Interactivity alone does not provide an adequate level of feedback, as usually, only a small part of the students respond.We believe that asking questions about the course material having been covered in the lesson,on the one hand, helps to retain knowledge, and on the other, gives direct feedback on the success of the learning process.
We distinguished two forms of feedback at the end of the lessons: online and written questions.Our experience has shown that online feedback is more effective because all students have smartphones, and they like that they can use it 'legally' in the lesson, it is not considered a test.The students see the results immediately and based on the feedback, the next lesson can already be made more successful.In the case of an online test, we get a more accurate picture of students' knowledge, because they surely answer there, while in the case of a short set of written questions, many students do not write anything on the paper; it is uncertain whether it is because they do not know the answer or because they do not even bother about it at the end of the lesson.The effect and effectiveness of the immediate feedback method are studied at several levels of education (Cooper et al., 2018).
Leydecker presents different possibilities for the activation of students in lectures with many participants including instant feedback for both sides on the learning progress.At Leibniz Universität Hannover, he examined a paper-based evaluation tool (EvaExam) and different online voting tools (Eduvote and arsnova.net).The first-year course Mathematics for Economy Students (1 and 2) is held in a group of approximately 600 students which consists of a two-hour lecture, a two-hour central tutorial and two-hour small tutorials weekly.According to the study, in large groups it is rather difficult to get direct and immediate feedback from all students about their understanding and their learning progress.Usually only a small number of students participate actively and answer the questions, a lot of students are quite passive and just listen and copy the contents from the board during the lecture.To cope with these problems, he tried different evaluation and voting tools to increase the interactivity in his lectures and to reach the following aims: (1) Getting direct information about the actual knowledge from most of the students and not only from some people who participate actively.All students should get the possibility to participate anonymously.
(3) Motivating the students to think and work on their own or together with a seatmate.
(4) Increasing attention during the lesson due to small active breaks.
Through his examinations, he has made a conclusion saying that the use of interactive elements has several advantages: (1) Large participation of the students; every student can participate without having the fear of being embarrassed.
(2) The students get immediate feedback about their knowledge and the knowledge of the whole group.
(3) The students can communicate with each other about mathematics during the lesson.
(4) The attention during the lecture is increased.
(5) The lecturer gets immediate feedback and can eventually make her/his explanations clearer.
However, the time needed for the regular written tests is an issue (Leydecker, 2017).
The Immediate Feedback Assessment Technique (IF-AT) 1 allows students to receive immediate feedback that assesses their knowledge.According to IF-AT, students receive partial credit for incorrect answers, which encourages them to reread the question and select the correct answer, thus, this technique requires more time than we can provide in our end-of-lesson tests.

The process of end-of-lesson feedback
In our study, we have developed a method of feedback.First-year mechanical engineering students of the Faculty of Engineering, University of Debrecen participated in it for two semesters.In the first semester, we used the immediate feedback method within the framework of the subject Mathematics I. We investigated the effect of the method on the students' results.The purpose of the surveys was not to evaluate the students.With the questions, we wanted to assess, on the one hand, how well the students understood the lesson material, whether they still remembered it at the end of the lesson, and how well they could recall the topics of the previous weeks.This is feedback on the efficiency of the educational process and creates an opportunity for quick correction.On the other hand, asking for feedback is part of our teaching method, it serves to deepen and establish the new information.In the questions, we generally did not ask them to do complicated calculations, but we were interested in whether the key concepts, methods and comments were in the students' memory.
We examined two groups; we applied the immediate feedback method in the experimental group, while in the control group we did not.Both groups had 32 people.Based on the t-test for the input test results, there was no significant difference between the initial preparation level of the two groups (Table 1).(The entrance test was multiple-choice and consisted of 12 questions.Students got 3 points for good answers, -1 point for bad answers, no answer meant 0 point.)We have prepared a four-question online test for each lesson of Mathematics I. We used the quiz function of the Kahoot application for the surveys.The program automatically evaluates the answers to the questions.The focus was on the knowledge that could be used to measure how much the students had understood the curriculum of the given lesson rather than on numerical calculations.In all cases, the text of the questions was a complex, specialised mathematical text and assumed the knowledge of the concepts involved.The sentence structure of the questions was simple and did not contain omissible parts.
The tasks were suitable for mechanical engineering students in terms of difficulty.The format of the answer was multiple-choice, one question had four answers, of which exactly one was correct.The questions were written in text form but could contain a formula.
In the next semester, we continued the study within the frame of the subject Mathematics II.Students having successfully completed Mathematics I were organised into groups for Mathematics II so that they had the same members of the experimental and control groups as in the case of Mathematics I.The number of students in the experimental group was 32 people, while the number in the control group was also 32.We asked for feedback at every lesson with the experimental group.
During the study, we had two hypotheses.
H1: The applied immediate feedback method generally increases the effectiveness of learning the course material, based on the results of mid-term and end-term tests.
H2: In the case of students with less preparation, the greater activity and better results shown during the immediate feedback increase the effectiveness of learning the course material, based on the results of mid-term and end-term tests.

The questionnaire
During the studied two terms, 24 tests were prepared, and we present two of them now.

Example 1
The course material was about the equations of the tangent and normal lines, the Taylor polynomial which is in the 3rd week of the term.It was assumed that the students had mastered the equations of the tangent and normal lines, the calculation of derivatives, and the concept of the Taylor polynomial.

Comparison of the results of the two groups (study of H1)
There were 32 students in both groups.During the term, each student wrote two papers, resulting in a total of 128 test results.A t-test was used to compare the average results of the two groups.Based on the t-test for the test results of the subject Mathematics I (all scores), there was no significant difference between the two groups (Table 2).

Study of immediate feedback within the group (study of H2)
We wanted to know whether the effect of the regular end-of-lesson questionnaires depends on the initial level of knowledge.
First, we examined how the test scores are related to the result of the entrance test (Figure 1).
By the data, there is no correlation between the two variables (R=0.225).Since the test results did not follow from the initial preparation, we examined whether dependence on the applied method could be demonstrated.
The students of the experimental group were divided into two groups based on their entrance test results.By the entrance test, the bottom 50% were classified into group A, while the top 50% were classified into group B. We examined whether the effect of regular immediate feedback differed in the two groups.To do this, we looked at the proportion of good answers on the end-of-lesson tests for each student, as well as the total score obtained based on the mid-term and endterm tests.By the results obtained, studying the effect of the immediate feedback  The correlation coefficient is R = 0.79 in Group A, and R = 0.11 in Group B. It can be concluded that, in the case of students with weaker foundations, the result of the tests (the total scores gained in the course) is significantly influenced by whether they paid attention to the lessons and whether they understood what was said.In other words, for them, it turned out to be more important to follow the lessons and to concentrate on end-of-lesson tests as well.The average score was much higher in Group B (76.4) than in Group A (58.9).In the group of students with lower scores on entrance test (Group A) the average score depended more on the results of the end-of-lesson tests.

Conclusion
By applying the presented method, we aimed to make mathematics education more effective for engineering students, since it plays an important role in engineering study programmes as the results are used directly.We believe that frequent checking of understanding, such as online feedback, is a tool to improve effectiveness, which we have explored in this paper.We believe that the application of the immediate feedback method helps the students to integrate their mathematical knowledge into their working memory, which we later integrate into the schemas in their long-term memory so that the students will be able to apply what they have learned at their engineering work with great competence.In this study, we have found that this method is useful particularly for students who come with a low level of knowledge and actively participate in the lessons.A feature of the feedback method is the possibility of quick correction, both for students and instructors.Since the prior mathematical knowledge of engineering students usually varies greatly, it is difficult to apply a general teaching method, so it is important that immediate feedback provides information on the students' individual progress.Since by our experience the immediate feedback method is most worth being applied for students coming with a lower level of knowledge, it can be useful particularly for catch-up courses.

Figure 2 .
Figure 2. Total test scores vs. end-of-lesson results in Group A (lower scores on entrance test)

Figure 3 .
Figure 3.Total test scores vs. end-of-lesson results in Group B (higher scores on entrance test)

Table 1 .
Two-sample t-test for the entrance test results

Table 2 .
Two-sample t-test (assuming unequal variances) for the test results (Math I)