A Function-Based Scoring Method for Evaluating Student Mental Models of Systems

Mental models are loosely-defined constructs people form to reason and make predictions about their surroundings. These models are an important aspect of systems thinking for engineers, a concept that emphasizes holistic thinking when working with complex systems which is increasingly important in multiple engineering disciplines. Methods to evaluate systems thinking and mental models of systems traditionally rely on questionnaires, or detailed interactive simulations of specific processes. This work presents a method based on functional modules for evaluating student responses to an instrument based on Lawson’s bicycle problem, intended to elicit students’ mental models of two systems. Students were given a simple outline of the two systems, a hair dryer and a car radiator, and were prompted to fill and label the components required for the system to fulfil the functionality described. This was done in two sessions, once before learning functional modeling, and once after, to utilize the method of scoring to evaluate any changes in their mental models due to exposure to functional modeling. The scoring method identifies common functional modules between two systems using Module Heuristics, and then identifies students’ recognition of those modules. This allows a direct comparison of the functional similarity between the two systems identified by the students and can capture a wider variety of correct answers than simply counting the components a student provides.