A fuzzy Bayesian model based semi-automated task analysis
The purpose of this research is to propose a new task analysis methodology that combines a statistical approach, the fuzzy Bayesian model, with classic task analysis methods, to develop a semi-automated task analysis tool to better help traditional task analysts identify subtasks. We hypothesize that this approach could help task analysts identify activity units performed by the call center agent. The term activity units, in our study, represents the subtasks the agents perform during a remote troubleshooting process. We also investigate whether this tool could help predict the activity units as well. With the help of the Remote Print Defect Diagnostics research team, an effort-intensive field-based approach for the call center's naturalistic decision making's environment was accomplished. A total of 126 customer calls were collected onsite. These calls provided a dataset of 5184 agent-customer conversation narratives containing approximate 165,000 words. The calls also represented a diversified working environment that involved 24 experienced agents, 55 printer models, 75 companies, 110 customers, and dealt with over 70 software and hardware issues. A human expert and an additional 18 Purdue juniors and seniors also participated in the validation of the 72 manually assigned subtasks. The machine learning tool's performance was then examined based on three models: classic Bayes, fuzzy Bayes (single word, two words, three words, four words, and their combinations), and hybrid Bayes, a combination of classic and fuzzy Bayes. Extensive calculations were carried out and substantial amounts of quantitative and qualitative results, including 1023 tables ranging from 41 records to 4.5 million records, were generated. By examining the preliminary results of the hybrid Bayes showing an overall hit rate of 57.21%, false alarm rate of 0.64%, and sensitivity value d' of 2.67, we are able to support our hypotheses that the fuzzy Bayesian based tool is able to learn subtask categories from the agent/customer narrative telephone conversations and to predict them as well. In addition to supporting the primary hypotheses the large volumes of data and results provided by this study have the promising potential of serving as a starting point for more in depth investigation of the application of task analysis tools to the naturalistic decision making environment. The abundant data and results have also the promise of providing the resources to explore exciting applications of hybrid Bayes models to many other areas of the naturalistic decision making environment.^
Mark Lehto, Purdue University.