How to Respond to Student Initiatives in Tutoring Systems 1

Farhana Shah and Martha W. Evens

Computer Science Department
Illinois Institute of Technology Chicago, IL 60616
farhana@steve.iit.edu
mwe@schur.math.nwu.edu

Abstract

The necessity for building a sophisticated human-machine interface for our Intelligent Tutoring System, called CIRCSIM-Tutor, has motivated us to explore natural language text understanding and generation. One complex task for the generator is crafting a response to student initiatives. A student initiative occurs when a student takes control of the tutoring session temporarily by saying something that forces the tutor to change the course of action and respond to the new situation. A question asked by the student in itself is considered to be one kind of initiative. We have analyzed tutor responses to student initiatives and developed twelve classes of tutor responses to be used in CIRCSIM-Tutor.

Introduction

Our research aims at accomplishing an intelligent tutoring system CIRCSIM-Tutor which helps students learn to solve problems in qualitative causal reasoning and improve their terminology. This system generates natural language dialogue and must be able to respond to student initiatives. We studied twenty eight transcripts of human keyboard-to-keyboard tutoring sessions. The tutors are two professors of Physiology at Rush Medical College, Joel Michael and Allen Rovick. They are teaching cardiovascular physiology with the goal of helping students learn to solve problems involving the negative feedback system triggered by the baroreceptor reflex. The students are first year medical students who already have some background in the subject matter from attending lectures or reading the text book. We used the conceptual categories for initiatives developed by Sanders [1992], who proposed eight classes containing various categories of initiatives. We analyzed responses to initiatives in twelve major classes based on the goals of the tutors and the tutoring strategies used.

Background

The transcripts we used in this research have been processed by several different programs. The first program, the Computer Dialogue System [Li et al., 1992] is used to record the tutoring session conducted between the tutor and the student. The file thus obtained is passed through another marking program that numbers it. For example "K4-st-84-1" in the following sentence:
	   K4-st-84-1: I don't think I understand the question.
indicates that this sentence comes from the fourth keyboard session, that the student is typing, that this is the 84th turn, and that it is the first sentence in that turn. In this case, the initiative belongs to the category which corresponds to features including when student does not understand something in the instruction from the tutor.

The session starts by describing a clinical scenario, in which some kind of perturbation disturbs the blood pressure. For example a mechanical heart pacemaker all of a sudden stops working properly and the heart rate is increased to 90 beats/ minute. The student is asked to make predictions about the changes experienced by four physical (haemodynamic) variables and three neurally controlled variables. The student is supposed to complete the predictions in three phases: Direct-Response (DR), Reflex-Response (RR) and Steady-State (SS). The DR phase includes the direct physical effects of the perturbation before the reflex begins to counteract the effect. The RR phase encompasses the changes brought by the reflex system to neutralize the effect. The SS phase shows the net effect of changes due to the direct effect of the perturbation and the changes made due to negative feedback.

Classification of Tutor Responses

Analysis of our transcripts shows that the tutor responses to student initiatives can be classified as follows:

  1. Hinting
  2. Directed Line of Reasoning
  3. Acknowledgment
  4. Confirmation
  5. Summary
  6. Instruction in the "Rules of the Game"
  7. Teaching the Sublanguage
  8. Teaching the Problem Solving Algorithms
  9. Help in Response to Pause
  10. Probing the Student's Inference Process
  11. Brushing Off
  12. Conversational Repair

We discuss these classes one by one with the aid of examples. The examples are extracted from different transcripts. We have expanded some of the abbreviations used by the tutor and the student in order to make the examples easier for our readers to understand.

1. Hinting

The tutors' desire to encourage active learning convinces them to prompt the student with hints. Hinting or reminding is a strategy that stimulates the recall of inert knowledge or activates the inferences needed in the completion of a task [Hume et al., 1996]. We noticed different types of hints in various forms embedded in the tutor's response. We divided them into Hume's categories:
	1.a. Convey Information Hint (CI-Hint)
	1.b. Point to Information Hint (PT-Hint)
The CI-Hints directly convey information to the student. For example:
EX. 1.a.  K25-st-161-1: Yes... so then you're saying that because reflex responses,
			for instance in map, are different than in the Direct
			Response that all/some/only that/ variable in Steady State
			reflect that in the Direct Response?
	  K25-tu-162-1: For any parameter that you look at across the three
			columns, ss almost always follows Direct Response
			(when Direct Response and Reflex Response are changing in
			different directions).

The PT-Hints point to pertinent information but do not explicitly convey information to the student. How the tutor points to a piece of information can be seen in the following example:

EX. 1.b.  K12-st-36-1: Yes I guess I am not sure then what happens to Total Peripheral
 		       Resistance
	  K12-tu-37-1: What is the primary mechanism of control of Total Peripheral
 		       Resistance?

2. Directed Line of Reasoning (DLR)

Some of the tutor responses are comprised of multiturn dialogue segments in which the tutor communicates by posing a sequence of questions in order to enhance student reasoning capability. The tutor avoids giving quick specific answers until the tactics of reminding and stepwise direction fail. Hume et al. [1996] call this kind of dialogue a directed line of reasoning (DLR). We find this in the following fragment of a dialogue.
EX.  2.	   K13-st-48-1: If Stroke Volume decreases every time, Heart Rate increases,
			how can Cardiac Output ever increase
	   K13-tu-49-1: What are the two parameters that determine Cardiac Output?.
	   K13-st-50-1: Stroke Volume and Heart Rate
	   K13-tu-51-1: Can you write the simple equation that relates Cardiac Output,
       	                Stroke Volume and Heart Rate?
	   K13-st-52-1: Stroke Volume times Heart Rate = Cardiac Output
	   K13-tu-53-1: Good.
	   K13-tu-53-2: Then if Cardiac Output is up and Stroke Volume is down what
			must have happened?
	   K13-st-54-1: Heart Rate increases more than Stroke Volume decreases
	   K13-tu-55-1: Right, that's exactly what happens.

3. Acknowledgement

An acknowledgement tells the student whether an explanation is correct or not.
EX. 3.	   K6-st-60-1: Does the direct affect steady state more than the reflexes?
	   K6-tu-61-1: Yes.

4. Confirmation

The class of response to a request for confirmation can be simple or combined with other categories. Simple confirmation of student initiatives may be made by a simple statement.
EX. 4.	   K4-st-48-1: How about the RAP, which may have an effect on how much blood
		       is reaching the ventricle.
	   K4-tu-49-1: Definitely, RAP affects ventricular filling.

5. Summary

The tutors choose to summarize often in all types of tutoring dialogue. They often use this strategy of reinforcing important concepts in responding to student initiatives.
EX. 5.	   K13-st-56-3: I am not sure if 120bpm is fast enough to cause that.
	   K13-tu-57-1: Probably not.
	   K13-tu-57-2: But more to the point, both tpr and cc change only when
			the reflex alters the activity in the ans
			(autonomic nervous system).
	   K13-tu-57-3: And since dr is BEFORE the reflex can act, both must be
			0 in dr.
	   K13-tu-57-4: Let's go on to the next column.

6. Instructions in the "Rules of the Game"

The tutor decides that student does not understand how the protocol is supposed to work. This response incorporates some instruction for the student directing her/him how to proceed.
EX. 6.	   K13-st-24-1: Cc increases maybe
	   K13-tu-25-1: No maybe's allowed.

7. Teaching the Sublanguage

The tutor is concerned about teaching correct usage of physiology language. Indeed this is one of the most important reasons for implementing a natural language dialogue in CIRCSIM-Tutor.
EX. 7.	   K12-st-46-1: Does the rate of blood removal from the central veins mean
			that blood entering the right atrium, if so i think venous
			return does go up immed.
	   K12-tu-47-1: We need to get our terminology straight

8. Teaching the Problem Solving Algorithms

The tutor responds according to the correct logical order of multiple initiatives taken by the student. A major goal of the tutor is making sure that the student understands how to solve problems.
EX. 8.	   K12-st-62-2: I'm just hesitant to say what comes first.
	   K12-st-62-3: I'll go with tpr i to slow blood flow back to heart
			(i don't really like this idea)
	   K12-tu-63-1: Well let's see if we can get at the first question
			I asked and then we'll come back to TPR.

9. Probing the Student's Inference Processes

The tutor encourages the student in active learning through self explanation. This also helps the tutor to update his model of the student. For example:
EX.  9.    K5-st-102-2: But I'll bet that's not right.
	   K5-tu-103-1: Well you're right in your bet.
	   K5-tu-103-2: Stroke Volume decreases because Cardiac Contractility
			decreases.
	   K5-tu-103-3: That doesn't mean that RAP has to be decreased!
	   K5-tu-103-4: Let me remind you again of the vascular function curve.
	   K5-tu-103-5: Does that help?
	   K5-st-104-1: RAP I.
	   K5-tu-105-1: Would you explain.
	   K5-tu-105-2: You're right but I just want to hear what you'r thinking.

10. Help in Response to Pause

When the tutor notices a delay on the student side, he offers his help. This is another tutor tactic to help the student in active learning. This response works as a rejoinder for the pause initiative.
EX.  10.   K5-st-45-1: I don	
	                      		big pause here
	   K5-ti-46-1: Need help?

11. Brushing Off

Whenever tutor fails to understand the student, they often give the student a "brush off."
EX.  11.   K12-st-100-4: But I am confused with the role of decreased Toatal
			 Peripheral Resistance  and Right Atrial Pressure.
	   K12-tu-101-1: I think that you haDr. Michael.
	   K12-tu-101-2: I think that I have kept you long enough.
	   K12-tu-101-3: Thanks very much for your help.

12. Conversational Repair

Repair is done to avoid misunderstanding and correct misconceptions. If the misunderstanding is not noticed at once, the conversation may break down at later stage. So it is very important to make an attempt to resolve the issue immediately.
EX.  12.   K5-tu-87-1: So?
	   K5-st-88-1: I don't understand.
	   K5-tu-89-1: How does CC D affect CO?

Length and Frequency of Responses

The following analysis is based on sixteen of the keyboard-to-keyboard sessions conducted during a one month period. Each session was two hours long. The maximum number of initiatives/ responses in a session was found to be 12 and the minimum was zero. The frequency and nature of the initiatives, as well as the length of the responses reflect not only the level of student understanding but also tutor's perspective. The tutor response varies in length (see Figure). The degree of detail and the type of content communicated by the tutor for the same category of student initiative, is different.


Figure: Response Length (content) by Initiative Categories.

Where 1d, 2a, 2b, and 3a are the categories coming out of Sanders' classification of student initiatives: 1. student asks a question, 2. student finds it hard to understand, 3. student requests repair, 4. student does repair, 5. student hedges, 6. student makes an explicit backward reference, 7. student takes initiative with respect to hardware/ software environment, 8. student asks administrivia type questions and the response length is calculated in terms of the number of sentences.

Summary

In this paper we have tried to analyze different transcripts to extract certain information regarding the tutor's attitude and tutoring style, especially when the tutor is engaged in responding to the student initiatives. We undertook this study in order to design responses to student initiatives. We have developed twelve classes of tutor responses. Most of them are based on intuitive criterion. The next question is what factors are responsible for producing conceptually different responses.

References

Hume, G. D., Michael, J., Rovick, A., Evens, M. (1996). Hinting as a tactic in one-on-one Tutoring. Journal of the learning Sciences, vol. 5, no. 1, pp. 23-47.

Li, J., Seu, J. H., Evens, M. W., Michael, J. A., and Rovick, A. A. (1992). Computer Dialogue System (CDS): A System for Capturing Computer-Mediated Dialogue. Behavior Research Methods, Instruments, and Computers, vol. 24, no. 4, pp. 535-540.

Sanders, G. A., Evens, M. W., Hume, G. D., Rovick, A. A., and Michael, J. A. (1992). An Analysis of How Students Take the Initiative in Keyboard-to-keyboard Tutorial Dialogues in a Fixed Domain. Proceedings of the Fourteenth Annual Meeting of the Cognitive Science Society. Indiana: Lawrence Erlbaum Associates, pp. 1086-1091.



1This work was supported by the Cognitive Science Program, Office of Naval Research under Grant No. N00014-94-1-0338, to Illinois Institute of Technology. The content does not reflect the position or policy of the government and no official endorsement should be inferred. Back to text