Computer science is a design driven discipline. ``Discussions'' about pedagogy in CS1 invariably dissolve into arguments over paradigm (language) or content (math first vs. programming first), with little serious attention (research) regarding the instructional methodologies and themes that transcend all of these arguments. I believe that problem-based instructional methodologies are most appropriate in the introductory context considering fundamental nature of computing, and have made my own course (``Introduction to LEGO Robotics'') a laboratory for exploring these methods in the classroom.

Design is a creationary, innovative process. This very simple, powerful statement lies at the heart of our discipline. The process of implementing quicksort involves, from the learner's perspective, creating code that never existed before, regardless of how many versions of the algorithm exist today. Likewise, proving that the worst-case running time of quicksort involves generating a proof that is, while available in most any text on algorithmic complexity, new to the student. Most important in both the creation of code or proof is the fact that the student is developing a new process for writing programs and formal mathematical arguments.

I believe that hands-on, problem-based instructional method is the most appropriate way to teach the process oriented, creative skills students encounter in computing. I have designed and developed a non-majors exploration course at Indiana University entitled Introduction to LEGO Robotics; I2LR is my laboratory for exploring authentic learning experiences in the classroom, and can be contrasted with current practice in six important ways:

1. It is uncommon for a lecture to last more than 20 or 30 minutes of a 1 1/4 hour course. Students spend considerable time in class engaged in some sort of hands-on activity (programming or building), often exploring concepts just presented in lecture. A great deal of stress is placed on writing and reflecting about their work.

2. A significant aspect of the couse are the laboratories that students engage in; these labs are presented as challenges to the students in terms of building and programming, often structured to require students to gain new knowledge to complete the challenge. They are expected to keep detailed journals of their progress in building and programming, noting not only what they did, but why---the reasoning and motivation behind their decisions.

3. Program design is very important in I2LR. The methodology employed for guiding students in tackling new programming challenges is based heavily on the PLT Scheme group's book How to Design Programs. Their approach is very similar in many respects to Polya's How to Solve It.

4. Students are required to work in groups for their laboratories; they wouldn't be able to complete the challenges in a reasonable amount of time otherwise. In addition, students are encouraged to work together outside of class, and in-class we employ Pair Programming, a concept put forward by the Extreme Programming design methodology and explored further in Laurie Williams's Collaborative Software Process.

5. Students are expected to write extensively as part of their laboratories, and are also asked in some assignments to reflect on the thought process they went through to come to their result.

6. Test-cases and good commenting practice is presented as a critical part of the problem solving process. This allows for discussion of conceptual problems in a student's efforts before they write a single line of code.

From a research perspective, this laboratory is over-broad. However, from a pedagogical perspective, it is nearly adequate for engaging students in the art and practice of design in the computing context.

I am done with coursework, collecting data, and reading intensely for theoretical perspectives and empirical research that related to what I am working on. It is especially difficult to find good literature in the sciences on applying problem-based strategies with adult learners.

Data collection is underway, and takes several forms. As a qualitative study, I am looking for a way to generate good, triangulatable data regarding my student's work in the classroom for purposes of assessing the instructional methods employed. I have not chosen an learning/instructional theoretical framework from which to base my analysis at this point---I will likely employ a grounded theory approach in my first analysis, and focus more intently on themes that arrise from that approach.

Data is collected via five primary conduits:

1. My journals.
   I write as much as possible about my experiences after any given day in the classroom; I focus as much as possible on observations regarding my interactions with the students, both when lecturing and when working with them 1-on-1 or with a particular group.

2. Student homeworks.
   Their lab writeups and personal journals of their process are very interesting, and provide a guide to the analysis of the very rich data that is being collected in the form of images and audio.

3. Audio.
   Each machine students work at is mic'ed, and runs an MP3 streaming engine that transmits a 22 Hz mono audio stream at 24 Kbits to a reflector server in our department. Each stream is then captured to disc for later analysis. This creates an effective ``field array'' of microphones, which are easily identified (spatially) by their source. One hour of MP3 at this level of quality/compression takes approximately 10 MB of space; therefore, a given class period with 45 minutes of ``lab time'' will generate
   45 minutes x 12 machines = 540 minutes, or 9 hours

   of audio. This takes approximately 90 MB of space.

4. Screen captures.
   The students begin their first few laboratories working with ROBOLAB, which is a visual programming language. They program in flowcharts (literally), and the highly visual nature makes screen captures a natural choice for imaging. In the past, students were asked to capture their own images as part of their work; now, a screen capture is made every 10-15 seconds and stored to a location accessible to students. This way, in their writeup, they can revisit the evolution of their own work. Additionally, I have a coarse-grained ``movie'' of their work in a given lab, time stamped and correllatable against conversations reagrding their work in the lab.

   Each screen capture is approximately 200 KB in size (as a JPEG compressed image). This ammounts to

   5 captures/minute * 45 minutes = 225 captures

   225 captures * 200 KB/capture = 45000 KB = 45 MB

   of screen captures for a given lab per machine, or 540 MB of images in a given lab.

5. Video.
   This is useless in a lab context with 30 students. It does, however, provide a time-track of sorts, and allows for seeing where students are as they move through the field array of microphones. Beyond this, the video is not of any particular value.

6. Still Images.
   The students are provded with a digital camera to document their construction. The course TA, when not assisting students, takes pictures of their constructions religiously, and will generate 80-100 JPEGs per hands-on class session. These pictures are also made available to students for use in their writeups.

There is, as can be seen, more data than I will ever be able to analyze in a lifetime (or two). The student's writeups will be used to guide the analysis. (If I ever spend a significant period of my life unemployed, I will begin on transcribing everything, and publish it to the public domain).

(Footnote regarding lots of data: another data source may or may not be incorporated into the study. As if I won't have enough...)

I'm currently taking on a life-project as a dissertation topic. If there were any sacred cows to be slain in this work, they've long been ground into hamburger and put into deep freeze. While that isn't a good idea, I know what I'm doing is really cool stuff, and the kids love it---and they cover an impressive amount of material besides. Last semester, we covered half of our intro course in programming in addition to building and playing with LEGO robots. That is impressive for a two credit non-majors course (compared to the intro course, which is a three-credit course for majors).

At this point, I need people to either attack my ideas with well documented research that undermines my position (which I hope I've made clear), or to help me narrow my scope (and again, I've not found enough related empirical research to satisfy myself). What I do not want is more references of a philosophical nature that attack (or support) this work. I'm trying, too, but it is possible that I don't know how to do an effective literature search.

Lastly, I have a want that didn't really happen from last year: some sense of community on-line, to fill the space between conferences. I have some ideas that I will have a slide or two on that I would like to present as ideas as to how the participants can start developing a sense of community in this discipline.

I've read a lot, because a lot is applicable, but not everything I've read has been useful. My BiBTeX database is in an unfortunate state of dissarray. Another day or two will be necessary (given my current to do list to get it on-line.

The only footnote currently regards an additional source of data that I came upon in December 2001. There's a bit of background, and the question comes at the end. If you'd like to read the last paragraph first, and then read from the 2nd to the penultimate paragraph, you may do so.

Several years ago, I wrote Vincent, a web-based Perl application for wrangling assignments in our department. It is undergoing it's final revision, for a number of reasons, most of them because of issues with the longevity of the previous version.

Vincent has the ability to run arbitrary programs against assignments as they come in. Our introductory course in progamming put then entire automated grader behind Vincent this semester, and made the homeworks worth a small percentage of the student's grade, but with large penalties for a failure to submit a working homework (please don't ask me the details, I don't know them all at this time).

The important thing is this: I believed (and still do) that a hand-in system should not ever throw away a student's homework once submitted. Regardless of how many versions they might submit, the previous versions are always important. So, Vincent never throws anything away. This, coupled with the full grader, created (inadvertantly), a very interesting data set.

Every student submitted, on average, 30-50 versions of each homework. Some students submitted 60-80 versions. This happend for each of the (approx.) 15 homeworks in the class.

I am told by our Human Subjects committee that obtaining permission to use this dataset is not going to be difficult, since it exists as part of a natural instructional process. Before obtaining the programs, they must be anonymized, but it should be possible to automate the collection of A) student status at IU (first-year, sophomore, etc.), B) college of enrollment (business, college of arts and sciences, etc.), and other demographics publically available on campus.

It would seem to be a 3 GB data set of a programming practice that I consider bad: hack and test, hack and test. This is particularly interesting compared to several of my students who submitted many of their homeworks written in pencil, and they were largely syntactically and semantically correct. I have reason to believe this is not because of the particular students in question, but the nature of the methodology employed, and think my arguments are strong enough that I can safely make that claim about these individuals.

I'm going to harvest the programs, as they are potentially a fascinating data set. The question is, are they a good ``counter-point'' for my own research?