Assessing 21st Century Assessment: A Meta with Peeps - Happy Easter

• Industry and business sends up the call for increased problem solving skills, collaborative ability, creativity. 
• College professors complain "kids these days cannot write!" but what they mean is that students have difficulty connecting primary assertions with supporting evidence in a coherent fashion (more of the critical-thinking thing).
• The schools mission demands we cultivate and nurture students who are loving, open to growth, religious, and committed to justice as well as intellectually competent -- these things are what we should write assessments to measure. 

Have you read the common core recently?

Having exact answers, and having absolute rights and wrongs, minimizes the necessity of thinking, and that pleases both students and teachers. For that reason, students and teachers alike prefer short-answer tests to essay tests; multiple-choice over blank short-answer tests; and true-false tests over multiple-choice.

 But short-answer tests are, to my way of thinking, useless as a measure of the student's understanding of a subject. They are merely a test of the efficiency of his ability to memorize.

- Isaac Asimov, "The Relativity of Wrong" 1989

(Full versions available online - did not link due to copyright)

Those of you who regularly read both this blog and @40ishoracle's know that, probably because we work about 15 feet away from eachother, we tend to talk about the same things. So it is probably not surprising that assessment has been the topic of choice for awhile.

Not safe for assessment?
marshmallowpeeps.com

my  #savethedinosaurs post became a little moot this week since NYC DOE backed off of its proposed list of 50 banned words. While I am hesitant to beat a dead tyrannosaurus (yeah, i had been saving that one), what I have been thinking is that this is an example where a lack of assessment goals (or possibly the wrong assessment goals) leads to some really awkward decisions about testing guidelines.

If the primary purpose of an assessment is to measure the ability to solve a certain level of equations, link a topic sentence to supporting sentences, and follow the basic rules of grammar, then it is easy to say that an assessment tool can be create which avoids uncomfortable topics like politics, evolution, and candy-peeps (mmmm...peeeepps).

Context: Computer Applications and HTML programming

At Brebeuf Jesuit, all freshman are required to take a digital citizenship course called "Computer Applications" (Course titles change about as fast as the Titanic could turn). One unit of this course is an introduction and overview of HTML programming. As the teachers discussed this unit in light of our new focus on "civilizing the digital natives", we began to unpack the assumptions in this topic.

• HTML programming gives students an opportunity to experience the rigor and close attention to detail that is necessary for the discipline (and useful across the curriculum).
• HTML programming, at a basic level, makes for more efficient blogs and other web communications that will become used more and more w/o relying solely on WYSIWYG editors. It is a communications imperative (and looks great on that intern job application).
• HTML programming gives students an idea about what it takes to build the webpages that they interact with everyday and this affective appreciation is helpful.

Looking at these objectives, a traditional assessment (multiple choice or matching to identify <tags>; a "revise the miswritten code" section; defining key terms like "attributes", etc) would not be able to successfully measure a student's ability beyond a basic understanding of the second objective.

Experience: Designing an Assessment Experience for HTML

Goals:

• Practical application of tags, attributes, and common processes such as links, formatting, and image control
• Use of critical thinking including disciplined attention to detail, consistent rechecking of method to results, interpretation of instructions
• Appreciation of the code that is working in the background of webpages

Assessment: 

Instagram of Old-School Programing

1. Students will complete a webpage based on instructions given in written from by the teacher. The written form will be a combination of text that should be replicated and highlighted instructions of how to format the text. for example:
   

   The Font of most of the text on this webpage is [Describe the color HERE]. Text should contrast with the background or it can be hard to read. [the word “Contrast” in the last sentence should be a link to this website: http://www.colorsontheweb.com/colorcontrasts.asp].

2. Student may use the web, notes, tutorials, etc. But the page must be created in NOTEPAD
3. Students will have one day to work on the test. At the end of the period students will have an opportunity to save progress to date and told that they are NOT to work on the exam at home (paper is collected) but that they may study and practice anything they would like.
4. Students turn in the completed (or nearly completed assessment) on the end of the second day

Reflection:  How well we have trained them not to think

"This was by far the hardest test we have had all year", remarked one student after the test.

As I have already been accused of being the hardest teacher in the school multiple times, I decided to pursue:

• "With a math test you are just following the steps until you get the answer and then you are done."
• "In English, even if you have to write, you just write about what you talked about in class. You don't have to constantly think about it."
• "This wasn't a test. You couldn't guess at anything. If it was wrong you had to figure it out."
• "I can't believe we have a test where you have to study in between the test. Heck, I had to study during the test!"

The students were thrown off for a variety of reasons, but primarily it fell into two categories:

 Outcome/Feedback Check: 

Web on left; Code on Right
Refresh to check

In designing a webpage, you typically (in notepad or in design-software) have your screen split to view the output on one half and the coding on the other. During the testing situation, the students did the same. They immediately knew if the coding was wrong because the output was incorrect. This would be the equivalent of completing a math problem, and immediately being told by the teacher it is wrong and should be corrected. Immediate feedback is disquieting for students used to the delayed feedback of the grade ("I totally bombed that test, but at least it is over").

Technology affords educators the opportunity to design assessments that are closer to the real world. There is seldom a final exam in a career-path job. Projects are assigned, drafted, submitted for feedback, returned, revised, given to colleagues. This process is essential to the collaboration and problem solving called for by the industrial complex, but is absent from traditional assessment.

 Assessment simulating Life, an open book:

In addition to the issue of immediate feedback, students had every resource available to them to complete the task. They had been given a week to work with simulations and had those same simulations available during the test and during the overnight study period. Deep down in dark places where excuses come from, each student knew that there was little reason to not be able to complete the exam perfectly. This created a lot of resentment and complaint. Indeed, the students who did the worst on the assessment instrument were not ones who did not know how to code; they were the ones who did not research in the intervening night, did not carefully check work, and did not have the care to complete the task.

Conversely, it also was fascinating to watch the thrilling smile creep across a student's face as she realized by poring over sample code and comparing it that her closing tag was in the wrong location. Like the feedback loop, there are few situations beyond traditional testing where you cannot refer to some other resource to help verify an answer or procedure. Knowing when you need to seek assistance may be more important than knowing the information initially.

Conclusions

Technology is only one of five 21st Century skills 

Our challenge than, as educators of the digital natives, is to take advantage of the technology and this critical moment in education to create a new kind of assessment. By clearly identifying the skills and knowledge that we expect students to have (OUTCOMES) and designing environments that demand access to that knowledge and use of those skills (ASSESSMENTS) then we are well on our way to transforming education.

The key becomes defining those outcomes, through discussion and collaboration (and maybe even student involvement - scandalous) in such a way that it acknowledges the broad strokes of 21st century skills and does not become mired in the details which, ultimately, can be looked up on Wikipedia.

Assessments grow out of those outcomes and, here again, we must be on guard to not sacrifice the big picture (career skills including flexibility and adaptablity; Learning skills including problem solving and systems thinking, etc.) for the easily scored details that should be the tools by which skills are demonstrated.

Because if they can think, let them eat Peeps.