The Language Assessment Process: A Multiplism Perspective

According to the author of this article there is a process that must take place for all language assessments. Here are the steps:
1. Determine the purpose of the assessment
2. Define language knowledge to be assessed
3. Selecting the assessment procedure
4. Designing items and tasks
5. Administering the assessment tool(s)
6. Determining the quality of the language sample/answers produced
7. Assessing the quality of the procedures
8. Interpreting the results
9. Reporting the results

To me this list should help a teacher who is struggling with formalizing the process of giving authentic assessments.

When I read the title of the article my first thought was 1) Is Multiplism a word? and 2) What does it mean. The author goes on to say that deciding which assessment tool to use depends on the purpose of the assessment and on how language knowledge is defined. She suggests that the use of a "multiplism" approach to language assessment, whereby multiple options are available at each phase of the assessment process.

The article discusses some of the same things as the test design chapter from the McNamara book---such as discrete point testing, and how the purpose of these assessments measure skills based type of language acquisition, rather than communicative competence. The types of assessments that the author recommends are the same as the authentic assessments we have been reading about.

The phases of assessment that I was particularly interested in learning about were the ones that had to do with what you do after you administer the assessment. Such as the multiple ways you can interpret and report results. I think having an assessment conference is a great idea. I also like the idea of reporting in the form of a narrative, rather than just giving scores or grades etc.

Basically, what I got from reading this article, is that there are some basic processes that should be done when giving assessments, but there are multiple ways of doing these phases that can and should be done depending on the purpose of your assessment.

Designing Authentic Assessment

Approaches to teaching and learning have changed in light of new views on how students learn. Where in the past we thought learning was linear, we now see that learning doesn't necessarily proceed by the accumulation of a common set of basic skills, but can follow multiple strategies and pathways. The view of constructivism believes that all individuals are thought to learn by constructing information about the world and by using active and dynamic mental processes. This newer view on how we learn has implications for how we must assess. Basically, as the author states: "If students construct information as they learn, and apply the information in classroom settings, assessment should provide the students with opportunities to construct responses and to apply their learning to problems that mirror their classroom activities in authentic ways."

The article discusses briefly several types of authentic assessments. It also mentions that many teachers DO use these assessments already, however they do it in such an informal way that it does not provide enough information about student learning, or the goals of instruction.

Here are the list of the authentic assessments the book provides, and a note on whether or not I've used them..

Oral interviews--I've used these, but informally

Story or Text Retelling--Yes I've used this, and developed a rubric (or used rubrics created professionally)

Writing Samples--Yes often, Use 6-traits rubric, and rubrics that align to state writing standards

Projects/Exhibitions--Yes, often develop my own scoring rubrics for these

Experimental/demonstrations--Haven't used

Constructed Response Items--Yes, I do use these for tests for my high schoolers on material that I teach in class

Teacher Observations--YEs, but informally (until I start my research!!)

Portfolios--No, but REALLY want to!! I had a portfolio in high school, and remember thinking it was a fair assessment of my varied skills and talents


If I had to assess myself on my awareness of authentic assessments (which I will do in more detail on the self assessment) I would say that I'm fairly aware of them, and I do use them, however I need to implement a more formal/systematic approach so that I can get more meaningful information from them, as well as use them more in instructional goal setting.

I like how the chapter gives step by step directions for designing authentic assessments--I'm kind of excited to try and work with some teachers (if they can accept me as a teacher, and not a district office spy..lol) and design some authentic assessments particularly for our students' writing. We used to have a writing portfolio as part of our "old" standards and I would love to see that put back in place with some careful designing by LA teachers.

McNamara-Chapter 2 Communication & Test Design

This chapter looks at language test design methods from the past to present. One thing that should be noted is that many of the "old" methods are still in use to some degree.

The first important term the chapter discusses is "test construct" which is basically the aspects of knowledge or skills which are being measured (can be social not psychological). The author says the test construct is determined or influenced by the test creators view of language. So depending on the views of language you may get a pen/paper test vs. a performance assessment.

The chapter talks about discrete point tests, which kind of came about in the 60s (but still can be seen today). These tests were mostly multiple choice, basically tested a person's knowledge of grammar, vocabulary, and pronunciation, tested different parts of communication separately (ie: reading, speaking, oral). Even though this type of test is supposed to be outdated, I know that school districts still use them, and most likely other organizations too.

Next the chapter discusses integrative tests which focus more on being able to communicate in a language rather than have knowledge about how the language works. These tests integrated the systematic features of a language.

The article also talks about Pragmatic tests: the ability to integrate grammatical, lexical, contextual, and pragmatic knowledge in test performance . and Communicative tests which have two main features: Performance tests that required assessment
to be carried during extended communication, and Attention to social roles candidates were likely to assume in real world settings.

Something I found important from the Ch.2 powerpoint was this shift of thinking when it comes to Language:

MAJOR SHIFT
Communicative tests represent a shift
from a psychological perspective on
language, which sees language as an
internal phenomenon, to a sociological
one, focusing on the external, social
functions of language.


I find it so interesting that in language testing (same with language teaching in many cases) that we have all this research that shows what methods are ineffective, and yet many schools still use these outdated practices. Do all tests fit into one of these categories in this chapter? I'm assuming that they do, or at least into a combination of categories. It's interesting how the views of what language is can change the method in which we design tests and assess students (or at least should change them!)

Assessment & Accommodations of ELLs--Abedi

This post has been a long time coming :)

I have been very interested in our discussions about assessment accommodations. Sometimes it just gets so hard to deal with giving students tests that we know are not written with them in mind--it feels like a losing battle, like there is just nothing we can do--but if you consider all the accommodations (side note: I hate, HATE, hate typing the word accommodations, because I almost always spell it wrong) that we can make to assist students in getting past the language barrier in testing instructions and test questions we just might make a positive difference for some of our students, and allow them to at least feel some success when it comes to state testing. Anyway, with that said, the Abedi article brings up several issues we must keep in mind when considering what is reported on AYP regarding our ELL students.

The six major issues are as follows:


1. Inconsistency in LEP classification across and within states
2. Sparse LEP population
3. Lack of LEP subgroup stability
4. Measurement quality of AYP instruments of LEP students
5. LEP baseline scores
6. LEP cutoff points



Studies have shown that academic achievement tests (such as the ones that are used to calculate AYP) are constructed and normed for native English speakers, and have lower reliability and validity for LEP populations. Schools/Districts with high LEP population are more likely to fall in the category for "needs improvement." Studies show that tests that are linguistically modified for LEP students produce better test scores for the students taking them.

After reading this article (or after analyzing your own district's AYP reports (which i have now done both) you will see that there are several flaws in the way that LEP data is reported in AYP (as with problems of other subcategories), and it seems like the best we can hope for is that policy makers and other people in power will see the problems that this type of reporting is having on small rural populations with large populations (at least relatively large) of ELL students, and make changes that allow states to determine what type of assessment and reporting should be used for their schools.