Instructions for starting up with LOBA

With new faculty workshops going on, I’ve had a couple of instructors ask how to get started with Learning Objectives Based Assessment (LOBA) and I thought this would be a good time to summarize how I got started. That was one of the daunting tasks when I started out, and I feel I’ve learned quite a bit this past year.

Writing Your Objectives

Obviously, this is the starting point.  In an ideal world, the best time to start writing is during the term before you start while teaching the class.  All the difficulties and issues students have are fresh in your mind and it is easy to write down a list of objectives you’d like the students to have.  This doesn’t usually work out, so I’d recommend sitting down with your syllabus, a stack of old exams or assignments, and the text.  I chunked everything by chapter, but you can organize the learning objectives whatever way best fits the flow of your class.  As you go through the text, old exams, or assignments, keep a pad of paper next to you and write down important ideas, concepts, or skills that you would have tested for.  For instance, if you are coming from a points-based grading system  and you take points off for students forgetting to include units with numbers, this would be a good skill to write down.  Anything you normally deduct points for, or award points for is a good candidate for a learning objective.

Once you have a list of skills and concepts, start writing out learning objectives.  It is best to write them from the student perspective.  For instance, “I can include appropriate units with all numbers”.  The learning objective should make it very clear what the student needs to do and shouldn’t be vague.  A poor learning objective would be something like “I can write up a good problem solution”.  This doesn’t tell the students what they need to study, or what is important.  The learning objective should make clear what you expect of the student.  With something like writing a problem solution, I would break it up into several small, specific learning objectives like “I can include a free-body diagram with my solution”, “I can list all the known and unknown quantities in a problem”, and “I can check that my answer has correct units and the number I get is reasonable”.

How Many Learning Objectives?

Fewer is always better, both for the students and for your own sanity.  It is way too easy to  write hundreds of learning objectives for a course, but that sets the hurdle too high.  For a 15 week term, 45 learning objectives works out to about 3 learning objectives each week.  The deciding factor in how many is too many comes down to home many learning objectives you can reasonably test in a single assessment.  I have several small learning objectives that can be tested in a single problem, such as the problem solution learning objectives above, so I tend to have more (around 60 or 70 total).

Determine Your Assessment and Reassessment Policy

Placing limitations on student assessment and reassessment is key to students not trying to tackle too much material at once, and to your own sanity.  I typically have two assessments in class each week and only allow reassessment on three days a week.  I should note that having two in-class assessments a week does cost me in terms of how much material I can “cover”, but I prefer a flipped-classroom so I wasn’t assessing I would have students working practice problems normally.  Research has shown that more frequent assessment does lead to better student performance, so I feel the class time “lost” to assessment is a large improvement for students.

The first term I taught I allowed students to specify which learning objectives they wanted to reassess on, but I ended up spending too much time writing up individualized reassessments for each student, which wasn’t feasible.  I know there are instructors out there who do manage to individualize reassessments, but it didn’t work for me.  I let students choose which chapter they want to reassess on, and I have a stack of reassessments ready for each chapter.

I have set up a Google form that students must fill out for reassessment.  The benefit of this is that 1) data from the form is dumped into a Google spreadsheet so each morning I can look up who needs reassessing for each day and 2) I have a ready record of who took which assessment, and when they took it.

Reassessment Request Google Form

It is pretty easy to create a form.  Just go to drive.google.com (assuming you have a Google account, which is free) and click on the “create” button in the upper left side of the screen.  One of the options will be “Forms”.  It is relatively straight forward to create a form, but feel free to email me if you need help.  This form will dump all the data into a spreadsheet on your Google drive.

I require students to make corrections to previous assessments before reassessing, and that they complete a series of homework problems to show me they have practiced the skills.  I also require that students reassess at least once within three weeks of the original in-class assessment.  They are free to continue reassessing as many times as they’d like after that, for as long as they would like, but I don’t want them waiting until the end of the term before they get started reassessing.

Assigning Grades

Most of us need to turn in a final grade at the end of the term so we need to figure out how to go from learning objectives to letter grade.  I have two levels of learning objectives, C-Level learning objectives, which I consider to be easier and more basic, and the A-Level learning objectives, which tend to be more challenging.  To get a C in my class, a student must complete at least 97% of all of the C-Level learning objectives.  If they get 75% of the C-levels, they only get a C-, 65% is a D and 50% or fewer is an F.  To get a grade above a C is determined by how many of the A-levels they have completed.

So how do you complete a learning objective?  For each problem on each assessment, I have a few learning objectives that are tested by the problem.  Each learning objective is given a score from 1 to 4, where 1 indicates a beginning skill level, 2 indicates developing skill, a 3 represents proficiency, and 4 is for advanced performance.  To complete a learning objective, a student must get a proficient or advanced score at least twice on a learning objective.  Ideally I’d like students to show proficiency more often, but I feel that two proficients indicates the student has some knowledge of the appropriate skill.

One thing to note about scoring: a student can get the problem wrong and still get a mark of proficient.  If the learning objective is “I can use F = ma to find the acceleration of a particle” and they have the right idea but make an arithmetic error, I would still give them a proficient mark.  It turns out it is much easier to determine proficiency that to determine how many points you should deduct for a particular error, so proficiency grading tends to be faster than points-based grading.

Keeping Track of Everything

When I first started out, I put boxes with the learning objective number next to each problem, but I found that students would memorize which learning objectives they needed, and that keyed them into how to solve each problem.  To combat this, and to make recording grades easier, I created cover sheets for each assessment like this one:

Example Assessment Grade Sheet

Having all of the learning objective scores on one sheet sped up grade entry, and insured that students couldn’t use the learning objective numbers to figure out how to solve a particular problem; it isn’t enough that a student can solve a problem using energy conservation, I want them to show they know WHEN to use energy conservation.

There are a couple of websites that allow you to track students progress (Blue Harvest or ActiveGrade, for example), and they are fairly good, but they currently don’t support the method I use for determining grades (edit:  Turns out ActiveGrade can do this..Thank you Riley Lark for correcting me).  I don’t average scores together because I don’t want students to think of their assessment scores as points.  I require students get a mark of proficient or higher at least twice to pass a learning objective.  I use a Excel spreadsheet to record individual assessment scores and the spreadsheet figures out how many times a student has passed each learning objective (Email me if you are interested in getting a copy of my spreadsheet code).

Write Your Assessments

If you’ve made it through all of these steps, congratulations, because you are now ready for class to start.  All that you need to do is write your assessments, which doesn’t have to be any different than writing a quiz or exam.  My assessments are really just like a quiz, but I call them assessments to put students in a different mind frame. Students are afraid of doing poorly on a quiz, since it tends to be a one-shot affair, but assessments can be retaken, so there is less stress.  I also like the term assessment because it can take other forms, such as oral presentations, webcasts, or simply answering a few questions in my office.  I’m not constrained by the traditional format of quizzes and exams.

I currently only have three or four versions of each assessment, with problems from widely varying contexts.  I keep track in my Google spreadsheet which version I gave each student so when they come in to reassess I don’t give them the same version.  I don’t doubt that some students might get copies of previous reassessments from other students, but I haven’t seen and evidence of it and for the most part I don’t think only having a few versions is a problem.  Each term I plan on adding another version or two, so that in short order I’ll have enough reassessments that I won’t need to worry about student’s sharing answers.

Sell It To The Students

Your students have spent at least a decade mastering how to get the most points with the least amount of effort and they will be very nervous with such a radically different grading paradigm.  You will need to explain to your students why you are doing this, spend time reassuring them that being able to reassess means less stress on them, and the learning objectives make it very clear exactly what they need to do.

You will also need to explain the grading on several different occasions over the first several weeks.  Even after three weeks, some of the students had questions about how things worked so I gladly set aside time to explain things to them.  Once they got the hang of it, though, many of them liked it.

Three Final Thoughts

The first time I taught using LOBA, I handed out a compete list of all learning objectives at the beginning of the term.  Unfortunately I found some objectives too easy, some too hard, and some just a waste of time and I changed some of them mid-term.  Big mistake.  The students felt I was moving the bar and were very stressed out by this.  If I had to start over again, I think I would only hand out a list of learning objectives for the next chapter or unit as we covered the material, which would allow me to change later learning objectives on the fly.  This also means you don’t have to have all of your objectives nailed down before the term starts, which can take some of the stress off of you.

I have found that my grade distribution is skewed towards higher grades.  I first thought that I wasn’t setting the bar high enough for A’s, but I realized that since students know what they need to do to get an A, they can push themselves.  I had one student who figured they were getting a B+ and where happy with that, but when they realized they only needed a few more learning objectives to reach the A- mark, they pushed to get it.  If a student knows exactly what they need to do to get a better grade, they will.

Lastly, I highly recommend checking out blog posts on the web.  I have a number of other posts on LOBA (or Standards Based Grading (SBG) as it is sometimes called) and there are a number of great resources.  I’d recommend Frank Noschese’s blog, Action-Reaction, John Burk’s Quantum Progress, and any of the SBG Gala’s (Gala #6 is here and has a list to earlier galas).  Check out these great resources.  Best of luck.  You are going to love it!

Addition:

A few people have asked for the Excel gradebook file so I thought I’d post it here: Gradebook – LOBA

Instructions:  On the “Totals” tab, enter the student names in the first two columns.  Copy and paste that list of names to the second tab labeled “Assessments”.  The order of the names matters because row 2 on the “Totals” tab is connected to row 4 on the “Assessment” tab.  Columns C and D on the “Totals” tab tells you how many A-level and C-level learning objectives each student has completed (scored 3 or 4 on that LO at least twice).  The top row of this tab is for the name or number of each learning objective.  On the “Assessments” tab, you can enter the scores for each student on each learning objective.  I hope this is helpful to some of you.

About these ads
This entry was posted in Teaching Physics and tagged , , , , . Bookmark the permalink.

12 Responses to Instructions for starting up with LOBA

  1. Riley Lark says:

    Thanks for making this post, and for mentioning ActiveGrade! Just an FYI: ActiveGrade *can* support your method of requiring two “proficient” scores and turning that into a binary result – we offer custom combination programming.

  2. Joss Ives says:

    Wonderful comprehensive post. Bookmarked! Approximately how long (in terms of time) are those twice-a-week in-class assessments and, on average, how many LOs do they assess?

  3. Andy "SuperFly" Rundquist says:

    great post, thanks for the level of detail! I wanted to second the notion of roughly one standard per class period. I found that really helped focus what I do in class. I point out that it’s the only class day on that standard and so they should get all their questions in.

  4. Karen King says:

    This is very helpful as I try this for the first time this fall (and thanks to Andy, too, for your help!). As a flipped class, do you find you need to motivate students with grading/points to do the reading and practice problems? I’d like to hear what others do. I’d like to make the grade only based on the standards, but I’m wondering if my students will do the work outside of class that they need to do to be prepared each day. Right now I was thinking of making the mastery of learning objectives 85% of the grade, and reading quizzes/practice problems worth 15%, just to have some external motivation. Have you found that to be a problem?

    • I don’t give any credit for reading or homework and I don’t do reading quizzes. I really do want them focusing on the learning objectives and on learning how to accomplish those learning objectives. At the start of the term very few students do the homework or reading, but I find that as the term progresses and they realize the importance of those things they start to do more of them. I do require they show they have worked homework problems to reassess, which does motivate them. Some of the students don’t learn to study, don’t bother reading or doing homework, and they end up failing. Since these are college students I feel that it is appropriate that they need to learn how to be self-sufficient learners. At the K12 level my philosophy might not be appropriate. A possible solution might be to have a learning objective that states, for example, “I can answer 75% of the reading quizzes questions correctly”. Of course you would then need to give them chances to reassess, which would mean making more reading quizzes. Does anyone else have any thoughts?

  5. Joss Ives says:

    I found your Fall 2011 standards, but I’m wondering if you would be willing to share your current set of standards for Mechanics and E&M? It sounds like they have probably become more finely honed after having them come into contact with students multiple times.

    • I haven’t taught the E&M yet, but I’ll talk to my colleague who has been teaching it using LOBA for the second time now and see if she’d let me post them.

      • Joss Ives says:

        Although I won’t be teaching Mechanics until at least next fall, I would be interested in seeing a topic’s worth of LOs to gain a better sense of what distinguishes your A- and C-level LOs.

    • I’ve posted my learning objectives and E&M learning objectives in a post. I hope this helps and if you use any of them let me know what improvements you end up making. Here is the post.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s