weighting rubric

PBL Rubric
I use project-based and problem-based learning a lot in my own teaching, as well as research it, recommend it, and present workshops on it.  Consequently, I am often asked about assessments and grading that come along with using PBLs and inquiry.  In conversations that I have with teachers and faculty members about using rubrics for grading student products, or learning artifacts, I consistently find there are two challenges that make rubrics not as effective as they could be.

Misalignment

The first challenge that I often observe or hear about is a misalignment between what the teacher or faculty member actually cares about and what is listed as criteria on a rubric.  For teachers, I usually see this as a mismatch between what they have listed on their lesson plans for the objectives or goals of the lesson and then what criteria they have listed in the rubric.

I try to remind teachers and faculty members that the objectives and goals of your lesson should be directly reflected (read as “obvious”) in the rubric.  That is, using the language from Bloom’s Taxonomy, such as “compare and contrast” or “analyze” or “explain,” that is embedded in your objectives or goals should be embedded overtly in the criteria for your rubric.

Often, I find that teachers and faculty members list criteria that are part of the requirements for the artifact, such as number of slides, number of pages/words, or spelling, but they fail to adequately list the very criteria for which they designed the lesson.  So, don’t forget to include criteria that delineate to students the quality you expect in their comparisons, analyses, or explanations.

Weighting

The second challenge I observe when teachers and faculty members use rubrics is inadequate weighting of criteria.  This is evident in a rubric when a student can do average or better (so maybe a “C” or better) with your rubric and still fail to understand the primary course content.

In my research, students told me it was easier to get a good grade with a project than it was on a test.  This calls into question the rigor of our rubrics.

Where I see this most prevalent is when teachers and faculty members use rubric maker tools.  I am a huge fan of tools such as iRubric, Rubric Maker and Rubistar.  I frequently recommend these tools to teachers as a beginning to building their own rubrics (see an overview at Edudemic).  I also always recommend that rubric designers (that’s teachers and faculty members) handle weighting in one of two ways inside of these tools.

Add a Column

The first way to handle weighting in a rubric is to add a column for weighting, such as a percentage or with points.  (I know this seems a little obvious.)  Many rubric tools, however, do not include this in their settings.  So, you have to go in and add this column.  I also encourage you to make sure that you weight most heavily what you care about most.  If the “scientific knowledge” or “analysis” or “articulation of symbolism” is what you care about most, make sure it is weighted appropriately.  Again, I believe that a student shouldn’t be able to do average or better on a performance-based assessment without demonstrating the knowledge and skills of the assignment. There is a great post by Pamela Flash that walks you through step-by-step of building a rubric and weighting is the second step.

Add Criteria

The second way to handle weighting is to add more criteria for the knowledge, skills, or requirements that you care most about.  For example, if you are emphasizing “compare and contrast,” then you may be able to break this down into more criteria, such as “identified…” and “supported…” and then “compared…”  If there is scientific knowledge that you are expecting students to “explain,” then you may be able to break this down into specific criteria.  Again, you’re adding criteria in order to increase the weight for the set of knowledge or skills that you care about.

Bonus Tip!

While I encourage teachers and faculty members to share their rubrics with students ahead of time (as well as consider having students collaborate in the creation of the rubric!), I also think it’s a great idea to turn the rubric into a self-check checklist for the students prior to submissions.

Following Up!

  1. Here’s a great article by Madeline Marcotte about rubrics and the pros and cons of using them.
  2. There’s another great post by John Larmer and the folks at BIE on how to use their rubrics for 21st Century Skills and PBL, so you may want to take a look at that one.
  3. Finally, Grant Wiggins offers up a thoughtful post on how rubrics can be created and how they should be used.  It’s thoughtful in that it offers some philosophical viewpoints about when and how rubrics should be used and what their purposes can be.

I would love to hear your thoughts on rubrics and how they work for you and what you struggle with when using them.  Please share your thoughts and any additional resources in the comments below. Good ideas are always welcome!

Creative Commons License Photo Credit: Kathy Cassidy via Compfight

I am super excited to be working with the Baptist College of Health Sciences here in Memphis.  I’ve been asked to present to their faculty as part of a faculty professional development day, so this is a great opportunity to share about problem-based learning, project-based learning, and some strategies to help with teaching online and hybrid courses.  This certainly overlaps with my work with the Interdisciplinary Journal of Problem-based Learning, as well as some of the teacher professional development I’ve been doing recently, too. Below are the two slidedecks that I will be using.

[slideshare id=25621707&doc=pbl-recovered-130826222508-phpapp02]

[slideshare id=25621802&doc=engaging-backup-130826223037-phpapp02]

If you happen to have questions about any of these, please let me know.

A few years ago, I wrote a book chapter with Janette Hill at The University of Georgia on the complexities of implementing student-centered pedagogies, like project-based learning and problem-based learning.  This chapter “Weighing the Risks with the Rewards: Implementing Student-centered Pedagogy within High Stakes Testing” was published in Understanding Teacher Stress in an Age of Accountability edited by Richard Lambert and Christopher McCarthy, and it seems to be even more relevant as we head toward Common Core implementations and PARCC assessments in Tennessee.

In addition, I feel that there is a growing interest in inquiry and student-centered pedagogies within STEM disciplines. So, I thought I would provide the chapter and link in case you’re interested.

http://www.academia.edu/894278/Weighing_the_Risks_with_the_Rewards_Implementing_Student-centered_Pedagogy_within_High_Stakes_Testing

While somewhat theoretical, this chapter is grounded in the work I’ve done over the years in project-based learning and problem-based learning with K-12 and higher education.  In addition, it presents a balanced view of how students and teachers must adjust and work within their environments.

A few days ago, MindShift, a site published by KQED, published an article about project-based learning (via What Project-Based Learning Is — and What It Isn’t | MindShift).  It focused on one teacher’s vision and goals of project-based learning.  Azul Terronez is an eighth grade teacher at High Tech Middle in San Diego, CA. You can see the description of Terronez’s project-based learning below in this excerpt from MindShift’s article:

When an educator teaches a unit of study, then assigns a project, that is not project-based learning because the discovery didn’t arise from the project itself. And kids can see through the idea of a so-called “fun project” for what it often is – busy work. “They don’t see it as learning; they see it as something else to do,” said Terronez. “They don’t see the value.”

For Terronez, the goal is to always connect classroom learning to its applications in the outside world. He’s found that when the project is based in the real world, addressing problems that people actually face, and not focused on a grade, students are naturally invested.

A Continuum, though

I can honestly say that I don’t disagree with Terronez’s description and goals at all for project-based learning.  In fact, it’s how I would prefer project-based learning to occur.  However, it does seem to discount the other possibilities for project-based learning, or “project-oriented learning” as it’s called in the article.  In my research, particularly with former student Dr. Suha Tamim, we found that there is for sure a continuum of which teachers implement project-based learning.

Some teachers do choose to do projects as Terronez’s describes, that is as the single method of instruction for students to learn new content.  However, we found that other teachers use projects as Terronez also mentions as reinforcement.  But we also found that:

  • some teachers use projects as a method to allow students to go deeper,
  • some teachers use projects as a method to allow students to represent their learning in multiple ways, and
  • some teachers use projects in all of these ways.

From our small research study that we hope to be published in the Interdisciplinary Journal of Problem-based Learning soon, it seemed likely that teachers used project-based learning with an alignment to their own pedagogical beliefs about how teaching and learning should occur.  Moreover, teachers felt constrained by their school, district, assessments, or curriculum to not do things differently.

And Value?

And maybe that’s the significant point to note from this article. High Tech Middle is school-wide initiative committed to project-based learning as Terronez describes, a variation on High Tech High’s model.  If an entire schools adopts project-based learning to this level, then the supports and scafffolds for teachers and students to teach and learn in this method are there. (Frankly, I’m jealous of the work happening at High Tech Highs and now High Tech Middles across the country. A real model that should be considered more.)

However, if a teacher is doing project-based learning alone, then variations are inevitable, because the same supports are not available to the teachers or the students.  But that doesn’t make the variations ineffective at achieving learning goals. Nor should they be discounted as valuable.

ijpbl-header-image-mgrant-60

I just wanted to provide a quick update on the stats for IJPBL from last month, because we continue to grow in use and downloads.  In October 2012, the Interdisciplinary Journal of Problem-based Learning had 5500 full-text downloads. The most popular papers were:

  1. Overview of Problem-based Learning: Definitions and Distinctions (1160 downloads)
    http://docs.lib.purdue.edu/ijpbl/vol1/iss1/3
  2. Storytelling as an Instructional Method: Definitions and Research Questions (338 downloads)
    http://docs.lib.purdue.edu/ijpbl/vol3/iss2/3
  3. Goals and Strategies of a Problem-based Learning Facilitator (336 downloads)
    http://docs.lib.purdue.edu/ijpbl/vol1/iss1/4

IJPBL now holds 83 articles, which have been downloaded a total of 182335 times. As a reminder, IJPBL’s acceptance rate has been around 11% in the past year, so it is a great place to publish your work for both rigor and access.

IJPBL is an open access journal that publishes articles on problem-based learning, project-based learning, case based learning, and inquiry methods. It is interdisciplinary, considering research from many fields, including medicine, engineering education, and education.  We are now indexed in PsychInfo.  Beginning in our spring 2013 issue, we will be launching a new section of the journal that represents more first-hand accounts of individual implementations of inquiry methods. Tentatively titled “In the Trenches,” this section will allow individual teachers, instructors, faculty members, or schools describe their implementations with more context and lessons learned.

ijpbl header image

As co-editor of IJPBL, I am sending out a Fall update for our journal.  There are some really exciting things happening with IJPBL, and I wanted to share those out with our followers, reviewers, and editorial board.

Good morning/afternoon, colleagues and friends!

Welcome back to a new academic year and fall (for most of us).  I hope your work is rewarding and it is going smoothly. I’m looking forward to seeing many of you at the annual AECT conference in Louisville next month. If you are attending AECT, please stop me in the hallway and say, hello. I would like to tell you thank you for your service in person.

Exciting things are happening, so I wanted to take this opportunity to update you on IJPBL.

New Co-Editor

I am pleased to announce that Dr. Krista Glazewski (glaze@indiana.edu) has been selected as the new Co-Editor of IJPBL.  Krista is an Associate Professor at Indiana University in their Instructional Systems Technology department (http://site.educ.indiana.edu/ProfilePlaceHolder/tabid/6210/Default.aspx?u=glaze ).  Please generously welcome Krista on board!

The slate of candidates for co-editor was extremely well qualified and their expertise deep.  So, this was a difficult decision.  I would also like to thank the current and former editorial board members, who provided feedback to Peg and myself, for helping us reach a decision.

Website Upgrades

If you have not visited http://ijpbl.org recently, I highly encourage you to surf on over.  During early summer, I (Michael) worked with our publisher Purdue Press to update our site and make it more contemporary.  Purdue Press and BePress (Berkeley Electronic Press that runs & supports the publishing system) consider IJPBL to be a model for open access publishing, achieving both rigor and sustainability. So, all of the updates were provided pro bono!  Also, to increase our visibility with your colleagues, peers, and students, consider adding a tagline to your email signature for IJPBL and a link to our new site.

Volume 6, Issue 2 was just published ahead of schedule with four articles and a book review.  If you haven’t had a chance to take a look, then I encourage you to take a look at our newest published research: http://docs.lib.purdue.edu/ijpbl/

New Features

Over the next year, we are looking to add two new features to IJPBL.  First, we will be working toward progressive publishing.  So, instead of publishing articles in an issue all at once, we will be experimenting with publishing online first and then gathering the articles together for the published issue.  This will help us take advantage of publishing to the web and open access better, as well as help authors get their works published sooner.  Be on the lookout for articles that are published online first!

Second, we are looking to add a Pedagogy or “In the Trenches” section to IJPBL.  This section would be in addition to the rigorous research we currently publish.  We are working on the guidelines over the next few weeks.  This section will represent more individual accounts of problem-based, project-based, case-based, and inquiry-based implementations.  If you have ideas about how this section may be crafted, then please forward those to me.  Also, if you or colleagues have a possible manuscript idea for this section, please let me know that as well.  We know there are folks who are experimenting with PBLs individually within courses, classes, or schools and may not be conducting rigorous evaluations, but they are learning a lot about what’s working.  We’d like to share those stories in a strategic manner.  I would love to talk with you more about this and how it might fit within our expanded publishing.

2014 Special Issue CFP

As a reminder, we are currently accepting proposals for our 2014 Special Issue on technology-supported PBL in teacher education.  This issue will be edited by Drs. Tom Brush (Indiana University) and John Saye (Auburn University).  You can find all the details by downloading the call for proposals at http://docs.lib.purdue.edu/ijpbl/vol6/iss2/7/ .  The deadline for proposals is October 15, 2012.  Please share this with your students and colleagues and across your professional/social networks.

Recruiting Reviewers

We are currently recruiting new reviewers for IJPBL.  Our best method for recruiting reviews is from recommendations by you.  So, I would like to encourage you to recommend two (just 2!) appropriate colleagues visit http://bit.ly/ijpbl-reviewers  and sign up to review.  If everyone does this, then our pool of reviewers would grow substantially.

Here is a sample email that you can edit-copy-paste to invite your colleagues:

On behalf of the Interdisciplinary Journal of Problem-based Learning, I would like to invite you to become a consulting editor for our journal. We believe your expertise would make an excellent contribution to our journal and offer the type of quality feedback authors deserve. Each consulting editor is invited to review approximately two manuscripts per year, and you are invited to attend our annual meeting held each spring in conjunction with the American Educational Research Association (AERA) conference. Your name will also be published in our Consulting Reviewers list at the beginning of each issue.

I hope that you will consider joining our reviewers’ board. Please fill out the form below.
http://bit.ly/ijpbl-reviewers

IJPBL continues to grow, and much of this is due to your work.  So, thank you for your continued support.  Of course, if you have questions about anything we’ve described here or comments about how to better improve IJPBL, please email us directly.

Have a great day!

I am super excited to let everyone know that Dr. Alistair Windsor and I have been awarded a science, technology, engineering, and math teacher professional development grant with Tipton County Schools and Lauderdale County Schools here in Tennessee.  This grant titled “Mobiles, Math, INquiry & Data (mMIND)” is funded by the Tennessee Department of Education‘s higher education commission and our state’s First to the Top funds as part of the Race to the Top federal program.  Below is a brief abstract of our professional development program. We are in the process of recruiting math and science teachers in Tipton and Lauderdale Counties.

The purpose of this project is to provide high quality, research-based teacher PD to 30 math and science teachers in Grades 7, 8, and 9 in Tipton and Lauderdale Counties of Tennessee. Specifically, this PD program will target (a) mathematical content knowledge, (b) mathematical pedagogical-content knowledge, (c) interdisciplinary inquiry through problem-based and project-based learning (PBLs) strategies, and (d) technology integration with mobile computing devices. This will be accomplished through summer teacher academies, continuing professional development, face-to-face classes, synchronous online sessions through Adobe Connect, and asynchronous online activities. By involving both math and science teachers, both content areas will be able to integrate CCSSM with pre-algebra/algebra, PBLs, authentic learning scenarios for applying math and science, and data analysis, representation, and interpretation methods, as well as active mobile teaching and learning strategies.

In addition, we will create professional learning communities (PLCs) at the building level and communities of practice at the district levels. With math and science teachers at the building level collaborating in a PLC, the teachers will be better situated to create interdisciplinary and complementary STEM lessons, which researchers suggest will prevent inert knowledge (Bereiter & Scardamalia, 1985) and increase generalizability of knowledge in multiple content areas (Grant & Branch, 2005; Spiro, Feltovich, Jacobson, & Coulson, 1992). At the district level, teachers will be able to collaborate and debrief across school boundaries, and share best practices.

Alert in pencile

Given my interest in problem-based and project-based learning, I thought I would share this video that came through my Zite feed late last week. It’s a TEDxLondon video from Ewan McIntosh in September 2011. Ewan blogged about the presentation and “The Problem Finders,” as he calls them recently on his NoTosh site. He says that,

I’ve sought out ways that we can give more of the learning process back to learners: so much of the hard work is done by teachers: scoping out what is ‘worth’ studying, preparing questions worth answering (or not worth answering!) and assessing the learning of students.

This is at the heart of problem-based and project-based learning. He goes on to say that

We’re working every week now with schools across the world in building The Design Thinking School, a pedagogical framework that borrows from enquiry-based learning and problem-solving curricula to bring new meaning and relevance to students, and we’re finding that such a framework works regardless of curriculum, country, culture or language. In independent schools with parents wanting top marks, in city schools where students are disengaged, in suburb schools were students are successful but bored… in every case it’s leading to more engaged students and better academic performance, in both elementary and high schools.

This is great. I hope you enjoy the video.

[youtube]http://www.youtube.com/watch?v=JUnhyyw8_kY[/youtube]

If my grant I just submitted for the Tennessee STEM teacher professional development is funded, I plan on using this as part of the professional development. What do you think of the video and work that Ewan is doing? Let me know your thoughts.

You may have just read about the new issue of IJPBL in my previous post.  I just wanted to let everyone know about this new call for manuscript proposals for a special issue of The Interdisciplinary Journal of Problem-based Learning. This special issue will celebrate the work of Howard Barrows, who is widely considered the founding father of problem-based learning.  From the call announcement:

The special issue will include papers that:

  • As a collection, represents the diversity of fields in which PBL is used.
  • Investigate PBL or particular facets of PBL, such as simulated patient assessments, or facilitation, in one of the following ways:
    • theoretical papers (limited number accepted)
    • empirical investigations (defined broadly as quantitative, qualitative, or mixed-methods)
    • literaturereviews (limited number accepted)

In addition the special issue editors (Andrew Walker, Heather Leary, and Cindy Hmelo-Silver) are interested in representing problem-based learning currently as “PBL is used as a label for a wide variety of interventions (1986). We want to represent that breadth as part of the special issue but want authors to be clear about how their own variations depart from Barrows’ norms.”

The deadlines for the submissions are:

  • Proposal (1000 words) submission deadline: November 1, 2011 (extended from previous announcement of October 1)
  • Full paper invitation notice: December 1, 2011
  • Full papers (3000-5000 words) submission deadline: March 15, 2012

If you are doing research in this area, then I highly encourage you to submit a proposal and review the full call on the IJPBL site at http://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=1252&context=ijpbl

Hey everyone! I wanted to let you know there is a new issue of  Interdisciplinary Journal of Problem-based Learning published today.  This is Volume 5, Issue 2.  Below is a list of the articles.

There a few real highlights to this issue. There is a tribute piece to Howard Barrows. There is an update from David Jonasen on how problem-solving has evolved. Plus, there is a call for proposals on problem-based learning! (Oh, and there happens to be an article by someone really familiar, too.)