Atomizing Assessments

Crowd-sourcing training and learning assessments
by Katin Imes

We have an interesting swirl of needs and resources appearing in the domains of learning and professional development, and I'm curious how we can add layers of efficiency and curation for learners via crowd-sourced or distributed methods. Learners helping learners, you could say.

The needs I see appearing:

  • in-the moment learning and skills development;
  • guidance for learning paths for new skills and knowledge;
  • measurement of accomplished learning or skill development.

Why Life-long Learning?

A skills and knowledge gap is keeping unemployment high.

UPDATE: This New York Times article titled, "Skills Don't Pay the Bills" at first blush seems to be painting a position opposite of my post, saying that American employers have lowered wages so low that it doesn't matter what skills you have, workers can't afford to work for them. However, if you read to the last line of the article, you'll see the conclusion is the same: we need to get better at learning at all ages and levels -- fast -- if the USA wants to stay a world-player in the 21st century.

We've all heard the mantra of "jobs, jobs, jobs." We hear the stories of companies they receiving hundreds of resumes for every job opening advertised. This naturally creates the idea that jobs are scarce out there.

But did you know that there are presently 3.6 MILLION job openings in America right now? (As of the last day of September, 2012, according to the U.S. Department of Labor's Job Openings and Labor Turnover Report, Nov 6, 2012.) And that number has remained unchanged since August.


21st-Century Basics: Maximizing Learning Environments

Learning Environment Strengths Comparison Chart

Here are some things we know:

  • Different people learn at different rates.
  • Different people have different connections, doorways, and interests related to any given class or topic.
  • We have amazing new technologies that allow us to shift, shuffle, relocate and deliver unprecedented amounts of information, video, materials, and communications to students, teachers, and parents.

Here are some questions you may have heard or might be asking yourself:

Now that Internet access and a modern computer are becoming common in American homes with students, we have new powerful teaching technology. On-demand access to lectures, examples, tutorials, simulations, and forums for learning just about anything you can imagine is a mind-blowing difference from just twenty years ago.

It really is a whole new world; it's just that many people don't know it yet.

The core principle to apply now is: make the best use of each modality and technology to serve students, teachers, parents and society.

In general, that boils down to making the very best learning environments possible. That means understanding the attributes of each asset -- the strengths and weaknesses -- and applying them accordingly.

For example, is learning on the computer a good replacement for mentoring, support and assessment that teachers can provide in the classroom? No, computer learning environments are not good at those things. So don't use computer learning for those things. Those are strengths of the classroom.

Meanwhile, is the classroom with a live teacher a good replacement for on-demand video? No. Teachers and classmates don't have pause or rewind buttons. Teachers don't enjoy repeating the same lecture for each class, over and over. It's a waste of the teacher's time; humans with college degrees and teaching credentials make for very expensive tape recorders / VCRs / CD-Players. It doesn't serve the students well, it doesn't serve the teachers well. So don't use classrooms for those kinds of things. These are strengths of Internet video.

"Blended Learning" is the term to mean a hybrid, part-computer, part-classroom approach. The goal is, of course, to create the most balanced, effective blended-learning mix possible. How do you know when you have a good mix? When students excel, learn faster and deeper, develop higher order skills, and report that they are enjoying the process. Some might be surprised to learn that when application and use of resources are balanced and optimized for best results, it is less expensive than inefficiently applying those resources - resources like classrooms and teachers.

What the end result? Well, it is the end result that is needed to be successful in the ever-more-complex, problem-addled 21st-century: better results at a lower cost. And when you apply that to educating more engineers, writers, leaders and knowledge workers, it means a huge impact on the success and wealth of America.

If you aren't familiar with the idea of a "flipped classroom", here's a good introduction.
Sal Khan Video Lecture on Digital Learning | Khan Academy
Also, see my resource page on Flipped Classrooms

Realize that the focus is on learning, not education.
Re-Imagining Learning in the 21st Century | MacArthur Foundation

Let's shift the focus of student activities away from consuming media to creating and producing it.
Rethinking Learning: The 21st Century Learner | MacArthur Foundation


Education: The Problem & The Solution

Note: Many of you, especially education wonks, will have heard all this before (except the solution video, below, may be new to you). For those outside of the educational system, these two videos succinctly yet positively summarize two foundational points. If you have kids in school right now, realize they are in the most difficult transition in educational history. As a parent, you'll want to take the extra effort to get your kids more 21st century education than 20th century education.

The Problem

The current school system, from kindergarten through college, was designed and evolved for an industrial nation, and an industrial age. It was good at some things, and not so good at others during that age.

Now, it seems the 20th century education system is good for very little. The high school drop-out rates are unacceptably high, especially for boys. Business is complaining that new graduates aren't hirable because they don't have the skills required, even in our time of record-high unemployment. Teachers complain they are little more than babysitters, and the laws restricting the disciplinary actions and liability of teachers and school districts mean that the police are being called in to handle classroom discipline. (Two examples from the WTF file are here and here, -- and it keeps happening, adding this to the list on Sept 27, 2012.) Things are a bit of a mess.

It is a system out of its time. Just as public education came into being in the late 1800's in order to support the Industrial Age, so will the next educational system come into being to support the Information Age. There is no choice here; it will change, it is already happening. The only question is: how painful will the change be for you and yours? Will you or your child be more 20th century or more 21st century when their schooling is completed?

Sir Ken Robinson explains the link between 3 troubling trends: rising drop-out rates, schools' dwindling stake in the arts, and ADHD in this RSA Animated Talk.

(You can watch the full, non-animated version, 55mins, here.) You can't hear him speak the stats on redefining genius and divergent thinking in the animation; it was displayed on a slide in the original talk at 49:14. The stats are:
Ages 3-5: 98%
Ages 8-10: 32%
Ages 13-15: 10%
Ages 25 and older: 2% (sample of 200,000 adults as a control)

The Solution

Kids need to be doing the kinds of activities and learning the kinds of skills that are foundational for an Information Age economy. Here's a terrific example.

This year, David Preston's high school students engaged online thought leaders and used open source principles and tools to create:

  • a microfinance operation
  • a hackspace
  • a venture incubator
  • an online art dealership
  • a P2P digital consulting practice
  • a game lab
  • an international digital research partnership
  • an alumni/friends community designed to evolve into a massively multiplayer open source learning environment that translates knowledge capital directly to the marketplace

Several students documented the experience in the following work-in-progress video, "We Are Superman."

David Preston holds a Ph.D. in Education Policy from the UCLA Graduate School of Education & Information Science. He has taught at universities and graduate institutes and consulted on matters of learning and organizational development for 20 years. For the past seven years David has also taught English for students of all ability levels in grades 9-12 at America’s fourth largest high school (in Los Angeles) and at a comprehensive high school on California’s central coast. More on David and his work at


Dr Edward de Bono at Creative Innovation 2010 - "Re-thinking the future"

As a young man in my 20's, discovering Edward de Bono's book, "Lateral Thinking: Creativity Step by Step" was a big deal. It was a new approach and seemed to amplify a kind of "smart" that hadn't been defined for me in school.

Here he presents an entertaining 18-minute talk that nicely summarizes his hypothesis that society's level and methods of thinking are "excellent for some things but not good enough."


TED launches TED-Ed!

When it comes to computer code and databases, simple combinations can make huge impact.

The new TED-Ed is an example. Take any YouTube video, frame it with teaching text about "Thinking Deeper", add a simple short quiz and a set of resources and text to "Dig Deeper", and you have an instant learning lesson suitable for anything you might be teaching (or learning!) for students of any age.