QStream review: a 3-minutes-a-day question stream

I've been experimenting with the learning tool QStream.com, formerly known as SpacedEd.com. I'm assuming here that QStream stands for Question Stream, as that is exactly what you get: every few days you get one or two questions in your mailbox, and they get out of the stream when answered correctly two times.

In their own words:

Qstream is a comprehensive and proven learning methodology developed with physicians, students and medical practitioners. The service bundles a complete learning platform for developing and delivering online learning that offers a convenient, fun and effective addictive learning experience directly to users in a simple “question a day” format. Anyone with access to the Web via a computer or mobile device can sign up and take courses.

In my words:

Three minutes of questions a day, keeps the trainer away...

 

What it does...

As an author you can go in and make a 'course'. (Yes, they still use the established education language.) Only, it is not a traditional course, it is essentially a collection of multiple choice questions. The editor is straightforward and completely browser based. You can leave most of the options as they are, but if you want to, you may tweak the default delivery schedule of the question pool and various other parameters. 

On the course level you specify the title, tags, category and description that will show up in the catalog. A course can also have multiple authors. The free version allows you only to publish publicly available courses. There are options to charge money, but I haven't tried them out and wouldn't know how that works.

The questions are either multiple choice, check all that apply or fill in the blank. When you add questions, keep in mind these are questions for developmental purposes, not for assessment! That is why the questions should relate as much to real work situations rather than being overly theoretical, and should come with proper feedback. When learners answer wrongly, it is the quality of feedback that will make them learn and avoid answering wrong again x days later.

As a learner, you can browse the catalog and enrol in a course. That will get you a schedule of daily questions. It won't take more than a few minutes of your time.


My course: feedback that works

But instead of reading about it, just experience it: I created a question pool "Feedback That Works". Go over there, make a free account en enrol in the course. You'll get a first set of questions and then a few more every few days. Wait, you'll say, I don't know anything about the topic, I can't just go into questions, can I? As it turns out, that was one of the things I wanted to find out with this little experiment. I have send out the link to people in the office, and quite a few were curious enough to enrol and give me feedback on the experience.

What I wanted to find out:

• Is this an effective way to continue the learning process started in the classroom learning, ie does it enhance retention and application?
• Does it work without having any learning on the topicbefore? (in other words, is the method of asking questions and getting feedback enough)
• Do people complete the journey? (or do they find it takes either too much time or isn't worthwile or isn't fun?)

The questions are taken from an elearning title with the same name by the Center for Creative Leadership. Feedback That Works is one of the more popular titles, training people on the Situation-Behavior-Impact feedback model. 

The reactions

Here are some reactions:

• I think I got most questions right because I knew them beforehand, not sure what would happen if the topic was unknown for me. However, I believe I will get right the questions I failed. (H)
• I think the tool is a very good one to keep you hooked without being invasive, very practical and straight to the point. I also think it would work to keep participants engaged in a soft way, particularly when we have several modules and we need to keep <...> in their minds in between. <...> I have liked it very much, I have enrolled to another (S)
• I have liked this method very much as it enabled me to really “digest” the information and therefore memorize much more easily than when I read the Guidebook. Although I would say that the questions reinforced the learnings I got from the book, I cannot say how efficient it would have been if I had absolutely no knowledge of the subject, it may simply have taken a bit longer for me to understand the subject and complete the course. (MA)
• I’m almost finished with the eLearning “Feedback That Works” course and wanted to share my thoughts with you on the experience. I’m not sure I was the most effective tester because the content was not new to me. I was already familiar with it, but what I liked was that it was a nice refresher for me and I definitely feel like it has been effective in terms of helping me remember some of the key points about giving feedback and WHY the best practices are effective. The explanation of why an answer is true or false is essential to the process.  <...> Other things that I liked specifically about the system include: The option to manually retire questions <...>, The option to change the email frequency <...> The thing I’m less sure about is whether I would have liked this course as much if the content was brand new to me. I’m a perfectionist and I don’t like to be wrong. Answering these questions feels a bit like I’m taking a test and all my life I’ve been taught that it’s bad to get test answers wrong. <...> I think the system has a lot of potential overall. It’s a novel way to introduce new content without a lot of lecture or reading, which is a nice change of pace. (K) 

In a 'guineapig gathering' session, the general thoughts on the tool were that it is indeed fun and engaging, doesn't take much time, and works to remind and prompts to apply in the work. But people also found that the tool would not work for them to learn a brand new topic without first being introduced to it. As for the third question on this experiment, the statistics to date tell me that 86% of people do all questions once, and 55% also complete them (a question is considered complete after two times giving a correct answer). I think that holds up pretty well compared to drop-out rates for self-paced elearning. Also, the whole point is that people can stop when they feel they've gotten enough out of the course.


Behind the interface

Currently the site has around 200 courses and 20000 members, of which mostly physicians. That's because the tool was introduced for medical students at the Harvard Medical School. You can read about that in the Time  article "The New Way Doctors Learn".

This innovative learning application is mainly based around

• The spacing effect (spaced repetition to beat the Ebbinghaus forgetting curve...)
• The testing effect (causing improved long term retention by testing)

They say their 'secret sauce' is their adaptive algorithm that tunes your daily content based on past performance. But the site also leverages features from social learning (commenting on questions, rating courses), gamification (leaderboard, scores), performance support (in the daily workflow, prompts), micro-learning (3 minutes or less) and push learning (emails).

If you are interested in the research conducted on the effectiveness of the tool, be my guest and hurry to the list of articles published or the publications of dr Price Kerfoot, one of the founding fathers of the site. Tests in the medical world have shown increased retention of knowledge by 170% compared to simple e-learning. I don't know what to make of that number, it seems very high, and those with a sound statistical and research background may verify it :-).

You can also get a private QStream site with your own logo etc for a substantial fee (think thousands of US$). Not an option for me, but it is the price to pay if you want to hide away your content from the public site.

What if...

So what can you do with a tool like QStream?
Well, here are some thoughts, feel free to add your own:

• Enrol people in a QStream after they had initial learning (eg in a class or old-style elearning course) to increase retention and remind people during their daily workflow.
• Have a group of people make their own question list after a learning event so they remind and test themselves on their own key learning points.
• Use it as a pre-test to see what should be covered in more depth or get more attention in other learning interactions.
• Use it as an ongoing assessment spread in time to measure the knowledge level of your staff and correct intervene where needed.

In a nutshell: ****

Easy. Fast. Effective. Fun. Cheap. Can supplement or replace traditional learning. May do with with more guidance on making proper developmental questions. (It is an art and crucial for the outcome.)
Try it to make up your own mind. I give it 4 stars.

There are mobile apps for QStream as well, but I couldn't get the Android app to work on my tablet, so I can't say how it compares to the site.