Future Directions for Problem Builder

Educators
1

Hi everyone,

We (OpenCraft, developers of the Problem Builder XBlock), would like to hear from anyone in the community who is using Problem Builder.

:arrow_right: If you are using Problem Builder, could you please reply below to let us know (A) that you’re using it, (B) what features you are using and why, and (C) if you’re interested in sponsoring any improvements. (If any of those involve confidential answers, feel free to omit details or to DM me).

Context

For context, Problem Builder is almost ten years old, and over the years we have built many features into it for various clients (see below). Some of these are unique features not available in any other XBlock. Others are now built in to the platform.

In recent years we have not had any clients requesting new features or sponsoring the development of Problem Builder. We continue to maintain it to ensure it’s compatible with the latest platform changes, but it hasn’t received any significant development in years.

Challenges

As I look to the future of Problem Builder, I see a few challenges:

  1. The big challenge that I’m aware of is that Problem Builder is pretty much the only course component XBlock out there that uses “XBlock children”, and future versions of the platform may not support XBlock children, in a bid to significantly simplify the runtime and improve performance. This means that in the future, Problem Builder may need a major refactor in order to continue to work.
    I don't think this is an insurmountable problem for Problem Builder, because it doesn't fundamentally *need* to use XBlock children - using XBlock children was just a semi-convenient way to borrow Studio's existing UI system and use it for stitching components together to build problems. (It would only truly need XBlock children if we wanted it to wrap arbitrary blocks, like putting an ORA2 inside a Problem Builder, but we don't support that really.) So Problem Builder could just as well use a visual editor or an XML config (as it did in the past) or a YAML template or any number of ways to define and display problems - it doesn't have to use XBlock children.
  2. Problem Builder has a large number of features, and we believe that some of them may no longer be used at all. But since we don’t know which features are used or not, the large feature set complicates maintenance, and also makes it more difficult to add new features or improve the UI.
  3. Both the author and learner interfaces are very basic, and could use some major UX improvements.

Possible Next Steps

If there is community interest and sponsorship, we’d love to plan the next evolution of Problem Builder; perhaps extracting the most valuable functionality into a simpler XBlock with a nicer UX, or deprecating and removing unused features, or simplifying the authoring workflow so that it doesn’t use “XBlock children”. On the other hand, if nobody is interested in sponsoring the development and the platform drops support for “XBlock children” in some future version, we may need to consider eventually deprecating Problem Builder at that time. But before we can make any decisions we need to hear from you - so please reply to this thread.

Current features of Problem Builder

For reference, here is a (possibly incomplete) list of Problem Builder’s features:

  1. Create problems similar to built-in (“capa”) problems, featuring:

    1. Long Answer (free text response)
    2. Multiple Choice / Multiple Response
    3. Sliding scale / slider - “indicate a numeric value on a sliding scale.”
    4. Swiping - “ask binary-choice questions with a swiping interface”
    5. Completion checkbox - indicate that a given assignment has been completed.

  2. Display questions in a linear set (“mentoring”) or one at a time / step-by-step (“step builder”)

  3. Include videos hosted by Ooyala within the problems

  4. Display “tips” for specific choices e.g. in multiple choice questions.

  5. Conditionally display messages on complete/incomplete/max. attempts reached/during review.

  6. Summarize/recap student answers to previous long answer questions from earlier in the course.

  7. Summarize/recap student answers in a table.

  8. Display a plot to summarizes answers to scale and/or rating questions.

  9. Display a visual dashboard that “fills in” as learners submit more answers
    dashboard-image → dashboard-image-overlay

  10. Instructor Tool to facilitate exporting student answers to a CSV file, for supported question types.

Attn @antoviaque @Agrendalath @john_curricume @Joan_Rodriguez

3 Likes
2

@braden as you know I use it for several courses. I appreciate the work you guys have put into that block and this post about its future direction.

There are a few key use cases where we use problem-builder:

  1. Answer “recaps”. We often use these to summarize important responses in the course, like if there is a big question at the end of each module and we recap all those at the end. To my knowledge, this is the only block that is capable of that.
  2. We use the step-builder functionality to present multiple MCQs in a row and then give feedback whether they got them all right or >= 1 wrong. It’s like a very short, simplified adaptive feedback exercise. For example, Q1) Is biomass available as a resource to this community Q2) Is biomass an efficient energy resource Q3) Given that X is generally interested in electric power, is biomass a good resource to use?

Those are the only two irreplaceable use cases I think we have. And I’d say we don’t use PB when a more supported component is available.

I think we could live in a world without problem-builder, as long as there was some notice about it’s deprecation.

3

Super timely, @braden , thanks for posting. Assessment is the first Core Capability that we’re looking at from a Core Product perspective.

It may make the most sense to fold this conversation in. In other words, if we can synthesize the “mission critical” features that Problem Builder delivers and add them as areas of investment for the Core Product Assessment capability, then we can ensure that the needs this block solves are accomplished via the Core, and can deprecate the rest (or find alternatives via more current LTI tools). What do you think?

Here is a very drafty Concept Note for the Core Product vision for assessment, and where we need to invest to get there. @john_curricume I’ve incorporated all of the feedback from the Educators WG brainstorm sesh into this doc. I’ll keep an eye on this space and transfer any irreplaceable use cases into this doc, and please feel free to comment directly in the doc as well.

2 Likes
4

@jmakowski It sounds to me like these ^ are the primary features of Problem Builder that facilitate use cases that the built-in problem and ORA currently cannot.

Would you mind helping me figure out where to add them to the concept doc?

I guess summarize/recap is not technically an assessment since the current implementation considers the long answer correct as soon as you write anything, and the “Recap” tool is read-only. Here are some screenshots from the documentation:

“Long answer” problem type earlier in the course:

863×384 6.57 KB

“Long answer recap” later in the course:

857×138 4.3 KB

And here’s a screenshot of the kind of assessment @john_curricume was mentioning (though he might be using different options that affect what info is shown):

More of that on this doc page.

1 Like