We (OpenCraft, developers of the Problem Builder XBlock), would like to hear from anyone in the community who is using Problem Builder.
If you are using Problem Builder, could you please reply below to let us know (A) that you’re using it, (B) what features you are using and why, and (C) if you’re interested in sponsoring any improvements. (If any of those involve confidential answers, feel free to omit details or to DM me).
Context
For context, Problem Builder is almost ten years old, and over the years we have built many features into it for various clients (see below). Some of these are unique features not available in any other XBlock. Others are now built in to the platform.
In recent years we have not had any clients requesting new features or sponsoring the development of Problem Builder. We continue to maintain it to ensure it’s compatible with the latest platform changes, but it hasn’t received any significant development in years.
Challenges
As I look to the future of Problem Builder, I see a few challenges:
The big challenge that I’m aware of is that Problem Builder is pretty much the only course component XBlock out there that uses “XBlock children”, and future versions of the platform may not support XBlock children, in a bid to significantly simplify the runtime and improve performance. This means that in the future, Problem Builder may need a major refactor in order to continue to work.I don't think this is an insurmountable problem for Problem Builder, because it doesn't fundamentally *need* to use XBlock children - using XBlock children was just a semi-convenient way to borrow Studio's existing UI system and use it for stitching components together to build problems. (It would only truly need XBlock children if we wanted it to wrap arbitrary blocks, like putting an ORA2 inside a Problem Builder, but we don't support that really.) So Problem Builder could just as well use a visual editor or an XML config (as it did in the past) or a YAML template or any number of ways to define and display problems - it doesn't have to use XBlock children.
Problem Builder has a large number of features, and we believe that some of them may no longer be used at all. But since we don’t know which features are used or not, the large feature set complicates maintenance, and also makes it more difficult to add new features or improve the UI.
Both the author and learner interfaces are very basic, and could use some major UX improvements.
Possible Next Steps
If there is community interest and sponsorship, we’d love to plan the next evolution of Problem Builder; perhaps extracting the most valuable functionality into a simpler XBlock with a nicer UX, or deprecating and removing unused features, or simplifying the authoring workflow so that it doesn’t use “XBlock children”. On the other hand, if nobody is interested in sponsoring the development and the platform drops support for “XBlock children” in some future version, we may need to consider eventually deprecating Problem Builder at that time. But before we can make any decisions we need to hear from you - so please reply to this thread.
Current features of Problem Builder
For reference, here is a (possibly incomplete) list of Problem Builder’s features:
Create problems similar to built-in (“capa”) problems, featuring:
Long Answer (free text response)
Multiple Choice / Multiple Response
Sliding scale / slider - “indicate a numeric value on a sliding scale.”
Swiping - “ask binary-choice questions with a swiping interface”
Completion checkbox - indicate that a given assignment has been completed.
Display questions in a linear set (“mentoring”) or one at a time / step-by-step (“step builder”)
Include videos hosted by Ooyala within the problems
Display “tips” for specific choices e.g. in multiple choice questions.
Conditionally display messages on complete/incomplete/max. attempts reached/during review.
Summarize/recap student answers to previous long answer questions from earlier in the course.
Summarize/recap student answers in a table.
Display a plot to summarizes answers to scale and/or rating questions.
Display a visual dashboard that “fills in” as learners submit more answers →
Instructor Tool to facilitate exporting student answers to a CSV file, for supported question types.
@braden as you know I use it for several courses. I appreciate the work you guys have put into that block and this post about its future direction.
There are a few key use cases where we use problem-builder:
Answer “recaps”. We often use these to summarize important responses in the course, like if there is a big question at the end of each module and we recap all those at the end. To my knowledge, this is the only block that is capable of that.
We use the step-builder functionality to present multiple MCQs in a row and then give feedback whether they got them all right or >= 1 wrong. It’s like a very short, simplified adaptive feedback exercise. For example, Q1) Is biomass available as a resource to this community Q2) Is biomass an efficient energy resource Q3) Given that X is generally interested in electric power, is biomass a good resource to use?
Those are the only two irreplaceable use cases I think we have. And I’d say we don’t use PB when a more supported component is available.
I think we could live in a world without problem-builder, as long as there was some notice about it’s deprecation.
Super timely, @braden , thanks for posting. Assessment is the first Core Capability that we’re looking at from a Core Product perspective.
It may make the most sense to fold this conversation in. In other words, if we can synthesize the “mission critical” features that Problem Builder delivers and add them as areas of investment for the Core Product Assessment capability, then we can ensure that the needs this block solves are accomplished via the Core, and can deprecate the rest (or find alternatives via more current LTI tools). What do you think?
Here is a very drafty Concept Note for the Core Product vision for assessment, and where we need to invest to get there. @john_curricume I’ve incorporated all of the feedback from the Educators WG brainstorm sesh into this doc. I’ll keep an eye on this space and transfer any irreplaceable use cases into this doc, and please feel free to comment directly in the doc as well.
@jmakowski It sounds to me like these ^ are the primary features of Problem Builder that facilitate use cases that the built-in problem and ORA currently cannot.
Would you mind helping me figure out where to add them to the concept doc?
I guess summarize/recap is not technically an assessment since the current implementation considers the long answer correct as soon as you write anything, and the “Recap” tool is read-only. Here are some screenshots from the documentation:
And here’s a screenshot of the kind of assessment @john_curricume was mentioning (though he might be using different options that affect what info is shown):
Hi @braden - thanks for bumping this thread with the Product Working Group. We read and discussed this as a group. You can find the recording of our meeting + notes here, although I was not at a computer and didn’t take notes, so this is my summary but I do recommend you watch the video if you can.
Basically, we discussed the deprecation process, and how it is a process: it’s not an immediate “this code will not work!!!” statement, but rather an opportunity for a maintainer to inform the community they will no longer be maintaining it, giving the community an appropriate amount of time to find another maintainer if the community finds it useful. I also don’t see a reason why a non-openedx repository can’t go through the Open edX official DEPR process.
In this case, I think if OpenCraft is no longer able to maintain this block, you should begin the deprecation process. You should follow OEP-21 in terms of communication and good faith effort to find a new maintainer. You should work with @jmakowski in making sure key interest groups have an opportunity to participate in the DEPR process.
We agree with you that facets of this XBlock could and perhaps should make their way into the Product Core, but at this time we can’t identify a product person to take on the definition work or a dev team to do the work. Perhaps part of the DEPR could see if there was strong interest, one or more groups could collaborate together to not maintain the old block but instead put effort/money into improving functionality in the product core.
Please let me know what other questions or thoughts you have!
Note. I had created a DEPR Issue for this a while back, but I think it’s fair to announce it here first, so there is more time for the community to respond. So I’ve posted in the forums as well here.
If you are using Problem Builder, could you please reply below to let us know (A) that you’re using it, (B) what features you are using and why, and (C) if you’re interested in sponsoring any improvements. (If any of those involve confidential answers, feel free to omit details or to DM me).
→ 
