Plugin-provided XBlock runtime services

Development Architecture
1

Hi all,

We’ve been working on openedx-ai-extensions, a standalone plugin that exposes LLM capabilities to the platform. As part of that work, we investigated whether a plugin can register a new XBlock runtime service — so that any XBlock could call self.runtime.service(self, “ai_extensions”) without importing plugin internals directly.

The short answer we found: there is currently no supported way to do this.

Service registration is entirely hardcoded inside openedx-platform’s runtime classes — either as an explicit Python dictionary (legacy runtime) or a hardcoded if/elif chain (modern
XBlockRuntime). Neither the XBlock library nor openedx-platform exposes an entry point, Django setting, filter, or signal that a plugin can use to inject a new service.

We explored three concrete approaches and documented them in ADR-0005 of our repo:

  1. Monkey-patching Runtime.service; plugin-only, no platform changes, but an anti-pattern we’re not willing to ship.
  2. A new openedx.xblock_service entry-point group; consistent with existing openedx.* entry points, but requires upstream platform changes we can’t own from this project.
  3. An XBLOCK_EXTRA_SERVICES Django setting; analogous to XBLOCK_EXTRA_MIXINS, but same constraint.
  4. Open-edx filters; consistent with openedx practices. Requires a new filter and modifications to the runtimes

We’re not pursuing any of these at this time. Beyond scope and timeline constraints on our end, we’re also aware that ADR-0006 (Role of XBlocks) points toward reducing XBlock’s dependence on runtime services. As such, any upstream proposal would need to make a case to the community.

So the question we’d like to bring to the community is:

  • Are we interested in having a plugin based approach to extending the services that the runtimes can expose to Xblocks?
  • If so, does any of the approaches above feel like the right direction? Are we missing an even better option?

Happy to hear if others have hit this wall, if there are approaches we missed, or if there’s existing discussion we should be reading.

2

What sort of functionality would your new service have, and how would you expect other XBlocks to make use of it?

If other XBlocks are going to be built to require the new AI service, then I would lean towards entry points because we’d want to express an actual pip installation dependency. Or possibly even creating a new platform-level service, if it’s going to be broadly needed.

FWIW, XBlock runtime services were broadly intended to be pluggable at some point, as this comment in the Service docstring illustrates:

But back then, we also had the notion that XBlock could run in other learning platforms. Once it became clear that they’d only run on our platform, pluggable services became less important as a goal, because most of the service layer was tied to platform anyway. The other consideration is that the service layer abstraction is most useful where there are plausibly different, pluggable implementations of the same service that might exist (e.g. on different LMS systems). If an XBlock really just needs a piece of functionality that your library can provide, it may be simpler if those XBlocks just include your library as an explicit dependency.

3

What we want to do is to expose some third party service call capabilities. In this case that third party is a LLM service. Naturally, the xblock could directly call the service, but the extension layer provided by the ai-extensions plugin has build in capabilities for author based configuration of how the LLMs are used and audit for the llm responses.

Also, having each xblock pin their own required version of the library would soon get into conflicting library versions such as OpenAI or a router like litellm. We already saw this with one xblock and the ai-extension plugin, so it might only get worse.

One of the use cases we are looking into is ORA grading. We have seen apetite for having a llm grading step that gives learners immediate feedback. This has even been explored by rg in an fork of ora2. This implementation pins the use of openai, requires a hard fork of ora and lacks a lot of the configuration that ai-extensions provides.

How we are envisioning it, the xblock would declare that it wants the ai_service @XBlock.wants("ai_extensions") and if it finds it, then it would present the UI for LLM grading. If the service is not present, then the xblock would still have all the other ORA2 capabilities. That in practice means that if an installation decides to add the ai-extensions plugin, then ora gets new options, but if the installation does not globally decide to add ai, then ora is what is has always been. By using the service instead of a hard ai library requirement we avoid having a dependency in an xblock that forces the whole platform to get AI connection.

There are alternatives for sure. Using pip install extras or even try catching some import, but having the xblock service looks cleaner. It would also allow for a different ai-plugin to implement the service differently and still get ORA to cooperate without having to fork.

4

Can you give an example of how high or low-level that API would be, from the consuming XBlock’s perspective (e.g. ORA)?

5

This is just an initial idea, but here is how I think about it so far.
If we are trying to have a xblock such as ORA have a grading step that uses AI, we would need to have the user_input and some instructions for the AI. This instructions include the prompt, but we have seen that we need more than just a prompt. In the ai-extensions project this normally includes a list of classes that know how to call the llm, extract content from the definition of the course and naturally the custom prompt that the course authors wrote for this. In there, we call this definition the ai_workflow_profile and the individual classes are called orchestrators and processors.

The simplest way to call the optional ai_service from an xblock would be to let the author configure and store select the profile they want to run when authoring the unit/component in studio and during runtime have something like:

ai_service = self.runtime.service(self, "ai_extensions")
results = ai_service.run_profile(profile_id, user_input)

I’m oversimplifying because we’d like to use async tasks eventually, but the gist of the idea is to completely abstract away the use of comercial llm services, api_keys, streaming, self-hosting models and even the act of writing and storing the llm prompt.

An even more abstract way of handling it would be to rely on the ai_workflow_scope model (not related to the xblock.scopes). This just has a way of resolving which ai_workflow_profile best matches the context of the calling function by using the course_id, location_id and a ui_placement_id. Then the selection of the profile happens at the ai-extensions code. We initially thought this is how we wanted to handle it, but I’m thinking now that we better select the profile directly given how the whole thing is to let authors have control.

I suppose that whenever we are working on having LLM feedback at ORA, we are going to need specific orchestrators and processors for this. Most of what we will require from authors is that they either select an existing prompt or write one for their own cases.

6

Coming back to this thread with what I think is a much smaller and more idiomatic proposal than the four options in our original ADR which inspired this thread. It takes the proposal in #2, but moves it to openedx/XBlock instead of edx-platform which I also find more fitting.

What we found digging deeper

While prototyping, we noticed that every runtime in the platform resolves services through the same base method in the XBlock library:

  • The legacy LMS/CMS/Studio wiring (block_render.py, preview.py,
    load_services_for_studio) only populates runtime._services; the lookup
    itself is ModuleStoreRuntime.service(), which delegates straight to
    xblock.runtime.Runtime.service().
  • The newer XBlockRuntime.service() (and OpenedXContentRuntime) runs its
    if/elif chain and then falls back to super().service() — the same base
    method.

So a fallback added to Runtime.service() in openedx/XBlock is automatically reached by all platform runtimes and the xblock-sdk workbench, with zero platform changes.

And the XBlock library already has everything needed to do this cleanly:
xblock/plugin.py is the generic stevedore entry-point loader used for xblock.v1 and xblock_asides.v1, with caching, AmbiguousPluginError on name collisions, an .overrides group for deliberate replacement, and a register_temp_plugin test helper. Even the docstring of xblock/reference/plugins.py:Service says the original intent was for services to “load through Stevedore, and have a plug-in mechanism similar to XBlock” — which matches what @dave said above about services being broadly intended to be pluggable.

The proposal: xblock.service.v1

A package offers a service by registering a provider class:

# setup.py of the providing package
entry_points={
    "xblock.service.v1": [
        "ai_extensions = openedx_ai_extensions.xblock_service:AIExtensionsService",
    ],
}

The entry-point name is the service name blocks already use with @XBlock.needs / @XBlock.wants. The change to Runtime.service() is roughly:

declaration = block.service_declaration(service_name)
if declaration is None:
    raise NoSuchServiceError(f"Service {service_name!r} was not requested.")
service = self._services.get(service_name)
if service is None:                                   # NEW
    service = self._load_service_from_entry_point(    # NEW
        block, service_name)                          # NEW
if service is None and declaration == "need":
    raise NoSuchServiceError(f"Service {service_name!r} is not available.")
return service

where the new helper resolves the class via a ServiceProvider(Plugin) loader and instantiates it as provider_class(runtime=self, xblock=block) (same constructor signature as xblock.reference.plugins.Service).

Key properties:

  • Runtime services always win. The entry-point group is only consulted when the runtime doesn’t provide the name, so a pip package can’t shadow user, field-data, i18n, etc.
  • needs/wants semantics unchanged. Plugin services are only handed to blocks that declared them; wants blocks degrade to None when the package isn’t installed.
  • Conflicts fail loudly. Two packages claiming the same name raise AmbiguousPluginError instead of last-write-wins; intentional override goes through xblock.service.v1.overrides.
  • Cost. Lookups (including misses) are cached by the existing Plugin.load_class machinery — steady state is one dict lookup.
  • Trust model. Same as XBlocks: installing the package is what activates its entry points.

The consuming XBlock API (answering Dave’s question)

From the XBlock author’s point of view nothing new is invented:

@XBlock.wants("ai_extensions")
class OpenAssessmentBlock(XBlock):

    def grade_with_ai(self, submission_text):
        ai_service = self.runtime.service(self, "ai_extensions")
        if ai_service is None:
            return None  # feature hidden when the plugin isn't installed

        ## the exact contract exposed by ai_service is still under development, but an example would be:
        return ai_service.run_profile(self.ai_profile_id, submission_text)

The block never imports the plugin, never pins its libraries, and works on installs without the plugin.

Status

I wanted to see what the fuss with claude fable was and I put it to the test today.
We have a working proof of concept: the ServiceProvider loader + fallback in a fork of openedx/XBlock (with tests covering entry-point loading, runtime shadowing, want/need semantics) and an openedx-ai-extensions branch registering ai_extensions through the new group.

I created a PR to XBlock here: Plugin extensible xblock services by felipemontoya · Pull Request #927 · openedx/XBlock · GitHub
This would be implemented by the ai-extensions repo by something like: [WIP] poc providing ai services to xblocks by felipemontoya · Pull Request #229 · openedx/openedx-ai-extensions · GitHub (as I commented above, this is not the final contract exposed by the ai_extensions service, we are working on it)

Specific questions we still have:

  1. Overall feedback, it this maintainable, secure and an elegant way of extending xblock services for the long term?
  2. Naming: xblock.service.v1 to mirror xblock.v1 / xblock_asides.v1?
  3. Per-call instantiation vs. memoizing per (runtime, service_name) — the old Service docstring warns against over-initialization.
7

For the goal of making XBlock runtime services pluggable, I think your proposal is perfectly reasonable. There might be some weird edge cases around initialization because of how runtimes are re-used/re-initialized in the LMS, but those could be managed. My larger concern is that I’m still not convinced that the runtime service abstraction is worthwhile for this use case, as opposed to having the AI framework as a library dependency.

Maybe this is something more fundamental that I’m not really grokking. I’m going to go over my assumptions, and please let me know where my disconnect is.

Assumption 1: Implementing as a runtime service does not give us more flexibility when it comes to version conflict management.

Python packages are already free to declare optional, extra dependencies. They can also declare acceptable version ranges for those dependencies, and it would be the job of pip (and later uv) to figure out the real version to pin to. If there are irreconcilable versioning conflicts, then that’s a real problem that’s better to figure out at build time than run time.

Assumption 2: Even if the runtime service is installed, we still have to deal with the “it’s not available” case.

Maybe it’s installed, but it’s been turned off by configuration at the site level. Or maybe there is no LLM configured. Or perhaps an LLM is configured, but is not available to the current user for whatever reason. Or maybe only a subset of functionality is enabled for this particular case, e.g. they can only run the cheap local model and not the pricy commercial one. Regardless, it’s never sufficient to just know whether or not the service is installed—clients must actually query the service in some way to and handle error cases.

Assumption 3: The installation overhead of libraries for LLM communication is small.

LLMs can be insanely expensive to run, and the communication process requires async tasks and possibly other operational complexity. But the actual overhead of the library code to manage that is small. In other words, if we install it and don’t use any of its features (because it’s not configured), there’s not much of a penalty in terms of system resources.

Assumption 4: We are not seriously entertaining the notion of competing AI frameworks for Open edX Platform integration, only competing backend processors/adapters.

We’re making a bet on our AI Extensibility Framework, and we’re not seriously thinking that the interface APIs are going to be re-implemented by competing frameworks. We may add a bunch of backends to enable various functionality with different LLM providers, but the interface glue is going to be defined in this one place. The only situation in which having a runtime service abstraction really helps us if we’re going to plausibly have competing implementations.

Assumption 5: Many things that are not XBlocks are going to want to use this framework.

Something like our courseware forum repo should be able to make use of the AI Extensibility Framework, far outside of the XBlock runtime.

Why is the AI Extensibility Framework a Plugin?

If my assumptions here are correct (and I realize that’s a big “if”), then I don’t understand why we’re working so hard to keep platform ignorant of the existence of the AI Extensibility Framework. The justification I see in the first ADR is:

The plugin approach is preferred because it’s faster to develop, maintains extensibility, and using events and filters makes it easier to maintain as the platform evolves. The decision to integrate into the default dependencies of the core can be made later.

I read “faster to develop” as “it’s hard to get things merged into openedx-platform”, which can be addressed by keeping the development in its current repo. If the point was to assess whether this is a thing that people will want, I think we’ve had adequate time assess that people definitely do want it.

We can make it opt-in, so that the default behavior in Willow is that no LLMs are configured and the API always just returns “unavailable” when queried. But I think we should accept that the framework for this is part of the core technical system. I think it’ll also make things easier for plugin developers (XBlocks or otherwise), if we can say something like “Willow ships with version XYZ of the AI Extensibility Plugin Framework, so do your CI against that”.

@Felipe: So in concrete terms, how would you feel about the following:

  1. We stop considering the AI Extensibility Framework itself a plugin, and have openedx-platform declare it as a normal dependency (like we do for openedx-core). It would have no LLM backends enabled by default.
  2. XBlocks could declare certain version ranges of openedx-ai-extensions as dependencies for themselves, and import any public APIs that the AI framework chooses to expose. There is no need to use XBlock runtime services as the access mechanism.
8

Thanks Dave for such thoughtful responses. This has been exactly the conversation I was hoping to have in order to reach a well tough out solution.

Let me go through your assumptions and then get to your concrete proposal at the end.

Assumption 1: Implementing as a runtime service does not give us more flexibility when it comes to version conflict management. Python packages are already free to declare optional, extra dependencies.
They can also declare acceptable version ranges for those dependencies, and it would be the job of pip (and later uv) to figure out the real version to pin to. If there are irreconcilable versioning conflicts, then that’s a real problem that’s better to figure out at build time than run time.

  • build-time selection of extras isn’t actually easy on the supported build path. pip install ora2[extra] works interactively, but in a Tutor image build you can’t flip on an extra for a dependency that openedx-platform already pins in its base requirements without forking it.
  • for xblocks the goal is to not carry the dependency at all. The xblock declares wants(“ai_extensions”), ships zero AI dependency, and binds only when the platform has the framework. That’s what completion does in an example below.

I do appreciate that pip/uv do resolve version ranges better than a runtime mechanism. Going over runtime services puts all the pressure on the interface created by the framework and it means we need to make it a lot more tight. Leaning into all the best practices set by oep-49. Maybe even offering a versioned API and a lot of documentation.

I think it would be good to look at what we do for other transversal dependencies such as grades/submissions/completion.

edx-sga uses:

from submissions import api as submissions_api
from submissions.models import StudentItem as SubmissionsStudent
from submissions.models import Submission

taken from edx-sga/edx_sga/sga.py at df94598d90560ef04f63722475c388d7f3b3f898 · mitodl/edx-sga · GitHub

xblock-flow-control extracts an lms code call in a runtime imported module with:

from flow_control.edxapp_wrapper.score import (
    score_module as ScoresClient,
)
# ---
from importlib import import_module
from django.conf import settings


def get_score_module_function(*args, **kwargs):
    """Get ScoreModule model."""

    backend_function = settings.FLOW_CONTROL_SCORE_MODULE_BACKEND
    backend = import_module(backend_function)

    return backend.get_score_module(*args, **kwargs)


score_module = get_score_module_function
# ---
from lms.djangoapps.courseware.model_data import ScoresClient

taken from flow-control-xblock/flow_control/edxapp_wrapper/backends/score_s_v1.py at bf27bf142397b63e675b914f8ff9691067ad50f9 · eduNEXT/flow-control-xblock · GitHub

Completion on the other hand is offered as a service:

completion_service = self.runtime.service(self, 'completion')

taken from openedx-platform/lms/djangoapps/lms_xblock/mixin.py at master · openedx/openedx-platform · GitHub

In recap:

  • sga → hard import of submissions (tight coupling),
  • flow-control → settings-swappable edxapp_wrapper backend (the hoops we devs jump through to avoid a hard import),
  • completion → runtime service (the clean one).

Assumption 2: Even if the runtime service is installed, we still have to deal with the “it’s not available” case.
Maybe it’s installed, but it’s been turned off by configuration at the site level. Or maybe there is no LLM configured. Or perhaps an LLM is configured, but is not available to the current user for whatever reason

Agree: we will have to handle happy and sad paths regardless of how the LLM call is being connected to the xblock code. The way I see it there are two separate layers. Whether the service is installed at all is a cheap binary check (wants + the service being None).
Whether AI is actually usable right now is answered by the API at call time and the xblock handles it the same way whether it reached that API by import or by service. There are a myriad of configurations that could cause an invalid response, but also operational issues with models, API KEY limitations, outages, models are removed constantly and much more. A good portion of that is what the ai-extensions-framework tries to solve. That’s why we are all in agreement that we prefer having xblocks reusing the implementation that ai-extensions offers rather than directly handling the llm calls.

Currently, there is no big switch to turn everything on or off. It is managed by the existence of DB stored scopes and profiles. This is a concern we could resolve by having a config/waffle switch that makes the runtime service return None which means it’s not available even if the code is there. Optionally it could return an api object that lets the xblock tell that ai services are turned off rather than uninstalled (I think this is a stretch, but for some consumers it might make sense).

This will be a lot of responsibilities for caller xblocks and it will build on top of the API design.

Which is a tangent, but also worth touching. What I’m expecting even if we go through a service or directly require the code, is that the call to the ai-extensions over the python API will go something like this (exposed through the api.py as well):

from openedx_ai_extensions import api as ai_service # in case we go over direct import
ai_service = self.runtime.service(self, 'ai_extensions') # in case we go over runtime services

ai_service.public_function(
    user_input=data,
    context_data=data,
)

The implementation of the function should be able to calculate the scope and thus find a suitable matching profile in the DB. The problem with this is that it requires admins to go define/configure profiles for every xblock they want to use.

We can alternatively support a profile argument to let the developers handle the config part directly, but this risks not respecting the administrator configurations to turn something on and off.
Also we could expose a way for xblocks (or api consumers) to save their profiles to the DB and then let admins take over and further control the definitions of the AI workloads.

This entire assumption further strengthens the case for a stable api object.

Assumption 3: The installation overhead of libraries for LLM communication is small. LLMs can be insanely expensive to run, and the communication process requires async tasks and possibly other operational complexity. But the actual overhead of the library code to manage that is small.

Not entirely. Agreed that an installed-but-unused library costs almost nothing at runtime. However, the cost is the dependency surface you take on platform-wide the moment this is part of the required install for everyone: disk, native wheels, package count, and above all the release cadence and version conflicts.

When we started working there were 2 projects that we knew that served as llm routers (litellm and langchain). After some evaluation we picked litellm and moved on. Later as time went on it turned out it is deceptively large as also langchain is.

Some comparison numbers current as of today. (Numbers below come from https://claude.ai/public/artifacts/1328550d-6008-4c20-820a-861a8c17b124)

LiteLLM (base) any-llm-sdk [openai,anthropic] any-llm-sdk [all] aisuite (base) mirascope instructor
Packages added 56 26 136 12 16 41
Disk (site-packages) 210 MB 77 MB 521 MB 23 MB 36 MB 83 MB
Own source lines 284,814 17,148 17,148 6,560 34,708 26,001
Own Python files 1,939 142 142 40 173 195
Mandatory Rust binaries tiktoken, tokenizers, pydantic_core, regex, rpds-py, jiter, hf-xet pydantic_core, jiter pydantic_core, jiter (+ provider-specific) none (base) pydantic_core, jiter pydantic_core, jiter
Releases (12 months) 189 70 70 3 24 17
Simultaneous active minor series 6 1 1 1 1 1
Has official CHANGELOG :cross_mark: :white_check_mark: (GitHub releases) :white_check_mark: :cross_mark: :white_check_mark: :white_check_mark:
openai v1 compatible :cross_mark: (requires v2) :white_check_mark: (floor at 1.99.3) :white_check_mark: :white_check_mark: N/A :white_check_mark:
Python 3.10 support :white_check_mark: :cross_mark: (3.11+ only) :cross_mark: :white_check_mark: :white_check_mark: :white_check_mark:
Forces huggingface_hub :white_check_mark: yes (mandatory) :cross_mark: no :cross_mark: no :cross_mark: no :cross_mark: no :cross_mark: no
Forces python-dotenv :white_check_mark: yes (mandatory) :cross_mark: no :cross_mark: no :cross_mark: no :cross_mark: no :cross_mark: no
Provider count 100+ 2 explicit + passthrough 42 15+ 20+ 15+
Fallback / load balancing :white_check_mark: :cross_mark: :cross_mark: :cross_mark: :cross_mark: :cross_mark:

Even the smallest library adds a few MBs, but litellm is a lot bigger. We would not want the whole jungle imported to run the unit tests in the gorila package.
If we ever decide to reduce the weight, we could switch to any-llm which is also well supported and select carefully which providers get installed by default, but that is a big refactor.

The churn of versions is even worse. Some of those routers and litellm in particular are designed for fast updaters which our 2 releases per year will struggle with. Not something that I was carefully consider in the ai-extensions release cycle, but I’ll be on top of that from now on.

Assumption 4: We are not seriously entertaining the notion of competing AI frameworks for Open edX Platform integration, only competing backend processors/adapters.
We’re making a bet on our AI Extensibility Framework, and we’re not seriously thinking that the interface APIs are going to be re-implemented by competing frameworks.

I mostly agree there won’t be competing frameworks. But I don’t think that’s where the abstraction earns its place. The value I’m after is the dependency boundary from Assumptions 1–2, the xblock targets a service name and carries no extra weight, and that holds even if ai-extensions is the only implementation that ever exists.

Though I wouldn’t rule competing implementations out either (they sort of already exist). MIT runs ask-tim on asides with their own backend, and WGU/ASU built ai-coach on litellm. A common service name (with the overrides group in the PR) is exactly what lets one xblock target “the AI service” and have either of those satisfy it.

Assumption 5: Many things that are not XBlocks are going to want to use this framework.

Yes, and I think this is the clearest way to see the whole thing: there should be only one public api.py, and consumers reach it the way that fits them.

Forums and other consumers will also have to deal with:

  • what happens if not available/turned off
  • how does the framework respond to ops issues (outages, model errors, incorrect output, …)
  • do we use profiles/scopes defined at the DB level?
  • how do operators choose their models/providers/keys
  • anything we devise for auditing mechanisms and human in the loop
  • supervision loops for the safety of llm responses

Why is the AI Extensibility Framework a Plugin?

The ADR answer was very on point when we started. We wanted to experiment and explore the ways in which LLM capabilities could be leveraged for the platform. Having this feature as a plugin made it easy to build rapidly for the nearly one year we have been working on it. I would however add:

  • because it allows adopters to have a faster cadence of upgrading the ai-extensions-framework without having to wait for the new release and also run a long migration. Some adopters are still running teak which is very well supported.

Finally I get to your concrete questions

  1. We stop considering the AI Extensibility Framework itself a plugin, and have openedx-platform declare it as a normal dependency (like we do for openedx-core). It would have no LLM backends enabled by default.

I think this could work, but there are nuances to consider.

First, ai-extensions today is more than a backend: it’s a django backend, a react frontend and a tutor plugin that wires it all together. Making it a normal platform dependency really means the backend becomes a dependency; the frontend still has to load as an MFE/slot, so that part stays plugin-shaped no matter what.

Second, I wouldn’t make the whole backend a hard dependency, because of the weight from Assumption 3. I’d split it in two:

  • a base library: the models (profiles, scopes, sessions), the openedx adaptors (content_libraries, submissions, location content) and the stable api.py. Light, no litellm. This is the piece that could become a normal platform dependency, like openedx-core.
  • a router library: the hard dependency on litellm, the llm_processors, the example profiles and the rest. Stays optional and upgrades on its own cadence.

That maps onto your proposal cleanly: the base library ships in the platform with no LLM backends, so the API is always importable and just returns “unavailable” until someone installs and configures a router. It also answers the weight concern, since the 210MB of litellm only lands on installs that opt into AI.

  1. XBlocks could declare certain version ranges of openedx-ai-extensions as dependencies for themselves, and import any public APIs that the AI framework chooses to expose. There is no need to use XBlock runtime services as the access mechanism.

Agreed that an xblock can just pin a version range of the base library and import the public API, and for some xblocks that’s the right call. I’d make the choice by how central AI is to the block:

  • if AI is a core, non-optional feature of the xblock, importing the base library and depending on it is perfectly fine.
  • if AI is an optional enhancement (ORA is my running example: it should keep working everywhere and only grow the AI grading step where the platform has the framework), then the runtime service is the cleaner door, because the xblock takes no dependency at all and degrades to None where the service isn’t there.

Either way it should be the same api.py underneath. The service isn’t a competing mechanism, it’s a thin optional adapter over the public API. So I think the question we’re actually left with is narrow: is that thin adapter (the xblock.service.v1 group in PR #927) worth carrying in XBlock core, given the base library is importable anyway? I’m happy to land wherever the community does on it.

9

One additional thing I want to mention: XBlock services don’t really have well-defined APIs. The API contract is “whatever the service provides is the API”, and it’s not really documented, defined, or enforced in any way other than by python at runtime.

In order to define a pluggable service, you need to define a generic API contract that each plugin implements, and ideally you would document that and enforce it via type checking / contracts / etc. This is something the XBlock ecosystem has never really achieved nor done properly - even the core APIs like self.runtime.x and author_view etc. have some inconsistencies.

So: I like your proposal, but I think it’s better to just hard-code use of the ai-extensions framework for now, and iterate with that, and only consider defining a pluggable API later on once you know more about what a stable service API would look like, and have shown that we definitely need at least two different implementations, and the XBlock layer is the best layer to integrate them at.

10

Thanks to both again.

Rather than keep arguing mechanism in the thread, I took the discussion and wrote it up as an ADR (0012) in the openedx-ai-extensions repo. The main thing it does is separate in two questions that derive from this thread.

  • What capability we expose to consumers (the API surface).
  • How that capability reaches them in code (the delivery mechanism).

Treating those as one decision made the runtime-service question look load-bearing when it does not need to be. Split apart, most of what you both raised lands on the what, and the how turns out to be deferrable.

On the “what” (Decision A). There’s a single supported surface: an api.py module that returns a typed result envelope — status, payload, metadata, session reference — never raw un-statused LLM text or a dict you have to inspect for shape. This is me taking Braden’s point directly: the reason XBlock services feel underdefined is that “whatever the service provides is the API.” So the contract lives in api.py, versioned and type-shipped, independent of any service seam. That’s the piece I want to define and harden first.

On the “how” (Decision B), and Braden’s advice to just hardcode for now. That’s reasonable and a possible next step. The ADR reflects it: we ship and stabilize the api.py contract, and we don’t build pluggability until a second real consumer justifies it. The runtime-service entry point stays the leading candidate for XBlocks, but it’s explicitly not committed to in this ADR. Crucially whatever seam we eventually pick hands back the same api.py surface, not a bespoke per-service object. The service becomes a thin door to reach the API.

On Dave’s assumptions, briefly:

  • Version conflicts / “figure it out at build time” (agreed), runtime services don’t resolve dependency conflicts, and I’m not asking them to. The answer is the light/heavy package split: a light package carrying types, statuses and stubs that’s always safe to depend on, and a heavy package carrying the LLM router. An install that doesn’t use AI doesn’t pay litellm’s ~200MB+ and 180-releases-a-year cadence.
  • Clients must still handle unavailability. The typed envelope is built for exactly that. The ADR sketches a draft taxonomy so downstream code can branch on a documented outcome instead of catching surprises. That’s a starting draft, not a commitment. I’d genuinely welcome feedback on the shape of it.
  • Non-XBlock consumers are treated as a first-class goal: inputs are plain and serializable with no XBlock/ORM shape, so the same call works from a web request, a Celery task, or an xblock.

I’d rather this ADR be a concrete thing to react to than a finished verdict, so it’s marked Provisional. If the “define the contract first, defer the seam” framing sounds right to you both, I’ll press ahead on api.py and the package split, and bring the runtime-service question back when there’s a second consumer to justify it.