Technology Oversight Committee Meeting Notes
Announcements and Updates [00:00:09 - 00:05:52]
Several project updates were announced at the beginning of the meeting.
AI Experimentation Framework
The Axim Open edX team launched into the second phase of AI experimentation work beginning at the end of February. Integration work was underway with the Badge Co-Writer tool for extracting credentials from existing legacy content as part of an experimental plugin to exercise the framework and ensure it functions properly.
A grant opportunity was co-applied for with the LASER team from George Washington University. The LASER team has developed AI tools that extract skills from content, which will integrate with the framework being built, providing additional proof points and potentially useful tools for platform users.
Enterprise Functionality Migration
2U’s Enterprise team was pulling functionality for the Enterprise offering out of the Open edX platform core into plugins using architectural changes made over the past year and a half. This work was characterized as decomplexifying the core and proving that the plugin framework is broadly usable across many use cases. The Enterprise functionality was not widely used in the Open edX community outside of 2U, with OpenCraft noted as having some clients using it.
Dynamic Extension Point Tools
Arunmozhi Periasamy of OpenCraft’s dynamic extension point catalog and navigator was highlighted as an interesting tool. It serves as proof that metadata for extension points has become standardized enough for someone to build a tool consuming that metadata to create a catalog, making it useful for development teams.
Conference Update
The Open edX conference was scheduled for the week of May 19th, with 102 talk submissions received after the Call for Proposals closed. Keynotes were being finalized and would be announced soon, consisting of a combination of plenary talks and panels similar to the previous year’s format.
Universal Open edX Mobile Application [00:06:01 - 00:23:51]
Concept and Current State
Discussion focused on developing a universal Open edX mobile application, similar to how Canvas and Moodle operate. Currently, the Open edX platform has native apps for Android and iOS used by a handful of organizations, requiring expertise in developing for both technologies and deploying to the Play Store and App Store. This complexity has led to a small number of actual mobile app deployments, with a one-to-one connection between mobile application deployment and catalog.
The proposed model would allow users to download a single Open edX-branded app that ranks highly in Play Store and App Store, then specify their school upon opening the app. The application would load the appropriate content, themes, colors, and branding for their institution, similar to Canvas Student/Canvas Teacher apps.
Technical Implementation
Zeit Labs had developed a production-ready prototype (not yet upstream-ready) requiring cleanup and standardization. Key technical components identified included:
-
Backend Support: Current support not generic enough for broader adoption
-
Feature Controls: On/off functionality for institutions to opt in
-
Dynamic Branding: Colors and branding partially runtime-capable, requiring full runtime implementation
-
Timeline: Estimated one to two months of work to make production-ready
Most implementation details would be bundled within the APK in plain text using asymmetric encryption keys in a well-known openindex.json file, with no protected secrets required.
Operational Considerations
Several operational requirements were identified for maintaining a universal app:
App Store Category: The application would fall into a browser-equivalent category with fewer requirements but necessitating processes for forwarding takedown requests to content sources if inappropriate content was reported.
Allow List: An allow list of sources would likely be maintained, requiring registration before content becomes available in the app to ensure accountability and enable contact if content violations occurred.
Metadata Collection: Some metadata inevitably gets collected by the Play Store about installs and usage, requiring consent from participating institutions.
Data Privacy: The universal app would not collect any user data directly; event stream data would go back to the original operational platform. SSO authentication would occur through the original instance using OAuth.
User Experience and Adoption
Feedback indicated that mobile-first usage was the norm for some regions and demographics:
-
Ethiopia and India: Strong interest in mobile-first content delivery, as phones were the primary way of interacting with online content
-
Spanish Universities: Lower priority as mobile deployment was perceived as too complicated and one more thing to manage, though younger learners tended to use everything on mobile phones
-
Computer vs. Mobile: Most current users access content via computer rather than mobile in some markets
Deployment Flexibility
The proposal maintained flexibility for custom deployments:
-
Large organizations like 2U could continue custom mobile app builds if desired
-
National-level deployments (e.g., National eLearning Center in Saudi Arabia) could deploy their own version with an allow list of tenants from specific universities
-
Most deployments would benefit from a very short path to getting up and running on mobile
-
Branding opportunities in the central app would be constrained compared to custom builds
Committee Response
The committee expressed general support with no significant objections. The approach was viewed as beneficial for many organizations as long as specific organizations could still choose more custom and branded experiences. The main value proposition was reducing barriers to mobile adoption while maintaining flexibility for those requiring deeper customization.
AI-Assisted Code Contributions [00:24:03 - 00:53:53]
Current Usage and Context
The assumption was confirmed that most teams were using AI code assistance to potentially write code, but certainly to assist with coding. At Axim, AI tools were proving highly valuable for large normalized refactoring projects that make the core software better but are tedious and error-prone for humans, while being easy for AI models to understand and repeat. This enabled technical debt reduction at unprecedented rates.
The organization intentionally waited to observe how other open source projects would respond to AI as author or co-author before establishing policies.
Key Distinctions: Assisted vs. Generated
Strong emphasis was placed on the distinction between AI-assisted and AI-generated code:
Assisted Code: Human-driven work with AI support, including robust human review to ensure content is not hallucinating. The importance lies in human review with appropriate checks and gates to catch quality concerns.
Generated Code: Code created by AI and submitted without meaningful human review, which was characterized as counterproductive. Simply asking AI to write code, then submitting it as a pull request without review was explicitly discouraged.
The consensus emerged that “assisted” was the key operative word, with human oversight being essential even as the level of required involvement may decrease over time.
Transparency and Attribution
Strong recommendations emerged around transparency requirements:
Co-Author Attribution: LLMs should be included as co-authors in Git commits when used for code assistance, enabling tracking of AI usage.
Interaction Logs: Some teams are including interaction logs showing prompts and how they adjusted the AI’s behavior through the process, providing transparency as both a learning tool and auditing mechanism.
Mandatory vs. Recommended: Co-author references in Git commits were suggested as mandatory requirements, while interaction logs would be recommended but not required.
Review Capacity and Quality Concerns
Concerns were raised about the cost and burden on reviewers:
Review Bottlenecks: The Open edX platform has grown substantially while the number of experts who deeply understand the system (knowing what will break, understanding history and future direction) remains limited. These experts’ time would become more constrained even if AI-generated contributions were rejected.
Inevitable Submissions: Even with a policy against accepting AI reviews, people would still submit code expecting at minimum a one-line response closing the request, creating workload regardless.
Scale Context: The Open edX project was characterized as one of the top projects ever open-sourced, including comparisons to Linux and Django, creating significant evaluation burden regardless of AI policy decisions.
AI as Force Multiplier
A counterargument positioned AI as potentially a force multiplier for key experts:
-
Expert reviewers were likely using AI to accelerate their work
-
AI assistance helps with code reviews and rote refactoring that would have been much more manual previously
-
High-quality code was being developed with immediate impact using tools like Cursor, Claude Code, and VS Code Copilot
-
The future involves directions provided in natural language producing new UIs or code functions with test harnesses
Quality Standards and Testing
Requirements were identified for maintaining quality:
Test Coverage: Need for robust, if not more robust, test coverage than currently exists to ensure quality when opening up the code generation pipeline.
Human Enforcement: Questions remained about having enough capacity to cover a step function increase in pull requests due to easier code writing.
Review Improvements: AI tools could potentially help with the code review bottleneck itself, though initial experiments with GitHub Copilot reviews were described as lackluster.
Standards, Skills, and Context
Discussion explored ways to improve AI-generated code quality:
Custom GPT and Skills: Skills or Claude Code capabilities could replace cookie cutters, automatically generating 90% of needed code for common patterns like new X blocks. Skills could also perform linting and check code against OEPs (Open edX Enhancement Proposals) for alignment.
Contextual Training: A significant quality jump exists between generic “review this pull request” prompts versus using a body of knowledge and training context. Developing Open edX-specific context-trained code review or development tools could make substantial differences.
Curated Information: Investment in environments to develop code for the Open edX platform with curated information sources could dramatically reduce development time compared to using general AI tools.
Documentation Strategy: Suggestion that documentation efforts should focus on custom GPT rather than traditional written documentation.
Tooling and Implementation
Specific tools and approaches were discussed:
Entire Tool: Recently announced tool by the former CEO of GitHub that includes additional metadata and context on commits to understand whether AI was used and how, enabling introspection at detailed levels.
Markdown Files: GitHub Copilot could be improved by editing Markdown files to make it more repository-specific, focusing on things like OEPs, ADRs, specific practices (e.g., wrapping in feature flags), though this approach showed mixed results initially.
Custom RAG Models: Potential for custom models with retrieval-augmented generation (RAG) specifically for the Open edX project, though this was acknowledged as vaporware at the time of discussion.
PR Review Automation
Strong interest emerged in using AI to address the pull request review bottleneck:
First-Pass Feedback: Offering automated pre-pass feedback upon PR submission rather than making contributors wait days for human review, helping address the “is this project alive?” question.
Review Quality: Concerns about automated reviews pushing people in the wrong direction, making contributions worse rather than better.
Bot Limitations: Initial experiences with automated review bots were lackluster, but the technology was evolving rapidly.
Maintenance Working Group: Identified as the appropriate forum for exploring AI assistance for maintenance tasks, as maintainer capacity was a major pain point.
Rapid Evolution and Future-Proofing
Recognition that the landscape was evolving extremely rapidly:
Outdating Documents: Acknowledgment that any written policy would quickly become outdated as the technology advances.
Agent Orchestration: Discussion of emerging patterns where principal engineers manage “Scrum teams” of AI agents rather than human-in-the-loop for every decision.
Model Improvements: Claude was noted as having dramatically outstripped other tools in recent months, with assessments from six months ago being wildly inaccurate three months later.
Infrastructure First: Emphasis on building infrastructure for the future even if not perfect initially (e.g., training an AI “Dave” to review like Dave Ormsbee might not be perfect at first but would improve substantially over six months).
Next Steps Identified
-
Develop recommendation for how to signal AI use in code contributions
-
Keep requirements lightweight to avoid burdening contributors
-
Focus on reducing maintenance burden through AI assistance
-
Finel to lead maintenance working group efforts on AI assistance
-
Share successful Copilot improvements via Markdown file tweaks across the project
-
Explore custom model and RAG opportunities for Open edX-specific AI tools
Front-End Base Migration [00:54:39 - 01:20:26]
Strategic Context and Goals
Discussion centered on a memo about front-end base migration, addressing the constellation of approximately 20 Micro-Frontends (MFEs) that people deploy routinely. These MFEs were originally created five years ago when edX focused on team autonomy, allowing independent team work but creating complexity in keeping them synchronized and upgraded.
The proposal aimed to reconsolidate MFEs while maintaining the benefits of front-end pluggability, so organizations would deploy fewer applications (e.g., one for instructors, one for students) while converting MFEs into components plugged into shell applications.
Technical Architecture
Key architectural elements included:
Slot-Based Extensibility: The front-end plugin framework currently working well in MFEs would be maintained as a core feature. Slots would continue to exist for component injection, though addresses for injection might change.
Component Modularity: Components would remain separate units of code that can be developed separately without requiring lockstep development, though they would be deployed together as part of larger applications.
Deployment Changes: The main trade-off was that independently deploying individual MFEs would become less easy. Deploying the learner experience would mean deploying the entire learner experience rather than individual components.
Future Pathways: While not currently being built, architectural pathways existed for dynamic loading of components from NPM or S3 buckets if independent deployment became a high priority again.
Customization Priorities
Strong concerns were raised about prioritizing customization capabilities:
Customization Over Out-of-Box: The position was articulated that customization ease for organizations using the Open edX platform should be prioritized over how the platform looks out of the box. Organizations should be able to make the product their own rather than just the product providers think they need.
Current Limitations: Current customization capabilities were characterized as “not high enough,” with specific examples cited including difficulty modifying video player surroundings and creating e-book reader experiences from course content.
Use Case Diversity: Recognition that every organization has different use cases, and if the goal is to grow the Open edX project, the platform cannot expect everyone to offer exactly the same service as MIT and edX (primarily MOOC provision).
Specific Use Cases Discussed
E-Book Reader Example: Faculty member requested help building an e-book platform, but the Open edX platform’s course interface doesn’t support the reading experience needed (sitting at night and reading). The question was whether organizations could easily modify the front end to create e-book-friendly experiences.
Video Player Customization: Attempts to change the video player and surrounding area received various responses about old MFEs, new X blocks, and compatibility issues, highlighting customization difficulties.
File Management: In campus use cases, file management was identified as a gap requiring workarounds like disguising X blocks because teachers cannot easily upload and link files when managing courses.
Platform vs. Products Philosophy
A fundamental philosophical discussion emerged about the future direction:
Platform-First Approach: The position was articulated that primary focus should be on the core platform that people build on top of, with a reference MOOC-oriented product built on that core. Additional products for campus, workforce, or K-12 could be built by various organizations on top of the core.
Market-Specific Products: Alternative view suggested picking specific markets (e.g., MOOCs, residential learning, e-books) and ensuring the platform is “really, really, really good” out of the box for 2-3 use cases rather than trying to be a universal tool.
Growth Strategy: Question raised about whether the Open edX project should prioritize ecosystem growth or making existing users’ lives better, with the position that the project is still small and should focus on rapid growth.
Legacy vs. Modern Code
Concerns were expressed about investment balance:
Over-investment in Modern Parts: Perception of potential over-investment in portions of the platform already relatively customizable (MFEs and front ends) while other parts remain legacy and hard to customize.
Growing Divide: The divide between easily customizable modern parts and difficult-to-customize legacy parts was described as getting bigger because focus remained on MFEs and front ends.
iframe Firewall: Reference to an “iframe firewall” between front ends and LMS content where customization is very difficult and needs improvement.
Operational Considerations
Team Distribution: Acknowledgment that advancing legacy contributions needs to be distributed across multiple teams in the ecosystem, not just borne by Axim, though Axim was currently doing a lot of this work.
WGU Investment: Western Governors University was dedicating half their team to advance this work, making it important from their deployment and customization perspective.
Maintenance Reduction: The move would allow retiring approximately 20 things currently being maintained, providing substantial value from a maintenance perspective.
Committee Perspectives
MIT Position: Strong sensitivity about prioritizing customization ease over out-of-box appearance, with specific concern that the platform should enable diverse use cases rather than prescribing specific offerings.
2U Perspective: Support for ensuring organizations can maintain custom and branded experiences while supporting improvements that help many organizations.
General Consensus: Agreement on the importance of customization as a first principle of architecture, even while recognizing that not all use cases can be supported equally.
Visibility and Documentation Gaps
Recognition that some customization difficulties might stem from awareness issues:
-
Developers may not know how much has changed or aren’t familiar with new tools
-
The option often feels like “adopt what exists or rewrite from scratch” when more gray area actually exists
-
Documentation, visibility, examples, and case studies could help close gaps
-
Custom GPT with curated documentation could help developers understand best approaches for customization
Next Steps Identified
-
Continue philosophical discussion at next meeting about whether the Open edX platform should be out-of-box solutions for specific markets or a universal customizable tool
-
Improve documentation and visibility of customization capabilities
-
Consider custom GPT or RAG-based tools to help developers understand customization options
-
MIT to provide specific details about video player customization challenges
-
Balance investment between modern customizable parts and legacy difficult-to-customize parts
University LMS Distribution [01:20:27 - 01:36:02]
Market Opportunity and Timing
Strong support was expressed for developing an Open edX distribution optimized for residential campus use cases, with characterization of a critical window of opportunity:
Global Demand: Multiple regions showing strong interest including Africa (800,000 students already using the Open edX platform for residential learning), Saudi Arabia (seeking to replace Blackboard across all universities), India (many universities using no LMS or old Moodle versions), and other mid-sized countries.
Market Window: A moment in time over the next year where international campus LMS platforms would be defined and chosen, and the Open edX platform having “a real shot” at capturing significant market share.
Target Governments: Focus on mid-sized governments (Brazil, Turkey, Saudi Arabia, Iraq) where an Open edX solution could be transformative, versus smaller countries that need any affordable option or huge countries (Japan, US, EU) that can afford to develop their own systems.
Funding Context: Multiple funders pursuing this in desperate (disparate) manners including MasterCard Foundation providing tranches to ASU, and NELC in Saudi Arabia investing, but efforts lacked coordination.
Technical Approach and Architecture
Discussion centered on how to implement university features:
Distribution Model: The envisioned approach was creating a distribution of the Open edX platform similar to how Ubuntu relates to Linux or Android relates to the general operating system—“making the Android” for universities specifically.
Open Questions: Technical implementation remained debatable—whether to use open core model, plugins, new MFEs, or complete distribution (e.g., Tutor distribution specifically for universities).
Cost Considerations: Implementation would become more expensive, and while providers had funded work so far, next steps might progress faster with dedicated funding to accomplish goals in 3-4 months.
Key Feature Gaps Identified
Several critical gaps were documented between current capabilities and university requirements:
Classroom and Semester Awareness: The Open edX platform lacks native recognition of classrooms and semesters, requiring hacks like CCX (Custom Courses for edX) which function more as workarounds than proper solutions.
Teacher-to-Student Communication: No easy way for one-to-one teacher-student communication, with no open source plugin available for this functionality.
Student Information System Integration: Integration story not well-known with lots of experimentation but no mainstream adoption of any approach.
Reports and Analytics: Current instructor dashboard expects 100,000 users for number crunching, appropriate for MOOCs and large sites but not universities where classes have maximum 300 students. Universities need different report types focused on individual students with navigable lists and pagination rather than aggregate statistics.
Live Classes: Recent improvements made but still not top-notch as a completely native feature. Issues include not retaining recording links, lacking interaction capabilities, and missing attendance tracking.
Classes and Sections: Whether implementing as one course per class or one course for 20 common classes, current solutions are hacks rather than proper classroom implementations.
Previous Efforts and Lessons
Spanish Consortium Work: Substantial work conducted through European grant funding, including interviewing teachers using Sakai and Moodle to understand on-campus usage patterns. Documentation existed covering what was addressed and what gaps remained, available for sharing.
Campus Working Group: Previous attempt gathered extensive requirements without a path to achieving realization, leaving participants’ valuable input unused without delivering results. This pattern needed to be avoided in future efforts.
File Management: Identified as a critical gap in previous projects, requiring creation of X blocks to disguise how files are managed because teachers cannot easily upload and link files when managing courses.
Funding and Collaboration Model
Strong recommendations emerged for a collaborative funding approach:
Coalition of the Willing: Preference for creating a consortium where multiple organizations each contribute funding amounts, then building a backlog based on that coalition rather than gathering requirements without resources to implement them.
University Participation: Suggestion to approach universities spending millions on Canvas (MIT, Harvard, others using combinations of Catsoup, the Open edX platform, and Canvas) to contribute quarter or half million dollars each to create a pool funding an open source Canvas competitor.
Foundation Support: Gates Foundation and other foundations could potentially contribute to the funding pool.
Provider Involvement: Organizations like 2U might be interested in participating in working groups even if their timeline is longer than others.
Skin in the Game: Organizations close to learners (Harvard, NELC, etc.) should put funding in because they already have skin in the game, ensuring they request the most important features and making the product useful quickly rather than conducting product research in a vacuum distant from university learners.
User Research Requirements
Emphasis placed on recruiting actual users to inform development:
Teacher Panels: Need to recruit teachers using both the Open edX platform and other LMSs (Canvas, Moodle, Sakai, Blackboard) who can articulate what the platform lacks.
Cross-Platform Experience: Focus on finding people with experience in multiple systems (like Hermann from Universidad AutĂłnoma de Madrid using both the Open edX platform and Moodle) who can provide comparative insights.
Real Usage Patterns: Rather than studying what’s on other platforms, directly observe how teachers actually use various LMSs in their universities to understand real needs versus assumed needs.
Parallel Workstreams: User research should proceed in parallel with funding acquisition rather than sequentially.
Backend Changes Required
Recognition that some necessary changes would need to occur in the backend, not just front-end customizations:
File Management: The way files are managed needs fundamental changes to support easy upload and linking by teachers managing courses.
Core Functionality: Two, three, or four different aspects of basic LMS functions need to be addressed differently in the Open edX platform core.
Decision Authority: Someone in the Open edX community needs to be willing to make important changes to accommodate basic LMS functions.
Current Progress and Next Steps
Documentation Status: Extensive documentation exists with raw materials to review from work ongoing since 2023, including feature comparisons and customer reviews/interviews.
Visual Artifacts: Next step identified as creating Figma files and Miro boards to make progress visible and gather feedback from interested parties.
Market Validation: Plans to show visual artifacts to customers and interested parties before finalizing architectural decisions about implementation approach.
Gap Assessment: Need to ensure comprehensive gap identification, as missing 5-10 requirements could prevent 30% of potential adopters even if most requirements are met.
Strategic Recommendations
Avoid Duplication: Ensure all existing work (Spanish consortium, ASU efforts, Saudi Arabia plans) gets coordinated rather than duplicated.
Prioritize Real Users: Build with organizations that actually have learners to serve rather than building in a vacuum.
Realistic Scoping: Accept that not all use cases can be supported equally—focus on specific campus LMS use cases rather than trying to be everything to everyone.
Market Timing: Recognize urgency of current window of opportunity before other platforms become entrenched.
Closing Discussion [01:36:02 - 01:36:02]
The meeting concluded with appreciation for the extended conversation and agreement that the university LMS distribution represented a significant strategic opportunity requiring coordinated action from multiple stakeholders.