### Abstract
This is the updated version of the "Improve Common Cartridge Imp…ort to Open edX" proposal. It includes decomposed milestones based on feedback from the first proposal
**Goal:** The primary objective of this initiative is to fully integrate the Common Cartridge (CC) standard into the Open edX platform. This enhancement aims to facilitate the effortless migration of digital learning content from widely-used Learning Management Systems such as Canvas, D2L Brightspace, Blackboard, and Moodle to Open edX. The initiative seeks to address the current challenges faced by educational institutions and businesses in migrating course data seamlessly and without technical barriers.
**Problem Statement:** Currently, Open edX does not support the Common Cartridge format natively, which leads to a cumbersome and error-prone process using a third-party conversion tool. This limitation acts as a barrier for institutions considering a switch to Open edX, especially given the popularity of Common Cartridge among major LMS providers.
**Proposed Solutions:**
**Quick Solution:** Enhance the existing cc2olx converter to improve its reliability and efficiency in transforming .imscc files into .tar.gz format suitable for import into Open edX.
**Strategic Solution:** Develop native support in Open edX for direct import of .imscc files, enabling a straightforward and user-friendly migration process directly within the Open edX Studio environment.
**Intended Outcomes:**
**Educational Institutions:** Facilitate easier transitions to Open edX, providing a viable and attractive alternative to current LMS options.
**Open edX Contributors and Community:** Simplify the content migration process, reduce the potential for data loss, and increase the platform's appeal and functionality, thereby attracting a broader user base and enhancing the platform’s community-driven development.
**Market Impact:** Position Open edX as a competitive player in the LMS market by removing interoperability barriers and catering to the needs of institutions looking for robust and flexible education technology solutions.
### Context & Background
### What is Common Cartridge?
**A Common Cartridge (CC)** is a set of open standards developed by the [1EdTech](https://www.imsglobal.org/activity/common-cartridge) member community that enable interoperability between content and systems. Basically, it is a standard for creating and sharing digital learning content in the education sector. This standard is supported by major LMS systems on the market like Blackboard Learn, Moodle, Brightspace, Canvas. As a result, users of these systems can effortlessly export course data in .imscc archive from one system and seamlessly upload it to another, ensuring fast and easy migration between systems. A Common Cartridge file contains features such as an IMS manifest file, roles metadata, LTI links, weblinks, embedded files, and question data.
### What is inside Common Cartridge?
A Common Cartridge file contains IMS manifest file, roles metadata, LTI links, weblinks, embedded files, and question data.
- **The manifest file** describes the content contained in the cartridge. For example, in a Common Cartridge compliant learning app, the manifest file informs which version is being used to package its content. The components that make up the manifest file may be in the package or external to the package.
- **Metadata** lets developers label and organize a cartridge’s content. The metadata can be in the IMS Learning Object Metadata (LOM) schema or any other metadata schema. Curriculum standards that are attributed to specific content or links packaged in the file can also be included in the LOM. **Common Cartridge Metadata file corresponds to the course.xml file in OeX.**
- **A Common Cartridge file** can contain LTI launch links and links to external web pages. The standard allows for data to be exchanged between externally linked applications and a LMS. LTI links can also be used as a method of authorizing users so that they can access remote content securely.
- The package can contain several pieces of **multimedia content**. This can include web content in an HTML format and common file types, such as PDFs, EPub textbooks, videos, and text documents.
- A Common Cartridge package can contain **tests, assessments, and question banks** that are formatted using the IMS Question and Test Interoperability (QTI) and Accessible Portable Item Protocol (APIP) standard.
- Common Cartridge files cannot contain **SCORM files**. However, SCORM content can be converted to Common Cartridges.
Common Cartridge Content Types
<img src="https://github.com/openedx/platform-roadmap/assets/167102272/a4f8bf67-9d21-4c5f-9ed5-8775f2d55fe8" width="500" />
.imscc file content example
<img src=https://github.com/openedx/platform-roadmap/assets/167102272/3f23c8a5-0036-4be9-baf0-1899765b62c4 width="300" />
### Common Cartridge Versions
| IMS Common Cartridge version | Summary |
|--------|--------|
| IMS Common Cartridge 1.0 | In the resource metadata context, includes the higher education value. Contains metadata for the intended user roles, Learner and Instructor. These are the user roles that can view the resource. Contains the intendedUse attribute for Web Content Resources. This attribute must contain one of the following values: lesson plan, syllabus, or unspecified. |
| IMS Common Cartridge 1.1 | Offers easy support and integration with IMS Learning Tools Interoperability (LTI) v1.0 resources. You can integrate rich learning applications or premium content with platforms or other systems from which applications can be launched, without additional customization. In the resource metadata context, includes the following values: higher education, school, training, and other. Adds metadata for the Mentor intended user role. The intendedUse attribute for Web Content Resources is optional in this Cartridge version. |
| IMS Common Cartridge 1.2 | Includes metadata for curriculum standards, which provides the option to associate the cartridge, resource, or question item with zero or more curriculum standard identifiers, from one or more providers. Users can use any curriculum standard, provided that it supports unique identifiers. The intendedUse attribute for Web Content Resources offers a new attribute called Assignment. |
| IMS Common Cartridge 1.3 |
Allows you to import multiple question banks within a single cartridge.Provides the ability to specify Common Cartridge XML descriptions within an imsmanifest.xml file.Provides the option to add metadata to LTI links. |
| Common Cartridge (CC)/Thin Common Cartridge (TCC) 1.4 | Expands upon the legacy specification to include new features: a K-12 profile, built from required metadata; the ability to align to CASE standards; support for the upcoming 1EdTech specification OpenVideo; and adds line items to LTI links. This version extends the interoperability already existing in CC/TCC and provides additional benefits to K20 institutions when publishers take advantage of the K-12 profile. |
### Market Research
Common Cartridge is employed by major LMS providers including Canvas, D2L Brightspace, Blackboard, and Moodle. These platforms collectively hold[ 88% of the Higher Ed LMS Market in the US and Canada](https://onedtech.philhillaa.com/p/state-of-higher-ed-lms-market-for-us-and-canada-year-end-2022-edition).
Most institutions that currently utilize a prominent platform often overlook Open edX as a potential alternative. For instance, dissatisfaction with Blackboard Learn LMS has prompted many users to seek migration options, as evidenced in this [reddit thread](https://www.reddit.com/r/Professors/comments/z6a4ee/changing_our_lms_currently_using_blackboard/). The challenge of seamlessly migrating courses may contribute to the underestimation of Open edX as a viable alternative.
### CC import to OeX — Current state
Open edX currently does not support the Common Cartridge standard, meaning tha courses exported in .imscc format cannot be seamlessly imported into the system. However, a [cc2olx](https://github.com/openedx/cc2olx) converter has been developed to transform Common Cartridge .imscc files into .tar.gz files, which can then be imported into Open edX Studio.
**Converter discovery**
Based on the [discovery](https://docs.google.com/spreadsheets/d/1eU3ZkGIPODRMUbgHx08xG3HRecWlmRerpQ6Aqi4iMYE/edit?gid=0#gid=0) made by 2U/edX team and provided by @cablaa77, the conversion success of specific file types appears to be uniform across all LMS platforms — if one type can be converted, it generally can be converted from any system in most cases. Below is a table comparing content types in Common Cartridge and number converted files. The discovery revealed conversions from Canvas (7 files converted), Moodle (1 file converted), Blackboard (1 file converted), and D2L Brightspace (1 file converted).
CC content | edX Content term | Files converted
-- | -- | --
Web Content | Files and Uploads | 10/10
Web Link | HTML Component | 10/10
Associated Content | LTI, other resource specific to external LMS**All non-native canvas content. This would include external tools and uploaded content as well. | 0/8
Multiple Choice | Problem Component (Multiple Choice Problem) | 5/5
Multiple Response | Problem Component (Checkbox Problem) | 3/3
Fill in the Blank | Problem Component (Text Input Problem) | 1/1More investigation is needed
Boolean | Problem Component (Text Input Problem) | 3/3
Pattern Match | | 0/3
Essay | ORA (Open Response Assessment) | 3/3
Discussion Topic | Discussion Component | 6/8Could not convert from Moodle and Brighspace.
**Raccoon Gang Experience**
During the latests process of using the converter to import courses from D2L Brightspace to Open edX, various issues were [encountered](https://discuss.openedx.org/t/cc2olx-converter-content/12461/3)
The imscc. archive contained a path with an unusual UTF-8 symbol, which appeared as “c” but was not recognized by Python’s filesystem library. The problem was resolved by renaming and repacking the exported course. The issue was related to MacOS filesystem, on Ubuntu the same bug wasn’t reproduced, so it may be usefull to enhance the user experience and create docker environment to run the conversion script.
The content included CDATA sections, leading to a ValueError. To fix this, a sanitization script prior to generating OLX was implemented.
The migrated course structure to Open edX consisted sections (modules in D2L) with one subsection and many units. However, initial course in D2L had included several subsections in each section (module in D2L).
Page description of Modules (for sections & subsections) were left empty after migration.
QTI Quizzes were not migrated due to problems during parsing. The algorithm to import QTI worked after changes in the parser algorithm.
PDFs were migrated as plain assets, advanced modules are not used during migration
Relative links in HTML for visuals (images, headings, icons) or external links aren’t valid after migration due to missing domain in URLs. The possible solution is to create absolute URLs by using user inputted configuration during migration.
Text blocks, Custom HTML embedded videos in Panopto & Discussions migrated well.
### Scope & Approach
**Scope**
From the current perspective, two ways of common cartridge conversion are possible.
1. Improve Converter
Enhance the current [cc2olx](https://github.com/openedx/cc2olx) converter to address any existing issues, allowing for a seamless transformation of courses from imscc format into .tar.gz files.
2. Integrate converter into Open edX
Integrate the capability to directly import .imscc files into Open edX through Studio functionality. This enhancement will enable effortless migration of courses from other LMS to Open edX, catering to users without technical expertise.
**Approach**
First, improving the converter to resolve known issues will increase the conversion rate and make the migration process from other LMS to Open edX smoother.
Second, integrating the capability to directly import IMSCC files into Open edX through Studio is a logical step.
After collecting feedback from users who try the new import tool, the Open edX community can identify primary areas for improvement and enhance the converter in future phases.
The benefits of this approach include:
1. Saving time and effort on identifying current issues with the cc2olx converter.
2. Increasing the amount of data collected from users utilizing imscc supported LMSs compared to technical discovery planned in the first version of proposal.
### Primary Beneficiaries
Integrating Common Cartridge files into Open edX will streamline the migration process and establish Open edX as an attractive option for educational institutions and businesses currently using platforms like Canvas, Brightspace, Blackboard, or Moodle.
**Beneficiaries of this enhancement include:**
1. **Educational Institutions:** Those utilizing platforms such as Canvas, Brightspace, Blackboard, or Moodle can now view Open edX as a practical alternative. This compatibility broadens their choices and facilitates a smoother transition to Open edX should they decide to switch.
2. **Open edX Contributors:** Contributors will benefit from a simplified and expedited process for migrating courses from the aforementioned platforms to Open edX. This efficiency not only saves time but also reduces the potential for errors during course transfers.
3. **Open edX Community:** The support of Common Cartridge is likely to attract a broader audience. An increase in users not only enriches the community but also fosters more diverse contributions to the platform’s development and enhancement.
### Milestones and/or Epics
### Phase 1: cc2olx converter improvement
In the first phase of cc2olx improvement we would like to resolve the known issues faced during latest usage of the cc2olx to convert CC courses to Open edX format.
- Algorithm enhancement should include the following:
- Enhance migration of the course structure, by fixing parts such as joined subsections and correctly migrate Section and Subsection titles if specified.
- Enhance HTML parcing and processing to fix issues with embeded Styles, JS scripts and other elements that might be already embed into CDATA.
- Enhance HTML parcing by allowing specifying additional user configurations to be used during conversion, such as Root domain for relative links. This might help migrating external resources with relative pass (docs, visuals, images, etc.)
- Support Submission type of the assement using file upload functionality in ORA xBlock in addition to currently supported Essay type.
- Discover possibility to recreate course Grading policy and configuration.
- Discover how to handle LMS specific XML for Canvas, D2L, Moodle, Blackboard, if implementation differs from the standard, e.g., (https://github.com/openedx/cc2olx/blob/master/src/cc2olx/external/canvas/module_meta.py#L7)
- Usability enhancements for the library will include:
- Enhance general output of the algorithm, provide more discriptive errors to users, track and log conversion steps.
- Optional specification for custom xBlocks to migrate content to. For example migrate PDFs to PDF xBlock instead of just link to asset. List of xBlocks that might be included for diferrent types of content:
- PDF xBlock
- Poll / Survey xBlock
- google-document xBlock
- feedback xBlock
- Tutor integration to run conversion in a dockerized environment. Possibly adding some additional utils to enhance opertors experience using conversion.
**Impact metrics**
- The result of this phase will be an increased percentage of content/content types converted from Common Cartridge to OLX. We expect the number reach over 80% of all content exported in imscc format for 90% of cases.
### Phase 2: Direct import in OeX Studio — Core Product Optimization _(DRAFT)_
As a user without technical knowledge, I want to effortlessly import courses in Common Cartridge format into Open edX using the user-friendly interface, so that I could smoothly migrate my course from Blackboard, Moodle, Brightspace, or Canvas to the Open edX platform.
Detailed description and Wireframes will be added soon.
AC:
1. User can import .imscc file in Course Import tab in Studio.
2. After conversion system shall give the report with information
a. what percentage of data was imported to course outline
b. what files was not imported into the system
c. error message if file was not imported
**Impact metrics**
This phase will allow to improve following metrics:
- Time needed to import courses in Common Cartridge format to Open edX
- Number of new users who transitioned to Open edX from Blackboard, Moodle, Brightspace, or Canvas.
### Phase 3: cc2olx converter further improvement (Draft)
Based on feedback collected from the .imscc importing tool beta launched during Milestone 1, the OeX community will now possess the ability to identify the primary issues of the current CC2OLX across different LMS systems, prioritize them, and systematically resolve.
### Other Options Considered
**CC Plugin to Open edX:** Implement third-party plugin to edx-platform, that will handle the conversion of the CC content via graphical interface, and allow this plugin to be optionally installed and enabled.
### Named Release
- First phase — Sumac.
- Second phase — Sumac and Teak.
### Timeline
TBD
### Proposed By
Raccoon Gang
###Additional info
- cc2olx converter for Common Cartridge - https://github.com/openedx/cc2olx/
Comments considered during proposal update:
- https://github.com/openedx/platform-roadmap/issues/352#issuecomment-2128066281
- https://github.com/openedx/platform-roadmap/issues/352#issuecomment-2161616423