Aspects superset-row-level-security patch

Hey all,
I’m currently working on assigning correct roles in the superset service added to openedx via the aspects plugin. The issue is around a patch I think may be broken or I’m just using it wrong. If you do not care about the background of the issue just scroll down to “exact problem”.

Based on this one page of documentation
Superset extra row level security — Open edX Aspects latest documentation (and some really old documentation from aspects v0.16.1 which I guess is irrelevant but it helped me a bit as well tutor-contrib-aspects · PyPI ) it is possible to add custom RLS rules to the platform.

My goal was to achieve a user who can create dashboards and charts but composed of data only from courses belonging to specific organizations (basically a user that would be given to a client company’s data analyst).

By default there are 3 RLS rules applied to the Instructor role however that role has no access to creating dashboards/charts.

First of all upon login into Open edX using a custom function added to the django auth pipeline I am assigning certain users staff permission on ONLY the courses belonging to their organizations.

Then to propagate that to a proper role in superset I am using the superset-sso-assignment-rules patch (Unfortunately group assignment logic is duplicated there - I’m not sure yet how to abstract it from here and the auth_pipeline). This has to be done since by default NO open edx funciton is mapped to the gamma function.

(can be seen inside superset container at /app/pythonpath/openedx_sso_security_manager.py in the get_user_roles function).

    if decoded_access_token.get("superuser", False):
        return ["admin", f"admin-{locale}"]
    elif decoded_access_token.get("administrator", False):
        return ["alpha", "operator", f"operator-{locale}", "instructor", f"instructor-{locale}"]
    else:
        # User has to have staff access to one or more courses to view any content
        # here. Since this is only called on login, we take the opportunity
        # to force refresh the user's courses from LMS. This allows them to bust
        # the course permissions cache if necessary by logging out and back in.
        courses = self.get_courses(username, force=True)
        if courses:
            if False:
                raise Exception(f"Instructor {username} tried to access Superset")
            return ["instructor", f"instructor-{locale}"]
        else:
            roles = self.extra_get_user_roles(username, decoded_access_token)
            if roles:
                if True and 'student' in roles:
                    raise Exception(f"Student access not allowed for {username} due to SUPERSET_BLOCK_STUDENT_ACCESS setting.")
                return roles

            if True:
                raise Exception(f"Student {username} tried to access Superset")
            else:
                return ["student", f"student-{locale}"]

So I am propagating based on some user values (like group ID)


hooks.Filters.ENV_PATCHES.add_item(

    ("superset-sso-assignment-rules", """

group = get_group_logic

if  group is analytics_people

    return ["gamma"]

return []

"""))

Having the Gamma role (access to all courses’ data since as I mentioned only Instructor has RLS built in) I want to apply exactly the same RLS to the Gamma user role as the default one for the Instructor role.

Previously I mentioned that I set the course_access_role to staff for analytics user. I did that since the can_view_courses function found at cd $(tutor config printroot)/env/plugins/aspects/apps/superset/pythonpath/openedx_jinja_filters.py only let’s users in superset access data if they have that role set (again, Admin and Alpha in the code snippet below are too wide for my case)

mentioned function.

for role in user_roles:
    if str(role) == "Admin" or str(role) == "Alpha":
        return ALL_COURSES

# Everyone else only has access if they're staff on a course.
courses = security_manager.get_courses(username)

Now the exact problem:
Apparently you can modify the RLS settings with this patch
"superset-row-level-security". It does actually modify the create_row_level_security.py file. (File at cd $(tutor config printroot)/env/plugins/aspects/apps/superset/pythonpath/openedx/create_row_level_security.py)
HOWEVER I think the patch is wrong. It starts AFTER the array of SECURITY_FILTERS and it always inputs code with an INDENT rendering the .py file unlaunchable.

Default part of file (pasted as photo since couldn’t get indentation to work)

After writing an example patch:

from tutor import hooks
# According to docs, this should just append a dict to the list.
# In reality, it breaks because Tutor/Jinja injects indentation before the brace.
hooks.Filters.ENV_PATCHES.add_item(

    ("superset-row-level-security", """

{

    "name": "gamma_rls_broken_demo",

    "schema": "event_sink",

    "exclude": [],

    "role_name": "Gamma",

    "group_key": "broken_demo",

    "clause": "1=1",

    "filter_type": "Regular",

},

"""))

The file looks as follows:

Could anyone please try to replicate this or help me solve my specific issue? This is Aspects 2.5.1. and I’m using the teak release. I also found a mismatch in the API call for when superset gets the user courses in openedx_sso_security_manager get courses function It calls permissions with an s where the actual API in this openedx version has no s… am I just fighting a version mismatch? Could not find Openedx x aspects compatibility table?

As a temporary (very ugly) solution I could overwrite the whole file using atexit from the plugin level however that is bound to break upon changing systems or updating aspects. I am looking for something more sustainable.

Hi @jakubkl - thank you so much for bringing this up, I think this might be an issue with the documentation. I’m assuming that the patch is loaded after the list is defined is because users might want to overwrite the defaults and use a completely different list of roles. The documentation will be updated to explain that you can define a complete list of roles, or append to the existing list.

What it sounds like you need to do, is format your patch to append the new role you want to add:

SECURITY_FILTERS.append({

    "name": "xxx",

    "schema": "xxx",

    "table_name": "xxx",

    "role_name": "xxx",

    "group_key": "xxx",

    "clause": "xxx",

    "filter_type": "xxx",

})

If that doesn’t work, please let me know!

As for get_courses - permission vs permissions - i’m seeing permissions in the Teak release openedx-platform/lms/djangoapps/courseware/courses.py at release/teak · openedx/openedx-platform · GitHub

Hey Sara,

thank you for the insightful answer and sorry for getting back to you so late, I was away from work.
I actually tried to patch the list by appending to it like you suggested before however the python code is unlaunchable because of the indentation. A plugin that starts like so:

hooks.Filters.ENV_PATCHES.add_item(

    ("superset-row-level-security", """


SECURITY_FILTERS.append({

    "name": "gamma_rls_event_sink",

    "schema": "event_sink",

    "exclude": ["user_pii"],

    "role_name": "Gamma",

    "group_key": "xapi_course_id",

    "clause": {% raw %}'{{ can_view_courses(current_username(), "course_key") }}'{% endraw %},

    "filter_type": "Regular",

}) ....

produces a config file like this:


Unfortunately I could not come up with any python hacks to make it work.

As to the permission vs permissions, it was just my mistake :sweat_smile: . When I called the API with permission the parameter was ignored and all courses visible for that user got retrieved. With the permissions=staff the parameter was correctly read and no courses were retrieved since I just assigned the CourseAccessRole wrong (via direct SQL instead of proper functions that do this).

I am still experiencing some difficulties with one more patch though. The superset-sso-assignment-rules populates a function that is only called AFTER the regular role assignment function and only under specific circumstances. The user has to:

  • not be an superuser/admin

  • not have the staff parameter in any courses

    If I do not assign the course-level staff role to any courses for the user I get into my extra logic and am able to assign the user the Gamma role upon login. The default jinja RLS filter however only returns courses for users that have the staff course-level role.

Seems like the solution is to not assign any extra staff roles to the user, give him Gamma during the extra roles assignment, then create a custom jinja filter (using the superset-jinja-filters patch) and then a custom RLS using that filter… That approach once again duplicates the role assignment based on user info (extract org. slug from email etc) and seems over-complex for what would seem to be a popular feature - creating accounts for data analysts with access to data from courses of only a specific client(s).

I will try that and give an update however let me know what you think.

Is it the indentation that is giving you an error? I was able to get it to work locally - I don’t think indents should affect python compiling.

I’m going to tag in @TyHob here - he knows a lot more about how RLS works and might be able to help.

Hi @jakubkl you might be the first group to try this functionality in a while, it’s definitely on the less tested end of things currently. I think the superset-row-level-security patch is just broken. When this change was made the indent should have been removed.

I agree with what you’re seeing with this use case and am curious to hear how your hacking goes. I’ve created an issue for the RLS patch. It would be great if you could write up an issue with a description of exactly what you’re trying to do on the superset-jinja-filters patch side of things we can hopefully clean that up and improve testing there as well.

Hey @TyHob thanks for resolving my doubts, I was unsure where to look for how the patch itself is programmed.

With the superset-jinja-filters I was trying to create a new clause used by the RLS that would return courses information from tables based on my needs.

I have managed to create a solution that fits in one plugin. Unfortunately it is most probably not sustainable across versions, however temporarily it works. It goes like this:

Goal: Upon login based on some grouping logic, assign the Gamma role to a data analyst account (Gamma because he has to be able to create charts/dashboard) however allow him to only view data from courses of a specific organization that he is linked to.

  1. Create a fresh account for the analyst - just a student status (he can be enrolled in the courses he’s analyzing but NO staff role - neither platform wide or course specific)
  2. Add a custom superset role assignment logic using the superset-sso-assignment-rules which is injected into the function extra_get_user_roles in openedx_sso_security_manager.py and invoked ONLY if the user has JUST a basic student status and no staff roles anywhere. Using that give him the Gamma role.
  3. Using the superset-jinja-filters patch create two new functions in the openedx_jinja_filters.py file. One is the filter itself very similar to the original, embedded into the aspects plugin - can_view_courses filter, I call it can_analyze_courses. The main difference is that it does not use the get_courses function found in the openedx_sso_security_manager but a custom get_courses_by_org function. This has to be done as the original one returns courses only where the user has the staff permissions (remember, we couldn’t give those to him because then we wouldn’t be able to automatically assign the Gamma role). Therefore the second function created using the patch returns courses from the API using the org parameter. We should get the organization parameter used in that call from the user’s attributes (again, user-organization assignment logic is duplicated..)
  4. Using the superset-config-docker the function is registered by being appended to the JINJA_CONTEXT_ADDONS dictionary (I read in the docs that this happens by modifying SUPERSET_EXTRA_JINJA_FILTERS however that didn’t work? I experimented by modifying the addons after seeing the original filter’s wrapper was listed there. Superset extra jinja filters — Open edX Aspects latest documentation )
  5. Finally create the RLS that use the new jinja filter with the broken patch. Here I mark the patched part by special #START and #END comments.
  6. Here is the dirtiest part - to fix the indent I just add a python .atexit function that finds the markers mentioned in 5. and reassembles the file without the indents.

All of this seems like total overkill but it works (at least at first glance when creating charts, I am yet to test everything out).

Here is the ready plugin. As a test the organization is just hardcoded (no extraction logic) and assigning Gamma is based on an email check, once again, any other sensible logic could be implemented here. USER_EMAIL_PLACEHOLDER and ORGANIZATION_PLACEHOLDER can be seen in the code.

I am open for suggestions and any critique :slight_smile:

from tutor import hooks
import os
import atexit
import textwrap

def fix_superset_rls_indentation():
try:
root_path = os.environ.get(“TUTOR_ROOT”, os.path.expanduser(“~/.local/share/tutor”))

    target_file = os.path.join(
        root_path, 
        "env", "plugins", "aspects", "apps", "superset", 
        "pythonpath", "openedx", "create_row_level_security.py"
    )

    if os.path.exists(target_file):
        with open(target_file, 'r') as file:
            content = file.read()

        if '#BEGINNING_OF_NEW_RLS' in content and '#END_OF_NEW_RLS' in content:
            pre_marker, rest = content.split('#BEGINNING_OF_NEW_RLS', 1)
            block, post_marker = rest.split('#END_OF_NEW_RLS', 1)

            fixed_block = textwrap.dedent(block)
            new_content = pre_marker + fixed_block + post_marker
            
            with open(target_file, 'w') as file:
                file.write(new_content)
            
except Exception:
    pass

atexit.register(fix_superset_rls_indentation)

hooks.Filters.ENV_PATCHES.add_item(
(“superset-sso-assignment-rules”, “”"
email = decoded_access_token.get(“email”, “”).lower()
if email == “USER_EMAIL_PLACEHOLDER”:
return [“gamma”]
“”"))

hooks.Filters.ENV_PATCHES.add_item(
(“superset-row-level-security”, “”"
#BEGINNING_OF_NEW_RLS
SECURITY_FILTERS.append({
“name”: “gamma_rls_event_sink”,
“schema”: “event_sink”,
“exclude”: [“user_pii”],
“role_name”: “Gamma”,
“group_key”: “xapi_course_id”,
“clause”: {% raw %}‘{{ can_analyze_courses(current_username(), “course_key”) }}’{% endraw %},
“filter_type”: “Regular”,
})

SECURITY_FILTERS.append({
“name”: “gamma_rls_reporting”,
“schema”: “reporting”,
“exclude”: 
,
“role_name”: “Gamma”,
“group_key”: “xapi_course_id”,
“clause”: {% raw %}‘{{ can_analyze_courses(current_username(), “course_key”) }}’{% endraw %},
“filter_type”: “Regular”,
})

SECURITY_FILTERS.append({
“name”: “gamma_rls_xapi”,
“schema”: “xapi”,
“exclude”: 
,
“role_name”: “Gamma”,
“group_key”: “xapi_course_id”,
“clause”: {% raw %}‘{{ can_analyze_courses(current_username(), "splitByChar(\’/\‘, course_id)[-1]") }}’{% endraw %},
“filter_type”: “Regular”,
})
#END_OF_NEW_RLS
“”")
)

hooks.Filters.ENV_PATCHES.add_item(
(“superset-config-docker”, “”"
from openedx_jinja_filters import can_analyze_courses

JINJA_CONTEXT_ADDONS.update({
‘can_analyze_courses’: can_analyze_courses,
})
“”")
)

hooks.Filters.ENV_PATCHES.add_item(
(“superset-jinja-filters”, “”"
from superset.extensions import cache_manager
from flask import session, current_app

def get_courses_by_org(sm, username, org, next_url=None, force=False):
log.info(f"[DEBUG RLS - API] Fetching org: ‘{org}’ for ‘{username}’")

cache = cache_manager.cache
cache_key = f"{username}+org+{org}"

if not next_url and not force:
    obj = cache.get(cache_key)
    if obj is not None:
        return obj

courses = []
provider = session.get("oauth_provider")
oauth_remote = sm.oauth_remotes.get(provider)

if not oauth_remote:
    log.error("[DEBUG RLS - API] No OAuth remote")
    return courses

token = sm.get_oauth_token()
if not token:
    log.error("[DEBUG RLS - API] No token")
    return courses

if next_url:
    url = next_url
else:
    raw_url = current_app.config["OPENEDX_API_URLS"]["get_courses"]
    base_url = raw_url.split('?')[0]
    url = f"{base_url}?username={username}&org={org}"

log.info(f"[DEBUG RLS - API] Calling URL: {url}")

try:
    resp = oauth_remote.get(url, token=token)
    log.info(f"[DEBUG RLS - API] API Status: {resp.status_code}")
    log.info(f"[DEBUG RLS - API] API Response: {resp.text}")
    
    resp.raise_for_status()
    response = resp.json()
except Exception as e:
    log.error(f"[DEBUG RLS - API] Request failed: {str(e)}")
    return courses

results = response.get("results", [])
for course in results:
    course_id = course.get("course_id")
    if course_id:
        courses.append(course_id)

if response.get("next"):
    next_courses = get_courses_by_org(
        sm, username, org=org, next_url=response["next"]
    )
    courses.extend(next_courses)

if not next_url:
    cache.set(cache_key, courses, timeout=300)

return courses

def can_analyze_courses(username, field_name=“course_id”, **kwargs):
log.info(f"[DEBUG RLS - JINJA] Evaluating for ‘{username}’")

user = security_manager.get_user_by_username(username)
user_roles = security_manager.get_user_roles(user) if user else []

if not user_roles:
    return NO_COURSES

for role in user_roles:
    if str(role) in ["Admin", "Alpha"]:
        return ALL_COURSES

user_org = "ORGANIZATION_PLACEHOLDER" 

try:
    courses = get_courses_by_org(security_manager, username, org=user_org, force=True) 
    log.info(f"[DEBUG RLS - JINJA] Found {len(courses)} courses")
except Exception as e:
    log.error(f"[DEBUG RLS - JINJA] Error fetching courses: {str(e)}")
    courses = []

if courses:
    course_id_list = ", ".join(f"'{course_id}'" for course_id in courses)
    return f"{field_name} in ({course_id_list})"
else:
    log.warning("[DEBUG RLS - JINJA] Course list empty. Returning 1=0.")
    return NO_COURSES

“”"))