SAML IdP "hinted links" not working in Lilac

mrtmm · December 15, 2021, 1:25pm

Hi all! We are facing a SAML issue with Lilac. After upgrading from koa to lilac, the SAML hinted sign in links are not working anymore, more precisely, the redirection after login does not work. It appears that the “next” param gets dropped and learners will land on /dashboard always instead of being taken to the correct page. It also appears that the issue is only with SAML (okta).

When SAML provider is in debug mode, we can see that the “next” parameter is there in the request but for some reason None on the response:

2021-12-15 08:15:34,123 INFO 22107 [common.djangoapps.third_party_auth.saml] [user None] [ip 127.0.0.1] saml.py:178 - SAML login request for IdP okta. Data: <QueryDict: {'auth_entry': ['login'], 'next': ['/courses/<course-id>/course/?tpa_hint=saml-okta'], 'idp': ['okta']}>. Next url /courses/<course-id>/course/?tpa_hint=saml-okta. XML is:

2021-12-15 08:15:34,696 INFO 21978 [common.djangoapps.third_party_auth.saml] [user None] [ip 127.0.0.1] saml.py:178 - SAML login response for IdP okta. Data: <QueryDict: {'SAMLResponse': ['<redacted>'], 'RelayState': ['okta']}>. Next url None. XML is:

Has anyone else run into this? What changed between koa and lilac that could affect the SAML hinted links?

What is different between our koa and lilac platforms is that for lilac we enabled the Account MFE, could that be somehow related? Before that we were not using any MFE’s, I am wondering about the following settings:

EDXAPP_CORS_ORIGIN_WHITELIST:
  - "{{ EDXAPP_CMS_BASE }}"
  - "{{ MFE_BASE }}"
EDXAPP_CSRF_TRUSTED_ORIGINS:
  - "{{ MFE_BASE }}"
EDXAPP_ENABLE_CORS_HEADERS: true
EDXAPP_ENABLE_CROSS_DOMAIN_CSRF_COOKIE: true
EDXAPP_CROSS_DOMAIN_CSRF_COOKIE_DOMAIN:
EDXAPP_CROSS_DOMAIN_CSRF_COOKIE_NAME:

Any help is much appreciated, thanks!

Tim_McCormack · December 15, 2021, 2:04pm

I haven’t looked closely but my gut sense is that one of these locations is relevant:

edx-platform/helpers.py at open-release/lilac.master · edx/edx-platform · GitHub
The is_safe_login_or_logout_redirect call (uses LOGIN_REDIRECT_WHITELIST) in edx-platform/helpers.py at open-release/lilac.master · edx/edx-platform · GitHub (but that should emit a warning log message…)

fghaas · December 15, 2021, 4:29pm

I work with @mrtmm on this issue, and I hope @jill doesn’t mind being tagged here as I’m hopeful she might have ideas on this issue, being the author of this PR:

github.com/edx/edx-platform

Adds settings.FEATURES['THIRD_PARTY_AUTH_HINT']

edx:master ← open-craft:jill/tpa-hint-setting

opened 08:04AM - 18 Jul 17 UTC

pomegranited

+126 -6

**Purpose** This feature allows the /login, /register, and course enrollment li…nks to be automatically redirected to the configured third-party authentication provider. This is useful for sites that exclusively use a third-party authentication system, and don't want anyone created accounts outside that system. Normal username + password logins and registrations are effectively disabled, though existing users can authenticate via the Django /admin URL. **Description** When provided, this setting appends `tpa_hint={value}` to the `next` URL provided for /login and /register page URLs. This allows the [existing `tpa_hint` feature](https://github.com/edx/edx-documentation/blob/master/en_us/install_operations/source/configuration/tpa/tpa_advanced_features.rst#hinted-sign-in) to be automatically added to login, registration, and enrolment URLs used throughout the platform. When combined with the existing TPA "Skip hinted login dialog" feature, the built-in Open edX login/register screens are bypassed entirely in favour of the SSO login/registration process. **JIRA tickets**: [OSPR-1836](https://openedx.atlassian.net/browse/OSPR-1836) **Sandbox URL**: Running 2ba3e49 * LMS: https://pr15587.sandbox.opencraft.hosting/ * Studio: https://studio-pr15587.sandbox.opencraft.hosting/ **Partner information**: 3rd party-hosted open edX instance **Deployment targets**: edx.org and edge.edx.org, though the feature is disabled by default. **Merge deadline**: None **Testing setup** These steps have already been run on the sandbox. 1. See **Settings** below for what to add to your `FEATURES` list in `/edx/app/edxapp/*.env.json`. Restart your edxapp service to effect these changes. 1. Set up an IDP SSO. You can set up an [oauth2 provider](http://edx.readthedocs.io/projects/edx-installing-configuring-and-running/en/latest/configuration/tpa/tpa_integrate_open/tpa_oauth.html), or follow the instructions below to set up SAML with TestShib. 1. Generate `saml.crt` (public key) and `saml.key` (private key) by running: ```bash openssl req -new -x509 -days 3652 -nodes -out saml.crt -keyout saml.key ``` 1. Add SAML Configuration using above keys. https://{lms}/admin/third_party_auth/samlconfiguration/add/ ``` Enabled: yes Entity ID: ensure is unique, e.g. https://saml.prXXXX.sandbox.opencraft.hosting Other config string: { "SECURITY_CONFIG": { "signMetadata": false, "metadataCacheDuration": "" } } Public key: Paste in content of `saml.crt` Private key: Paste in content of `saml.key` ``` 1. Download SAML auth XML, to a unique name: https://{lms}/auth/saml/metadata.xml *Note* On devstack, I've had to wait a few min for the metadata.xml to appear 1. Upload XML to http://www.testshib.org/register.html This will create a backend called `tpa-saml`. 1. Add a Provider Configuration (SAML IdPs): https://{lms}/admin/third_party_auth/samlproviderconfig/add/ ``` Enabled: Yes Name: `TestShib` Skip hinted login dialog: yes Backend name: `tpa-saml` Idp Slug: `testshib` # must match the second part of the THIRD_PARTY_AUTH_HINT Entity ID: `https://idp.testshib.org/idp/shibboleth` Metadata source: `https://www.testshib.org/metadata/testshib-providers.xml` ``` 1. After submitting, the Metadata Available flag should go to Green. You may have to refresh to see this. **Testing instructions**: Use an Incognito tab to make it easier to fully logout of the TestShib auth provider. 1. Visit the Register page. Note redirect to testshib login form. 1. Finish registering an account with testshib: 1. Fill in testshib login, and you'll be redirected back to the LMS to finish registration. 1. <strike>Click "Create an Account" to</strike> fill in the details not provided by TestShib, and accept the Terms & conditions. (issue fixed with 8277856). 1. Note that a new LMS account is registered. 1. Close your Incognito window to logout. 1. Visit the Login page. Note redirect to testshib login. 1. Fill in testshib login, and note that you're logged into your new account. 1. Close your Incognito window to logout. 1. Visit the Courses page, and Enrol in a course. Note that you're first redirected through the testshib login process, and then enrolled in the course. **Author notes and concerns**: 1. Feature needs to be added to [edx-documentation](https://github.com/edx/edx-documentation/blob/master/en_us/install_operations/source/configuration/tpa/tpa_advanced_features.rst#hinted-sign-in). CC @edx/doc 1. The `settings.ENTERPRISE_SERVICE_WORKER_USERNAME` user isn't being created by default yet (cf https://github.com/edx/configuration/pull/3974), so had to create it manually: ```bash sudo -u edxapp -Hs source ~/edxapp_env ~/edx-platform/manage.py lms shell --settings=devstack from django.contrib.auth import get_user_model get_user_model().objects.create(username='enterprise_worker') ``` **Reviewers** - [x] @snowcrshd - [x] @mattdrayer - CC @gsong **Settings** ```yaml EDXAPP_FEATURES: ENABLE_COMBINED_LOGIN_REGISTRATION: true ENABLE_THIRD_PARTY_AUTH: true THIRD_PARTY_AUTH_HINT: "saml-testshib" # if using SAML, or "oa2-google-oauth2", if G+ OAuth2 ```

If we’re able to see the next URLs in the memcached entries corresponding to a session, is it safe to assume that the SAML TPA pipeline should be able to populate the next_url variable from that cached session entry, once it processes the successful SAML response? In other words, if it does not, is it OK to assume that something broke between Koa and Lilac, and that this is more likely to be a bug than a configuration issue?

mrtmm · December 15, 2021, 7:02pm

mrtmm:

What is different between our koa and lilac platforms is that for lilac we enabled the Account MFE, could that be somehow related? Before that we were not using any MFE’s, I am wondering about the following settings:
EDXAPP_CORS_ORIGIN_WHITELIST:
  - "{{ EDXAPP_CMS_BASE }}"
  - "{{ MFE_BASE }}"
EDXAPP_CSRF_TRUSTED_ORIGINS:
  - "{{ MFE_BASE }}"
EDXAPP_ENABLE_CORS_HEADERS: true
EDXAPP_ENABLE_CROSS_DOMAIN_CSRF_COOKIE: true
EDXAPP_CROSS_DOMAIN_CSRF_COOKIE_DOMAIN:
EDXAPP_CROSS_DOMAIN_CSRF_COOKIE_NAME:

We can scratch that, it shouldn’t be related. I’ve spun up an instance with everything MFE disabled and still was facing the same issue…

@Tim_McCormack thanks for the suggestion! I’ve been looking at those bits of code quite a bit but yes, we should be seeing something in the logs if the issue was coming from there. Also, I don’t see any logic changes there between koa and lilac

Zia_Fazal · December 16, 2021, 10:50am

I think next_url is lost because of the session is lost when user lands back on LMS after authentication. The reason of session lost could be DCS_SESSION_COOKIE_SAMESITE setting. can you make sure

DCS_SESSION_COOKIE_SAMESITE = “None”
when
SESSION_COOKIE_SECURE = True

OR
DCS_SESSION_COOKIE_SAMESITE = “Lax”
when
SESSION_COOKIE_SECURE = False

fghaas · December 16, 2021, 11:05am

We set:

DCS_SESSION_COOKIE_SAMESITE: Lax
SESSION_COOKIE_SECURE: true

But on that note, what does this mean exactly?

DCS_SESSION_COOKIE_SAMESITE_FORCE_ALL: true

Could this somehow break the use of next=?

Zia_Fazal · December 16, 2021, 11:11am

@fghaas could you please try setting

DCS_SESSION_COOKIE_SAMESITE="None"

This PR and discussion has some context on this setting and why next_url is dropped.

fghaas · December 16, 2021, 7:06pm

Right, that worked for us, thank you!

(Also, thanks for the edit on the earlier suggestion; that makes it clearer. )

I’d like to point out that in testing this change, we ran into the issue that it is apparently un-testable in Devstack.

In Devstack, we had this in /edx/etc/lms.yml:

# grep DCS_SESSION_COOKIE_SAMESITE /edx/etc/lms.yml 
DCS_SESSION_COOKIE_SAMESITE: None
DCS_SESSION_COOKIE_SAMESITE_FORCE_ALL: true

But checking with manage.py lms shell showed:

# ./manage.py lms shell
Python 3.8.10 (default, Jun  2 2021, 10:49:15) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django.conf import settings
>>> print(settings.DCS_SESSION_COOKIE_SAMESITE)
Lax

(And yes, we checked if quoting 'None' made any difference; it did not. That setting apparently just can’t be overridden from lms.yml in Devstack.)

It looks as though that is due to this commit:

github.com/edx/edx-platform

Fix issue with `SameSite=None` cookies from being blocked when `django_cookies_samesite.middleware.CookiesSameSite` is enabled for `devstack_docker` environment.

edx:master ← CUCWD:ztraboo.master/samesite-cookie-session-default

opened 03:03AM - 24 Jul 20 UTC

ztraboo

+6 -0

This is a fix for `devstack_docker` default value set to `Lax` for `DCS_SESSION_…COOKIE_SAMESITE`. It was defaulting to `SameSite=None` which requires a secure site which `localhost` site does not by default. Setting this `SameSite` cookie attribute to something other than `None` will continue to allow login to the LMS for `devstack_docker` environment. Regards to #23671 and https://discuss.openedx.org/t/lti-xblock-and-samesite/759/16

The rationale behind that change isn’t quite clear to me. I just wanted to mention it here in case others run into the same issue, and fruitlessly attempt to make that configuration change in Devstack.

Thanks again for the help!

mrtmm · December 17, 2021, 6:52am

Thanks @Zia_Fazal!

Tim_McCormack · December 17, 2021, 1:44pm

Wow, great catch, Zia!

system · March 17, 2022, 1:45pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Please test open-release/koa.test02 Build-Test-Release	0	366	December 2, 2020
Upgrade from koa to lilac migration error Tutor Help tutor , koa , lilac	5	339	February 22, 2023
LTI issue with Lynda (LinkedIn Learning) Development koa	3	727	February 10, 2022
Error in Custom Theme When Upgrading from Koa -> Nutmeg Tutor Help koa , nutmeg	2	271	February 12, 2023
Error during Nutmeg -> Olive upgrade Tutor Help	1	275	April 17, 2024

SAML IdP "hinted links" not working in Lilac

Related Topics