Tutor CONFIG_UNIQUE filter not overwriting (16.1.4)

Site Operators Tutor Help
1

Hello!

I have a number of secrets that are being programmatically fetched from AWS Secrets Manager using boto3 and then saved into our config.yml file using a tutor plugin:

@hooks.Filters.CONFIG_UNIQUE.add()
def fetch_secrets_from_aws(config):
    secrets = s.do_config()
    
    for secret in secrets:
        config.append(
            (secret, secrets[secret])
        )
        
    return config

secrets is a dictionary ex

{
  'MY_SECRET': 'VALUE'
}

which is then converted into a tuple of the requisite form in that for loop
('MY_SECRET', 'VALUE')

Something that I’ve noticed is that when I update a value either in AWS or directly into the s.do_config() helper (for testing purposes) that the values in config.yml are not updating/being overwritten to match

ex

@hooks.Filters.CONFIG_UNIQUE.add()
def fetch_secrets_from_aws(config):
    config.append(
        ('MY_SECRET', None)
    )
        
    return config

Will not delete/remove the existing key

@regis
Is this a bug, is there another filter I should be using instead to set values/update values that may already exist, am I manipulating the config list incorrectly, or am I just doing something that isn’t intended to be done? Some insight would be appreciated!

Thanks!

2

Hi,
It’s very cool to see some creative use of the Filter API. I think that fetching secrets from AWS makes a lot of sense.

What you are observing is “normal” – in the sense that it’s the expected behaviour of the CONFIG_UNIQUE filter. Any value that is written to the user’s config.yml file is not going to be very easily overridden. Actually, the only way to override a value that is already present in config.yml is to run tutor config save --set KEY=VALUE. The goal here is to prevent Tutor from transparently overriding some important value, such as a password.

Yet, I understand that overriding a password is precisely what you are trying to achieve. To figure out what’s the best solution here, we need to know if you want to fetch setting values from AWS every time you run tutor config save? If yes, then we will need to create a new filter. If not, then maybe you want to consider creating a new tutor myplugin fetch-settings command? (using the CLI_COMMANDS filter) Neither are very difficult to implement. The former will require some changes to Tutor core (again: nothing too complicated), while the latter is all up to you.

3

I had previously tinkered around with using click commands to fetch the passwords but fetching on config save ended up working better for our deployment pipeline (and had the side effect of making our devs remember to fetch passwords more frequently haha), so a new Filter I think would definitely be preferable on this.

Do you have any docs or examples that I could look at for something like that? Right now its not a massive issue that this gets updated/fixed/whatever since the passwords are relatively static, but we’re also using it for things like branch selection for our MFE forks and that’s something that changes more often

4

What you are trying to achieve is a fairly advanced new feature, so the only documentation is the Tutor source code. You will need to do the following:

  1. Create a CONFIG_USER filter in tutor.hooks.catalog, similar to CONFIG_UNIQUE and CONFIG_OVERRIDES, and document it.
  2. Rewrite the tutor.config.get_user function to make use of that new filter.
  3. Add a callback for that filter in a plugin that you create. That callback will be responsible for fetching the AWS secrets and overriding existing values in the user configuration.

Does that make sense?

5

Yeah, thanks for pointing where to look!

You made the action/filter system really similar to Wordpress so I’d imagine it won’t be too complicated to sort out adding a new one myself, especially with the CONFIG_UNIQUE context to look at

Thanks!

1 Like