How to update Caddyfile using tutor plugin

I am trying to increase the upload content size. Default size is 100MB.
I am trying to change it to 500MB.
I am able to change in the openedx-cms-production-settings using tutor plugin.
I have to update the same in Caddyfile located at the the location “$(tutor config printroot)/env/apps/caddy/Caddyfile” for the following entry studio.lms-dev.techsophy.com{$default_site_port}
I have directly edited the file and changed the value of max_size to 500MB and it is working.

How to I apply this change using tutor plugin? Need some hint

Hello @partha, welcome!

There are several patch hooks in the Caddyfile you can take advantage in a plugin. For instance, caddyfile-cms. I’m no Caddy expert, but I believe you could create a plugin like so, and it would get you what you need:

from tutor import hooks

hooks.Filters.ENV_PATCHES.add_item(
    (
        "caddyfile-cms",
        """
    request_body {
        max_size 500MB
    }
"""
    )
)

Let us know if it works!

I don’t think using tutor plugin would work because the request_body.max_size is appended at the bottom of the settings and it will be ignored by the default value.

One workaround way that I can think about is to modify the tutor Caddyfile template at
/home/ubuntu/.local/lib/python3.8/site-packages/tutor/templates/apps/caddy/Caddyfile
but you have to manually set this value everytime you upgrade tutor.

@partha can you please check whether implementing the “caddyfile-cms” and “caddyfile-lms” patches works for you? If not, would they work if the patch was located before the import proxy "lms:8000"/import proxy "cms:8000" statements?

The plugin did not work though the Caddy file was updated as bellow

 request_body {
        max_size 250MB
    }

    import proxy "cms:8000"


        request_body {
            max_size 500MB
        }

Even after modifying the Caddyfiles as suggested by @regis it did not work. Following are the changes I made.
I have manually edited the Caddyfile at location
“$(tutor config printroot)/env/apps/caddy/Caddyfile”
moved the request_body section before import as shown bellow

request_body {
            max_size 500MB
        }
    import proxy "cms:8000"

Thank you @partha. And you did restart caddy every time to take your changes into account, right? (tutor local restart caddy)

Yes I did restart the services after the changes.

Sorry it took me so long to get back to you @partha. I propose that we move the “caddyfile-cms” and “caddyfile-lms” patches just before the import proxy line. I believe this should resolve your issue, as you would be able to create a patch that overrides the max_size. Right?

EDIT: see this pull request feat: overridable lms/cms max upload size by regisb · Pull Request #792 · overhangio/tutor · GitHub As explained in the PR description, once it is merged you should be able to modify the upload max size by adding the following to the “caddyfile-cms” patch:

handle_path /import/* {
    request_body {
        max_size 500MB
    }
}
1 Like

YES! @regis that would solve the issue we currently have and that I mentioned way back in Caddy issues with ID Verification

Thank you!

@partha did you ever find a lasting solution for max_size as a plugin?

@regis, I’m by no means an expert in the plugins system but having had a look at the template and the list of available patches, it appears to me that the max_size value is hard coded into the template and so cannot be defined by a patch as that will simply add a value to CaddyFile instead of overwriting the defaults, this might break syntax or at the very least introduce duplication of directives for Caddy. (thinking back to running apache/nginx sites in the past these kind of things can cause issues but it’s been a long time since I did, I’m anticipating that Caddy might behave similarly)

Additionally, insofar as I can tell, the following patch (unless I misunderstood it’s function) has no effect in relation to Caddy’s max_size directive (whether or not that’s how it’s supposed to be I’m unsure)
openedx-cms-production-settings: |
MAX_ASSET_UPLOAD_FILE_SIZE_IN_MB

I think I might have some ideas how to work around this but not sure how/if it would interact with anything in unexpected ways. Note I have not tested any of this, it’s just theory I came up with in my head, please don’t implement unless I’m not a complete idiot (see footnote at bottom)

Possible option:
Add new patch directives for each max_size value in the template, example:

{{ LMS_HOST }}{$default_site_port}, {{ PREVIEW_LMS_HOST }}{$default_site_port} {
    @favicon_matcher {
        path_regexp ^/favicon.ico$
    }
    rewrite @favicon_matcher /theming/asset/images/favicon.ico

    # Limit profile image upload size
    handle_path /api/profile_images/*/*/upload {
        request_body {
            {{ patch("caddyfile-lms-profile_images_max_size")|indent(4) }}
        }
    }

    import proxy "lms:8000"

    {{ patch("caddyfile-lms")|indent(4) }}

    handle_path /* {
        request_body {
            {{ patch("caddyfile-lms-max_size")|indent(4) }}
        }
    }
}

{{ CMS_HOST }}{$default_site_port} {
    @favicon_matcher {
        path_regexp ^/favicon.ico$
    }
    rewrite @favicon_matcher /theming/asset/images/favicon.ico

    import proxy "cms:8000"

    {{ patch("caddyfile-cms")|indent(4) }}

    handle_path /* {
        request_body {
            {{ patch("caddyfile-cms-max_size")|indent(4) }}
        }
    }
}

Though if my understanding is right this will break the build for everyone who is NOT using a patch as it won’t have default values to assign.
Perhaps default values can be kept in the file with a condition (similar to the Global config at the top like {% if not ENABLE_WEB_PROXY %}) where we can have an option in config.yml to choose
eg: CADDYFILE_USEDEFAULT_MAX_SIZE: [true|false] (where false implies to use patch and true specifies use built-in defaults.
Maybe something like this?:

{{ CMS_HOST }}{$default_site_port} {
    @favicon_matcher {
        path_regexp ^/favicon.ico$
    }
    rewrite @favicon_matcher /theming/asset/images/favicon.ico

    import proxy "cms:8000"

    {{ patch("caddyfile-cms")|indent(4) }}

    handle_path /* {
        request_body {
            {% if not CADDYFILE_USEDEFAULT_MAX_SIZE %}
            max_size 250MB
    {% else %}
    {{ patch("caddyfile-cms-max_size")|indent(4) }}
    {% endif %}

            
        }
    }
}

Though please DO assume that I might be an idiot who doesn’t know what they’re talking about, my level of expertise with caddy/tutor is not great, this is all guesswork. The patch/variable names I supplied are for illustrative purposes and not intended to be a final suggestion

Hello,

I made this plugin

from tutor import hooks

hooks.Filters.ENV_PATCHES.add_item(
    (
        "caddyfile-cms",
        """
handle_path /import/* {
    request_body {
      max_size 500MB
    }
}
"""
    )
)

but when I apply it it always returns a 404 error in the caddy log

tutor_local-caddy-1  | {"level":"error","ts":1704386774.3679094,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_ip":"192.168.88.62","remote_port":"4705","proto":"HTTP/1.1","method":"GET","host":"studio.dev.lan","uri":"/import/course-v1:edX+DemoX+Demo_Course"},"user_id":"","duration":0.087692554,"size":2608,"status":404}

How should I correct this error in the plugin?

@regis or @partha , i just go through disscussion but still when i save caddy file and these are changes which reflects due to plugin

studio.local.edly.io{$default_site_port} {
    @favicon_matcher {
        path_regexp ^/favicon.ico$
    }
    rewrite @favicon_matcher /theming/asset/images/favicon.ico

    import proxy "cms:8000"

    handle_path /import/* {
	request_body {
	   max_size 500MB
	}
   }

    handle_path /* {
        request_body {
            max_size 250MB
        }
    }
}

here is my plugin which i made

from tutor import hooks

hooks.Filters.ENV_PATCHES.add_item(
    (
        "caddyfile-cms",
        """
    handle_path /import/* {
	request_body {
	   max_size 500MB
	}
   }
"""
    )
)

Could you please provide me a solution , as your disscussion seems to have high understanding of tutor and i joined recently , so i done what i understand through your disscusion