@mahyard, in order to enable custom templates for certificates, you’ll need to set settings.FEATURES["CUSTOM_CERTIFICATE_TEMPLATES_ENABLED"] = True in your lms.env.json.
After that, if you have a custom theme enabled, you can override the certificate templates in your custom theme.
The base files are found in edx-platform/lms/templates/certificates. You can copy these into your custom theme, and make changes as necessary to fit your needs.
Yes, the Preview Certificate function will direct to the regular certificate view in the LMS and should show these changes if the custom theme and templates are configured correctly.