Hi all, I’m hoping somebody can help me. I seem to be running into exactly this unsolved problem. When I try to export a course, it stays stuck on “Exporting…”. I also tried sending an email from a course, which stays stuck on:
So this makes me assume that Celery is somehow broken. I can’t find any errors in the logs, though. Nothing in the lms-worker and cms-worker logs, nor are there any errors in the lms and cms logs when I try to run these tasks…
I’ve been trying to debug celery from inside one of the containers, but there I do get errors. I’m not sure if I’m just doing it wrong, though…
$ tutor local exec cms-worker -- celery status
docker-compose -f /home/tutor/.local/share/tutor/env/local/docker-compose.yml -f /home/tutor/.local/share/tutor/env/local/docker-compose.prod.yml -f /home/tutor/.local/share/tutor/env/local/docker-compose.tmp.yml --project-name tutor_local exec cms-worker celery status
Traceback (most recent call last):
File "/openedx/venv/lib/python3.8/site-packages/amqp/transport.py", line 188, in _connect
entries = socket.getaddrinfo(
File "/opt/pyenv/versions/3.8.12/lib/python3.8/socket.py", line 918, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -9] Address family for hostname not supported
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/openedx/venv/lib/python3.8/site-packages/kombu/connection.py", line 446, in _reraise_as_library_errors
yield
File "/openedx/venv/lib/python3.8/site-packages/kombu/connection.py", line 433, in _ensure_connection
return retry_over_time(
File "/openedx/venv/lib/python3.8/site-packages/kombu/utils/functional.py", line 312, in retry_over_time
return fun(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/kombu/connection.py", line 877, in _connection_factory
self._connection = self._establish_connection()
File "/openedx/venv/lib/python3.8/site-packages/kombu/connection.py", line 812, in _establish_connection
conn = self.transport.establish_connection()
File "/openedx/venv/lib/python3.8/site-packages/kombu/transport/pyamqp.py", line 201, in establish_connection
conn.connect()
File "/openedx/venv/lib/python3.8/site-packages/amqp/connection.py", line 323, in connect
self.transport.connect()
File "/openedx/venv/lib/python3.8/site-packages/amqp/transport.py", line 129, in connect
self._connect(self.host, self.port, self.connect_timeout)
File "/openedx/venv/lib/python3.8/site-packages/amqp/transport.py", line 197, in _connect
raise (e
File "/openedx/venv/lib/python3.8/site-packages/amqp/transport.py", line 213, in _connect
self.sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/openedx/venv/bin/celery", line 8, in <module>
sys.exit(main())
File "/openedx/venv/lib/python3.8/site-packages/celery/__main__.py", line 15, in main
sys.exit(_main())
File "/openedx/venv/lib/python3.8/site-packages/celery/bin/celery.py", line 217, in main
return celery(auto_envvar_prefix="CELERY")
File "/openedx/venv/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/openedx/venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/openedx/venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/openedx/venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/celery/bin/base.py", line 134, in caller
return f(ctx, *args, **kwargs)
File "/openedx/venv/lib/python3.8/site-packages/celery/bin/control.py", line 80, in status
replies = ctx.obj.app.control.inspect(timeout=timeout,
File "/openedx/venv/lib/python3.8/site-packages/celery/app/control.py", line 294, in ping
return self._request('ping')
File "/openedx/venv/lib/python3.8/site-packages/celery/app/control.py", line 106, in _request
return self._prepare(self.app.control.broadcast(
File "/openedx/venv/lib/python3.8/site-packages/celery/app/control.py", line 741, in broadcast
return self.mailbox(conn)._broadcast(
File "/openedx/venv/lib/python3.8/site-packages/kombu/pidbox.py", line 328, in _broadcast
chan = channel or self.connection.default_channel
File "/openedx/venv/lib/python3.8/site-packages/kombu/connection.py", line 895, in default_channel
self._ensure_connection(**conn_opts)
File "/openedx/venv/lib/python3.8/site-packages/kombu/connection.py", line 433, in _ensure_connection
return retry_over_time(
File "/opt/pyenv/versions/3.8.12/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/openedx/venv/lib/python3.8/site-packages/kombu/connection.py", line 450, in _reraise_as_library_errors
raise ConnectionError(str(exc)) from exc
kombu.exceptions.OperationalError: [Errno 111] Connection refused
Error: Command failed with status 1: docker-compose -f /home/tutor/.local/share/tutor/env/local/docker-compose.yml -f /home/tutor/.local/share/tutor/env/local/docker-compose.prod.yml -f /home/tutor/.local/share/tutor/env/local/docker-compose.tmp.yml --project-name tutor_local exec cms-worker celery status
Any advice is welcome! Thanks!
I’m running Nutmeg, with
tutor --version
tutor, version 14.0.5