Does The Number Of Celeryd Processes Depend On The --concurrency Setting?

We are running Celery behind Supervisor and start it with celeryd --events --loglevel=INFO --concurrency=2 This, however, creates a process graph that is up to three layers deep a

Solution 1:

You shouldn't have 7 processes if --concurrency is 2.

The actual processes started is:

• The main consumer process

  Delegates work to the worker pool

• The worker pool (this is the number that --concurrency decides)

So that is 3 processes with a concurrency of two.

In addition a very lightweight process used to clean up semaphores is started if force_execv is enabled (which it is by default i you're using some other transport than redis or rabbitmq).

NOTE that in some cases process listings also include threads. the worker may start several threads if using transports other than rabbitmq/redis, including one Mediator thread that is always started unless CELERY_DISABLE_RATE_LIMITS is enabled.