Job/Batch engine

Job, van Gogh

The kernel of has a job engine that can execute many batches simultaneously in multiple threads. This can be used, for example, when many jobs need to be done, such as calculating the MD5 hash for 100,000 files or similar.

Each individual job is its own target. Targets are very lightweight, so it is possible to create and register very many of them. Targets can send messages network-wide, process responses, use timers, etc. You can start one or many successor jobs, control other targets, count operations, check results, and branch.

A job engine can include any number of threads, and a thread can register any number of job targets simultaneously. The maximum number of job targets in the threads can also be limited. As soon as this number is fallen below, further targets from the waiting queue are registered and started.

You can launch different targets in the same job engine, or you can create multiple job engines at the same time.

Internally, messages are used for any communication between engine, threads and jobs. Many of the functions of these classes are for convenience, as they only send messages.

See also - Innovative Distributed System