[SOLVED] PHP architecture - Individual cron job per user

Issue

I'm building a website in PHP and need an API to be checked on a regular basis for EACH USER individually.

In a nutshell it's a SaaS to steer a user account on another website with additional/automated options. A web based bot if you want.

So basically, I need to dynamically create a new cron job with its individual interval for each user, and these should be executed in parallel (it would take too long to put all queries into one cron job if theres a lot of users).

Also, it might become necessary that each request is done with a different IP. Reason is that it is possible that the API provider is annoyed by us and wants to block us. Since the API key is public they will most likely do it by simply blocking our IP. When we change that frequently, that should help a lot.

Is something like that possible? What would this require? Any option that doesn't get too expensive?

I thought of RabbitMQ for example, but that wouldn't quite tackle all issues and I'm wondering if there's some better/smarter solution.

Thanks!

Solution

Look at temporal.io open source project. It supports practically an unlimited number of such periodic jobs. The PHP SDK can be found here.

Answered By - Maxim Fateev