Issue
How can I queue the multiple API request in FastAPI.
For example if there is an API which takes 10 seconds to complete the request and there are 10 API requests at same time then how can We handle that part.
If anyone can help on this then it would be a great help
Thanks
I tried to use the selenium using fastAPI and Gunicorn and its working fine if there one request at a time but if I will do multiple request the selemium generate error. So I'm trying to find a solution like queue processing in FastAPI.
Solution
Queueing is done by the server (gunicorn in your case), not by FastAPI.
Gunicorn has a backlog where the requests are being queued. You can control it using --backlog=N switch. The default backlog size is 2048, so the error is probably due to a timeout on the client. Try to increase it.
You can spawn multiple workers in Gunicorn, use --workers=N switch. If you worker is CPU-bound, you should set N to the number of available cores. It's a better solution.
And, if your worker is IO bound, use an asynchronous model to process more requests at once. This is the best solution.
Answered By - szrg Answer Checked By - Senaida (WPSolving Volunteer)