Skip to main content
Software Engineering

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

5
  • Yes – and you don't have to write it yourself. Some operating systems have built-in "job" and "workload" handling capability. Linux/Unix has a great many open-source systems. (You will find both back-end processing and front-end user interface stuff, already done. Clusters ... networks ... AWS ... it's all "a thing already done." Commented Apr 7, 2020 at 17:47
  • Might want to elaborate on when async would be used (for example waiting a few seconds) versus waiting a few minutes (queue approach). Commented Apr 7, 2020 at 17:57
  • hmmm those seconds add up fast and will crash your server under load Commented Apr 7, 2020 at 18:17
  • Okay, I think I'm going to do something similar. I started researching the topic. One thing that comes up my mind is how my "worker" application will be notified when a file or db record is created with the request, would RabbitMQ or something similar work?Or maybe I can set up my linux server to periodically fire a request to see if something new is inserted and if so, send the data for processing (maybe send POST request with it). Anyway, I still have a lot of research to do, thanks Commented Apr 7, 2020 at 21:12
  • @stubborn Just write the data somewhere, And from the writer apps point of view, it Is done. What happens to the data than, who picks them up And when or how often, Is no concern of that writer app. The Worker app Will check either periodically or whenever a Worker thread (from Its pool) Is free to process new set of data, or it can actively wait for new data like with rabbitmq. Commented Apr 9, 2020 at 4:32