Alternative ways to transfer records from one environment to another

Question

I'm working on an application which has a feature of syncing records between two environments. For example, a record A is created in environment A. After a user verifies it, the user can use the sync feature to transfer the record from environment A to B. The solution has been implemented using Django API and requests module. The environment file consists of the two variables, currEnv and deployEnv. Based on their values in each environment, the records would either be sent to another environment or stored in the database.

Sample code:

import requests;
class SyncRecord():
    def post(self,request):
        if env.currEnv!=env.deployEnv:
            requests.post('http://www.example.com',json.loads(request.POST))
        else:
            Model.save(request.POST)

This is currently working, but I was looking for other ways to do it. I have considered using queuing systems like Kafka and SNS, but couldn't apply to my service as there is neither a producer nor a consumer. The current solution is taking a lot of time since it needs to transfer from one environment to another and pass through different conditions.

What changes could be done to improve my API?

Is your current solution sufficient for the requirements of the system you are building, and if not, why not? We can't answer how to "improve" something unless we know the problem you're trying to solve. — Philip Kendall
– Philip Kendall, Commented Jan 28, 2024 at 17:14
@PhilipKendall the current solution meets the requirements. I'd like to know how to reduce the time it takes to transfer the record. Transaction failures have also been handled using status codes. — Lucifer Darknight
– Lucifer Darknight, Commented Jan 28, 2024 at 17:50

J_H · Accepted Answer · 2024-01-28 22:30:33Z

It's unclear why this takes a "long" time:

    requests.post('http://www.example.com', json.loads(request.POST))

If you're looking for async bulk solutions, they are easy to implement.

(1.) Write the request to data/YYYY-MM-DD/HH-MM-SS-GUID.json, and use rsync -av data/ someserver:/data/ to send recent updates to another server. Or focus on a daily directory, to speed things up.

(2.) INSERT each request to your record table, which has an indexed updated timestamp column. Then sending recent updates is a matter of SELECT * FROM record WHERE updated > recent, and doing an INSERT of such records into another server's table.

Stack Exchange Network

Alternative ways to transfer records from one environment to another

1 Answer 1

Hot Network Questions

Alternative ways to transfer records from one environment to another

1 Answer 1

Related

Hot Network Questions