51
votes
Am I right that switching programming languages will have little impact on the scalability of a CRUD API?
At a global level, you're wrong - language does matter, or at the very least you will spend more $$$ on compute if you write it in a less computationally efficient language.
While I'm not at liberty ...
- 26.1k
42
votes
Accepted
Am I right that switching programming languages will have little impact on the scalability of a CRUD API?
There's a good video (warning, contains really a lot of swearing, but worth it) that will give you a good basis for arguing against this consultant... Solving Imaginary Scaling Issues (at Scale), by ...
- 830
20
votes
Am I right that switching programming languages will have little impact on the scalability of a CRUD API?
You should remove this consultant from touching any of your system. A consultant suggesting a complete rewrite of an application in a different language are going to cause a lot of damage to the ...
- 12.5k
15
votes
Am I right that switching programming languages will have little impact on the scalability of a CRUD API?
With a simple CRUD api, your bottlenecks are probably the network and the database. So the code you use for mediating and translating between the two layers is probably not that performance-critical.
...
- 23.5k
11
votes
Saving high-frequency events to a connection-limit constrained database
My guess is that you need to explore more carefully an approach that you have rejected
Enqueue the events on our server
My suggestion would be to start reading through the various articles published ...
- 34.7k
10
votes
Accepted
How can I scale a database that requires that writes are consistent?
The basic tradeoff of distributed system is that if you increase the number of write replicas that is needed to be updated for commitment, you improve reliability, but increase latency. On the flip ...
- 12.5k
9
votes
Accepted
Saving high-frequency events to a connection-limit constrained database
Input stream
It is not clear if your 1000 events/second represent peaks or if it's a continuous load:
if it's a peak, you could use a message queue as buffer to spread the load on the DB server over ...
- 82.2k
8
votes
Should data transformation be on the front or on the back end in this scenario?
The decision to do the transformation on the front-end or the backend should depend much more on the kind of transformation that you want to do rather than the number of items involved in the ...
8
votes
Should data transformation be on the front or on the back end in this scenario?
Where to transform data?
Should data transformation be on the front or on the back end?
The question is slightly nonsensical, or at least glossing over certain necessary consideration. Conversion ...
- 59.5k
7
votes
Developing your application as microservices or as a monolithic app - which is cheaper when deploying and scaling?
There's no "normally". Whichever of two options is cheaper to deploy, operate and scale depends on many factors. Often the deployment of a monolith is cheaper, but for example if you need changes in ...
- 18.6k
7
votes
Accepted
How can adding redunancy adversely affect performance
Scenario A: You have a system that solves problem X. Scenario B: You have a system that solves problem X and has to ensure that it is always synchronized with the redundant backup.
It is pretty clear ...
- 105k
7
votes
Designing a microservice architecture
To meet this requirement, I need several microservices.
Not really. You don't 'need' microservices at all. There may be good reason to decompose your architecture into smaller loosely coupled ...
- 30.9k
6
votes
Saving high-frequency events to a connection-limit constrained database
If I understand correctly the current flow is:
Receive and event (I assume through HTTP?)
Request a connection from the pool.
Insert the event to the DB
Release the connection to the pool.
If so I ...
- 30.9k
6
votes
Accepted
What time series database can support high cardinality?
Performing complex queries in real time on large time series data is not scalable. It puts an unreasonable load on your store, and will not perform well regardless of the database used. The only time ...
- 9,247
6
votes
How to adapt the architecture of a website and have suitable hosting along with it to meet increasing users?
Just a quick note on terminology:
Scaling up refers to improving the hardware.
Scaling out refers to adding more hardware in a load balanced environment.
In the web world, we tend to scale out the web ...
- 27k
6
votes
Accepted
Sharding rule updating on DB scaling
Starting from V11,
Postgres supports Declarative Partitioning where you can divide the table into partitions with built-in Hash partitioning strategy (modulus or round-robin).
It also allows you to ...
- 3,242
6
votes
Am I right that switching programming languages will have little impact on the scalability of a CRUD API?
By that they seem to mean that a backend API written in .NET will handle the growing performance requirements much better than a backend API written in a scripting language like JavaScript, Ruby or ...
- 110k
5
votes
Shopping Cart Concurrency Question
The problem is of course the database access on the 'add to basket' request.
Customers will add stuff to their basket all the time, they aren't always going to be buying that item. Plus at times of ...
- 84.4k
5
votes
Saving high-frequency events to a connection-limit constrained database
Assumptions
I'm going to assume that the load you describe is constant, as that is the more difficult scenario to solve for.
I'm also going to assume you have some way of running triggered, long-...
- 51
5
votes
Should we share a database between two 'Microservices'?
The reason why it's generally preferable to separate out writes and reads is because the problems that arise from them are different, and depending on whether your application is read or write heavy ...
- 509
5
votes
Accepted
How do atomic updates work at scale?
First things first, based on this question it appears you do not at all understand the concept of atomicity. I’d recommend doing a bit more work to understand that before moving on to distributed ...
- 110k
5
votes
Performance implications of a game project monorepo on *modern* Git
Git just does not scale to huge projects. Repos should not generally grow beyond 5GB including the entire history if you want a good user experience (or if you want to use commercial hosting services)....
- 136k
5
votes
How can I efficiently handle click events and sequentially related operations (like user–URL deactivation) in a scalable URL shortener?
But with thousands or millions of clicks, this becomes a bottleneck
Well, have you tested it to be able to assert that?
You can't just guess bottlenecks in software development—chances are, you'll ...
- 139k
4
votes
Accepted
Best way to design file storage for image processing system
You basically have two "directions" in which you can scale file system storage: vertically and horizontally.
Vertical scaling is basically just making a drive faster. The obvious move here is from ...
- 44.8k
4
votes
Should data transformation be on the front or on the back end in this scenario?
Actually it's the reverse. You want to push as much processing to the client as possible.
Assuming they are on a modern desktop PC, they may well have more resources than your server, and as a group, ...
- 84.4k
4
votes
How can I scale a database that requires that writes are consistent?
You can:
Distribute your inventory across your nodes — the airlines do this, they give out some seats on each plane to code share partners or travel agents so they can operate somewhat ...
- 34.8k
4
votes
Can I manage thousands of concurrent connections with a non-Node stack?
Always be prepared to do things over, sometimes you have to.
But also don’t let perfect be the enemy of good enough.
Since this is a first attempt, program it using the languages and libraries you ...
- 1,132
4
votes
How can adding redunancy adversely affect performance
Two simple examples:
Adding an index to a database table is a common way to introduce redundancy, with the intention of speeding up read operations on that table. However, if a table is more ...
- 220k
4
votes
Accepted
How can users be notified of changes during collaborative editing in a system like Google Docs?
To solve the question of how Google Docs works:
Here is a Blog post by the guy who designed the syncing system of gdocs: https://neil.fraser.name/writing/sync/
The link point to a detailed writeup of ...
- 110
4
votes
Scaling Heavy Writes When Fetching Data From External API
tl;dr: Break this "too big" problem into smaller chunks.
and saves that data in a postgres DB.
Good!
You are building atop an excellent foundation.
our containers have crashed quite a lot ...
- 7,891
Only top scored, non community-wiki answers of a minimum length are eligible