In the past couple of months I've worked with both Apache Spark and Druid for some projects. As I've gone through the process of learning how to use these tools, I've spent some time reading through their documentation and peaking into their inner workings (the under-the-hood code rather than the API level code).
In both projects, they mention both on-heap memory and -- to use a project generic term -- "worker-memory" (the amount of memory a particular worker has when performing a job/task). I'm interested in learning more about the latter.
In these projects, did the developers create classes to model contiguous memory itself (perhaps using Arrays)? If not, I can't quite figure out how they'd go about allowing the "worker-memory" setting to be defined in the first place.
If anyone has any insight into the particulars revolving around this, I'd be interested in learning about it.
