Timeline for What is a suitable format for writing large amounts of data within few milliseconds
Current License: CC BY-SA 4.0
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 19 at 10:33 | comment | added | gnasher729 | @TZubiri I remember reading a "programming note" by Apple which just stated "the file system is not a database". | |
| Jun 14 at 12:18 | comment | added | TZubiri | The requirement was that the data needs to be written, no other information was given. A filesystem is a key, value store, which is probably what would be implemented with a json or csv anyways. What would the header field be? "Id,float"? "float"? Just skip the lossy and problematic float to string conversion, and store it as is. Welcome to programming, this is literally a hello world level challenge. | |
| Jun 13 at 13:00 | comment | added | Greg Burghardt | @TZubiri - ok, but is that a standard format that solves the OP's problem? That's the key point, in my opinion. There are other standards, too. And writing data is only half the story. Data is also typically read, and then processed. The OP should consider those needs and requirements as well. Instead, my answer describes an approach the OP can use to discover a format that suites their needs. | |
| Jun 13 at 5:45 | comment | added | TZubiri | Standard format exists, and it's just the format used in memory. Array of floating points. You know, mantissa, exponent, all that. Hardware limitations don't quite matter for 480KB/s, any modern write device can handle that and has been able since the 2000s. | |
| Jun 13 at 0:22 | history | answered | Greg Burghardt | CC BY-SA 4.0 |