60
votes
Accepted
What is the advantage of log file rotation based on file size?
"I have never had any issues with large files" is not synonymous with "there is never a problem with large files". Your lack of experience with problems does not prove that there ...
- 59.5k
30
votes
What is the advantage of log file rotation based on file size?
Log file rotation is not only related to technical issues with the file itself. It's also related with operational considerations, like:
being able to incrementally backup the logs (especially in ...
- 82.2k
16
votes
What is the advantage of log file rotation based on file size?
The other answers already provided very useful reasons, but there is one more I'm surprised I haven't seen outside of a comment yet:
To be able to delete old log files to free disk space
A very busy ...
- 322
13
votes
The best way to handle exceptions?
For the purposes of this method that returns a list, just don't catch exceptions. Let it throw.
Unhandled exceptions in your application still need to be handled, but the error handling occurs much ...
- 46.2k
13
votes
The best way to handle exceptions?
Sometimes the best way to handle an exception is not at all.
Since I only want the error and stack trace to be printed in the console and the application to be terminated, what is the best approach ...
- 120k
10
votes
Accepted
Storing images in base64 vs binary, in cloud
With base64, you get a 33.3% overhead because of the 4:3 ratio. This means that for a 3 MB image, you will pay 4 MB of storage, and possibly, depending on the pricing model, 32 Mbps of network ...
- 139k
9
votes
How is a software able to read a network file faster than it appears to be possible?
A network of 100 Mbps (mega bits per seconds) conveys 12,5 MB per seconds, including payload and protocol overhead. A file of 165 MB needs at least 13,5 seconds (In fact, it would require slightly ...
- 82.2k
9
votes
Load and process (compressed) data from filesystem in the blink of an eye
profiling measurements
It sounds like you do not yet know where the CPU cycles go,
or, more importantly, the sources of I/O + network delay.
I didn't see any numeric figures,
let alone 98th percentile ...
- 7,891
8
votes
What is the advantage of log file rotation based on file size?
never had any issues with large files
and cannot think of reasons why we might set an arbitrary limit
Every system I have ever ssh'd into has had limited budget and limited attached storage.
If you ...
- 7,891
6
votes
First time having users - how to deal with backwards compatibility?
A good start would be to resist the urge to keep changing the file format.
If you really must, think about backward compatibility. Adding new features to an existing file format doesn't stop you ...
- 9,782
6
votes
Is there a standardised way of adding custom metadata to image and other filetypes?
For WAV, AIFF, PNG and other IFF-like file formats, there is a standard way: define additional chunk types for your additional metadata.
For the other file formats in question, you can use format-...
- 5,477
6
votes
Accepted
Constantly writing a JSON file
The last time I had a requirement like that, the process looked like
When starting a new file, write the characters [\n to it to open a JSON array.
When appending to an existing file, seek to the end....
6
votes
How does resuming downloads work?
For tracking the progress: the response to the first download request will have a Content-Length header telling the download manager how big the entire file is. That info will be persisted (and ...
- 4,638
6
votes
Accepted
Checking if a file exists before writing. Always avoid, or sensible with the right use case?
When opening a file (regardless if it is opened for reading or writing), one has to do proper exception / error handling for the specific "open" statement (as well as for any further I/O). ...
- 220k
6
votes
Allow-Rendering-Prevent-Download Architecture
Fundamentally, you are trying to solve an impossible problem - there is no difference between "rendering a file in the client's browser" and "download the file". They both involve ...
- 26.1k
6
votes
What is the advantage of log file rotation based on file size?
I worked on a bug last year where an action in production was suddenly failing due to a timeout. Fast, near instantaneous in dev and QA, failing in production. Took weeks of on again off again work ...
- 11.2k
5
votes
Difference between '\n' and '\r\n'
Here is an answer from the best source - Microsoft.
Why is the line terminator CR+LF?
This protocol dates back to the days of teletypewriters. CR stands for
"carriage return" - the CR ...
- 349
5
votes
Accepted
How to deal with file events with microservice that exposes a REST API
The best solution is to change whatever generates the files so that it sends them to your api rather than saving them to files.
This allows your api to be triggered on a new file being added and do ...
- 84.4k
5
votes
Folder structure to store image/file uploads
This is really a question of: What are your requirements?
It's quite likely that content your team creates comes from a source control system and is assembled as part of a build process, so you ...
- 4,629
5
votes
The best way to handle exceptions?
Let the exceptions bubble up to your caller, by just adding a throws clause to the ler() method.
If someone calls your ler() method, it's important for them to know whether that was successful, ...
- 6,276
5
votes
The best way to handle exceptions?
Since I only want the error and stack trace to be printed in the console and the application to be terminated, what is the best approach to deal with this?
If you don't want to handle a checked ...
- 30.9k
4
votes
How do I deal with file downloads?
Both approaches have advantages and disadvantages, you need to weigh them according to your requirements. Direct links can be faster or use less server resources, may even scale better if you use a ...
- 18.6k
4
votes
Applying function to file line by line or read entirely into structure first?
You can always measure, but you might be surprised at the results, especially for sequential access. People don't think about optimizations done at lower levels of abstraction. For example, your ...
- 149k
4
votes
First time having users - how to deal with backwards compatibility?
First things first. Version your file format.
Write a version number or other identifier as the first thing in the file, then work out which class you need to be able to read that format.
One of the ...
- 13.1k
4
votes
Accepted
Efficient way to separate text file to header, tail lines and the leftover in between
The head lines are essentially a non-problem, so I don't see a need to discuss it.
If you want your program to be able to handle streaming data, there's no way around buffering. But if you only want ...
- 18.6k
4
votes
Load and process (compressed) data from filesystem in the blink of an eye
I assume you are a junior programmer because there are a lot of red flags in your question that show you don't know what you are doing. You need to profile first.
Especially you need to find out
how ...
- 702
4
votes
Looking for clarification on the process of playing audio files
The topic is quite expansive, you started out good and veered off-course around here:
As it's a compressed format, it can't just be directly translated to
audio data, and hence needs to be ...
- 2,212
3
votes
Accepted
Set per-process open file limit in code or in system configuration?
I would not have the application change the system settings. What if you have two such applications running? they would fight!
Instead, I would have the application check the max allowed value and ...
- 84.4k
3
votes
Accepted
Design of file I/O -> processing -> file I/O system
Using multiple threads is most helpful when your program is CPU bound and you want to parallelize your program to use multiple CPU cores.
This is not the case for I/O bound problems, such as your ...
- 136k
Only top scored, non community-wiki answers of a minimum length are eligible