Timeline for Increase speed of Bash script which used grep into a while loop
Current License: CC BY-SA 3.0
4 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 9, 2018 at 8:24 | comment | added | Gordon Davisson |
This is the right answer. The slowest thing in the original script is creating subprocesses for each external and/or piped and/or captured ( $(cmd) ) command in the loop. I count 18 subprocesses per loop! You can decrease that count significantly, but you're much much better off switching to a language that can do complex operations without needing to start other programs to do the work. Nearly any other language could churn through the entire file in a single process.
|
|
| Feb 27, 2018 at 9:55 | comment | added | Stephen Kitt | Text Processing in Python is an old book covering the topic, but you’d be better off learning text processing in Python 3 at this stage. Personally I really like Modern Perl. | |
| Feb 27, 2018 at 9:37 | comment | added | Shred | I'm a beginner, as can be seen by the quest, in bash as long as with python. Could you please give me some references on how to do a work like this with python? | |
| Feb 27, 2018 at 9:22 | history | answered | Stephen Kitt | CC BY-SA 3.0 |