1

I have two files; file1 containing -

hello world
hello bangladesh  

and file2 containing -

 Dhaka in Bangladesh
 Dhaka is capital of Bangladesh  

I want to update file2 as -

hello world
hello bangladesh 
Dhaka in Bangladesh
Dhaka is capital of Bangladesh

This is done by -

cat file1 file2 >> file3
mv file3 file2  

But, I don't want to create a new file. I guess using sed it may be possible.

6
  • 3
    Almost all tools that edit in-place create a temporary file, just so you know. Commented Jan 3, 2018 at 1:29
  • @don_crissti, a valid point, but such tools do so a lot more safely than many shell script authors. Also relevant: How can I safely create and access temp files from shell scripts? Commented Jan 3, 2018 at 1:35
  • @don_crissti That means if I want to do this for keeping hard disk usage same at that time, this doesn't work. Commented Jan 3, 2018 at 1:40
  • @alhelal, how big is this file, anyway? Since you're not deleting file1, you can't keep your hard disk usage the same. Perhaps you should append file2 to file1 and then move it over the top of file2. That way you don't create a new file, you just overwrite the old. Commented Jan 3, 2018 at 1:57
  • 1
    @alhelal, aha. Well, in common UNIX/Linux filesystems, appending a couple of lines to a multi-gigabyte file is much, much faster than PRE-pending those lines to the same file. In the latter case, in most if not all filesystems, the entire file needs to be rewritten. Commented Jan 3, 2018 at 2:15

4 Answers 4

4

Sure it's possible.

(Unless you are also concerned about temporary files being potentially created "under the hood," as virtually all text editors work that way. I don't say it is flatly impossible to avoid any possibility under-the-hood temp file creation, but it's not covered in this answer.)

printf '%s\n' '0r file1' x | ex file2

This is a POSIX-compliant command using ex, the POSIX-specified non-visual predecessor to vi.

printf is only used here to feed a command to the editor. What printf outputs is:

0r file1
x

x is save and exit.

r is "read in the contents of the named file."

0 specifies the line number after which the read-in text should be placed.

7
  • This works. But, Is this increase my hard-disk usage? Commented Jan 3, 2018 at 1:37
  • @alhelal, that depends on the file sizes and your memory utilization. Obviously since you're not deleting file1 your hard disk usage will have a net increase (although even that's not actually true in all cases). But will you have a temporary increase in hard disk utilization? That's a different layer of abstraction, not answerable from the data given. Commented Jan 3, 2018 at 1:55
  • Does this create temporary file? Commented Jan 3, 2018 at 1:59
  • @alhelal, again, that depends on the file size and your memory utilization. Commented Jan 3, 2018 at 2:01
  • 2
    In modern systems ex is usually implemented as part of vim. vim creates a swap file. I'm not sure if ex does same but if you add -n to a ex/vim invocation that will prevent swap file creation. So you might want to do that depending on what your requirements are. Commented Jan 3, 2018 at 2:22
3

There aren't a lot of ways to modify files in place using standard tools. Even if they appear to do so they may be using temporary files (i.e. GNU sed -i).

ex on the other hand, will do the trick:

 ex -n -c '0r file2' -c wq file1

ex is a line editor and vim evolved from it so these commands may look familiar. 0r filename does the same thing as :0r filename in vim: insert the specified file after the given address (line number). The line number here is 0 which is a kind of virtual line that represents the line before line 1. So the file will be inserted before any existing text.

Then we have wq which saves and quits.

If you notice the comment below about this being "very brittle" take it with a grain of salt. If you want to use this in a script just be sure to do proper pre-condition validation (files exist, are readable/writable, etc.). Good scripters would be doing that anyways.

Update: I've added -n to address OP's concerns about avoiding any temporary file creation. Assuming that your ex is implemented as part of vim (as is the case on most modern systems) this flag will suppress the normal swap file creation that vim does.

6
  • Note that (a) this is not fully POSIX compliant as ex is not mandated to support +command syntax, and also (b) it is very brittle if used in scripting as the non-existence of file2 will hang ex up waiting on user input. See my answer for a version that avoids these pitfalls. Commented Jan 3, 2018 at 1:32
  • @alhelal Sure. Does it make sense now? Commented Jan 3, 2018 at 1:56
  • Good edit, but just FYI it is still not script-worthy. If file2 is not readable ex will still wait forever. If ex is unable to write the changes to file1 it will wait forever. Any error will make ex wait for user input. For a scripted use of ex, you should never do otherwise than feeding it commands in a way that guarantees it will receive an EOF (such as using a pipe or heredoc). Commented Jan 3, 2018 at 2:22
  • Do you add a "not POSIX compliant" comment to every answer that's not POSIX compliant. Because the convention I've noticed is the inverse: the person posting the answer indicates POSIX compliance. You can assume the rest aren't. I'm thinking of that "o"-word again. Commented Jan 3, 2018 at 2:27
  • Unless it's stated in the answer that it's not portable, or unless (even better) it's stated on which specific systems the command works, then yes, I usually do. If you say, "on Linux systems" or "using GNU Awk" or similar, then obviously you don't need to say it's not POSIX-compliant. I picked up the habit from this site of being very specific about where a command will or will not work—and it's saved me a lot of trouble, so I do such comments myself just as I've seen others do. Feel free to ping me in chat; let's not clutter up the comments. Commented Jan 3, 2018 at 2:48
0
all=$( cat file1 file2 )
echo "$all" > file2

only works with small-ish files.

0

Similar to Jasen's answer, this is a basis for a utility.

The code sponge in package moreutils is designed to allow an in-place capability to almost any program, albeit externally. Essentially it soaks up input from STDIN and then writes the collected output to a filename, but NOT with re-direction.

We liked the idea, so we created a work-alike that collects data in memory, as opposed to a file as the real sponge does.

Here is a snippet of shell script to illustrate this:

# Utility functions: print-as-echo, print-line-with-visual-space.
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }

pl " Input files data2, data3:"
head data[23]
pe
ls -gGli data[23]

pl " Results, re-write file from memory, preserving inode:"
cat data[23] | ./absorb-memory-public data3
head -v data3
pe
ls -gGli data3

Producing:

-----
 Input files data2, data3:
==> data2 <==
hello world
hello bangladesh 

==> data3 <==
Dhaka in Bangladesh
Dhaka is capital of Bangladesh  

1051395 -rw-r--r-- 1 30 Jan  7 08:58 data2
1052221 -rw-r--r-- 1 53 Jan  7 08:58 data3

-----
 Results, re-write file from memory, preserving inode:
==> data3 <==
hello world
hello bangladesh 
Dhaka in Bangladesh
Dhaka is capital of Bangladesh  

1052221 -rw-r--r-- 1 83 Jan  7 08:58 data3

The core perl code is quite short (especially without error checking, the addition of which may be desirable):

use warnings;
use strict;
use Carp;

# Avoid hang for argument matching "-version","--version", etc.
exit(0) if @ARGV && $ARGV[0] =~ /-version/;

my ( $file, $f, $memory );
$file = shift || croak("Need a filename.");

$/ = 0777;      # Slurp the entire STDIN
$memory = do { local $/; <> };

open( $f, ">", $file ) || die " Cannot open file \"$file\" for write.\n";

print $f "$memory";

Run on a system like:

OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.9 (jessie) 
bash GNU bash 4.3.30
perl 5.20.2

Best wishes ... cheers, drl

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.