Switching writing from file to stdout using "sink()" in R

Question

I implemented the following procedure that aims to write some files and print a message in the end of each file when the writing is done:

# Print one file per piaf
output_dir_piafs <- "OUTPUT_dataset_piafs"
unlink(output_dir_piafs, recursive = TRUE, force = TRUE)
dir.create(output_dir_piafs)
for (i in 1:length(lst_sorted)) {
    sink()      # Generates warnings...
    filename <- paste(output_dir_piafs, "/piaf_", i, ".txt", sep="")
    sink(file = filename, append = TRUE)
    sink(type = "message")
    cat("  ", colnames(file1), "\n")

    for (j in 1:length(lst_sorted[[i]])) {
        cat(j, " ")
        lapply( lst_sorted[[i]][[j]], 
                function(x) { 
                    cat(as.character(x), " ")
                }
        )
        cat("\n")
    }

    ## back to the console
    sink()
    cat(paste(filename, "done !\n"))
    #flush(stdout())        # Tested, no particular effect
}

My problem is that, if I do not add a sink() at the very beginning of the loop, the final writing on the standard output (cat(paste(filename, "done !\n"))) has no effect. On the other hand, adding this early sink() generates warnings that I would like to avoid:

There were 50 or more warnings (use warnings() to see the first 50)
> warnings()
Warning messages:
1: In sink() : no sink to remove
2: In sink() : no sink to remove
3: In sink() : no sink to remove

Does anyone has an idea on how sink() behaves, and/or how to get rid of those warnings?

Note: I also tried try(sink(), silent=TRUE), but the silent option prevents only from errors...

Just a silly thought. Have you tried your code from a fresh R session? Perhaps you have some connections open, try sink.number("message"). — Roman Luštrik
– Roman Luštrik, Commented Nov 7, 2013 at 13:23
Good thinking! sink.number("message") doesn't return anything, but from a fresh new session, the standard output gets written fine. It is only when I stop the process (Ctrl-C), then re-run it, that the standard output fails. So, I think you're right. My bad: interrupting the process might have some bad consequences on the stdout connection. Moreover, if I type sink() right before re-running the process, it still doesn't work... — Gauthier Boaglio
– Gauthier Boaglio, Commented Nov 7, 2013 at 13:52
I made some other tests. If I let the process terminate properly, then re-run it, everything works well. This is definitely the Ctrl-C interruption which generates the issue. Any suggestion about how to make the things right, after having interrupted that way? — Gauthier Boaglio
– Gauthier Boaglio, Commented Nov 7, 2013 at 14:15
Been there, done that, bought the t-shirt, didn't like it. I'm glad you solved your problem. — Roman Luštrik
– Roman Luštrik, Commented Nov 7, 2013 at 15:55

Gauthier Boaglio · Accepted Answer · 2013-11-08 17:38:15Z

4

You don't have to use sink, have a look at ?cat, which can directly write to a file.

The following code should work:

output_dir_piafs <- "OUTPUT_dataset_piafs"
unlink(output_dir_piafs, recursive = TRUE, force = TRUE)
dir.create(output_dir_piafs)
lst_sorted <- c(1,2,3)
file1 <- c(a=1, b=2, c=3)
for (i in 1:length(lst_sorted)) {
  filename <- paste(output_dir_piafs, "/piaf_", i, ".txt", sep="")
  cat("  ", colnames(file1), "\n", file=filename, append=T)
  for (j in 1:length(lst_sorted[[i]])) {
    cat(j, " ", file=filename, append=T)
    lapply( lst_sorted[[i]][[j]], 
           function(x) { 
             cat(as.character(x), " ", file=filename, append=T)
           }
           )
    cat("\n", file=filename, append=T)
  }
  cat(paste(filename, "done !\n"))
}

Please note that I set the variables lst_sorted and file1 in order to make the code reproducible.

Does the cat solution work for you?

edited Nov 8, 2013 at 17:38

Gauthier Boaglio

10.3k5 gold badges52 silver badges85 bronze badges

answered Nov 7, 2013 at 13:23

user1981275

13.5k11 gold badges78 silver badges104 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Gauthier Boaglio Over a year ago

Yep, it works, but it also takes twice the time sink takes to write the data into the files. Which is quite annoying... I suppose doing this results in opening/closing the file connection at each cat.

Karl Forner Over a year ago

concerning the speed, just create f <- file(filename, 'w') before your loop, then call cat() with file=f

Gauthier Boaglio Over a year ago

@KarlForner Ok, thanks! That could make the cat user1981275's solution viable. We'll see if with the Ctrl-C interruption, we still need to close the file in a tryCatch...

Gauthier Boaglio Over a year ago

@user1981275 Thanks for your help! Just a remark: the cat("\n"), should be written in the file as well: cat("\n", file=filename, append=T).

Community · Accepted Answer · 2017-05-23 11:52:24Z

I finally fixed the issue by adding a sink(type="output") when catching the interruption - Ctrl-C - (this makes the things right for later usage of the standard output - which would remain locked/diversion otherwise):

tryCatch({

    # Print one file per piaf
    output_dir_piafs <- "OUTPUT_dataset_piafs"
    unlink(output_dir_piafs, recursive = TRUE, force = TRUE)
    dir.create(output_dir_piafs)
    for (i in 1:length(lst_sorted)) {
        filename <- paste(output_dir_piafs, "/piaf_", i, ".txt", sep="")
        sink(file = filename, append = TRUE)
        sink(type = "message")
        cat("  ", colnames(file1), "\n")

        for (j in 1:length(lst_sorted[[i]])) {
            cat(j, " ")
            lapply( lst_sorted[[i]][[j]], 
                    function(x) { 
                        cat(as.character(x), " ")
                    }
            )
            cat("\n")
        }

        ## back to the console
        sink(type="output")
        cat(paste(filename, "done !\n"))
    }

}, interrupt = function(ex) {

    ##cat("An interrupt was detected.\n")
    sink(type="output")            # Restore the standard output !
    ##print(ex)

}) # tryCatch()

The other way to do (based on the help of user1981275 and Karl Forner), would have been:

tryCatch({

    # Print one file per piaf
    output_dir_piafs <- "OUTPUT_dataset_piafs_2"
    unlink(output_dir_piafs, recursive = TRUE, force = TRUE)
    dir.create(output_dir_piafs)
    for (i in 1:length(lst_sorted)) {
        filename <- paste(output_dir_piafs, "/piaf_", i, ".txt", sep="")
        f <- file(filename, 'w')
        cat("  ", colnames(file1), "\n", file=f)

        for (j in 1:length(lst_sorted[[i]])) {
            cat(j, " ", file=f)
            lapply( lst_sorted[[i]][[j]], 
                    function(x) { 
                        cat(as.character(x), " ", file=f)
                    }
            )
            cat("\n", file=f)
        }

        flush(f)
        close(f)
        ## back to the console
        cat(paste(filename, "done !\n"))
    }

}, interrupt = function(ex) {

    closeAllConnections()

}) # tryCatch()

This solution has the great advantage to not make a diversion of the standard output, meaning that, even without catching the interruption, the standard output remains active.

Note that Keyboard Interruption handling is still necessary to avoid warnings like:

> warnings()
Warning message:
closing unused connection 3 (OUTPUT_dataset_piafs_2/piaf_16.txt)

Accordingly to ?closeAllConnections:

‘closeAllConnections’ closes (and destroys) all user connections, restoring all ‘sink’ diversions as it does so.

Last Note: No real speed difference regarding file writing between those two methods.

Collectives™ on Stack Overflow

Switching writing from file to stdout using "sink()" in R

2 Answers 2

4 Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Related