Identify if output goes to the terminal or is being redirected in Golang May 16 2021

Good command-line tools are a pleasure to work with. A feature I'm always grateful for is when the developer took the time to provide an output that is human readable and an output that is easy to pass to another tool for additional processing. A neat trick to differentiate between these two cases is by having the application identify if the output is being "piped" to another program or not. If the process's output is not being piped or redirected, we can assume that the user is looking at the results via a terminal. If that is not the case, our application could behave differently and show output that is easier to parse by another program. In this post, I'll show you how to determine if the stdout of a program is being redirected or piped using the Go programming language.

First, we'll have to talk about Character devices and Block devices.

Character devices and Block devices a brief introduction

The general task of the operating system is to manage the hardware resources on a computer. To access the hardware, we use device drivers. Device drivers are the interface between the operating system and the hardware. The drivers implement an API that the Operating System understands and expects. For example, some of the operations we can expect on the device are the following(There are more supported system calls, but you can explore them on your own time man 2 syscall):

On a Unix based operating system, we have two major divisions on device drivers, block and character. For example, block devices would be hardware like hard drives, RAM Disks, CD-ROMs (remember those?). Examples of character devices could be the keyboard, mouse, printers, tape drives, ttys, etcetera. Block devices typically have physical addressable storage, so the Input/Output (I/O) operations can be done to specific addresses on the device. For Character devices, the I/O is usually performed in a stream, i.e. stream of bytes.

You might be asking: "how is all this related to our quest for identifying if our program's output is being read by the user or by another program?"

Well, we could check if the standard output(stdout) of our program is a character device. If stdout is a character device, we can assume the terminal output is being displayed to the User.

Before we can do that, let's first explore a little bit more what stdout represents.

STDIN, STDOUT, STDERR File descriptors

A long time ago, at the time of the teletypes, we had a file associated with the keyboard for input and a file associated with the screen for the output. The standard was that the stdin file descriptor referenced the file descriptor used for the keyboard, and the screen (or paper in some older cases) was the stdout. So we read from stdin and write to stdout. Nowadays, these file descriptors are no longer tied to physical devices. Streams emulate them. But we can still find them in the /dev directory, which contains all our system's devices.

If we list the files /dev/std* we'll see that they are link to some file descriptors:

In macOS:

1
2
3
4
$ ls -l /dev/std*
lr-xr-xr-x  1 root  wheel     0B May  8 07:41 /dev/stderr@ -> fd/2
lr-xr-xr-x  1 root  wheel     0B May  8 07:41 /dev/stdin@ -> fd/0
lr-xr-xr-x  1 root  wheel     0B May  8 07:41 /dev/stdout@ -> fd/1

In Linux:

1
2
3
4
$ ls -l /dev/std*
lrwxrwxrwx 1 root root 15 May  8 06:57 /dev/stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root root 15 May  8 06:57 /dev/stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root root 15 May  8 06:57 /dev/stdout -> /proc/self/fd/1

Interesting. We can see it points to a file, which in turn is a link to another file! (I'll show the macOS results, but you can do the same on your Linux box if you are curious).

1
2
$ ls -l /dev/fd1
crw--w----  1 derik  tty   16,  22 May 16 12:44 /dev/fd/1

The interesting part is that it shows that it is a character device! (see the first letter c on the description. Let's see what we get if I list my disk:

1
2
$ $ ls -l /dev/disk0s1
brw-r-----  1 root  operator    1,   1 May  8 07:41 /dev/disk0s1

As we were expecting, it shows it is a block device.

So far, so good. Aside from stdin and stdout, there is a third file descriptor stderr. stderr is where diagnosis messages are sent. In Unix based operating systems, the kernel maintains a per-process file descriptor table to keep track of all opened file descriptors by the current process. The table starts with index 0, so we have:

Let's play a little with Go, and print some messages.

println, fmt.Println, and fmt.Fprintln

There are many ways we could print a message on the screen, but let's start with the built-in println. Create a file called print_demo.go

1
2
3
4
5
package main

func main() {
    println("Hello, I'm an output from go!")
}

We can run it:

1
2
$ go run print_demo.go
Hello, I'm an output from go!

We can see on the screen the message "Hello, I'm an output from go!" but is that the stdout? Let's find out.

Bash redirect

We can redirect the output from our program to a file using the > redirect operator in bash. We want to redirect the stdout, identified with the file descriptor 1. If we don't specify the file descriptor on the redirect, the default is 1, so we can avoid typing it, but I'll keep it for clarity.

1
2
$ go run print_demo.go 1> output.log
Hello, I'm an output from go!

Oh, we still see it on the screen, and if we check output.log, it'll be an empty file. Let's use file descriptor 2, which corresponds to stderr, and see if that works.

1
$ go run print_demo.go 2> output.log

This time we didn't see any output on the screen. Let's see the content of output.log.

1
2
$ cat output.log
Hello, I'm an output from go!

Interesting. Now we know that the built-in println prints to stderr. Remember, stderr is used for diagnosis messages, so it makes sense. Now, let's try with fmt.Println. Let's create a new file, fmt_print_demo.go, with the following content:

1
2
3
4
5
6
7
package main

import "fmt"

func main() {
    fmt.Println("Hello, using fmt.Println!")
}

If we run it and try to redirect the stdout:

1
$ go run fmt_print_demo.go 1> output.log

This time we didn't see any message on the screen. Let's check output.log:

1
2
$ cat output.log
Hello, using fmt.Println!

Nice! So fmt.Println prints to stdout. And also, > overwrites the content of the file. If we want to append, we can use the operator >>. Now let's see how we can use fmt.Fprintln.

The function Fprintln from the fmt package has the following signature:

1
func Fprintln(w io.Writer, a ...interface{}) (n int, err error)

So it receives an io.Writer as the first parameter, and it can receive a variable number of elements that comply with the empty interface. I won't explain what the empty interface is, but, in a general sense, all structs in go implement the empty interface, so it accepts elements of any type. And it returns the bytes written and an error object. With it, we can specify where do we want to print our output.

Let's create a new file fprintln_demo.go, with the following content:

1
2
3
4
5
6
7
8
9
10
11
package main

import (
    "fmt"
    "os"
)

func main() {
    fmt.Fprintln(os.Stdout, "Hello, STDOUT!")
    fmt.Fprintln(os.Stderr, "Hello, STDERR!")
}

Let's run it and redirect the stdout to one file and the stderr to another.

1
2
3
4
5
$ go run fprintln_demo.go 1> output_stdout.log 2> output_stderr.log
$ cat output_stdout.log
Hello, STDOUT!
$ cat output_stderr.log
Hello, STDERR!

If we want to redirect both stdout and stderr to the same file, we could redirect stdout to a file and tell bash to redirect stderr to stdout, which points to a file adding both to the same file.

1
2
3
4
$ go run fprintln_demo.go  1> output.log 2>&1
$ cat output.log
Hello, STDOUT!
Hello, STDERR!

We can simplify that by just using the shorthand &>

1
2
3
4
$ go run fprintln_demo.go  &> output.log
$ cat output.log
Hello, STDOUT!
Hello, STDERR!

OK, one last detour before we get back to our initial quest to identify if our file's output is the terminal or another process.

A pipe (|) or the terminal

If we use the pipe (|) operator, the stdout of a program becomes the stdin of another. Using pipes, we can mix and match different command-line tools to create complex workflows. The Unix philosophy for command-line tools is to do one thing but do it well. Good command-line tools follow that philosophy in their design, and they can easily be put together to obtain beautiful results.

For example, let's imagine we want to generate random names for containers, but we want them to be sorted. We could use the /usr/share/dict/words file that comes with most Unix based operating systems as a source for our names. Then we can use a combination of shuf(1) and sort(1).

First, we could extract the words from the file:

1
$ shuf -n 10 /usr/share/dict/words

You should see ten random words, but we require them to be sorted. To comply with that requirement, we can use the command sort:

1
$ shuf -n 10 /usr/share/dict/words | sort

After running that command, we get ten random words sorted alphabetically. It was a contrived example, but it exemplifies the use of a pipe. Now it is time to get back to our quest.

How to identify if we are displaying to the terminal or to another program

First, let's revisit the use case. We want to present a human-readable interface when viewed by a user, and we want to offer an easily parsed view to programs. For example, let's display the contents of an Array in a human-readable way and a comma-separated-value format to be parsed by other programs.

We start by creating a new file called show_list.go, with the following content:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
package main

import "fmt"

func main() {
    ml := [10]string{
        "asimen",
        "dichotomize",
        "motey",
        "noneternity",
        "overcarefully",
        "painlessness",
        "prerejoice",
        "suffection",
        "tatouay",
        "tuberculate",
    }
    fmt.Println("This is the content of our array")
    for _, v := range ml {
        fmt.Printf("* %s\n", v)
    }
}

If we run the file, we get an easy-to-read message that shows the content of the array. Now we want an output that shows the content of the array as a comma-separated list of values.

1
asimen, dichotomize, motey, noneternity, overcarefully, painlessness, prerejoice, suffection, tatouay, tuberculate

We need a way to identify what type of device our stdout is attached to. For that, we'll make use of the Stat function of the os package that has the following signature:

1
func (f *File) Stat() (FileInfo, error)

It is a function applied to a File, and it returns a FileInfo structure. From the os source code, we can see that the File info returns the following information:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
// The defined file mode bits are the most significant bits of the FileMode.
// The nine least-significant bits are the standard Unix rwxrwxrwx permissions.
// The values of these bits should be considered part of the public API and
// may be used in wire protocols or disk representations: they must not be
// changed, although new bits might be added.
const (
    // The single letters are the abbreviations
    // used by the String method's formatting.
    ModeDir        = fs.ModeDir        // d: is a directory
    ModeAppend     = fs.ModeAppend     // a: append-only
    ModeExclusive  = fs.ModeExclusive  // l: exclusive use
    ModeTemporary  = fs.ModeTemporary  // T: temporary file; Plan 9 only
    ModeSymlink    = fs.ModeSymlink    // L: symbolic link
    ModeDevice     = fs.ModeDevice     // D: device file
    ModeNamedPipe  = fs.ModeNamedPipe  // p: named pipe (FIFO)
    ModeSocket     = fs.ModeSocket     // S: Unix domain socket
    ModeSetuid     = fs.ModeSetuid     // u: setuid
    ModeSetgid     = fs.ModeSetgid     // g: setgid
What we are looking for --> ModeCharDevice = fs.ModeCharDevice // c: Unix character device, when ModeDevice is set
    ModeSticky     = fs.ModeSticky     // t: sticky
    ModeIrregular  = fs.ModeIrregular  // ?: non-regular file; nothing else is known about this file

    // Mask for the type bits. For regular files, none will be set.
    ModeType = fs.ModeType

    ModePerm = fs.ModePerm // Unix permission bits, 0o777
)

We can see that we can obtain if the file is a character device. In reality, we can see that it returns a bitmask that we can use to verify if it is a character device. Let's run some small tests before. Create a new file bitmask.go with the following content:

1
2
3
4
5
6
7
8
9
10
11
12
package main

import (
    "fmt"
    "os"
)

func main() {
    o, _ := os.Stdout.Stat()
    fmt.Printf("%b\n", o.Mode())
    fmt.Printf("%b\n", os.ModeCharDevice)
}

If we run it, we get:

1
2
3
$ go run bitmask.go
69206416
2097152

That doesn't help us much, but let's see how it looks in binary, modify the code to display the numbers in binary:

1
2
3
4
5
6
7
8
9
10
11
12
package main

import (
    "fmt"
    "os"
)

func main() {
    o, _ := os.Stdout.Stat()
    fmt.Printf("%b\n", o.Mode())
    fmt.Printf("%b\n", os.ModeCharDevice)
}

Let's run it:

1
2
3
$ go run bitmask.go
100001000000000000110010000
1000000000000000000000

Better, but not quite. Let's add some padding, so both numbers align and are easy to spot the match.

1
2
3
4
5
6
7
8
9
10
11
12
package main

import (
    "fmt"
    "os"
)

func main() {
    o, _ := os.Stdout.Stat()
    fmt.Printf("%032b\n", o.Mode())
    fmt.Printf("%032b\n", os.ModeCharDevice)
}

Rerun it:

1
2
3
4
$ go run bitmask.go
00000100001000000000000110010000
00000000001000000000000000000000
          ^---This is the bit we are interested in

If you remember your boolean algebra, we can extract it using an and operation &. If we use an And(&) operation between the stdout mode and the os.ModeCharDevice, we would get the following:

1
2
3
4
  00000100001000000000000110010000
& 00000000001000000000000000000000
----------------------------------
  00000000001000000000000000000000

Remember "and" is only true if both elements are true, and the bitmask os.ModeCharDevice has only one 1 element, so the rest would be zero and what we would get is the same bitmask. So we could use a validation like the following to identify if it is a Terminal or a Pipe:

1
2
3
4
5
6
    o, _ := os.Stdout.Stat()
    if (o.Mode() & os.ModeCharDevice) == os.ModeCharDevice { //Terminal
    //Display info to the terminal
  } else { //It is not the terminal
    // Display info to a pipe
  }

Ok, let's rewrite our show_list.go file with the following code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
package main

import (
    "fmt"
    "os"
)

func main() {
    ml := [10]string{
        "asimen",
        "dichotomize",
        "motey",
        "noneternity",
        "overcarefully",
        "painlessness",
        "prerejoice",
        "suffection",
        "tatouay",
        "tuberculate",
    }
    o, _ := os.Stdout.Stat()
    if (o.Mode() & os.ModeCharDevice) == os.ModeCharDevice { //Terminal
        fmt.Println("This is the content of our array")
        for _, v := range ml {
            fmt.Printf("* %s\n", v)
        }
    } else { //Pipe
        s := ""
        for _, v := range ml {
            s += fmt.Sprintf("%s,", v)
        }
        if s[len(s)-1] == ',' {
            s = s[:len(s)-1]
        }
        fmt.Printf(s)
    }
}

And that's it. Let's test it in the terminal:

1
2
3
4
5
6
7
8
9
10
11
$ go run show_list.go
This is the content of our array
* asimen
* dichotomize
* motey
* noneternity
* overcarefully
* painlessness
* prerejoice
* suffection
* tatouay

Now let's redirect it to a file:

1
2
3
$ go run show_list.go > list.csv
$ cat list.csv
asimen,dichotomize,motey,noneternity,overcarefully,painlessness,prerejoice,suffection,tatouay,tuberculate 

Success!

Final Thoughts

We covered a lot of ground to understand how to identify if we are displaying to a terminal or through a pipe or redirect. The code is simple:

1
2
3
4
5
6
    o, _ := os.Stdout.Stat()
    if (o.Mode() & os.ModeCharDevice) == os.ModeCharDevice { //Terminal
    //Display info to the terminal
  } else { //It is not the terminal
    // Display info to a pipe
  }

But understanding what it means is the exciting part. Now you know a bit of block and character devices, redirection and pipes, even some simple bit masking. I hope you had fun and find it helpful.

References and resources


** If you want to check what else I'm currently doing be sure to follow me on twitter @rderik or subscribe to the newsletter. If you want to send me a direct message you can send it to derik@rderik.com.