Identify if output goes to the terminal or is being redirected, in Golang May 16 2021
Good command-line tools are a pleasure to work with. A feature I'm always grateful for is when the developer took the time to provide an output that is human readable and an output that is easy to pass to another tool for additional processing. A neat trick to differentiate between these two cases is by having the application identify if the output is being "piped" to another program or not. If the process's output is not being piped or redirected, we can assume that the user is looking at the results via a terminal. If that is not the case, our application could behave differently and show output that is easier to parse by another program. In this post, I'll show you how to determine if the stdout of a program is being redirected or piped using the Go programming language.
First, we'll have to talk about Character devices and Block devices.
Table of Contents
Bash Beyond Basics Increase your efficiency and understanding of the shell
If you are interested in this topic you might enjoy my course Bash Byond Basics. This course helps you level up your bash skills. This is not a course on shell-scripting, is a course on improving your efficiency by showing you the features of bash that are seldom discussed and often ignored.
Every day you spend many hours working in the shell, every little improvement in your worklflows will pay dividends many fold!
Learn moreCharacter devices and Block devices a brief introduction
The general task of the operating system is to manage the hardware resources on a computer. To access the hardware, we use device drivers. Device drivers are the interface between the operating system and the hardware. The drivers implement an API that the Operating System understands and expects. For example, some of the operations we can expect on the device are the following(There are more supported system calls, but you can explore them on your own time man 2 syscall):
openreadwrite
On a Unix based operating system, we have two major divisions on device drivers, block and character. For example, block devices would be hardware like hard drives, RAM Disks, CD-ROMs (remember those?). Examples of character devices could be the keyboard, mouse, printers, tape drives, ttys, etcetera. Block devices typically have physical addressable storage, so the Input/Output (I/O) operations can be done to specific addresses on the device. For Character devices, the I/O is usually performed in a stream, i.e. stream of bytes.
You might be asking: "how is all this related to our quest for identifying if our program's output is being read by the user or by another program?"
Well, we could check if the standard output(stdout) of our program is a character device. If stdout is a character device, we can assume the terminal output is being displayed to the User.
Before we can do that, let's first explore a little bit more what stdout represents.
STDIN, STDOUT, STDERR File descriptors
A long time ago, at the time of the teletypes, we had a file associated with the keyboard for input and a file associated with the screen for the output. The standard was that the stdin file descriptor referenced the file descriptor used for the keyboard, and the screen (or paper in some older cases) was the stdout. So we read from stdin and write to stdout. Nowadays, these file descriptors are no longer tied to physical devices. Streams emulate them. But we can still find them in the /dev directory, which contains all our system's devices.
If we list the files /dev/std* we'll see that they are link to some file descriptors:
In macOS:
1
2
3
4
$ ls -l /dev/std*
lr-xr-xr-x 1 root wheel 0B May 8 07:41 /dev/stderr@ -> fd/2
lr-xr-xr-x 1 root wheel 0B May 8 07:41 /dev/stdin@ -> fd/0
lr-xr-xr-x 1 root wheel 0B May 8 07:41 /dev/stdout@ -> fd/1
In Linux:
1
2
3
4
$ ls -l /dev/std*
lrwxrwxrwx 1 root root 15 May 8 06:57 /dev/stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root root 15 May 8 06:57 /dev/stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root root 15 May 8 06:57 /dev/stdout -> /proc/self/fd/1
Interesting. We can see it points to a file, which in turn is a link to another file! (I'll show the macOS results, but you can do the same on your Linux box if you are curious).
1
2
$ ls -l /dev/fd1
crw--w---- 1 derik tty 16, 22 May 16 12:44 /dev/fd/1
The interesting part is that it shows that it is a character device! (see the first letter c on the description. Let's see what we get if I list my disk:
1
2
$ $ ls -l /dev/disk0s1
brw-r----- 1 root operator 1, 1 May 8 07:41 /dev/disk0s1
As we were expecting, it shows it is a block device.
So far, so good. Aside from stdin and stdout, there is a third file descriptor stderr. stderr is where diagnosis messages are sent. In Unix based operating systems, the kernel maintains a per-process file descriptor table to keep track of all opened file descriptors by the current process. The table starts with index 0, so we have:
STDIN- index 0STDOUT- index 1STDERR- index 2
Let's play a little with Go, and print some messages.
println, fmt.Println, and fmt.Fprintln
There are many ways we could print a message on the screen, but let's start with the built-in println. Create a file called print_demo.go
1
2
3
4
5
package main
func main() {
println("Hello, I'm an output from go!")
}
We can run it:
1
2
$ go run print_demo.go
Hello, I'm an output from go!
We can see on the screen the message "Hello, I'm an output from go!" but is that the stdout? Let's find out.
Bash redirect
We can redirect the output from our program to a file using the > redirect operator in bash. We want to redirect the stdout, identified with the file descriptor 1. If we don't specify the file descriptor on the redirect, the default is 1, so we can avoid typing it, but I'll keep it for clarity.
1
2
$ go run print_demo.go 1> output.log
Hello, I'm an output from go!
Oh, we still see it on the screen, and if we check output.log, it'll be an empty file. Let's use file descriptor 2, which corresponds to stderr, and see if that works.
1
$ go run print_demo.go 2> output.log
This time we didn't see any output on the screen. Let's see the content of output.log.
1
2
$ cat output.log
Hello, I'm an output from go!
Interesting. Now we know that the built-in println prints to stderr. Remember, stderr is used for diagnosis messages, so it makes sense. Now, let's try with fmt.Println. Let's create a new file, fmt_print_demo.go, with the following content:
1
2
3
4
5
6
7
package main
import "fmt"
func main() {
fmt.Println("Hello, using fmt.Println!")
}
If we run it and try to redirect the stdout:
1
$ go run fmt_print_demo.go 1> output.log
This time we didn't see any message on the screen. Let's check output.log:
1
2
$ cat output.log
Hello, using fmt.Println!
Nice! So fmt.Println prints to stdout. And also, > overwrites the content of the file. If we want to append, we can use the operator >>. Now let's see how we can use fmt.Fprintln.
The function Fprintln from the fmt package has the following signature:
1
func Fprintln(w io.Writer, a ...interface{}) (n int, err error)
So it receives an io.Writer as the first parameter, and it can receive a variable number of elements that comply with the empty interface. I won't explain what the empty interface is, but, in a general sense, all structs in go implement the empty interface, so it accepts elements of any type. And it returns the bytes written and an error object. With it, we can specify where do we want to print our output.
Let's create a new file fprintln_demo.go, with the following content:
1
2
3
4
5
6
7
8
9
10
11
package main
import (
"fmt"
"os"
)
func main() {
fmt.Fprintln(os.Stdout, "Hello, STDOUT!")
fmt.Fprintln(os.Stderr, "Hello, STDERR!")
}
Let's run it and redirect the stdout to one file and the stderr to another.
1
2
3
4
5
$ go run fprintln_demo.go 1> output_stdout.log 2> output_stderr.log
$ cat output_stdout.log
Hello, STDOUT!
$ cat output_stderr.log
Hello, STDERR!
If we want to redirect both stdout and stderr to the same file, we could redirect stdout to a file and tell bash to redirect stderr to stdout, which points to a file adding both to the same file.
1
2
3
4
$ go run fprintln_demo.go 1> output.log 2>&1
$ cat output.log
Hello, STDOUT!
Hello, STDERR!
We can simplify that by just using the shorthand &>
1
2
3
4
$ go run fprintln_demo.go &> output.log
$ cat output.log
Hello, STDOUT!
Hello, STDERR!
OK, one last detour before we get back to our initial quest to identify if our file's output is the terminal or another process.
A pipe (|) or the terminal
If we use the pipe (|) operator, the stdout of a program becomes the stdin of another. Using pipes, we can mix and match different command-line tools to create complex workflows. The Unix philosophy for command-line tools is to do one thing but do it well. Good command-line tools follow that philosophy in their design, and they can easily be put together to obtain beautiful results.
For example, let's imagine we want to generate random names for containers, but we want them to be sorted. We could use the /usr/share/dict/words file that comes with most Unix based operating systems as a source for our names. Then we can use a combination of shuf(1) and sort(1).
First, we could extract the words from the file:
1
$ shuf -n 10 /usr/share/dict/words
You should see ten random words, but we require them to be sorted. To comply with that requirement, we can use the command sort:
1
$ shuf -n 10 /usr/share/dict/words | sort
After running that command, we get ten random words sorted alphabetically. It was a contrived example, but it exemplifies the use of a pipe. Now it is time to get back to our quest.
How to identify if we are displaying to the terminal or to another program
First, let's revisit the use case. We want to present a human-readable interface when viewed by a user, and we want to offer an easily parsed view to programs. For example, let's display the contents of an Array in a human-readable way and a comma-separated-value format to be parsed by other programs.
We start by creating a new file called show_list.go, with the following content:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
package main
import "fmt"
func main() {
ml := [10]string{
"asimen",
"dichotomize",
"motey",
"noneternity",
"overcarefully",
"painlessness",
"prerejoice",
"suffection",
"tatouay",
"tuberculate",
}
fmt.Println("This is the content of our array")
for _, v := range ml {
fmt.Printf("* %s\n", v)
}
}
If we run the file, we get an easy-to-read message that shows the content of the array. Now we want an output that shows the content of the array as a comma-separated list of values.
1
asimen, dichotomize, motey, noneternity, overcarefully, painlessness, prerejoice, suffection, tatouay, tuberculate
We need a way to identify what type of device our stdout is attached to. For that, we'll make use of the Stat function of the os package that has the following signature:
1
func (f *File) Stat() (FileInfo, error)
It is a function applied to a File, and it returns a FileInfo structure. From the os source code, we can see that the File info returns the following information:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
// The defined file mode bits are the most significant bits of the FileMode.
// The nine least-significant bits are the standard Unix rwxrwxrwx permissions.
// The values of these bits should be considered part of the public API and
// may be used in wire protocols or disk representations: they must not be
// changed, although new bits might be added.
const (
// The single letters are the abbreviations
// used by the String method's formatting.
ModeDir = fs.ModeDir // d: is a directory
ModeAppend = fs.ModeAppend // a: append-only
ModeExclusive = fs.ModeExclusive // l: exclusive use
ModeTemporary = fs.ModeTemporary // T: temporary file; Plan 9 only
ModeSymlink = fs.ModeSymlink // L: symbolic link
ModeDevice = fs.ModeDevice // D: device file
ModeNamedPipe = fs.ModeNamedPipe // p: named pipe (FIFO)
ModeSocket = fs.ModeSocket // S: Unix domain socket
ModeSetuid = fs.ModeSetuid // u: setuid
ModeSetgid = fs.ModeSetgid // g: setgid
What we are looking for --> ModeCharDevice = fs.ModeCharDevice // c: Unix character device, when ModeDevice is set
ModeSticky = fs.ModeSticky // t: sticky
ModeIrregular = fs.ModeIrregular // ?: non-regular file; nothing else is known about this file
// Mask for the type bits. For regular files, none will be set.
ModeType = fs.ModeType
ModePerm = fs.ModePerm // Unix permission bits, 0o777
)
We can see that we can obtain if the file is a character device. In reality, we can see that it returns a bitmask that we can use to verify if it is a character device. Let's run some small tests before. Create a new file bitmask.go with the following content:
1
2
3
4
5
6
7
8
9
10
11
12
package main
import (
"fmt"
"os"
)
func main() {
o, _ := os.Stdout.Stat()
fmt.Printf("%b\n", o.Mode())
fmt.Printf("%b\n", os.ModeCharDevice)
}
If we run it, we get:
1
2
3
$ go run bitmask.go
69206416
2097152
That doesn't help us much, but let's see how it looks in binary, modify the code to display the numbers in binary:
1
2
3
4
5
6
7
8
9
10
11
12
package main
import (
"fmt"
"os"
)
func main() {
o, _ := os.Stdout.Stat()
fmt.Printf("%b\n", o.Mode())
fmt.Printf("%b\n", os.ModeCharDevice)
}
Let's run it:
1
2
3
$ go run bitmask.go
100001000000000000110010000
1000000000000000000000
Better, but not quite. Let's add some padding, so both numbers align and are easy to spot the match.
1
2
3
4
5
6
7
8
9
10
11
12
package main
import (
"fmt"
"os"
)
func main() {
o, _ := os.Stdout.Stat()
fmt.Printf("%032b\n", o.Mode())
fmt.Printf("%032b\n", os.ModeCharDevice)
}
Rerun it:
1
2
3
4
$ go run bitmask.go
00000100001000000000000110010000
00000000001000000000000000000000
^---This is the bit we are interested in
If you remember your boolean algebra, we can extract it using an and operation &. If we use an And(&) operation between the stdout mode and the os.ModeCharDevice, we would get the following:
1
2
3
4
00000100001000000000000110010000
& 00000000001000000000000000000000
----------------------------------
00000000001000000000000000000000
Remember "and" is only true if both elements are true, and the bitmask os.ModeCharDevice has only one 1 element, so the rest would be zero and what we would get is the same bitmask. So we could use a validation like the following to identify if it is a Terminal or a Pipe:
1
2
3
4
5
6
o, _ := os.Stdout.Stat()
if (o.Mode() & os.ModeCharDevice) == os.ModeCharDevice { //Terminal
//Display info to the terminal
} else { //It is not the terminal
// Display info to a pipe
}
Ok, let's rewrite our show_list.go file with the following code:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
package main
import (
"fmt"
"os"
)
func main() {
ml := [10]string{
"asimen",
"dichotomize",
"motey",
"noneternity",
"overcarefully",
"painlessness",
"prerejoice",
"suffection",
"tatouay",
"tuberculate",
}
o, _ := os.Stdout.Stat()
if (o.Mode() & os.ModeCharDevice) == os.ModeCharDevice { //Terminal
fmt.Println("This is the content of our array")
for _, v := range ml {
fmt.Printf("* %s\n", v)
}
} else { //Pipe
s := ""
for _, v := range ml {
s += fmt.Sprintf("%s,", v)
}
if s[len(s)-1] == ',' {
s = s[:len(s)-1]
}
fmt.Printf(s)
}
}
And that's it. Let's test it in the terminal:
1
2
3
4
5
6
7
8
9
10
11
$ go run show_list.go
This is the content of our array
* asimen
* dichotomize
* motey
* noneternity
* overcarefully
* painlessness
* prerejoice
* suffection
* tatouay
Now let's redirect it to a file:
1
2
3
$ go run show_list.go > list.csv
$ cat list.csv
asimen,dichotomize,motey,noneternity,overcarefully,painlessness,prerejoice,suffection,tatouay,tuberculate
Success!
Final Thoughts
We covered a lot of ground to understand how to identify if we are displaying to a terminal or through a pipe or redirect. The code is simple:
1
2
3
4
5
6
o, _ := os.Stdout.Stat()
if (o.Mode() & os.ModeCharDevice) == os.ModeCharDevice { //Terminal
//Display info to the terminal
} else { //It is not the terminal
// Display info to a pipe
}
But understanding what it means is the exciting part. Now you know a bit of block and character devices, redirection and pipes, even some simple bit masking. I hope you had fun and find it helpful.
References and resources
- A more in-depth explanation of Character device Drivers.
- Learn more about bash redirection.
- If you want to read a little bit more about bit-masking.