Understanding the Swift Argument Parser and working with STDIN

Operating systems have provided command-line interfaces for decades now, and all but the simplest command-line tools require argument parsing. Due to years of use and refinement, there are many expectations on how command-line tools should handle arguments. Because of these popular conventions, creating an argument parser is not as simple as we might think.

Creating a bespoke argument parser might not be where we would like to spend most of our time. The good news is that you don’t have to, Apple open-sourced the Swift Argument Parser (SAP). In this post, we’ll learn how the Swift Argument Parser works, and how to use it for handling STDIN for composable command-line tools.

The Swift Argument Parser (SAP) relies heavily on property wrappers. To better understand how the Argument Parser works, let’s first have a look at what property wrappers are.

**Note: You can find the full code on the GitHub Repository

NOTE Before we get into property wrappers, let’s clear a possible confusion. I would like to make clear that the Swift Argument parser and TSCUtility’s Argument Parser are different. In a previous post, I explained how to use the Argument Parser that the Swift Package Manager (SPM) uses. The SPM Argument Parser works differently, not better or worst, just different. There are many solutions to the same problem. In my opinion, the Swift Argument Parser feels more Swifty, so depending on your preferences, you might feel more comfortable using one or the other.

**Note: If you want to read about other argument parser options I’ve written about:

Property wrappers

If you are familiar with Property Wrappers, you can skip to the next section.

Property wrappers were introduced in Swift 5.1. The concept is straightforward, a property wrapper “wraps” a property to encapsulate additional logic that affects that property. That might sound abstract, so let’s see a couple of examples so we can better understand what property wrappers are all about.

To begin with, let’s talk about properties. Properties belong to classes, structs, and enumerations. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

//Class 
class Person {
    let name: String //name is a property of Person Objects
    
    init(name: String) {
        self.name = name
    }
}

//Struct
struct User {
    var username: String //username is a property of User
    var password: String //password is a property of User
}

//enums
enum ReadError {
    case fileNotFound //fileNotFound is a property of ReadError
    case permissionDenied //permission Denied is a property of ReadError
    var errorMsg: String { //errorMsg is a computed property of ReadError
        switch self {
            case .fileNotFound:
                return "File not found"
            case .permissionDenied:
                return "You don't have the required permissions for this file"
        }
    }
}

As you probably know, Swift provides many ways to enhance properties (lazy properties, computed properties, property observers, etcetera). We are going to focus on property wrappers to understand better how the Swift Argument Parser defines the arguments it supports.

Property wrappers encapsulate additional logic that we would like to apply to a property. If our property is simple, we don’t need property wrappers. As with any other design decision, you need to evaluate if using property wrappers makes sense for your specific case.

Ok, let’s see an example. Imagine that we need a property to represent a logged-in user. The behaviour of our user property is as follows: the property should use the username “anonymous” if it’s not logged-in, and if it is logged-in, it should use the logged username. Let’s create the property wrapper using a class. Remember, classes, are reference types, while structs and enums are value types. We’ll use a class because we might want to pass the user around.

In general, to define a property wrapper, we need to define a class, struct, or enum that contains a property named wrappedValue. We use the @propertyWrapper annotation to identify our class/struct/enum as a property wrapper. Let’s first define a User struct, which we’ll use as our base:

1
2
3
4

struct User {
    var username: String
    var password: String
}

Simple enough. Now, let’s create the property wrapper LoggedUser. Our LoggedUser will fictitiously authenticate our user. In our case a user is logged-in if it has a password, else it’ll be anonymous.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

@propertyWrapper class LoggedUser {
    var user: User
    var wrappedValue: User {
        get { user }
        set {
            //User login validation code that we are not going to implement
            //We are going to only check if it has a password
            if newValue.password.isEmpty {
                user = User(username: "anonymous", password: "")
            } else {
                user = newValue
            }
        }
    }

    init(){
        user = User(username: "anonymous", password: "")
    }

    init(user: User){
        self.user = user
    }
}

Now we can use @LoggedUser anywhere we need to work with authenticated users without having to worry about the implementation details. Imagine we would like to represent a user session:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

struct Session {
    var id: String
    @LoggedUser var user: User
    init(_ id: String){
        self.id = id
    }

    func logUser(_ username: String, password: String){
        user = User(username: username, password: password)
    }

    func logout() {
        user = User(username: "anonymous", password: "")
    }
}

This example is contrived, I know. But it is just to show you how property wrappers work. We used a type defined by us (our struct User), but property wrappers can be used with any other type.

Let’s see one last example, this time using a String property. Imagine we need a property that holds the path to a file, but we want to make sure that it supports filenames with spaces on it. We need our property to escape the spaces.

1

"My Special File" => "My\ Special\ File"

We could use property wrappers to fulfil the requirement. We are going to create a property wrapper name EscapedFilename. Check the following implementation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

@propertyWrapper struct EscapedFilename {
    private(set) var filePath: String
    var wrappedValue: String {
        get { filePath }
        set {
            filePath = (newValue as NSString).replacingOccurrences(of: " ", with: "\\ ")
        }
    }

    init() {
        filePath  = ""
    }
}

We can now use the property wrapper whenever we need an escaped filename. For example:

1
2
3
4
5
6
7
8

struct ReferenceFile {
    @EscapedFilename var path: String
}

var file = ReferenceFile()
file.path = "My Special File"
print(file.path)
//My\ Special\ File

There is a lot more to property wrappers, but this should be enough for us to understand the logic behind the Swift Argument Parser. I would encourage you to read the Swift.org documentation on Property Wrappers, so you can learn more about them.

With the basics out of the way, we can move to discuss the Swift Argument Parser.

The Swift Argument Parser model

First, let’s look at how the Swift Argument Parser models commands, subcommands and arguments. The SAP basic unit of work is ParsableArguments. The ParsableArguments protocol defines any supported argument. An object that conforms to the ParsableArguments could include arguments represented by the following property wrappers:

The ParsableCommand protocol represents commands and sub-commands. ParsableCommand implements ParsableArguments, so any object that conforms to the ParsableCommand could make use of the property wrappers that define arguments. Also, an object that conforms toParsableCommand should implement the run function. The run function contains the logic that will run when we execute the command. Commands can include other commands allowing us to implement sub-commands.

With that structure in mind, let’s create a command-line tool that makes use of the Swift Argument Parser.

Example command-line tool

We are going to build a command-line tool that uses Escape sequences to colour text. Nice and simple. The name of our command-line tool will be colorico. For reference, here is our desired usage:

1

USAGE: colorico <text> [--good] [--no-good] [--output-file <output-file>]

Ok, we are ready to start. I’ll use the Swift Package Manager to create our tool. You can use Xcode if you want to. First, let’s create our directory and initialises our package:

1
2
3

$ mkdir colorico
$ cd colorico
$ swift package init --type executable

That should generate everything we need. Let’s add the Swift Argument Parser as a dependency to our Package.swift. The Package.swift should look like the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

// swift-tools-version:5.2
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "colorico",
    dependencies: [
        .package(url: "https://github.com/apple/swift-argument-parser", from: "0.2.0"),
    ],
    targets: [
        .target(
            name: "colorico",
            dependencies: [
                .product(name: "ArgumentParser", package: "swift-argument-parser"),
            ]),
        .testTarget(
            name: "coloricoTests",
            dependencies: ["colorico"]),
    ]
)

Ok, let’s build it and run it just to make sure that the dependencies are ok.

1
2
3
4
5
6
7
8
9

# our package is built as part of `run`, but you can run it manually if you want
# `swift build`
$ swift run
Fetching https://github.com/apple/swift-argument-parser
Cloning https://github.com/apple/swift-argument-parser
Resolving https://github.com/apple/swift-argument-parser at 0.2.0
[35/35] Linking colorico
Hello, world!
$

Perfect, dependencies ready, we can begin working on our CLI tool.

Let’s edit the colorico/Sources/colorico/main.swift, import the ArgumentParser module, and create our Colorico command.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

import ArgumentParser

struct Colorico: ParsableCommand {
    @Argument(help: "text to colour.")
    var text: String

    mutating func run() throws {
        print("\u{1B}[33m\(text)\u{1B}[0m")
    }
}

Colorico.main()

We can now run our command:

1

$ swift run hello

And you should see hello in yellow, great!

I’ll just show how to add a @Flag and @Option and then move to more interesting cases for building command-line tools, not only command-line applications.

There is no point to repeat the documentation given in the Swift Argument Parser repository; their documentation is very clear. You can also check the Argument Parser Announcement blog post on Swift.org for more examples. For the sake of completeness, we’ll add an @Flag and @Option.

We are going to use a flag called --good which will display our text in green. We can also pass the option --output-file if we would like to store the result in a file. Let’s also add the inversion option to --good so if we pass --no-good we print the text in red.

The following is how our main.swift will look:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

import ArgumentParser
import Foundation //For FileManager

struct Colorico: ParsableCommand {

    static var configuration = CommandConfiguration(
        abstract: "Colorico adds colour to text using Console Escape Sequences",
        version: "1.0.0"
    )

    enum Colour: Int {
        case red    = 31
        case green  = 32
    }

    @Argument(help: "text to colour.")
    var text: String

    @Flag(inversion: .prefixedNo)
    var good = true

    @Option(name: [.customShort("o"), .long], help: "name of output file(the command only writes to current directory)")
    var outputFile: String?


    mutating func run() throws {
        var colour = Colour.green.rawValue
        if !good {
            colour = Colour.red.rawValue
        }
        let colouredText = "\u{1B}[\(colour)m\(text)\u{1B}[0m"
        if let outputFile = outputFile {
            let path = FileManager.default.currentDirectoryPath

            //Lets prevent any directory traversal
            let filename = URL(fileURLWithPath: outputFile).lastPathComponent
            let fullFilename = URL(fileURLWithPath: path).appendingPathComponent(filename)
            try colouredText.write(to: fullFilename, atomically: true, encoding: String.Encoding.utf8)
        } else {
            print(colouredText)
        }

    }
}

Colorico.main()

We can now run our command-line tool and test it:

1
2
3
4

$ swift run colorico --good hello
# you should see "hello" in green
$ swift run colorico --no-good bye
# you should see "bye" in red

Let’s test our output file.

1
2
3

$ swift run colorico "This is my message to you" -o output.txt
#we can cat the file and see the output in green colour
$ cat output.txt

NOTE: We added some mitigation for directory transversal. Imagine we run our colorico tool from the ~/Desktop/, and a cheeky user decides to pass the output file ../.bash_profile. If we hadn’t removed the path, we would have overwritten the user’s ~/.bash_profile. Our code removes all the previous paths and just leaves .bash_profile, but there are more considerations you should take (e.g. maybe force an extension or a working directory). I made sure to include this here because often security is just an afterthought, but it should be at the forefront of your concerns if you are creating command-line tools. The command-line offers a lot of power, so you, as a command-line developer, should make sure to write safe tools/applications.

Ok, we have a working command-line program. Let’s talk about my opinion on the difference between command-line application and command-line tools.

Differences between command-line tools and command-line applications

First, I want to make clear that this is not a standard definition. I’ve never heard this differentiation before, but I like to think there are more people out there that agree with me on this. I differentiate between a command-line tool and a command-line application, in the following manner:

Most of the examples I see for the Argument Parser work perfectly for a command-line application. But how do we make it also work for a command-line tool? We need to make sure that our command-line tool is composable.

Let me illustrate the composition with an example, using sort(1).

Imagine we have the file characters.txt, with the following content:

1
2
3
4
5
6
7

Matrim
Lanfear
Fortuona
Birgitte
Logain
Zarine
Aginor

We can tell sort(1) to sort it:

1
2
3
4
5
6
7
8
9

$ sort characters.txt
Aginor
Birgitte
Fortuona
Lanfear
Logain
Matrim
Zarine
$

As we can see, sort(1) can receive a filename as an argument, and it will sort the content and displays it on the screen (standard output - STDOUT). Let’s take a look at another tool, cat(1). We can use cat(1) to shows the content of a file:

1
2
3
4
5
6
7
8
9

$ cat characters.txt
Matrim
Lanfear
Fortuona
Birgitte
Logain
Zarine
Aginor
$

We can compose cat(1) and sort(1), and make the output of one be the input for the other one. For this we can use a pipe:

1
2
3
4
5
6
7
8

$ cat characters.txt | sort
Aginor
Birgitte
Fortuona
Lanfear
Logain
Matrim
Zarine

What if we would like to create a list of users from those names? We would need to remove the capitalisation. So let’s use perl(1) to do the downcasting.

1
2
3
4
5
6
7
8

$ perl -pe '$_=lc' characters.txt 
matrim
lanfear
fortuona
birgitte
logain
zarine
aginor

Alright, we have the character names in lower case, we can now sort them:

1
2
3
4
5
6
7
8

$ perl -pe '$_=lc' characters.txt |sort
aginor
birgitte
fortuona
lanfear
logain
matrim
zarine

Perfect. I think you get the idea. How is it that sort(1) manages to read from the STDIN (Standard Input) in some cases and some others from a parameter? Well, let’s implement this functionality to our colorico tool so we can learn how to do it.

Working with STDIN

We, command-line users, have some conventions for command-line tools:

And that’s it, makes sense, right? You can try it with some of your command-line tools. If you didn’t know these rules before, now that you know them, you’ll see them in the tools you use.

Ok, let’s try to implement this behaviour in colorico.

Making colorico a composable command-line tool

To have our command-line tool handle the cases where we know it should get the input from STDIN, we need to take control of the arguments before Argument Parser.

With that in mind, we need to replace the call to Colorico.main(). While using .main() works great for command-line applications, it is not adequate for our command-line tool(maybe we can contribute and add this functionality to the Argument Parser project). So what we are going to do is use only the argument parsing part of the Argument Parser, and build our own initial logic.

Let’s start by defining a function to read from the STDIN.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

func readSTDIN () -> String? {
    var input: String?

    while let line = readLine() {
        if input == nil {
            input = line
        } else {
            input! += "\n" + line
        }
    }

    return input
}

The readSTDIN function is simple. We read from the STDIN until there are no more lines to read and return the lines we read.

We also need a variable to store the text we read, so let’s create that:

1

var text: String?

Is optional because we might not get any input from the command-line.

Now comes the interesting part. Remember, the cases we would like to read from the STDIN are:

We have access to the arguments passed to our program through CommandLine.arguments. If you’ve used Swift for Scripting you probably used this before. So our first validation looks like this:

1
2
3
4

if CommandLine.arguments.count == 1 || CommandLine.arguments.last == "-" {
    if CommandLine.arguments.last == "-" { CommandLine.arguments.removeLast() }
    text = readSTDIN()
}

If we didn’t get any arguments or if the last argument is -, we read the text from the STDIN. If you are wondering why we check if CommandLine.arguments.count == 1 and not zero? It is because the first argument is always the name of the program. An empty command has at least one argument, its name.

Ok, let’s build the arguments so we can pass it to our Colorico struct.

1
2
3
4

var arguments = Array(CommandLine.arguments.dropFirst())
if let text = text {
    arguments.insert(text, at: 0)
}

We now have our arguments ready, either with the STDIN content or with the arguments passed to the command.

We are going to parse the command and call run if everything is ok.

1
2
3
4
5
6
7

let command = Colorico.parseOrExit(arguments)
do {
    try command.run()
} catch {

    Colorico.exit(withError: error)
}

And that’s it. Here is the full main.swift:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78

import ArgumentParser
import Foundation //For FileManager

struct Colorico: ParsableCommand {

    static var configuration = CommandConfiguration(
        abstract: "Colorico adds colour to text using Console Escape Sequences",
        version: "1.0.0"
    )

    enum Colour: Int {
        case red    = 31
        case green  = 32
    }

    @Argument(help: "text to colour.")
    var text: String

    @Flag(inversion: .prefixedNo)
    var good = true

    @Option(name: [.customShort("o"), .long], help: "name of output file(the command only writes to current directory)")
    var outputFile: String?


    func run() throws {
        var colour = Colour.green.rawValue
        if !good {
            colour = Colour.red.rawValue
        }
        let colouredText = "\u{1B}[\(colour)m\(text)\u{1B}[0m"
        if let outputFile = outputFile {
            let path = FileManager.default.currentDirectoryPath

            //Lets prevent any directory traversal
            let filename = URL(fileURLWithPath: outputFile).lastPathComponent
            let fullFilename = URL(fileURLWithPath: path).appendingPathComponent(filename)
            try colouredText.write(to: fullFilename, atomically: true, encoding: String.Encoding.utf8)
        } else {
            print(colouredText)
        }

    }
}

func readSTDIN () -> String? {
    var input: String?

    while let line = readLine() {
        if input == nil {
            input = line
        } else {
            input! += "\n" + line
        }
    }

    return input
}

var text: String?

if CommandLine.arguments.count == 1 || CommandLine.arguments.last == "-" {
    if CommandLine.arguments.last == "-" { CommandLine.arguments.removeLast() }
    text = readSTDIN()
}

var arguments = Array(CommandLine.arguments.dropFirst())
if let text = text {
    arguments.insert(text, at: 0)
}

let command = Colorico.parseOrExit(arguments)
do {
    try command.run()
} catch {

    Colorico.exit(withError: error)
}

We can now run it and see if it works properly:

1
2
3
4

$ swift run colorico hello
#You should see hello in green
$ swift run colorico hello --no-good
#You should see hello in red

Ok, at least it works as before. Let’s now test the two cases where we would read the content from STDIN. Beginning with no arguments passed to the tool.

1
2

$ swift run colorico
# the program is waiting for your input

After adding a couple of lines you can press [Ctrl+D] (this is the End Of File - EOF) on an empty line, and you should see the text in green. What if we want it red? We make use of the second case, passing - as the last argument.

1
2

$ swift run colorico --no-good -
# the program is waiting for your input

Now, after you press [Ctrl+D], you should see the text you entered in red.

Ok, the last test. Let’s compose it with other commands.

Create a file name characters.txt with the following content:

1
2
3
4
5
6
7

Matrim
Lanfear
Fortuona
Birgitte
Logain
Zarine
Aginor

And, let’s use the following command:

1

$ cat characters.txt | swift run colorico --no-good -

You should see the list of characters in red.

Congratulations! You built a composable command-line tool.

Final thoughts

Creating an Argument Parser is not easy. So, first of all, thanks to everyone that has contributed to the Swift Argument Parser project.

I explained a few usages of the argument parser, but it can handle a lot of different cases. For example:

Just to name a few. I love working on the command-line and creating command-line tools/applications. Having an argument parser like this makes it very pleasant to create more. I’ve written before about using other Argument Parsers:

From all of the options, my favourite so far has been the Swift Argument Parser. I hope you see why.

I think the group behind it did an excellent job. I’ll try to contribute to the project, but currently, my time is limited. But if Apple would like to hire me to spend more time on it, I will pack and move to San Jose the next day. In all seriousness, I encourage you to have a look at the code to learn more. Also, you can contribute by using it in your projects and letting the team know of anything that is missing or could be improved.

Ok, that’s it for this post. Let me know what you think, and if it was useful. I always appreciate getting messages from people that read my posts.

Bonus for the readers that want to know more than what is necessary. If you wonder how does the Argument Parser knows which properties it has that require parsing, look into this function, especially to the Mirror call. Have fun!

**Note: You can find the full code on the GitHub Repository

References


** If you want to check what else I'm currently doing, be sure to follow me on twitter @rderik or subscribe to the newsletter. If you want to send me a direct message, you can send it to derik@rderik.com.
$