Understanding the Swift Argument Parser and working with STDIN Jul 7 2020
Operating systems have provided command-line interfaces for decades now, and all but the simplest command-line tools require argument parsing. Due to years of use and refinement, there are many expectations on how command-line tools should handle arguments. Because of these popular conventions, creating an argument parser is not as simple as we might think.
Creating a bespoke argument parser might not be where we would like to spend most of our time. The good news is that you don't have to, Apple open-sourced the Swift Argument Parser (SAP). In this post, we'll learn how the Swift Argument Parser works, and how to use it for handling STDIN for composable command-line tools.
The Swift Argument Parser (SAP) relies heavily on property wrappers. To better understand how the Argument Parser works, let's first have a look at what property wrappers are.
**Note: You can find the full code on the GitHub Repository
NOTE Before we get into property wrappers, let's clear a possible confusion. I would like to make clear that the Swift Argument parser and TSCUtility's Argument Parser are different. In a previous post, I explained how to use the Argument Parser that the Swift Package Manager (SPM) uses. The SPM Argument Parser works differently, not better or worst, just different. There are many solutions to the same problem. In my opinion, the Swift Argument Parser feels more Swifty, so depending on your preferences, you might feel more comfortable using one or the other.
**Note: If you want to read about other argument parser options I've written about:
- Command-line argument parsing using Swift Package Manager's TSCUtility module
- Building a CLI tool using Swift and Vapor's Console module
Table of contents
Property wrappers
If you are familiar with Property Wrappers, you can skip to the next section.
Property wrappers were introduced in Swift 5.1. The concept is straightforward, a property wrapper "wraps" a property to encapsulate additional logic that affects that property. That might sound abstract, so let's see a couple of examples so we can better understand what property wrappers are all about.
To begin with, let's talk about properties. Properties belong to classes, structs, and enumerations. For example:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
//Class
class Person {
let name: String //name is a property of Person Objects
init(name: String) {
self.name = name
}
}
//Struct
struct User {
var username: String //username is a property of User
var password: String //password is a property of User
}
//enums
enum ReadError {
case fileNotFound //fileNotFound is a property of ReadError
case permissionDenied //permission Denied is a property of ReadError
var errorMsg: String { //errorMsg is a computed property of ReadError
switch self {
case .fileNotFound:
return "File not found"
case .permissionDenied:
return "You don't have the required permissions for this file"
}
}
}
As you probably know, Swift provides many ways to enhance properties (lazy properties, computed properties, property observers, etcetera). We are going to focus on property wrappers to understand better how the Swift Argument Parser defines the arguments it supports.
Property wrappers encapsulate additional logic that we would like to apply to a property. If our property is simple, we don't need property wrappers. As with any other design decision, you need to evaluate if using property wrappers makes sense for your specific case.
Ok, let's see an example. Imagine that we need a property to represent a logged-in user. The behaviour of our user property is as follows: the property should use the username "anonymous" if it's not logged-in, and if it is logged-in, it should use the logged username. Let's create the property wrapper using a class. Remember, classes, are reference types, while structs and enums are value types. We'll use a class because we might want to pass the user around.
In general, to define a property wrapper, we need to define a class, struct, or enum that contains a property named wrappedValue
. We use the @propertyWrapper
annotation to identify our class/struct/enum as a property wrapper. Let's first define a User
struct, which we'll use as our base:
1
2
3
4
struct User {
var username: String
var password: String
}
Simple enough. Now, let's create the property wrapper LoggedUser
. Our LoggedUser
will fictitiously authenticate our user. In our case a user is logged-in if it has a password, else it'll be anonymous.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
@propertyWrapper class LoggedUser {
var user: User
var wrappedValue: User {
get { user }
set {
//User login validation code that we are not goign to implement
//We are going to only check if it has a password
if newValue.password.isEmpty {
user = User(username: "anonymous", password: "")
} else {
user = newValue
}
}
}
init(){
user = User(username: "anonymous", password: "")
}
init(user: User){
self.user = user
}
}
Now we can use @LoggedUser
anywhere we need to work with authenticated users without having to worry about the implementation details. Imagine we would like to represent a user session:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
struct Session {
var id: String
@LoggedUser var user: User
init(_ id: String){
self.id = id
}
func logUser(_ username: String, password: String){
user = User(username: username, password: password)
}
func logout() {
user = User(username: "anonymous", password: "")
}
}
This example is contrived, I know. But it is just to show you how property wrappers work. We used a type defined by us (our struct User
), but property wrappers can be used with any other type.
Let's see one last example, this time using a String
property. Imagine we need a property that holds the path to a file, but we want to make sure that it supports filenames with spaces on it. We need our property to escape the spaces.
1
"My Special File" => "My\ Special\ File"
We could use property wrappers to fulfil the requirement. We are going to create a property wrapper name EscapedFilename
. Check the following implementation:
1
2
3
4
5
6
7
8
9
10
11
12
13
@propertyWrapper struct EscapedFilename {
private(set) var filePath: String
var wrappedValue: String {
get { filePath }
set {
filePath = (newValue as NSString).replacingOccurrences(of: " ", with: "\\ ")
}
}
init() {
filePath = ""
}
}
We can now use the property wrapper whenever we need an escaped filename. For example:
1
2
3
4
5
6
7
8
struct ReferenceFile {
@EscapedFilename var path: String
}
var file = ReferenceFile()
file.path = "My Special File"
print(file.path)
//My\ Special\ File
There is a lot more to property wrappers, but this should be enough for us to understand the logic behind the Swift Argument Parser. I would encourage you to read the Swift.org documentation on Property Wrappers, so you can learn more about them.
With the basics out of the way, we can move to discuss the Swift Argument Parser.
The Swift Argument Parser model
First, let's look at how the Swift Argument Parser models commands, subcommands and arguments. The SAP basic unit of work is ParsableArguments
. The ParsableArguments
protocol defines any supported argument. An object that conforms to the ParsableArguments
could include arguments represented by the following property wrappers:
@Argument
: represents positional arguments that will be supported by our command.@Option
: Key value pair arguments, e.g.--src=file.csv
.@Flag
: toggles or flags, e.g.ls -l
. Flags or toggles don’t require a value; their presence in the arguments give enough information to make them useful.
The ParsableCommand
protocol represents commands and sub-commands. ParsableCommand
implements ParsableArguments
, so any object that conforms to the ParsableCommand
could make use of the property wrappers that define arguments. Also, an object that conforms toParsableCommand
should implement the run
function. The run
function contains the logic that will run when we execute the command. Commands can include other commands allowing us to implement sub-commands.
With that structure in mind, let's create a command-line tool that makes use of the Swift Argument Parser.
Example command-line tool
We are going to build a command-line tool that uses Escape sequences to colour text. Nice and simple. The name of our command-line tool will be colorico
. For reference, here is our desired usage:
1
USAGE: colorico <text> [--good] [--no-good] [--output-file <output-file>]
text
- is the text to apply the colours to.--good/--no-good
: this flags dictate which color to use(good
= green,no-good
= red).--output-file
: If we pass this option, we save the result to a file in the current directory.
Ok, we are ready to start. I'll use the Swift Package Manager to create our tool. You can use Xcode if you want to. First, let's create our directory and initialises our package:
1
2
3
$ mkdir colorico
$ cd colorico
$ swift package init --type executable
That should generate everything we need. Let's add the Swift Argument Parser as a dependency to our Package.swift
. The Package.swift
should look like the following:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
// swift-tools-version:5.2
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription
let package = Package(
name: "colorico",
dependencies: [
.package(url: "https://github.com/apple/swift-argument-parser", from: "0.2.0"),
],
targets: [
.target(
name: "colorico",
dependencies: [
.product(name: "ArgumentParser", package: "swift-argument-parser"),
]),
.testTarget(
name: "coloricoTests",
dependencies: ["colorico"]),
]
)
Ok, let's build it and run it just to make sure that the dependencies are ok.
1
2
3
4
5
6
7
8
9
# our package is built as part of `run`, but you can run it manually if you want
# `swift build`
$ swift run
Fetching https://github.com/apple/swift-argument-parser
Cloning https://github.com/apple/swift-argument-parser
Resolving https://github.com/apple/swift-argument-parser at 0.2.0
[35/35] Linking colorico
Hello, world!
$
Perfect, dependencies ready, we can begin working on our CLI tool.
Let's edit the colorico/Sources/colorico/main.swift
, import the ArgumentParser
module, and create our Colorico
command.
1
2
3
4
5
6
7
8
9
10
11
12
import ArgumentParser
struct Colorico: ParsableCommand {
@Argument(help: "text to colour.")
var text: String
mutating func run() throws {
print("\u{1B}[33m\(text)\u{1B}[0m")
}
}
Colorico.main()
We can now run our command:
1
$ swift run hello
And you should see hello in yellow, great!
I'll just show how to add a @Flag
and @Option
and then move to more interesting cases for building command-line tools, not only command-line applications.
There is no point to repeat the documentation given in the Swift Argument Parser repository; their documentation is very clear. You can also check the Argument Parser Announcement blog post on Swift.org for more examples. For the sake of completeness, we'll add an @Flag
and @Option
.
We are going to use a flag called --good
which will display our text in green. We can also pass the option --output-file
if we would like to store the result in a file. Let's also add the inversion
option to --good
so if we pass --no-good
we print the text in red.
The following is how our main.swift
will look:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
import ArgumentParser
import Foundation //For FileManager
struct Colorico: ParsableCommand {
static var configuration = CommandConfiguration(
abstract: "Colorico adds colour to text using Console Escape Sequences",
version: "1.0.0"
)
enum Colour: Int {
case red = 31
case green = 32
}
@Argument(help: "text to colour.")
var text: String
@Flag(inversion: .prefixedNo)
var good = true
@Option(name: [.customShort("o"), .long], help: "name of output file(the command only writes to current directory)")
var outputFile: String?
mutating func run() throws {
var colour = Colour.green.rawValue
if !good {
colour = Colour.red.rawValue
}
let colouredText = "\u{1B}[\(colour)m\(text)\u{1B}[0m"
if let outputFile = outputFile {
let path = FileManager.default.currentDirectoryPath
//Lets prevent any directory traversal
let filename = URL(fileURLWithPath: outputFile).lastPathComponent
let fullFilename = URL(fileURLWithPath: path).appendingPathComponent(filename)
try colouredText.write(to: fullFilename, atomically: true, encoding: String.Encoding.utf8)
} else {
print(colouredText)
}
}
}
Colorico.main()
We can now run our command-line tool and test it:
1
2
3
4
$ swift run colorico --good hello
# you should see "hello" in green
$ swift run colorico --no-good bye
# you should see "bye" in red
Let's test our output file.
1
2
3
$ swift run colorico "This is my message to you" -o output.txt
#we can cat the file and see the output in green colour
$ cat output.txt
NOTE: We added some mitigation for directory transversal. Imagine we run our colorico
tool from the ~/Desktop/
, and a cheeky user decides to pass the output file ../.bash_profile
. If we hadn't removed the path, we would have overwritten the user's ~/.bash_profile
. Our code removes all the previous paths and just leaves .bash_profile
, but there are more considerations you should take (e.g. maybe force an extension or a working directory). I made sure to include this here because often security is just an afterthought, but it should be at the forefront of your concerns if you are creating command-line tools. The command-line offers a lot of power, so you, as a command-line developer, should make sure to write safe tools/applications.
Ok, we have a working command-line program. Let's talk about my opinion on the difference between command-line application and command-line tools.
Differences between command-line tools and command-line applications
First, I want to make clear that this is not a standard definition. I've never heard this differentiation before, but I like to think there are more people out there that agree with me on this. I differentiate between a command-line tool and a command-line application, in the following manner:
- command-line application: a self-contained program that only deals with a specific domain. Think of the command-line application
docker
. Its only purpose is to manage Docker containers and resources. - command-line tool: built to accomplish a specific task, but it can be composable. Think of the command-line tools
grep(1)
,find(1)
,cat(1)
,sort(1)
.
Most of the examples I see for the Argument Parser work perfectly for a command-line application. But how do we make it also work for a command-line tool? We need to make sure that our command-line tool is composable.
Let me illustrate the composition with an example, using sort(1)
.
Imagine we have the file characters.txt
, with the following content:
1
2
3
4
5
6
7
Matrim
Lanfear
Fortuona
Birgitte
Logain
Zarine
Aginor
We can tell sort(1)
to sort it:
1
2
3
4
5
6
7
8
9
$ sort characters.txt
Aginor
Birgitte
Fortuona
Lanfear
Logain
Matrim
Zarine
$
As we can see, sort(1)
can receive a filename as an argument, and it will sort the content and displays it on the screen (standard output - STDOUT). Let's take a look at another tool, cat(1)
. We can use cat(1)
to shows the content of a file:
1
2
3
4
5
6
7
8
9
$ cat characters.txt
Matrim
Lanfear
Fortuona
Birgitte
Logain
Zarine
Aginor
$
We can compose cat(1)
and sort(1)
, and make the output of one be the input for the other one. For this we can use a pipe:
1
2
3
4
5
6
7
8
$ cat characters.txt | sort
Aginor
Birgitte
Fortuona
Lanfear
Logain
Matrim
Zarine
What if we would like to create a list of users from those names? We would need to remove the capitalisation. So let's use perl(1)
to do the downcasting.
1
2
3
4
5
6
7
8
$ perl -pe '$_=lc' characters.txt
matrim
lanfear
fortuona
birgitte
logain
zarine
aginor
Alright, we have the character names in lower case, we can now sort them:
1
2
3
4
5
6
7
8
$ perl -pe '$_=lc' characters.txt |sort
aginor
birgitte
fortuona
lanfear
logain
matrim
zarine
Perfect. I think you get the idea. How is it that sort(1)
manages to read from the STDIN (Standard Input) in some cases and some others from a parameter? Well, let's implement this functionality to our colorico
tool so we can learn how to do it.
Working with STDIN
We, command-line users, have some conventions for command-line tools:
- If the tool is called without any parameters, the tool will try to read from the STDIN (this is not always the case, in some cases, people prefer to display the tool's USAGE, but we'll assume that this is our desired behaviour).
- If the tool's last argument is a single dash (
-
) read the argument from the STDIN.
And that's it, makes sense, right? You can try it with some of your command-line tools. If you didn't know these rules before, now that you know them, you'll see them in the tools you use.
Ok, let's try to implement this behaviour in colorico
.
Making colorico
a composable command-line tool
To have our command-line tool handle the cases where we know it should get the input from STDIN, we need to take control of the arguments before Argument Parser.
With that in mind, we need to replace the call to Colorico.main()
. While using .main()
works great for command-line applications, it is not adequate for our command-line tool(maybe we can contribute and add this functionality to the Argument Parser project). So what we are going to do is use only the argument parsing part of the Argument Parser, and build our own initial logic.
Let's start by defining a function to read from the STDIN.
1
2
3
4
5
6
7
8
9
10
11
12
13
func readSTDIN () -> String? {
var input: String?
while let line = readLine() {
if input == nil {
input = line
} else {
input! += "\n" + line
}
}
return input
}
The readSTDIN
function is simple. We read from the STDIN until there are no more lines to read and return the lines we read.
We also need a variable to store the text we read, so let's create that:
1
var text: String?
Is optional because we might not get any input from the command-line.
Now comes the interesting part. Remember, the cases we would like to read from the STDIN are:
- If the tool is called without any parameters it'll try to read from the STDIN.
- If the tool's last argument is a single dash (
-
) read the argument from the STDIN.
We have access to the arguments passed to our program through CommandLine.arguments
. If you've used Swift for Scripting you probably used this before. So our first validation looks like this:
1
2
3
4
if CommandLine.arguments.count == 1 || CommandLine.arguments.last == "-" {
if CommandLine.arguments.last == "-" { CommandLine.arguments.removeLast() }
text = readSTDIN()
}
If we didn't get any arguments or if the last argument is -
, we read the text from the STDIN. If you are wondering why we check if CommandLine.arguments.count == 1
and not zero? It is because the first argument is always the name of the program. An empty command has at least one argument, its name.
Ok, let's build the arguments so we can pass it to our Colorico
struct.
1
2
3
4
var arguments = Array(CommandLine.arguments.dropFirst())
if let text = text {
arguments.insert(text, at: 0)
}
We now have our arguments ready, either with the STDIN content or with the arguments passed to the command.
We are going to parse the command and call run
if everything is ok.
1
2
3
4
5
6
7
let command = Colorico.parseOrExit(arguments)
do {
try command.run()
} catch {
Colorico.exit(withError: error)
}
And that's it. Here is the full main.swift
:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
import ArgumentParser
import Foundation //For FileManager
struct Colorico: ParsableCommand {
static var configuration = CommandConfiguration(
abstract: "Colorico adds colour to text using Console Escape Sequences",
version: "1.0.0"
)
enum Colour: Int {
case red = 31
case green = 32
}
@Argument(help: "text to colour.")
var text: String
@Flag(inversion: .prefixedNo)
var good = true
@Option(name: [.customShort("o"), .long], help: "name of output file(the command only writes to current directory)")
var outputFile: String?
func run() throws {
var colour = Colour.green.rawValue
if !good {
colour = Colour.red.rawValue
}
let colouredText = "\u{1B}[\(colour)m\(text)\u{1B}[0m"
if let outputFile = outputFile {
let path = FileManager.default.currentDirectoryPath
//Lets prevent any directory traversal
let filename = URL(fileURLWithPath: outputFile).lastPathComponent
let fullFilename = URL(fileURLWithPath: path).appendingPathComponent(filename)
try colouredText.write(to: fullFilename, atomically: true, encoding: String.Encoding.utf8)
} else {
print(colouredText)
}
}
}
func readSTDIN () -> String? {
var input: String?
while let line = readLine() {
if input == nil {
input = line
} else {
input! += "\n" + line
}
}
return input
}
var text: String?
if CommandLine.arguments.count == 1 || CommandLine.arguments.last == "-" {
if CommandLine.arguments.last == "-" { CommandLine.arguments.removeLast() }
text = readSTDIN()
}
var arguments = Array(CommandLine.arguments.dropFirst())
if let text = text {
arguments.insert(text, at: 0)
}
let command = Colorico.parseOrExit(arguments)
do {
try command.run()
} catch {
Colorico.exit(withError: error)
}
We can now run it and see if it works properly:
1
2
3
4
$ swift run colorico hello
#You should see hello in green
$ swift run colorico hello --no-good
#You should see hello in red
Ok, at least it works as before. Let's now test the two cases where we would read the content from STDIN. Beginning with no arguments passed to the tool.
1
2
$ swift run colorico
# the program is waiting for your input
After adding a couple of lines you can press Ctrl+D on an empty line, and you should see the text in green. What if we want it red? We make use of the second case, passing -
as the last argument.
1
2
$ swift run colorico --no-good -
# the program is waiting for your input
Now, after you press [Ctrl+D], you should see the text you entered in red.
Ok, the last test. Let's compose it with other commands.
Create a file name characters.txt
with the following content:
1
2
3
4
5
6
7
Matrim
Lanfear
Fortuona
Birgitte
Logain
Zarine
Aginor
And, let's use the following command:
1
$ cat characters.txt | swift run colorico --no-good -
You should see the list of characters in red.
Congratulations! You built a composable command-line tool.
Final thoughts
Creating an Argument Parser is not easy. So, first of all, thanks to everyone that has contributed to the Swift Argument Parser project.
I explained a few usages of the argument parser, but it can handle a lot of different cases. For example:
- Enumerated options
- Custom types
- Definition of short and long-form of arguments
Just to name a few. I love working on the command-line and creating command-line tools/applications. Having an argument parser like this makes it very pleasant to create more. I've written before about using other Argument Parsers:
- Vapor - Building a CLI tool using Swift and Vapor's Console module
- Swift Package Manager - Command-line argument parsing using Swift Package Manager's TSCUtility module
From all of the options, my favourite so far has been the Swift Argument Parser. I hope you see why.
I think the group behind it did an excellent job. I'll try to contribute to the project, but currently, my time is limited. But if Apple would like to hire me to spend more time on it, I will pack and move to San Jose the next day. In all seriousness, I encourage you to have a look at the code to learn more. Also, you can contribute by using it in your projects and letting the team know of anything that is missing or could be improved.
Ok, that's it for this post. Let me know what you think, and if it was useful. I always appreciate getting messages from people that read my posts.
Bonus for the readers that want to know more than what is necessary. If you wonder how does the Argument Parser knows which properties it has that require parsing, look into this function, especially to the Mirror call. Have fun!
**Note: You can find the full code on the GitHub Repository
References
- A more detailed Control Escape Sequences resource from docs.microsoft.com
- Building a CLI tool using Swift and Vapor's Console module - By me
- Command-line argument parsing using Swift Package Manager's TSCUtility module - By me
- Swift Argument Parser - GitHub repo
- Argument Parser - Examples - GitHub
- Argument Parser - Documentation - GitHub
- Announcing ArgumentParser - Swift.org
- Property wrappers in Swift - By John Sundell
- Property Wrappers - Swift.org Documentation Read the whole section on Properties.
- Property Wrappers Proposal - SE-0258 , read the initial proposal gives insights on what problems they solve.
- A Look into Argument Parser - Federico Zanetello . Federico explains technical aspects of the Argument Parser, entertaining read.
- Swift Property Wrappers - NSHipster, nice examples.
- Using swift for scripting - Rderik. If you want to use swift for scripts.