Understanding the RunLoop model by creating a basic shell Oct 1 2019 Latest Update: Jul 6 2020
When we find ourselves listening for events, reacting to those events, and then going back to listening for more events, we have ourselves an Event-Loop. Having an event-loop is a common scenario. So common that Apple decided to provide us with a model to handle event-loops consistently. In this post, we are going to explore how RunLoops
work and use them to build basic shell
.
Let's get started by creating a program that makes use of an event-loop.
** You can check the full code of the shell
in the GitHub repository.
Table of Contents
Event-Loops
One of the most common cases for an event-loop is a REPL (Read Eval Print Loop). And probably one of the most used REPLs out there are shells, your shell (bash
, zsh
, fish
whichever you prefer) is the classic example of an event-loop. The shell waits for your input, once it reads your input, it executes the commands you send, and finally, it goes back to listening for more input.
Let's create a basic shell that has only one useful command (two if we count exit
), list the files in the current directory using ls
.
We can create our application using the Swift Package Manager.
1
2
3
$ mkdir rdshell
$ cd rdshell
$ swift package init --type executable
Add this content to main.swift
:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
import Foundation
func list() -> [String] {
let fm = FileManager.default
let content :[String]
do {
try content = fm.contentsOfDirectory(atPath: ".")
} catch {
content = [String]()
}
return content
}
func processCommand(_ fd: CFFileDescriptorNativeDescriptor = STDIN_FILENO) -> Int8 {
let fileH = FileHandle(fileDescriptor: fd)
let command = String(data: fileH.availableData,
encoding: .utf8)?.trimmingCharacters(
in: .whitespacesAndNewlines) ?? ""
switch command {
case "exit":
return -1
case "ls":
print(list(), terminator:"\n$ ")
return 0
case "":
return 1
default:
print("Your command: \(command)|", terminator:"\n$ ")
return 0
}
}
print("Welcome to rdShell\n% ", terminator: "")
outerLoop: while true {
let result = processCommand()
switch result {
case -1:
break outerLoop
case 1:
print("Error reading command")
break outerLoop
default:
break
}
}
print("Bye bye now.")
Our shell supports the following commands:
ls
- list files in current directory-
exit
- finishes the shell session and exits
Any other input from the user gets echoed back.
We are going to use this simple event-loop to explore how the RunLoop
works.
First, let's make clear what is happening when we run our program. When we execute our program, we start a process
. A process works as a container for all the resources associated with it(file descriptors, virtual memory space, etcetera) and all the execution threads. A process can have one or many threads of execution.
In our example, we only have one thread, also known as the main
thread. We defined two functions, one that lists the contents of the current directory, and a second one that is in charge of processing the input from the Standard Input(stdin
). After the declaration of our functions, we have a loop that is in charge of our event-loop (The Read Eval Print Loop). And that should give us a very basic shell
implementation that we'll use as a base to explore RunLoop
s.
You can run it and see it working.
1
$ swift run
We are going to replace that infinite loop (while true
) with a RunLoop
. But first, let's begin by understanding some key concepts about RunLoop
s.
The RunLoop
In macOS, we get a RunLoop
object with every thread, no matter how the thread is created (via pthread
or Thread
). Also, it is not mandatory to use the RunLoop
, but your thread has access to creating one on-demand if you need it.
The main elements of a RunLoop
are:
Input Source: this will be the asynchronous source of the events. There are different types of input sources:
- Port-based: The events come asynchronously from ports. Examples of port-based input sources are, Mach-ports, sockets, message ports, and file descriptors.
- Timer: For this input source the events are fired with a timer object.
- Perform selector: This input source allows us to call a selector on any thread we have access to.
- Custom: As the name implies, we have to implement all of the signalling for this input source.
Mode: The mode is a string that allows us to tag the input source and observers. We can use the mode to filter out which events are being processed based on the mode. We have the following predefined modes:
- Modal - used by the system to tag events coming from modal panels.
- Event tracking - used to restrict events during user interface tracking loops like mouse drags.
- Connection - used when we are working with
NSConnection
events. - Common modes - groups modes together. For Cocoa applications, this set includes by default the following modes:
default
,modal
, andevent tracking
. You can add more modes to the "common modes" if you need. - Default - this is the mode you'll typically use. It is the mode used by most operations.
Observers: The observers will be triggered not directly by the events coming from the input source but changes in the state of the
RunLoopp
. For example, we can set an observer to be triggered when a loop is about to process a timer.
In summary, we can use a RunLoop
to monitor input for different sources. Also, we can define observers to be notified when a RunLoop
changes state. Every input source and observer should always be associated with at least one mode
, by which we can filter the events being processed by the RunLoop
. Also, when we run our RunLoop
we define the mode it will run in. Let's see some examples:
1
2
3
4
5
6
7
8
9
10
11
12
13
// get our the `RunLoop` associated to our current thread
let currentRL = RunLoop.current
//Run our current `RunLoop` on default mode
currentRL.run() //We could run it directly RunLoop.current.run()
//Let's define a custom mode
let customMode = "com.rderik.myeventmode"
//Run our current `RunLoop` on a specif mode
currentRL.run(mode: RunLoop.Mode(customMode),
before: Date.distantFuture)
Important note: a RunLoop
can only run on a specific mode if there is at least one input source or timer to monitor. If there are none, the RunLoop
will not run (or stop after all the input sources and/or timers have completed).
Ok, I think that should give us a good base to begin working with RunLoop
s. Let's start by replacing the infinite while
in our previous example with a RunLoop
.
Using a RunLoop
We need an input source for our RunLoop
. The good news is that Core Foundation
provides us with utility functions to generate RunLoopSource
s from the most common objects.
CFFileDescriptorCreateRunLoopSource
-
CFSocketCreateRunLoopSource
-
CFMachPortCreateRunLoopSource
-
CFMessagePortCreateRunLoopSource
For our case, we are going to read from the stdin
through a CFFileDescriptor
, and we are going to use CFFileDescriptorCreateRunLoopSource
to generate our source info.
Alright, let's do this.
Updating our code to use a RunLoop
.
Let's start by defining a function to register our CFFileDescriptor
as an input source. A CFFileDescriptor
provides an opaque type that can be used to monitor read/write activity using a RunLoop
. The CFFileDescriptor
uses a CFFileDescriptorCallBack
when there is activity in the CFFileDescriptor
, we'll define our call back function later. First, let's register our input source.
1
2
3
4
5
6
7
8
9
10
func registerStdinFileDescriptor() {
let fd = CFFileDescriptorCreate(kCFAllocatorDefault, STDIN_FILENO, false, fileDescriptorCallBack, nil)
CFFileDescriptorEnableCallBacks(fd, kCFFileDescriptorReadCallBack);
let source = CFFileDescriptorCreateRunLoopSource(kCFAllocatorDefault, fd, 0)
// customMode is defined outside as:
//let customMode = "com.rderik.myevents"
let cfCustomMode: CFRunLoopMode = CFRunLoopMode(customMode as CFString)
CFRunLoopAddSource(RunLoop.main.getCFRunLoop(), source, cfCustomMode);
}
We begin by creating our CFFileDescriptor
. As you can see we are sending fileDescriptorCallBack
as our CFFileDescriptorCallBack
, we'll define it later. Then we generate the input source by calling CFFileDescriptorCreateRunLoopSource
. And finally, we add the input source to the main RunLoop
. Nothing too complicated here, just boilerplate.
Now let's create the function we would like to run when there is something to process from the stdin
. We'll name our function fileDescriptorCallBack
, as the name implies, it'll be the registered callback to our CFFileDescriptor
.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
func fileDescriptorCallBack(_ fd: CFFileDescriptor?, _ flags: CFOptionFlags, _ info: UnsafeMutableRawPointer?) {
let nfd = CFFileDescriptorGetNativeDescriptor(fd)
let result = processCommand(nfd)
switch result {
case -1:
print("Bye bye now.")
default:
registerStdinFileDescriptor()
// customMode is defined outside as:
//let customMode = "com.rderik.myevents"
RunLoop.main.run(mode: RunLoop.Mode(customMode),
before: Date.distantFuture)
}
}
The function relies on our previous function processCommand(_: CFFileDescriptorNativeDescriptor = STDIN_FILENO)
to process the commands we receive from the user via the file descriptor. Based on the result from processCommand
we take different actions. If the result
is -1
it means that the user typed exit
so do nothing, just display a goodbye message. The default
behaviour on the switch
statement calls our function registerStdinFileDescriptor
that registers our CFFileDescriptor
as an input source, and runs the main
RunLoop in our customMode
("com.rderik.myevents").
You might be wondering, why did we re-register our input source and re-run the RunLoop
?
We do this because once our input source triggered the read
on CFFileDescriptor
, there were no more input sources to monitor for the main RunLoop
. Remember the Important Note
Important note: a
RunLoop
can only run on a specific mode if there is at least one input source or timer to monitor. If there are none theRunLoop
will not run (or stop after all the input sources and/or timers have completed).
That is the reason we have to re-register the input source again and re-run the RunLooop
.
Next, you can see how our whole file would look:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
import Foundation
func list() -> [String] {
let fm = FileManager.default
let content :[String]
do {
try content = fm.contentsOfDirectory(atPath: ".")
} catch {
content = [String]()
}
return content
}
func processCommand(_ fd: CFFileDescriptorNativeDescriptor = STDIN_FILENO) -> Int8 {
let fileH = FileHandle(fileDescriptor: fd)
let command = String(data: fileH.availableData,
encoding: .utf8)?.trimmingCharacters(
in: .whitespacesAndNewlines) ?? ""
switch command {
case "exit":
return -1
case "ls":
print(list(), terminator:"\n$ ")
return 0
case "":
return 1
default:
print("Your command: \(command)|", terminator:"\n$ ")
return 0
}
}
func fileDescriptorCallBack(_ fd: CFFileDescriptor?, _ flags: CFOptionFlags, _ info: UnsafeMutableRawPointer?) {
let nfd = CFFileDescriptorGetNativeDescriptor(fd)
let result = processCommand(nfd)
switch result {
case -1:
print("Bye bye now.")
default:
registerStdinFileDescriptor()
// customMode is defined outside as:
//let customMode = "com.rderik.myevents"
RunLoop.main.run(mode: RunLoop.Mode(customMode),
before: Date.distantFuture)
}
}
func registerStdinFileDescriptor() {
let fd = CFFileDescriptorCreate(kCFAllocatorDefault, STDIN_FILENO, false, fileDescriptorCallBack, nil)
CFFileDescriptorEnableCallBacks(fd, kCFFileDescriptorReadCallBack);
let source = CFFileDescriptorCreateRunLoopSource(kCFAllocatorDefault, fd, 0)
// customMode is defined outside as:
//let customMode = "com.rderik.myevents"
let cfCustomMode: CFRunLoopMode = CFRunLoopMode(customMode as CFString)
CFRunLoopAddSource(RunLoop.main.getCFRunLoop(), source, cfCustomMode);
}
let customMode = "com.rderik.myevents"
print("Welcome to rdShell\n$ ", terminator: "")
fflush(__stdoutp)
registerStdinFileDescriptor()
RunLoop.main.run(mode: RunLoop.Mode(customMode),
before: Date.distantFuture)
Good, now you can run it, and it'll work as the old one did.
Ok, that doesn't seem as easy as our previous infinite loop implementation. It looks more convoluted, why would we use it?
Yea, I agree, for a simple case we might not need a RunLoop
, but there are some advantages even for a simple case as ours.
When we run infinite loops, as before, the main thread is always active, while when we used the RunLoop
the main thread went to sleep until there was something to read. So, in general, this implementation is more efficient than the previous. Also, the RunLoop
s shines when we can make use of the thread to run other tasks instead of waiting idly.
In a command-line tool, like ours, "blocking" the main thread is not that noticeable. But if you were to block the main thread in a macOS app or an iOS app, your user will have a terrible experience. For example, continually fetching content from an external API over the internet takes a few seconds. Imagine if we executed the fetch in our main thread, the user will be unable to interact with anything in our app until the fetch is completed. You can see how that would be poor user experience. If it's a single fetch we could probably use an async call through GCD
, but an event-loop is better served by a RunLoop
. In any case, you can see the benefit of using a non-blocking solution.
Ok, let's make better use of our RunLoop
by improving our shell.
Making use of our non-blocking implementation
At the moment, our shell looks as if it's the same as our previous implementation with an infinite loop. It even seems more trouble to get the same results. I told you that our implementation is non-blocking. But how can we prove that?
If our implementation is non-blocking, we should be able to run other tasks while there is nothing to read. Correct?
To prove that, we are going to run another task. We are going to display the current time in our shell's prompt and update it every second. On our original implementation, the one with the infinite loop, this couldn't be done because the read
operation will block the main thread and we can't do anything else.
Ok, let's get started.
Our prompt will look like this:
1
2
# [HH:MM:SS] $
[17:22:25] $
First, let's prepare our main to create a place holder for our prompt. Also, you might have noticed that some times the prompt was not showing up correctly. We'll fix that by "flushing" our stdout
so it displays everything we had put on the buffer.
Our main file will look like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
import Foundation
let promptPlaceHoder = "[00:00:00] $ "
func list() -> [String] {
let fm = FileManager.default
let content :[String]
do {
try content = fm.contentsOfDirectory(atPath: ".")
} catch {
content = [String]()
}
return content
}
func processCommand(_ fd: CFFileDescriptorNativeDescriptor = STDIN_FILENO) -> Int8 {
let fileH = FileHandle(fileDescriptor: fd)
let command = String(data: fileH.availableData,
encoding: .utf8)?.trimmingCharacters(
in: .whitespacesAndNewlines) ?? ""
switch command {
case "exit":
return -1
case "ls":
print(list(), terminator:"\n\(promptPlaceHoder)")
fflush(__stdoutp)
return 0
case "":
return 1
default:
print("Your command: \(command)", terminator:"\n\(promptPlaceHoder)")
fflush(__stdoutp)
return 0
}
}
func fileDescriptorCallBack(_ fd: CFFileDescriptor?, _ flags: CFOptionFlags, _ info: UnsafeMutableRawPointer?) {
let nfd = CFFileDescriptorGetNativeDescriptor(fd)
let result = processCommand(nfd)
switch result {
case -1:
print("Bye bye now.")
default:
registerStdinFileDescriptor()
// customMode is defined outside as:
//let customMode = "com.rderik.myevents"
RunLoop.main.run(mode: RunLoop.Mode(customMode),
before: Date.distantFuture)
}
}
func registerStdinFileDescriptor() {
let fd = CFFileDescriptorCreate(kCFAllocatorDefault, STDIN_FILENO, false, fileDescriptorCallBack, nil)
CFFileDescriptorEnableCallBacks(fd, kCFFileDescriptorReadCallBack);
let source = CFFileDescriptorCreateRunLoopSource(kCFAllocatorDefault, fd, 0)
// customMode is defined outside as:
//let customMode = "com.rderik.myevents"
let cfCustomMode: CFRunLoopMode = CFRunLoopMode(customMode as CFString)
CFRunLoopAddSource(RunLoop.main.getCFRunLoop(), source, cfCustomMode);
}
let customMode = "com.rderik.myevents"
print("Welcome to rdShell\n\(promptPlaceHoder)", terminator: "")
fflush(__stdoutp)
registerStdinFileDescriptor()
RunLoop.main.run(mode: RunLoop.Mode(customMode),
before: Date.distantFuture)
If you run it now, you'll see the prompt correctly showing up.
Now let's create our PromptTier
. Create the file PromptTimer.swift
in your sources directory. And add the following content:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import Foundation
class PromptTimer {
func start() {
let timer = Timer(timeInterval: 1.0, target: self, selector: #selector(printTime), userInfo: nil, repeats: true)
RunLoop.current.add(timer, forMode: RunLoop.Mode("com.rderik.myevents"))
}
@objc func printTime(timer: Timer) {
let formatter = DateFormatter()
formatter.dateFormat = "HH:mm:ss"
let currentDateTime = Date()
let strTime = formatter.string(from: currentDateTime)
print("\u{1B}7\r[\(strTime)] $ \u{1B}8", terminator: "")
fflush(__stdoutp)
}
}
As you can see in our start
function, we are setting a Timer
and adding it to our current RunLoop
using the same mode
we have for our input source. The timer
will trigger printTime
every second.
The printTime
function might have some code that you haven't seen before if you haven't used console escape sequences. Escape sequences are an interesting concept. You've most likely used them before, but here they are used in a more sophisticated way. The most common escape sequence you probably encounter is \n
which tells the terminal to display a new line and move the cursor to the next line. Here we are using control escape sequences
, we are telling the terminal that the following should be interpreted as a control escape sequence
by preceding it with \u{1B}
.
For example, the control sequence 7
saves the current cursor position and the control sequence 8
restores the saved cursor position. As you can see, first, we save where the user had its cursor. Then we move to the beginning of the line (using \r
carriage return) and print our prompt. After that is done, then restore the cursor to where the user had it before. Clever right?
Ok, that's all with TimerPrompt
, let's integrate it to our main
file. After the registration of our stdin
input source, we'll create a TimePrompt
and start it.
1
2
3
4
registerStdinFileDescriptor()
//After the registration add our PromptTimer and start it
let pt = PromptTimer()
pt.start()
That's it. You can see the complete main.swift
next:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
import Foundation
let promptPlaceHoder = "[00:00:00] $ "
func list() -> [String] {
let fm = FileManager.default
let content :[String]
do {
try content = fm.contentsOfDirectory(atPath: ".")
} catch {
content = [String]()
}
return content
}
func processCommand(_ fd: CFFileDescriptorNativeDescriptor = STDIN_FILENO) -> Int8 {
let fileH = FileHandle(fileDescriptor: fd)
let command = String(data: fileH.availableData,
encoding: .utf8)?.trimmingCharacters(
in: .whitespacesAndNewlines) ?? ""
switch command {
case "exit":
return -1
case "ls":
print(list(), terminator:"\n\(promptPlaceHoder)")
fflush(__stdoutp)
return 0
case "":
return 1
default:
print("Your command: \(command)", terminator:"\n\(promptPlaceHoder)")
fflush(__stdoutp)
return 0
}
}
func fileDescriptorCallBack(_ fd: CFFileDescriptor?, _ flags: CFOptionFlags, _ info: UnsafeMutableRawPointer?) {
let nfd = CFFileDescriptorGetNativeDescriptor(fd)
let result = processCommand(nfd)
switch result {
case -1:
print("Bye bye now.")
default:
registerStdinFileDescriptor()
// customMode is defined outside as:
//let customMode = "com.rderik.myevents"
RunLoop.main.run(mode: RunLoop.Mode(customMode),
before: Date.distantFuture)
}
}
func registerStdinFileDescriptor() {
let fd = CFFileDescriptorCreate(kCFAllocatorDefault, STDIN_FILENO, false, fileDescriptorCallBack, nil)
CFFileDescriptorEnableCallBacks(fd, kCFFileDescriptorReadCallBack);
let source = CFFileDescriptorCreateRunLoopSource(kCFAllocatorDefault, fd, 0)
// customMode is defined outside as:
//let customMode = "com.rderik.myevents"
let cfCustomMode: CFRunLoopMode = CFRunLoopMode(customMode as CFString)
CFRunLoopAddSource(RunLoop.main.getCFRunLoop(), source, cfCustomMode);
}
let customMode = "com.rderik.myevents"
print("Welcome to rdShell\n\(promptPlaceHoder)", terminator: "")
fflush(__stdoutp)
registerStdinFileDescriptor()
let pt = PromptTimer()
pt.start()
RunLoop.main.run(mode: RunLoop.Mode(customMode),
before: Date.distantFuture)
Important Note this won't work correctly on the Xcode
console because the debug console is not a full terminal and doesn't support all the control escape sequences
. This makes sense because the Xcode
debugger is for debugging not to run a terminal. So, you'll have to run your application from a terminal. You can drag your application from the Products
section in the Xcode
navigator, and drop it in your Terminal
. Or swift run
if you are using the Swift Package Manager.
Now run the program, and you should be able to see the time updating in the prompt, and also you'll be able to input commands :).
Congratulations, your shell is looking great!
** You can check the full code of the shell
in the GitHub repository.
Final Thoughts
I think you can now see how useful having access to the RunLoop
is. I think it's quite an intelligent approach to handling event-loops. We were able to run our RunLoop
in a custom mode (com.rderik.myevents
) and add two different sources to the RunLoop
one triggered by a Timer
, and another triggered by a CFFileDescriptor
.
We didn't get to see any example using other input sources, like CFMachPort
or CFMessagePort
, but I hope you got a good idea of how to use RunLoop
s. And if you need it, you can adapt what we created here to any other input source. We also didn't get to Observers
, but I encourage you to explore that on your own.
Also, I hope you can now see why it's not a good idea to run blocking tasks on the main thread in your apps (macOS and *OS alike). And that now you can take advantage of using the RunLoop
on your threads when dealing with event-loops.
Ok, that's it for this post. I hope you find it useful. As always, feedback is welcome. And let me know any tips you have for using RunLoops
.
One last thing, you might not have noticed, but I have a newsletter. I send a short message every week with what I've been doing during the week and some additional tips. You might find it useful. If you like my posts, I think you'll enjoy the newsletter too.
Related topics/notes of interest
- Apple's Threading programming guide - RunLoops.
- Swift's corelibs - Implementation of RunLoop.
- Apple's Overview of CFRunLoopSource.
- Apples description of CFFileDescriptor.
- Paul Hudson - Timer examples Good for
Timer
reference. - Stack overflow answer - giving some examples of Control Scape Sequences.
- ANSI/VT100 Terminal Control Escape Sequences
- A more detailed Control Escape Sequences resource from docs.microsoft.com