Understanding the RunLoop model by creating a basic shell Oct 1 2019

When we find ourselves listening for events, reacting to those events, and then going back to listening for more events, we have ourselves an Event-Loop. Having an event-loop is a common scenario. So common that Apple decided to provide us with a model to handle event-loops consistently. In this post, we are going to explore how RunLoops work and use them to build basic shell.

Let’s get started by creating a program that makes use of an event-loop.

** You can check the full code of the shell in the GitHub repository.

Event-Loops

One of the most common cases for an event-loop is a REPL (Read Eval Print Loop). And probably one of the most used REPLs out there are shells, your shell (bash, zsh, fish whichever you prefer) is the classic example of an event-loop. The shell waits for your input, once it reads your input, it executes the commands you send, and finally, it goes back to listening for more input.

Let’s create a basic shell that has only one useful command (two if we count exit), list the files in the current directory using ls.

We can create our application using the Swift Package Manager.

1
2
3

$ mkdir rdshell
$ cd rdshell
$ swift package init --type executable

Add this content to main.swift: 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

import Foundation

func list() -> [String] {
  let fm = FileManager.default
  let content :[String]
  do {
    try content = fm.contentsOfDirectory(atPath: ".")
  } catch {
    content = [String]()
  }
  return content
}

func processCommand(_ fd: CFFileDescriptorNativeDescriptor = STDIN_FILENO) -> Int8 {
    let fileH = FileHandle(fileDescriptor: fd)
    let command = String(data: fileH.availableData,
                         encoding: .utf8)?.trimmingCharacters(
                            in: .whitespacesAndNewlines) ?? ""
    switch command {
    case "exit":
      return -1
    case "ls":
        print(list(), terminator:"\n$ ")
        return 0
    case "":
        return 1
    default:
        print("Your command: \(command)|", terminator:"\n$ ")
      return 0
    }

}


print("Welcome to rdShell\n% ", terminator: "")
outerLoop: while true {
    let result = processCommand()
    switch result {
    case -1:
        break outerLoop
    case 1:
        print("Error reading command")
        break outerLoop
    default:
        break
    }
}
print("Bye bye now.")

Our shell supports the following commands:

Any other input from the user gets echoed back.

We are going to use this simple event-loop to explore how the RunLoop works.

First, let’s make clear what is happening when we run our program. When we execute our program, we start a process. A process works as a container for all the resources associated with it(file descriptors, virtual memory space, etcetera) and all the execution threads. A process can have one or many threads of execution.

In our example, we only have one thread, also known as the main thread. We defined two functions, one that lists the contents of the current directory, and a second one that is in charge of processing the input from the Standard Input(stdin). After the declaration of our functions, we have a loop that is in charge of our event-loop (The Read Eval Print Loop). And that should give us a very basic shell implementation that we’ll use as a base to explore RunLoops.

You can run it and see it working.

1

$ swift run

We are going to replace that infinite loop (while true) with a RunLoop. But first, let’s begin by understanding some key concepts about RunLoops.

The RunLoop

In macOS, we get a RunLoop object with every thread, no matter how the thread is created (via pthread or Thread). Also, it is not mandatory to use the RunLoop, but your thread has access to creating one on-demand if you need it.

The main elements of a RunLoop are:

In summary, we can use a RunLoop to monitor input for different sources. Also, we can define observers to be notified when a RunLoop changes state. Every input source and observer should always be associated with at least one mode, by which we can filter the events being processed by the RunLoop. Also, when we run our RunLoop we define the mode it will run in. Let’s see some examples:

1
2
3
4
5
6
7
8
9
10
11
12
13

// get our the `RunLoop` associated to our current thread
let currentRL = RunLoop.current

//Run our current `RunLoop` on default mode
currentRL.run() //We could run it directly RunLoop.current.run()


//Let's define a custom mode
let customMode = "com.rderik.myeventmode"

//Run our current `RunLoop` on a specif mode
currentRL.run(mode: RunLoop.Mode(customMode),
                 before: Date.distantFuture)

Important note: a RunLoop can only run on a specific mode if there is at least one input source or timer to monitor. If there are none, the RunLoop will not run (or stop after all the input sources and/or timers have completed).

Ok, I think that should give us a good base to begin working with RunLoops. Let’s start by replacing the infinite while in our previous example with a RunLoop.

Using a RunLoop

We need an input source for our RunLoop. The good news is that Core Foundation provides us with utility functions to generate RunLoopSources from the most common objects.

For our case, we are going to read from the stdin through a CFFileDescriptor, and we are going to use CFFileDescriptorCreateRunLoopSource to generate our source info.

Alright, let’s do this.

Updating our code to use a RunLoop.

Let’s start by defining a function to register our CFFileDescriptor as an input source. A CFFileDescriptor provides an opaque type that can be used to monitor read/write activity using a RunLoop. The CFFileDescriptor uses a CFFileDescriptorCallBack when there is activity in the CFFileDescriptor, we’ll define our call back function later. First, let’s register our input source.

1
2
3
4
5
6
7
8
9
10

func registerStdinFileDescriptor() {
    let fd = CFFileDescriptorCreate(kCFAllocatorDefault, STDIN_FILENO, false, fileDescriptorCallBack, nil)
    CFFileDescriptorEnableCallBacks(fd, kCFFileDescriptorReadCallBack);
    let source = CFFileDescriptorCreateRunLoopSource(kCFAllocatorDefault, fd, 0)

    // customMode is defined outside as:
    //let customMode = "com.rderik.myevents"
    let cfCustomMode: CFRunLoopMode = CFRunLoopMode(customMode as CFString)
    CFRunLoopAddSource(RunLoop.main.getCFRunLoop(), source, cfCustomMode);
}

We begin by creating our CFFileDescriptor. As you can see we are sending fileDescriptorCallBack as our CFFileDescriptorCallBack, we’ll define it later. Then we generate the input source by calling CFFileDescriptorCreateRunLoopSource. And finally, we add the input source to the main RunLoop. Nothing too complicated here, just boilerplate.

Now let’s create the function we would like to run when there is something to process from the stdin. We’ll name our function fileDescriptorCallBack, as the name implies, it’ll be the registered callback to our CFFileDescriptor.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

func fileDescriptorCallBack(_ fd: CFFileDescriptor?, _ flags: CFOptionFlags, _ info: UnsafeMutableRawPointer?) {
    let nfd = CFFileDescriptorGetNativeDescriptor(fd)
    let result = processCommand(nfd)
    switch result {
    case -1:
        print("Bye bye now.")
    default:
        registerStdinFileDescriptor()
        // customMode is defined outside as:
        //let customMode = "com.rderik.myevents"
        RunLoop.main.run(mode: RunLoop.Mode(customMode),
                         before: Date.distantFuture)
    }
}

The function relies on our previous function processCommand(_: CFFileDescriptorNativeDescriptor = STDIN_FILENO) to process the commands we receive from the user via the file descriptor. Based on the result from processCommand we take different actions. If the result is -1 it means that the user typed exit so do nothing, just display a goodbye message. The default behaviour on the switch statement calls our function registerStdinFileDescriptor that registers our CFFileDescriptor as an input source, and runs the main RunLoop in our customMode (“com.rderik.myevents”).

You might be wondering, why did we re-register our input source and re-run the RunLoop?

We do this because once our input source triggered the read on CFFileDescriptor, there were no more input sources to monitor for the main RunLoop. Remember the Important Note

Important note: a RunLoop can only run on a specific mode if there is at least one input source or timer to monitor. If there are none the RunLoop will not run (or stop after all the input sources and/or timers have completed).

That is the reason we have to re-register the input source again and re-run the RunLooop.

Next, you can see how our whole file would look:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

import Foundation

func list() -> [String] {
  let fm = FileManager.default
  let content :[String]
  do {
    try content = fm.contentsOfDirectory(atPath: ".")
  } catch {
    content = [String]()
  }
  return content
}

func processCommand(_ fd: CFFileDescriptorNativeDescriptor = STDIN_FILENO) -> Int8 {
    let fileH = FileHandle(fileDescriptor: fd)
    let command = String(data: fileH.availableData,
                         encoding: .utf8)?.trimmingCharacters(
                            in: .whitespacesAndNewlines) ?? ""
    switch command {
    case "exit":
      return -1
    case "ls":
        print(list(), terminator:"\n$ ")
        return 0
    case "":
        return 1
    default:
        print("Your command: \(command)|", terminator:"\n$ ")
      return 0
    }

}

func fileDescriptorCallBack(_ fd: CFFileDescriptor?, _ flags: CFOptionFlags, _ info: UnsafeMutableRawPointer?) {
    let nfd = CFFileDescriptorGetNativeDescriptor(fd)
    let result = processCommand(nfd)
    switch result {
    case -1:
        print("Bye bye now.")
    default:
        registerStdinFileDescriptor()
        // customMode is defined outside as:
        //let customMode = "com.rderik.myevents"
        RunLoop.main.run(mode: RunLoop.Mode(customMode),
                         before: Date.distantFuture)
    }
}

func registerStdinFileDescriptor() {
    let fd = CFFileDescriptorCreate(kCFAllocatorDefault, STDIN_FILENO, false, fileDescriptorCallBack, nil)
    CFFileDescriptorEnableCallBacks(fd, kCFFileDescriptorReadCallBack);
    let source = CFFileDescriptorCreateRunLoopSource(kCFAllocatorDefault, fd, 0)

    // customMode is defined outside as:
    //let customMode = "com.rderik.myevents"
    let cfCustomMode: CFRunLoopMode = CFRunLoopMode(customMode as CFString)
    CFRunLoopAddSource(RunLoop.main.getCFRunLoop(), source, cfCustomMode);
}

let customMode = "com.rderik.myevents"
print("Welcome to rdShell\n$ ", terminator: "")
fflush(__stdoutp)
registerStdinFileDescriptor()
RunLoop.main.run(mode: RunLoop.Mode(customMode),
                 before: Date.distantFuture)

Good, now you can run it, and it’ll work as the old one did.

Ok, that doesn’t seem as easy as our previous infinite loop implementation. It looks more convoluted, why would we use it?

Yea, I agree, for a simple case we might not need a RunLoop, but there are some advantages even for a simple case as ours.

When we run infinite loops, as before, the main thread is always active, while when we used the RunLoop the main thread went to sleep until there was something to read. So, in general, this implementation is more efficient than the previous. Also, the RunLoops shines when we can make use of the thread to run other tasks instead of waiting idly.

In a command-line tool, like ours, “blocking” the main thread is not that noticeable. But if you were to block the main thread in a macOS app or an iOS app, your user will have a terrible experience. For example, continually fetching content from an external API over the internet takes a few seconds. Imagine if we executed the fetch in our main thread, the user will be unable to interact with anything in our app until the fetch is completed. You can see how that would be poor user experience. If it’s a single fetch we could probably use an async call through GCD, but an event-loop is better served by a RunLoop. In any case, you can see the benefit of using a non-blocking solution.

Ok, let’s make better use of our RunLoop by improving our shell.

Making use of our non-blocking implementation

At the moment, our shell looks as if it’s the same as our previous implementation with an infinite loop. It even seems more trouble to get the same results. I told you that our implementation is non-blocking. But how can we prove that?

If our implementation is non-blocking, we should be able to run other tasks while there is nothing to read. Correct?

To prove that, we are going to run another task. We are going to display the current time in our shell’s prompt and update it every second. On our original implementation, the one with the infinite loop, this couldn’t be done because the read operation will block the main thread and we can’t do anything else.

Ok, let’s get started.

Our prompt will look like this:

1
2

# [HH:MM:SS] $
[17:22:25] $

First, let’s prepare our main to create a place holder for our prompt. Also, you might have noticed that some times the prompt was not showing up correctly. We’ll fix that by “flushing” our stdout so it displays everything we had put on the buffer.

Our main file will look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

import Foundation

let promptPlaceHoder = "[00:00:00] $ "

func list() -> [String] {
  let fm = FileManager.default
  let content :[String]
  do {
    try content = fm.contentsOfDirectory(atPath: ".")
  } catch {
    content = [String]()
  }
  return content
}

func processCommand(_ fd: CFFileDescriptorNativeDescriptor = STDIN_FILENO) -> Int8 {
    let fileH = FileHandle(fileDescriptor: fd)
    let command = String(data: fileH.availableData,
                         encoding: .utf8)?.trimmingCharacters(
                            in: .whitespacesAndNewlines) ?? ""
    switch command {
    case "exit":
      return -1
    case "ls":
        print(list(), terminator:"\n\(promptPlaceHoder)")
        fflush(__stdoutp)
        return 0
    case "":
        return 1
    default:
        print("Your command: \(command)", terminator:"\n\(promptPlaceHoder)")
        fflush(__stdoutp)
      return 0
    }

}

func fileDescriptorCallBack(_ fd: CFFileDescriptor?, _ flags: CFOptionFlags, _ info: UnsafeMutableRawPointer?) {
    let nfd = CFFileDescriptorGetNativeDescriptor(fd)
    let result = processCommand(nfd)
    switch result {
    case -1:
        print("Bye bye now.")
    default:
        registerStdinFileDescriptor()
        // customMode is defined outside as:
        //let customMode = "com.rderik.myevents"
        RunLoop.main.run(mode: RunLoop.Mode(customMode),
                         before: Date.distantFuture)
    }
}

func registerStdinFileDescriptor() {
    let fd = CFFileDescriptorCreate(kCFAllocatorDefault, STDIN_FILENO, false, fileDescriptorCallBack, nil)
    CFFileDescriptorEnableCallBacks(fd, kCFFileDescriptorReadCallBack);
    let source = CFFileDescriptorCreateRunLoopSource(kCFAllocatorDefault, fd, 0)

    // customMode is defined outside as:
    //let customMode = "com.rderik.myevents"
    let cfCustomMode: CFRunLoopMode = CFRunLoopMode(customMode as CFString)
    CFRunLoopAddSource(RunLoop.main.getCFRunLoop(), source, cfCustomMode);
}




let customMode = "com.rderik.myevents"
print("Welcome to rdShell\n\(promptPlaceHoder)", terminator: "")
fflush(__stdoutp)
registerStdinFileDescriptor()
RunLoop.main.run(mode: RunLoop.Mode(customMode),
                 before: Date.distantFuture)

If you run it now, you’ll see the prompt correctly showing up.

Now let’s create our PromptTier. Create the file PromptTimer.swift in your sources directory. And add the following content:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

import Foundation

class PromptTimer {

    func start() {
        let timer = Timer(timeInterval: 1.0, target: self, selector: #selector(printTime), userInfo: nil, repeats: true)
        RunLoop.current.add(timer, forMode: RunLoop.Mode("com.rderik.myevents"))
    }

    @objc func printTime(timer: Timer) {
        let formatter = DateFormatter()
        formatter.dateFormat = "HH:mm:ss"
        let currentDateTime = Date()
        let strTime = formatter.string(from: currentDateTime)
        print("\u{1B}7\r[\(strTime)] $ \u{1B}8", terminator: "")
        fflush(__stdoutp)
    }
}

As you can see in our start function, we are setting a Timer and adding it to our current RunLoop using the same mode we have for our input source. The timer will trigger printTime every second.

The printTime function might have some code that you haven’t seen before if you haven’t used console escape sequences. Escape sequences are an interesting concept. You’ve most likely used them before, but here they are used in a more sophisticated way. The most common escape sequence you probably encounter is \n which tells the terminal to display a new line and move the cursor to the next line. Here we are using control escape sequences, we are telling the terminal that the following should be interpreted as a control escape sequence by preceding it with \u{1B}.

For example, the control sequence 7 saves the current cursor position and the control sequence 8 restores the saved cursor position. As you can see, first, we save where the user had its cursor. Then we move to the beginning of the line (using \r carriage return) and print our prompt. After that is done, then restore the cursor to where the user had it before. Clever right?

Ok, that’s all with TimerPrompt, let’s integrate it to our main file. After the registration of our stdin input source, we’ll create a TimePrompt and start it.

1
2
3
4

registerStdinFileDescriptor()
//After the registration add our PromptTimer and start it
let pt = PromptTimer()
pt.start()

That’s it. You can see the complete main.swift next:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

import Foundation

let promptPlaceHoder = "[00:00:00] $ "

func list() -> [String] {
  let fm = FileManager.default
  let content :[String]
  do {
    try content = fm.contentsOfDirectory(atPath: ".")
  } catch {
    content = [String]()
  }
  return content
}

func processCommand(_ fd: CFFileDescriptorNativeDescriptor = STDIN_FILENO) -> Int8 {
    let fileH = FileHandle(fileDescriptor: fd)
    let command = String(data: fileH.availableData,
                         encoding: .utf8)?.trimmingCharacters(
                            in: .whitespacesAndNewlines) ?? ""
    switch command {
    case "exit":
      return -1
    case "ls":
        print(list(), terminator:"\n\(promptPlaceHoder)")
        fflush(__stdoutp)
        return 0
    case "":
        return 1
    default:
        print("Your command: \(command)", terminator:"\n\(promptPlaceHoder)")
        fflush(__stdoutp)
      return 0
    }

}

func fileDescriptorCallBack(_ fd: CFFileDescriptor?, _ flags: CFOptionFlags, _ info: UnsafeMutableRawPointer?) {
    let nfd = CFFileDescriptorGetNativeDescriptor(fd)
    let result = processCommand(nfd)
    switch result {
    case -1:
        print("Bye bye now.")
    default:
        registerStdinFileDescriptor()
        // customMode is defined outside as:
        //let customMode = "com.rderik.myevents"
        RunLoop.main.run(mode: RunLoop.Mode(customMode),
                         before: Date.distantFuture)
    }
}

func registerStdinFileDescriptor() {
    let fd = CFFileDescriptorCreate(kCFAllocatorDefault, STDIN_FILENO, false, fileDescriptorCallBack, nil)
    CFFileDescriptorEnableCallBacks(fd, kCFFileDescriptorReadCallBack);
    let source = CFFileDescriptorCreateRunLoopSource(kCFAllocatorDefault, fd, 0)

    // customMode is defined outside as:
    //let customMode = "com.rderik.myevents"
    let cfCustomMode: CFRunLoopMode = CFRunLoopMode(customMode as CFString)
    CFRunLoopAddSource(RunLoop.main.getCFRunLoop(), source, cfCustomMode);
}

let customMode = "com.rderik.myevents"
print("Welcome to rdShell\n\(promptPlaceHoder)", terminator: "")
fflush(__stdoutp)
registerStdinFileDescriptor()
let pt = PromptTimer()
pt.start()
RunLoop.main.run(mode: RunLoop.Mode(customMode),
                 before: Date.distantFuture)

Important Note this won’t work correctly on the Xcodeconsole because the debug console is not a full terminal and doesn’t support all the control escape sequences. This makes sense because the Xcode debugger is for debugging not to run a terminal. So, you’ll have to run your application from a terminal. You can drag your application from the Products section in the Xcode navigator, and drop it in your Terminal. Or swift run if you are using the Swift Package Manager.

Now run the program, and you should be able to see the time updating in the prompt, and also you’ll be able to input commands :).

Congratulations, your shell is looking great!

** You can check the full code of the shell in the GitHub repository.

Final Thoughts

I think you can now see how useful having access to the RunLoop is. I think it’s quite an intelligent approach to handling event-loops. We were able to run our RunLoop in a custom mode (com.rderik.myevents) and add two different sources to the RunLoop one triggered by a Timer, and another triggered by a CFFileDescriptor.

We didn’t get to see any example using other input sources, like CFMachPort or CFMessagePort, but I hope you got a good idea of how to use RunLoops. And if you need it, you can adapt what we created here to any other input source. We also didn’t get to Observers, but I encourage you to explore that on your own.

Also, I hope you can now see why it’s not a good idea to run blocking tasks on the main thread in your apps (macOS and *OS alike). And that now you can take advantage of using the RunLoop on your threads when dealing with event-loops.

Ok, that’s it for this post. I hope you find it useful. As always, feedback is welcome. And let me know any tips you have for using RunLoops.

One last thing, you might not have noticed, but I have a newsletter. I send a short message every week with what I’ve been doing during the week and some additional tips. You might find it useful. If you like my posts, I think you’ll enjoy the newsletter too.

Related topics/notes of interest


** There is no comment system yet, but you can send me a message on twitter @rderik or send me an email: derik[at]rderik[dot]com.