Newsletter #39 - Fooled by labels Dec 5 2023
Hello, and welcome to issue #39!
This issue includes a new short story. The idea behind this story came from a Capture The Flag (CTF) game I was playing a few weeks ago. One of the flags could be obtained using a reasoning similar to what is presented in the story. It also seems to be a potential security issue often overlooked in projects starting to use Kubernetes.
I won't go into much detail now; I would prefer you to read the story, and we can discuss it after you've read it. But I would like you to apply what you learn from the story to other aspects of your day-to-day life. Question what you think, and you might be able to catch a glimpse of something that wasn't obvious before.
I hope you like it.
Secrets and the Illusory Nature of Reality - 0x2
"Que Dios te bendiga,”
Said his mum as a goodbye and a blessing for his first day as a security analyst.
Kevin couldn't help but smile as his mum gave her usual farewell blessing. He was excited. His first assignment was to review the security of a mid-size company. They run their applications on EKS, the AWS-managed Kubernetes service.
Luckily for him, in a previous job, he was in charge of modernising a legacy project to improve its delivery speed. He started using Kubernetes during that project, so he was familiar with the tools and workflows.
Without too much trouble, but still, after some bureaucracy, he was onboarded to the client's systems and given a small area to conduct his work. He was located with all the other engineers in an open-plan office.
He was ready to start. He began by listing what were the capabilities of the user he received:
1
kubectl auth can-i --list
He had the same access as a junior developer, just the necessary permissions to complete the standard software development life cycle tasks. He checked all the resources and possible misconfigurations in the cluster that his permissions allowed.
He was so entranced in his work that he didn't notice when day turned into night. He was the only one left at the office, or at least that was what he thought.
It seemed that there was one other person, a janitor, doing his rounds of cleaning.
Kevin was getting frustrated. He hadn't found any glaring misconfigurations or issues. He had such high hopes. This was supposed to be an easy task for him. Complex systems, like Kubernetes, are often misconfigured. The surface area is vast, so he expected to have something to report by now.
He started pacing and talking to himself. Such was his distress that he didn't notice the janitor standing next to him.
"Padre Santo!" He exclaimed, getting startled by the unexpected companion.
"Oh, hablas español!" said the janitor.
"Mucho gusto, mi nombre es Hernan"
They continued in Spanish.
"Hey, you gave me a fright there", said Kevin.
"Sorry. You seemed a little bit stressed, so I came to check if you were okay", replied Hernan.
"Don't worry about it. It is my first day, and I don't seem to be able to do my job."
"Ha! I know exactly how that feels. But what is it that you are doing? Maybe talking to someone about it might loosen some ideas. At least that was what my grandpa always said".
Something about Hernan was very charming. Or maybe it was a desperate attempt to try something different than getting frustrated in the terminal. Kevin started:
"Well, it's something about computers. It is quite technical, so I don't know how to explain it to you. No offence."
"None taken. I understand. But maybe, just try to explain the general idea. I used to be very clever in solving my grandpa's riddles. And there are only a few things harder than those riddles."
Kevin laughed at his good nature companion and continued:
"Okay, let me see if I can explain it in simple terms."
"I've checked all the system configurations, and everything seems to be okay. I'm checking for security vulnerabilities but can't find any."
"what do you mean by everything?" interrupted Hernan.
"You know, pods, volumes, nodes, secrets, the whole cluster" replied Kevin.
Hernan grinned and said:
"There you have it! You just found it! The key is the secrets. That sounds important."
For a moment, Kevin thought Hernan had given him a clue. But no, that was silly. He was just grasping at straws. He replied to the still-grinning Hernan: "No, Hernan, you don't understand. Secrets are, well, secrets. You can't look at them."
"Says who?" the affable janitor said.
"Well... I mean... eh..." "You know, why would it be called secret if it wasn't safe and secret?" replied the confused Kevin.
"The way I see it, names are just that. They hint at something, but often, people put too much stock in names. Just because something is a secret doesn't mean you can't split it open and see what it keeps as a secret."
"You know, I've seen many people get that confused; they hear something described and just believe it. They believe the description and never question whether it is true or not. They even do that with their own thoughts. People believe their thoughts only because they thought of them, not because they have evaluated their truthfulness."
All of a sudden, Hernan stopped and smiled again.
"Sorry, I just started rambling. I'll return to my work and let you return to yours."
"I hope you find what you are looking for." "Hasta luego."
Kevin was still processing what the janitor said when he remembered an article about secrets containing the value they stored encoded, not encrypted.
He replied with an absent-minded "Gracias" and waved to the departing man. Still thinking about it, he noticed the error:
People believe that a secret can't be read just because of its name. In Kubernetes, secrets are not secured on their own. They depend on the Role-based access control (RBAC) to specify who can do what on the specific resource, or you have to encrypt the secret content so it can only be decrypted by someone with the correct key.
He listed all the secrets in the namespace.
1
kubectl get secrets
He saw an interesting secret called database_connection_string
He then fetched the details of the secret:
1
kubectl get secrets database_connection_string -o json
He saw that the value was there. It was just encoded in base64, just a text encoding, not encrypted.
He extracted the value:
1
kubectl get secrets database_connection_string -o jsonpath='{.data.connection_string}' | base64 -d
Bingo! He got the connection string that included the user and password to connect to the database. Finally, he had a finding to report.
After all that excitement, he relaxed, and his thoughts returned to the janitor, working his way to the end of the hallway.
His eyes focused on the janitor. Was he pulling a USB out of one of the computers?
"Hey!" yelled Kevin and ran after the janitor.
The janitor slowly turned around the corner, and when Kevin reached the corner, he was gone.
"Dios mío! That was definitely another finding".
Final Thoughts
It is interesting how easily we deceive ourselves by labels. The deception can be seen from two different angles. One, if you are causing the deception, and two, if you are being deceived. If you are the person creating the deception, you take advantage of preconceived ideas from the "victim". This is exemplified in social engineering and magic tricks using misdirection.
I think it is harder to be aware when we are being deceived, either by someone else or by our own preconceptions. This applies when we are debugging or searching for vulnerabilities. You have to ask yourself what the code actually does, not what you think it does. Once you know what it does, you can start thinking of finding ways to fix the behaviour or exploit the flaw.
It sounds easy. You just:
- Stop and think carefully about which area of the codebase you will focus on.
- Gather data. You need to have as many facts as possible on your object of observation and its context.
- Analyse. Use the collected data to identify possible areas of interest or behaviours that seem out of place or areas that you don't understand.
- Test/experiment. Test the hypothesis that you came up with when doing the analysis. One trick is to try to search for data or cases that invalidate your hypothesis. Sometimes, we get so invested in the hypothesis that we trick ourselves into finding anything that validates our idea, even if it is wrong. Check confirmation bias.
- Repeat step 1.
Technically, those steps are sound, but in reality, debugging and security research have a component of creativity and stubborn determination that is hard to put into easy steps to follow. Think of it as meditation or as if you are attempting to lucid dream. You need to constantly evaluate what's on your mind or check if you are sleeping for lucid dreaming. You practice regularly until it becomes second nature, and you can stop and observe what is really happening. You give yourself time to think.
Anyway, I hope you enjoy this short story.
Until next time,
Derik P.S. If you know of some good stories, please send them my way. I'm always looking for interesting stories.
Boring Admin News - Change of Email provider
Well, it finally happened. TinyLetter, the email service I use for this newsletter, is being sunset. That means I need to switch to a new provider.
This issue will probably be the last newsletter I send using TinyLetter, so expect the following issue to use a different provider, maybe Beehiiv. Let me know if you have recommendations for a provider that might be a good fit.