Privilege Escalation Deconstructed: Attacks and How to Stop Them

Christophe Limpalair (00:00): Awesome. Thank you. I appreciate the introduction, Joseph. Yeah, this is one of my favorite topics. I've been talking a lot about this topic, so I'm excited to dive into it and get started. Today we're going to be talking about privilege escalation, deconstructed attacks, and how to stop them. So let me go ahead and talk about the agenda and exactly what we'll be covering today. So first, of course, introductions. Who am I? Why am I talking about this? And why should you listen? And then after that we'll dive straight into the meat of the conversation, which is what are AWS, IAM privilege escalation attacks. And we'll even do a live demonstration where I've got a lab environment spun up that's going to demonstrate an attack that we've been seeing being used in the wild. And then after that, we'll put on our blue team hat, shifting gears to the defensive side of how to find potential perverse paths, also how to prevent attacks, and then finally, how to detect attacks. (00:53): And then at the conclusion, I will share some links with a bunch of resources that we'll be talking about in this presentation and also some extra resources that can help you learn more about this topic and learn how to take what we're talking about today and apply it to your cloud environments. So this really applies to cloud in general. We're talking Azure, GCP, AWS, everything else in between. But most of my experience is with AWS, so I'm going to be focusing on that side of the conversation today. But like Joseph said, my name's Christophe Limpalair. No worries about messing up my last name, I mess it up all the time as well. It's just that type of last name. So no worries there. But like Joseph mentioned, I'm Christophe Limpalair, I'm the founder and a trainer at Cybr, cybr.com, which is a training platform that focuses on hands-on AWS security learning. (01:44): So whether you're an individual or whether you're a team, we help you build a deep expertise and practical expertise in AWS security. So the summary team had me fill in a few different facts about myself, and after spending about an hour just staring at a blank screen, I went to my wife and I was like, "Please help me write some fun facts. I cannot come up with anything at all." And here's what she came up with, and I agree a hundred percent. I am a professional hobbyist. My current hobby is homebrewing. I actually just brewed a beer last weekend, and so I'm excited to see how it turns out just in time for Halloween. And then before that, I was growing mushrooms, so everything from medicinal to gourmet mushrooms. So that was an interesting hobby as well, learned a lot from that. Otherwise, I'm a secret baker with a very large sweet tooth. (02:30): I definitely like sweets a lot, especially baked sweets. I don't bake that often, but when I do, it usually turns out pretty good, so I'm a secret baker. And then the last fun faculty we came up with is that I'm actually French. I was born and raised in France, but I currently live in Denver, Colorado. So if you live in the area or if you're ever visiting for a conference or something like that, let me know. I'd love to go skiing or hiking or even just grab a bite to eat or coffee. But that's enough about me. Let's actually dive into what AWS IAM privilege escalation attacks are, what they look like. We'll do that live demo and then we'll talk about the defensive side of things. So just in case you're not already familiar, we'll go through some basic concepts and then build on top of that. (03:12): But privilege escalations are absolutely not new to the cloud. They've been around for as long as technology has been around. And in very simple terms, it's when you have an attacker that's going from lower-level privileges to higher-level privileges. But in terms of the cloud and AWS specifically, what we're talking about is when an attacker has found a way through a user or role or some other type of resource, but their current access is relatively limited, and so they're trying to escalate their privileges to higher level privileges in order to achieve their objective, whatever that objective may be. So they will want to look for that path that goes from their current permissions to some sort of higher-level permission. But the thing is, a lot of times when we talk about this, people kind of assume that we're talking about somebody finding an account or permissions that basically can't do anything at all and going all the way to getting admin rights. (04:08): And while a lot of case studies that we see of these attacks in the wild, that's the case because maybe they just have a very badly misconfigured environment that happens fairly often, unfortunately. But technically speaking, especially in the cloud nowadays, you do not need, or threat actor does not need to gain admin level permissions to do a serious amount of damage. Some examples of things that they can still do in the cloud without admin level permissions are things like launching EC2 instances, really, really big GPU intensive EC2 instances for crypto mining. We see this all the time. Or they might do things like try to access secret values. So maybe they have access to secrets manager or some other type of database that's storing some sort of secret, maybe access keys or like API keys, I mean things like that. Or maybe even worse, they get access to some sort of code that you have that's running in Lambda or that's running in ECS, like in containers or anything like that. (05:05): And you have hard-coded credentials in the code, and so they can do a lot of damage with that. Or in the case of this presentation, what we're going to look at is exfiltrating sensitive data, which is something that we see happening fairly often with these types of attacks. So maybe they're accessing Amazon S3 where there's some customer data or some sort of proprietary information, backups, whatever. Maybe they can access another database like RDS or DynamoDB or some other database that you've got running on EC2 or something like that. Or like we mentioned, maybe they get access to code and then that either contains sensitive information in the code or in S3 or in the database or having access to those things grants them access to sensitive data, which they can then either exfiltrate and try to ransom you for, or just sell it on the black market and things like that. (05:54): Now of course, these are just three different examples of issues and breaches that we've seen, but it could evolve into other things as well. We see a lot of threat actors that will gain access to an AWS environment and then maybe they use SES to run phishing campaigns, for example. Using your domains and trashing your domain authority essentially and your reputation. So admin rights are absolutely not required to have successful privacy in AWS, and I think that's something that's really important to remember. So speaking of, let's talk about the live demonstration and the scenario and what we're going to be looking at today. So this was just happenstance, very good coincidence, but I started writing this presentation before August of this year, and then in August I woke up to an email of an article called Leaked Environment Variables Allow Large Scale Extortion Operation in Cloud Environments. (06:47): And it's a report by unit 42 of Palo Alto Networks where they work with one of their clients, I believe, and they uncovered this attack and did a detailed report of it, and that attack used the exact attack that we're going to demonstrate in this demo and live environment. There's a QR code on the screen there. There's also that URL that you can go straight to and that will show you the article I'm talking about. And I thought that was just a very helpful and timely use case for this. So our scenario is going to be similar to that attack that I just mentioned, but it's going to have some differences for the sake of demo. So for our intended use case, let's say that you're a support developer, you're working on a team with access to a non-sensitive S3 bucket or maybe multiple S3 buckets, and you're accessing that through an IAM role. (07:36): So you're assuming the role to be able to access whatever you need to access for your job. Because you need to be able to create more buckets at any point in time and change those settings for your support function on those buckets or on new buckets without constantly having to go to the teams like the security team that's already too busy and having to request permissions, your manager ends up giving you the ability to update your role policy through the attach role policy action, which is the attack technique that we're going to exploit. This is actually a fairly common example, especially in development environments and things like that where you don't want to tighten up restrictions too much on your developers. They need flexibility, they need to be able to innovate, and they don't want to get bogged down with always asking for permissions. So this is a fairly common scenario, and this is a common scenario that can be done very securely unless it's misconfigured. (08:29): So in this case, unfortunately, there are a couple of issues. A, you are using an access key still, which is a long-term credential that you should not be using anymore and unfortunately ended up getting leaked through environment variables. A common example is either leaking through end variables, like environment variables or you're committing to a GitHub repository, maybe you got phished, maybe you have malware. Those are very, very common ways that these access keys get compromised, so that's going to be our entry point today. And then also the second mistake was that your policy that was granted to you to give you that ability was misconfigured, and it allows permissions that were not intended for your user, but unfortunately, your manager did not realize they did that and you didn't realize it either, but the threat actor ends up discovering it. So the attacker is able to assume that support role. (09:21): And that role has a policy that grants the attach role policy, which is a very valid policy, especially in development environments where you want give your developers that flexibility. But the threat actor is able to use that attach role policy that was misconfigured to then attach a managed policy called Amazon S3 full access. So if you use AWS, you probably know what this managed policy is and roughly what it does. But this is a policy that's created by Amazon Web Services that's managed by them, updated by them, et cetera, and it's available in every single AWS account. So whenever you spin up an account, you have access to those types of managed policies by default. This one happens to give you, as the name implies, full access to the Amazon S3 service. So probably a little bit overly permissive, and unfortunately, the threat actor was able to attach that role, or excuse me, that policy to the role because of the misconfigured attach role policy. (10:18): So then, once they've done that, they assume the role and instead of just being able to access those non-sensitive S3 buckets that we mentioned at the beginning of this scenario, they're now able to access buckets that contain sensitive customer data. Now in this scenario, I say that the S3 buckets are in the same account. Ideally, you're not doing that ideally, you're storing sensitive data in completely separate accounts, although we see a lot of organizations that don't do that still, but in this case, it could also apply to cross-account permissions. You could still see this being an issue with cross-accounts. So it's not necessarily that you put sensitive customer data in separate accounts and you're 100% safe. This could still be leveraged for cross-account as well, but in this case, we'll keep it simple and we'll demonstrate in just the same AWS account. So that's the background context about our scenario. (11:09): Now, let's actually jump in to the AWS CLI and let's demonstrate this attack. So like I said, the technique that we're demonstrating today is attachable policy. This is just a hands-on lab that we have demonstrating this type of attack on cyber. And so I've just gone ahead and I've launched the account. The reason I'm showing you this is because, remember when I said if I go back just for a second, remember how I said that they were compromised access keys, through the environment variables? Well, this is essentially mimicking that. So even though I'm showing it as a lab environment, just pretend like I am a threat actor and I found your GitHub, and within your GitHub you had this access key ID and the secret access key. So then what I could is I could just go AWS configure and then I can pass in those credentials and I'll finish configuring like that. (12:01): So once I do that, now we're at the stage where I'm the threat actor, I found these credentials, I've got these credentials loaded in my CLI, I'm now ready to enumerate, excuse me. And so the first thing that we'll do is in enumeration, is to try and find a lot of information about who I have access as or what I have access as. Maybe I'm a role, maybe I'm a user, and then what can I do with these credentials? And also of course, are these credentials still actually valid or did they expire or get rotated? Or something like that. So that's going to be one of the first questions that we'll want to answer. And so a common way that attackers do this is, is with the call abs-sts-get-call-right-initity. There's nothing wrong with this call necessarily. You can use this on a daily basis. I use this on a daily basis for legitimate purposes. (12:53): So a lot of times people are like, "Well, if attackers use this consistently, why don't we alert on it?" And unfortunately, if you do that, you'll probably get flooded with alerts. So this is a very common call that many of you are probably using who are in the audience right now. But the reason that an attacker would run this as one of the first commands, it's because it tells you exactly A, I have valid credentials and B, here's who I'm authenticating as, which in this case is a user. You can ignore this part here that's just part of the lab, bu-t my user is going to be support Mike. So remember, I'm a support developer. Now that I know that I have some valid credentials and that I'm a user, I can try to enumerate what permissions I have. There's a number of different ways that we can do that. (13:34): But for example, as a starting point, I can do list, user policies just like this, and then I can pass in my username by just copy and pasting this. Now keep in mind, I'm going to skip a lot of enumeration steps here. Obviously you could run automation scripts and stuff like that. You'd probably need to do a lot more than what I'm doing because this is a black box environment for you. But for the sake of time, I already know a lot of stuff, so I'm skipping through some of the extra enumeration we would do and just kind of go into the meat of it. So I could run list user policies, I see there are no policy names. That's fairly common, a lot of times you don't want to attach policies directly to users. Instead, you attach them to groups. So what you would want to run instead, is I AM list group policies just like this. And then what you would want to type in is your username that we just copied and pasted here as well. (14:25): Actually, sorry, meant to say group name. So first we'd want a group, I skipped a step. We can't list group policies if we don't yet know what group we're part of. So we'll do list group for users and that's when autocomplete comes in really handy. And we'll do dash username and then we'll pass in username. Of course, I need to know what group I'm part of, if I'm even part of a group before I can list policies for that group. So, I would go through and I would find the group name here, and then I can run that prior command that I tried to run earlier. So we'll do group name and then passing that information. So of course that tells you, hey, there's a policy that's attached to this group, but what is within that policy? I don't know yet. So I need to type in a command, like get group policy. (15:18): Of course, every time I'm live like this or I'm trying to talk and type, I cannot multitask, so I always just type over like crazy. So in this case, we're going to do get group policy to retrieve that policy and then we'll pass in group name, which I think I still have copied. And then we need to pass in the policy name itself, so we'll go like that. Cannot be found. So we've got get group policy, get group name and policy name. What did I do wrong here? So we're getting group, we've got get group policy, group name. Let me make sure I'm copying the right group name. And then we'll pass it in, policy name. And this is the group policy. (16:16): Okay, I must have had to type over something, I'm not entirely sure what, I'll have to go back through and take a look at it. Like I said, I cannot multitask, you can ask my wife. I just am not a multitasker. So now we're retrieving that policy from the group itself. I'm just scrolling through so we have all the texts on the screen and we can break it down and take a look at it. Like I said, I'm skipping some steps here, because I need to get to the punchline, but this is just giving you some basic permissions where you can do what we're doing right now with enumeration. What we're really interested in this case into is A, we can list roles B, we can assume a role. So I gave you the scenario, you already knew this ahead of time, but as an attacker I wouldn't know that yet. (16:54): So I'm just now discovering that I'm able to assume the role of support role. That's important information because now I know that there's this role in this account and I have that ability. Also, in addition to that, I have the ability to attach role policy to that very role. And there's a condition in here that tells me that I'm allowed to attach this Amazon S3 full access. Now that's just for our lab, just to restrict things and give you a sandbox. In real life, you'd probably wouldn't have this. You may have an asterisk for all policies or you may have zero conditions at all. So in this case, you can cheat a little bit because it tells you exactly what policy you're able to add. But in the real world, you may have an asterisk here or nothing at all. (17:36): But, long story short, now I know that there's this role, I know I can assume it and I know I can attach stuff to it. So I can go straight to listing roles in this account and instead of actually listing potentially hundreds or thousands of roles in here, I can query for that exact role name because I saw that role name within that policy. Now this one I sometimes get wrong. I forget if that's the correct syntax. I think that's right. We're about to find out. So I can do AWS IAM list roles. I can filter for query of that support role name, and I'll get information back about the role name, the ARN, et cetera. And importantly, also, I get back the trust policy. So within this trust policy, I can confirm that I'm allowed to assume this role because this is my principal ARN, this is my user. (18:24): You may have multiple users in here. You may have other types of principles, but that confirms what we saw in the prior policy that I am indeed allowed to assume this role and perform actions through the role. So the next thing I want to know is, hey, what can I do with support role? So we would do the same thing. We would go through and we would enumerate some role policies for that exact role. So we'd pass in role name, which is going to be this support role. (18:52): We can see that we have a policy name of access, not sensitive buckets, very descriptive name in this case. And so we'll do, get role policy and then we'll pass in the role name of support role, and then we'll pass in the policy name of access sensitive bucket. And this will tell you exactly what you have access to. You can list buckets for a bucket called cyber operations data. You can also get objects for cyber operations data, and then you can list all of your buckets and get bucket location, which is a common permission because otherwise it breaks things in S3. So a lot of times you'll see that and have that ability to list other buckets even if you don't have access to those buckets necessarily. So, recapping, I know I can access a role. I know what the role can do. Let's go ahead and assume this role and let's get started with this privilege escalation attack. (19:44): So we can do something like AWS Assume role. If you are working in AWS day to day, you've already seen this command, you probably know exactly what this does. But this allows you to assume this role by passing in the role ARN, which we enumerated above here. So we'll pass in our role ARN that we collected before. We'll set a role session name of let's do Access. We get back our credentials, and so now we can configure our CLI for those credentials. We can do AWS configure profile support role. We'll pass in the access key ID, the secret access key fill in the rest of that. And then of course we have to configure and set our AWS session token. So we'll pass in that session token that you see here. And then we'll set profile of support role. And so now we can do AWS STS, get caller identity for that profile of support role. And we can confirm that we've assumed that support role. (20:43): So everything is normal, no progress has happened yet, this is all standard, this is all day-to-day stuff. Where we start to get a little creative here is we can actually run S3 LS and then we can pass in that bucket that we know we have access to. So I can pass this in just to confirm that my permissions are right. So we'll do support role, and of course, it's three LS not L3. So now we can list out content in this, we can see that we have a bucket. We can see that we have access to that bucket and we're able to list stuff within this bucket. We can also do AWS, S3 LS Oops, I forgot to pass in profile. (21:28): And now we can see that there's two buckets. So we have access to this one. I want to try the same thing for that customer data one. So we'll modify this to slash slash and then it was the customer data one that I wanted. We get access denied. So we are able to access cyber operations data, we are not able to access cyber customer data. I can also remove the profile to show you that the user also does not have access to this, and again, we get access denied. But because I've enumerated and because I know that I can attach that Amazon S3 full access policy to this, because I have the attach role policy access, all I have to do to execute this very simple PrivEsc but very dangerous PrivEsc is I can just do AWS IAM, attach role policy. Then I'll pass in my role name of support role, and then I'll pass in the policy ARN, which was up here. (22:24): And I'm going to copy and paste that because I do not remember the syntax. So it's this, where is it? It's going to be this one right here. So I'll pass in that managed policy ARN of Amazon S3 full access for our support role. I'm not using the roles credentials, I'm using my user credentials. But now if I rerun that LS command using our role assumption, so if I'm running this one here with profile support role, all of a sudden I have access to customers.txt and socialsecuritynumber.csv. I'm not going to download them. If you want to go through the lab on your own time and download them and see what's inside and capture the flag, go for it. But the fact of the matter here is that, obviously within minutes, because I know the environment, it would take longer if I'm a threat actor, not necessarily though if I have automation. (23:14): But within just a few minutes, I went from not being able to access an SQ bucket that I was not supposed to have access to, now being able to access that customer data and then exfiltrating that access data or that customer data, excuse me, since I also have download permissions. I don't just have read permissions, I also have write permissions. I can download the objects, I can exfiltrate them, I could encrypt them or delete them and then leave a ransom note. Literally just because I have that attachable policy, I am permission that had no boundaries, that had no restrictions, that did not restrict what this user and assuming the role is allowed to do. So it's very basic, a lot of times when I show this, people are like, this is so simple, "Make something more complicated." But the fact of the matter is you don't need to make something more complicated. Thi-s is not necessarily nation-state-level stuff. This is stuff that a lot of people know about now, especially threat actors, and they can attack your environments using this if they can get that initial entry point. (24:13): It's that easy, and it can be that damaging. Because if this were a multimillion-dollar company that I just went into and they had all that customer data, this could be worth hundreds of millions of dollars in fines and data breaches and so on and so forth, literally within just a few minutes. That's how dangerous this type of PrivEsc can be, and that's how easy it can be to attack if you know some of these attack techniques. So that's the demo, that's the attack, that's the red team side. Now let's actually put on our blue team hats for a second and let's talk about what do I do about this? Great, "You showed me something very scary, you showed me something very dangerous. How do I take this back to my company? How do I take this back to my team? And how do I prevent it, detect it, and fix it?" (24:56): So let's talk about finding potential PrivEsc paths, like what we just did here. I want to give you a few different options. By no means is this comprehensive, but as a good starting point, I highly recommend checking out the AWS native solution called IAM Access Analyzer. If you're not already familiar with it, this is a great way to get started in achieving or trying to achieve at least simplifying least privilege. As we all know in the audience here, getting to true least privilege is extremely difficult. And so something like this, a service like Access Analyzer can be very helpful in paring that down. Another thing it can do though is it can help you identify external access. So for example, if you misconfigure an S3 bucket, or if you have somebody that's creating a backdoor to an S3 bucket, something like Access Analyzer can help you detect that, find it, and remediate it. (25:45): The thing with Access Analyzer though, is it's a great starting point for this, but it is fairly limited in terms of feature set. However, I do see that AWS is launching more and more features with Access Analyzer. So it's definitely something that's worth considering. And we're actually going to talk about one of the features that they recently launched at the prior re-Invent or two features actually that are very helpful with this in just a minute. Of course, the next thing you can do is you can pent-test your permissions. Whether you have an internal pent-test team or you engage with an external team, you want to make sure that it's a team that's fairly well-versed in cloud obviously, but that also understands these Proveska attack paths, and they can help you find a lot of these types of issues. And of course, you can run security assessments. (26:24): You've got open source tooling, things like Prowler, etc etc, that can be very good at finding the low-hanging fruit, which is a great starting point, but it won't necessarily find some of the more complex attack types that we'll talk about in just a minute. So again, like I said, I'll find low-hanging fruit, but not necessarily everything that you could possibly find. This is such a growing problem and a common problem that we're seeing a lot more tooling being developed in this area, both commercially and open source. And so for example, some of the ones that, I know the founding team, I've worked with them before, they know IAM very well, are going to be K9 security and CodeShiled, and those are just two examples, there are many others as well. They can look at that sort of solution that can help you with finding some of these PrivEsc paths. But finding them, detecting them, understanding them is very different from actually preventing the attacks. (27:13): So once you find them, how do you prevent these attacks? Or even without finding them, how do you prevent them from happening in the first place? One of the best ways, and I'm seeing more people using permission boundaries, but I think still it's a topic that some folks struggle with a little bit, but a great way to prevent PrivEscs and the exact PrivEscs that I showed you today is using IAM permission boundaries. The thing is, especially in your development environments, you don't want to restrict your devs to the point where they can't do anything without having to raise their hand waiting hours or waiting days just to do simple stuff. You want them to innovate, you want them to have a certain level of flexibility and permissions without endangering your AWS environments. So using something like permission and boundaries can set limits on what permissions someone or something like a resource or role can receive and can execute against no matter what. (28:06): So in our example there, even if they try to set Amazon S3 full permissions or full access, you can have a permission boundary that prevents that from happening, even if you give them that attach role, policy permission. So attach role policy is very legitimate. It's something that you can give to your developers and should in a lot of environments, maybe not production, but in a lot of those other more flexible environments, you should be granting that, so they have flexibility. But you should be enforcing boundaries and giving them restrictions that they can't break out of. And IAM permission boundaries is an excellent way of doing that, but I think a lot of times people have a hard time wrapping their head around it. So that's something to look into. The other thing that I was talking about before that AWS just recently launched at Reinvent is IAM Access Analyzer, but Custom Policy Check APIs. (28:57): So they have check no new access to API, definitely worth checking looking into. It checks whether new access is allowed for an updated policy when compared to the existing policy that's already in place before you deploy it. And then the other API call is check access not granted. So you can create a deny list, an organizational policy deny list that says, "I never want these actions allowed for these types of policies." And so before you deploy a new policy, you can call this API and it will tell you whether you are going against that deny list or not. So these two calls are fantastic things to add to your CIAC pipelines because they can block deployments from going forward if anything like that is detected. Of course, you can also use SCPs. I'm not going to go too in-depth with SCPs because we did do a free workshop with the Sonrai team, with the CTO of Sonrai actually. (29:51): Sandy, you did a great job with this presentation, helped me with it as well, give me some great tips for it, and this is a great way to block organizational unit or AWS org level or account level access. Whereas when we talked about permission boundaries, that's going to be more at the identity level. So definitely worth looking into. And there's a QR code for the free workshop if you're interested. Another great option that you may not be as familiar with is CIEM, not to be confused with a SIEM, very different purposes, but CIEMs help inventory identities so you can keep track of that over time. They can help reveal effective permissions, which can be very hard to wrap your head around when you have hundreds or thousands of identities and permissions going on. (30:31): And then also it helps you identify the pathways between your identities and your data, which again is extremely difficult to do at a large scale. So a CIEM can really help you with that. Which means they can help you catch more complex attack paths that you may not be ab-le to get with just a basic security assessment scan, for example. Otherwise, something else that the Sonrai team just recently launched, and I partnered with them to create this cheat sheet that I'll have a link to at the end of the conclusion there. They have their cloud permissions firewall, and I bring it up because A, it can help you implement important STPs that we just talked about. It can also do things like quarantine zombies. So if you have resources that are just collecting dust, it can help you quarantine those resources. (31:13): It can help you log down unused services. So even if somebody is able to PrivEsc, they can't just launch some sort of feature or resource or whatever in that service that you never otherwise use. Of course they can help limit excessive privileges, which is important for preventing PrivEscs, and then they can implement something like permissions on demand or POD where maybe you do want in certain types of environments to have your team have to request that additional permission before the security team or your manager or whoever will grant that access. So that's super helpful as well. (31:46): So I know I'm running a little bit lower on time here, so we'll go a little bit shorter on the detection side. The reason I'm also going shorter on the detection side for these attacks is because honestly, talking about logging, monitoring, incident response, threat detection, stuff like that is a topic we could talk about for hours on end. So instead of giving you a bunch of recommendations, what I'll do is, I'll just ask you a few questions, and those are questions that I want you to keep in the back of your head as you walk away from this presentation. For example, I'm sure many of you know what CloudTrail is. If you're using AWS, we know it can collect events and a lot of data, but are you actually doing anything practical with that data? Is it just going into S3 collecting dust, or are you ingesting that? (32:28): Are you doing an anomaly detection? Are you looking at maybe ways that AI can also help with that? I know it's still in the early stages, but it can help you do stuff like dashboards or anomaly detection. So there anything you can implement that can help you with that. Do you have any sort of alerting in place? So if somebody is trying to do some of the stuff that we looked at today in production environments especially, but also in some cases in development environments, do you have any alerting in place that can alert your security team to look into it? And more importantly, how regularly are you testing those alerts and incident response playbooks? I actually did this recently for cyber. I was looking at a potential attack path and I'm like, I don't know that we have a good thing in place to be able to handle this type of attack, so let me try to run a scenario and let me try to run a response playbook and see what happens. (33:15): And in the process of doing that, I discovered that I just did not have good visibility or an effective response to that type of attack. And so that got me thinking like, what other solutions, either custom-built or ideally something else that's already built and you can open the box and use out of the box that I can implement to get that visibility, that extra visibility for that attack path and to more effectively and quickly respond if there is an ongoing attack. So those are just some questions I wanted to wrap up with to ask you because I think a lot of times these are overlooked. We're all very busy, there's a lot going on. It can be very easy to put these on the back burner, but they're very, very important. So as a conclusion, I just want to thank you for attending this conversation. (34:00): I hope this was helpful. Hopefully you've never seen this type of attack before, or if you have, hopefully I gave some additional tips that maybe you haven't heard before. I'd love to get your feedback, and so I've got a few things here at the end. We've got cheat sheets and resources that can help you learn more. So stuff I've talked about in this presentation that you can go to that URL or scan the QR code for. You can download the cheat sheet I was talking about earlier and show it on the slide and a whole lot more at this link. So I'll leave this up for maybe a few more seconds. If you want to screenshot real quick or if you want to scan the QR code. And like Joseph said, this is recorded, so don't worry if you miss it now, you can always get it back later. (34:41): I've got two more QR codes. One is for my LinkedIn profile and one is for our website. I put them on there in case you're interested in checking it out, but also in case you have any feedback for this presentation or any follow-up questions, I'm very active on LinkedIn. Here's my email as well. And we have a contact form on Cyber.com. If you want to reach out and ask any questions, any tips, tricks, anything like that? I've definitely gotten burned a few times by PrivEscs, especially because of the nature of that we offer where we're basically inviting people into our environments and we're saying, "Hey, here's a vulnerable environment. Go have fun but go have fun within our bounds." (35:17): It's something that I've gotten wrong many times before and it's cost me some money. So it's something where I very passionate about because I think we need more content... And we [inaudible 00:35:33] reach out. I'm a very approachable guy. Again, thank you for attending. I really appreciate it and I hope this was helpful. I think we've got time for questions now, so I'll hang around here for the next nine or so minutes as well. Joseph Barringhaus (35:44): We do have some questions. Christophe, there's a couple in the Q&A, at least one or two in the Q&A. Throw some in the chat as well for Christophe, anything privilege, escalation related, and AWS related throw it. And now Christophe, my first question, which is not listed. Was it or was it not a crypto miner that got access to your environment? Christophe Limpalair (36:03): Yeah, so funny story, I'll try to keep it short though, is I think this is back in 2017 or 2018. IAM was good, but it was nowhere near as mature as it is today, at least not in AWS. And so we had a lab where we wanted to teach people how to use auto-scaling. So launching EC2 instances, building them out, shrinking them back in, et cetera. And IAM had no way, believe it or not, to restrict what instance type you granted access to. So if you granted access to auto-scaling, they could launch any instance they wanted. Literally as long as it's available to the account, they could launch it. And some people knew that, so they would go into our environments and they would spin up the largest GPU instances, which are laughable now. They probably don't even exist anymore, they'd be so slow. (36:51): But back then it was just so expensive. And so they would launch a dozen of those, the maximum amount you could launch in the account. And of course they would do it in the middle of the night. So I'd wake up the next morning, alerts blaring and everything, and I probably should have had them wake me up, but wake up and it's like thousands of dollars in billing. And so we had to build custom solutions. There were just no way to do that in IAM. We reshot to AWS, they ended up fixing it a couple of months later. So they're fairly responsive to that, which is great. But yeah, it's come a long way and I've definitely gotten burned a few times for sure. Joseph Barringhaus (37:22): Yeah, you're about to get your pager duty alert, that's everyone. Everyone's biggest nightmare is in the middle of the night here in that alert. Yeah, hope you got your bill refunded. Same. Awesome, thanks Christophe. Sorry, a question in the Q&A section here. Main issues are cross-account or cross-cloud. What's the best way to defend against those? I'm assuming this is in reference to privilege escalation. Christophe Limpalair (37:45): Cross-account or cross-cloud honestly is very similar to the other recommendations that I would have. As long as you're looking at it and starting higher up and just breaking it down into manage manageable chunks. So a way that I typically try to think about it is, if you're dealing with a production environment, just be a lot more restrictive about that regardless of whether it's in that single account or multi-account. Just implement those SCPs, implement permission boundaries, lock things down super tight. In the development environments, try to look at it in terms of which tools can I implement so that as we add more accounts or as they shrink or whatever, we still continue to have that visibility. So when I was talking about, if you're doing a pen test or something like that, it can help you find some of those issues. But try to find or build tooling that scales as your environments scale. (38:38): And so that's why you see a lot of tooling like CIEMs and things like that, that are coming out because it does get exponentially more difficult as you add more accounts, as you add more identities and everything like that. So I would say that the same tips apply. Most of the resources that I shared about in this presentation can be used multi-account. You can use stuff like IAM Access Analyzer multi-account, it can be deployed that way. If you're using AWS organizations, you can use CIEMs across multiple accounts. Some of those other tools that I mentioned, like Nine Security, et cetera, I'm pretty sure those also work with multi-account. So a lot of what I try to give advice on doesn't just apply to single accounts because I run multiple accounts, I run dozens of accounts. So a lot of the advice I give for that is definitely multi-account friendly as well. Joseph Barringhaus (39:24): I think what you call that too, if you just need to see your problem access analyzer, we've heard it a few times today is a really good way to see and whatever your primary cloud is, that one problem. If you try it to start expanding beyond multiple clouds, that's where you start looking into getting a solution that gives you that visibility across. The other piece too, like we had talked about, you mentioned the firewall. Thank you for the subtle plug. The firewall is meant to take action on those things. So you don't have a ton of alerts, you go solve the problems instead of just getting alert on it. So you wouldn't even get your middle of the night call because it would've been blocked. No problem. Christophe Limpalair (39:53): There you go. Where were you in 2018? Joseph Barringhaus (39:56): Yeah, well don't worry. We're here now Christophe, we got you. I know Christophe won't do it other than putting the QR code on the site, but I'm going to say right now, go check out cybr.com. It is an awesome resource for everyone. A lot of cloud security tips, trainings, hands-on labs experience. Go check it out. We worked with Christophe for a while now and honestly, I've done a bunch of cyber trainings myself too and certainly enjoy them, so. Christophe Limpalair (40:19): Thank you. I appreciate it. Joseph Barringhaus (40:20): Like I said, go blow up his LinkedIn. We just connected just now even though we're working together for a while. Not that I'm upset about that, but this was an awesome session, Christophe, thank you so much for being here. Privilege escalation. You just saw it's real, it can happen really quickly. And always if we don't get to your question or if you have a question later, please reach out to Christophe on LinkedIn, send us a message, our team is happy to chat as well. We have an amazing session lined up next as well called JIT versus long-lived roles. So we're going to talk about just-in-time and just-in-time provisioning versus long-lived roles, pros and cons of both. Certainly plays into the privilege escalation conversation we just had. It's going to start here in just a second. Christophe, any final words for the group here? Christophe Limpalair (41:04): No, just happy learning and hopefully this was helpful and go ahead and apply it. Take what we just learned today and go apply it and let me know if I can help you with that. But thanks for having me on access, I really appreciate it guys. Joseph Barringhaus (41:14): Absolutely. Keep an eye on the chat here in the just few moments. Open those up. There's a couple of blogs on privilege escalation, all of the content from Christophe as well that he shared today. All of the links are found in the link in the chat as well. And of course the policies that we shared throughout the event as well. So, if you have any questions, feel free to reach out. If not, go grab a cup of coffee. Stay tuned, you will automatically get shifted to the next session as soon as it is live. So be ready, we're going to get started here in just a moment. Thanks so much, Christophe. Christophe Limpalair (41:41): Thanks. Bye.