we literally just waited for somebody to come to the back alley of this bank to open the door we walked up in a shredman uniform of the company that managed the shred bins we had found a shirt at a thrift store that looked like the same color it was a button up and we made our own patch for it we walked in said we were here to pick up the shred bin and we just walked out and put the shred bin in our truck and took off and inside the shred bin hadn't been any anything shredded and had a bunch of passwords on sticky notes and all this kind of stuff and so immediately we got like more access than pretty much we had gone through the fishing campaign and the external and stuff like that that was this week's gift Jason hedex casually explaining how him and his team managed to hack into a bank this episode is truly awesome and you do not want to miss it Jason is someone that commands so much respect in the world of penetration testing and security and I'm thrilled that he decided to sit down with Dwayne and my myself to talk about his journey Jason's hacking Journey started off as a bit of a miscreant teenager before honing his skills and truly becoming one of the best penetration testers in the world and even getting that coveted number one spot on the bug crowd back in 2014 but Jason's done something that a lot of penetration testers and hackers haven't been able to do and let's move all the way through into the executive table and boardroom of some truly massive and awesome companies Today Jason is the sizo and hacker in charge at battlebot butterbot is an awesome company it's a veteran and minority-owned organization that specializes in emulating real world attack with a truly world-class team if you want to protect yourself against attackers the best form of defense is to hire those attackers and let them show you where your weak points are and that's exactly what butterbot does before battlebot Jason was a sizeo at Ubisoft and has some pretty cool stories to share about his time there and he's also been an executive and held many leadership roles in some pretty awesome companies along the way in this episode Jason shares with Dwayne and myself his journey from becoming a hacker to becoming a sizo and she has some pretty ludicrous stories about his time along the way I can't wait to get into it so without further Ado here's the conversation that I had with Dwayne McDaniel and Jason haddocks the hacker in the boardroom what made you start to kind of get into cyber security what made you start getting into this this field and kind of how old were you when it all when the bug hits you the security bug I've always been a little bit of a hustler I would say even even in you know grade school and high school and um uh you know in my early 20s I actually did a dark net Diaries episode about some of this but my early 20s I was trying to make fake IDs basically I bought one from a friend of mine to just get into bars before I was 21 and um and I got it and it was horrible and obviously didn't pass and I got busted at the door of uh you know one of the bars I tried to go to and I got taken away I didn't get in trouble or anything like that they just took it away but uh but I got really mad at the fact that you know it was so crappy and so I ended up um kind of on what was you know before the dark net you know the Forum ecosystem for uh you know Carter's and people who made fake IDs which back then was um you know counterfeit library and Shadow crew and stuff like that I ended up trying to make my own fake IDs and so I ended up doing that just for me and my friends and um that kind of put me into this world where you know there are several ecosystems there there is the ecosystem people who are making fake IDs they're the ecosystems of people who are making fake credit cards and they're the ecosystems of the hackers who are hacking you know what they call the dumps of the credit cards and it still works this way today um and the hackers Supply the Carters you know to put the cards on the fake credit cards and then the Carters need the IDS to cash out at retail stores to match the credit name on the credit card that they make that's fake and so is this ecosystem and so I got to know a lot of the hackers and I was like wow you know this stuff is really cool this was you know back when web hacking was just you know getting you know kind of like popular and um you know Network hacking had been predominantly the you know the um the industry at that point and so you know that was I was getting exposure to it I wasn't really like super interested or anything like that and then uh Shadow crew the main Forum site that I was you know on got busted and taken down by the FBI and the Secret Service and it was a worldwide raid on uh with like Interpol and like the Canadian mounties showed up in Canada and like basically took down all the uh the highest ranking members of Shadow crew and uh put them in jail um and it was a big thing it was the first big bust of one of the dark net forms that had ever really happened um they continued that pattern today that you know like one will get too big and they'll bust it down and um but I wasn't like you know I had sold you know like a couple IDs or something like that on the Forum really it was it was more like an art project to me right like I found the challenge of making these things like um like really interesting and you know had some mystique and danger to it but after that um I was taking the semester after all that happened I was taking a uh College elective class and or I was looking at my electives list and there was a a class called ethical hacking and network defense which is the first time I'd ever seen anything like that I'm like oh go check it out because I was doing Tech stuff I was doing Cisco stuff and so I took this elective and my teachers like teaching this stuff and I'm like this is old already compared to what people in the forums were doing this is years old um and this is not what modern hacking looks like at least from my exposure to it and so uh I talked to him a lot and he was like he's like you you're very knowledgeable on this you could do this for a living and I'm like what do you mean you could do this for a living he's like there's this thing called penetration testing and you basically act like a bad guy but you are a good guy and you get paid a lot of money for it and I was like yo that sounds epic and so ever since then I've been hooked and I went on a tear and I just I mean I beg borrowed and stole everything I could on Modern penetration testing and made friends in the industry and you know I've been in it for 15 years and offensive security um and I've never let it go even when I transitioned to being a security leader it's always been in my blood is hacking that's an awesome back story literally to be part of I mean it's not a good thing but yeah like an early raid like that and for something so International and yeah honestly famous um maybe not in the best way Infamous probably the better word there um that's such an interesting path and it's interesting to say that about Academia being behind it it always feels like it lagged behind I was recently at a conference where it was very evident how far behind like they're still writing standards on how to migrate from on-prem to the cloud which feels like that's 2008 which actually came out um I'm very curious on the next part of your journey so pen testing makes total sense how do you jump from that seat to cso I know there's a lot of listeners out there who are wondering like how do you get doing the exact Suite in the first place but from the pin testing perspective that seems like a really unusual path yeah absolutely so um I'm a big believer in that um like you really can't be an effective security leader unless you sat in the seat of someone who's done security and I and I mean that in the sense of not specifically application uh you know offensive security but you know either red or blue or you know purple but like you can't just come in there as an exit like a ha ha executive I call them um and just lead like a security organization because you have no idea exactly what's going on you have no idea how to orient or prioritize your program and this is actually where I get into it with a lot of other csos out there because they believe that you can manage it completely from a risk perspective and um I just don't believe that's true anyway my path there um you know I did offensive security for a long time um at smaller consultancies and then I went to HP and so when I went to HP the industry was just starting to get big for web and mobile um and mostly you know dynamic testing basically and um at that point it was Dan Kaminsky um who was amazing and not with us anymore which is really sad uh acquaintance of mine um he was the director of penetration testing and Io active which back then was the most legit consultancy that existed in the scene Dan was doing you know like worldwide talks and and I just had this goal of being Dan Kaminsky basically um and and building a team of uh great offensive security people and so that's what I I did at HP I built the small consultancy inside of HP called Shadow labs and um once you know I realized that like oh building a consultancy uh is not just about testing it's about hiring and about culture and about leadership I had to learn all those things and I cut my teeth there um and so you know me and a buddy of mine ended up building this thing called Shadow Labs instead of HP that was wildly successful um we were on The Cutting Edge of mobile testing for a long time back when it was um Objective C and we had like you know I think the second binary analysis engine I wrote for iOS applications that could identify security things it was right behind varicode basically um and so it was uh it was really dope but I got to learn um a lot of things about leadership and building the team and then I've always been pretty good at speaking and breaking down really technical Concepts to non-technical people um and I'm good under pressure and so those intangibles basically kept me at a leadership spot at HP then after HP um I migrated into kind of the bug Bounty world and I took a leadership position there um and then a leadership position after that uh was my you know I managed security at bug crowd as well as some operations and then after that you know these are five year stints right but after that I went to Ubisoft which was my big like uh like corporate security ciso job um and that's a 22 000 person organization and you really learn about um you know like what a real CSO has to deal with not just like a startup CSO and I have War Stories from there and I feel like my three and a half years there was honestly 10 years um and yeah and so you know that was my path uh but I think the intangibles got me there I think I think being able to break down uh and present are two really big things I think understanding the tech at a level no one else around me did but having those intangibles on top of that really got me to where I did um you know learning about Sound Security Programs and what those look like uh you know being good Under Pressure I think all those things got me kind of into the leadership path um to this day honestly and obviously so now at butterbot right on thanks for thanks for walking us through your resume you're hired story um for real it's being able not just to talk the talk but walk the walk as they say um like knowing both sides of that it's so important I've met so many especially researchers who they know their stuff inside and out but wow they can't explain it to you at all and they can't explain the business value or align it it sounds like that your your background as a pen tester has someone an attacker um really sets you up for Success here would you say that's like I I know you already said that you prefer that in your Executives but do you think it's really a necessity to have that kind of background or can you think csos can reach that some other other path yeah I mean you can you can hire around you right like really great experts and and advise you but then the CSO job is is really just a management job which is like not the kind of Cesar job I want really um uh I want to be involved in strategy and and um you know really build a program so uh yeah I think that I think that having some technical um relative experience whether it's you know you came up you know at some point being an admin or sock or threat intelligence or testing or even a developer uh who understands the product really well who could threat model it and you know something like that I think that it's a tremendous Advantage for csos these days um and like I said it's specifically a tremendous Advantage because you can prioritize your security program in the correct way instead of just the way that every other CSO is telling you to do it right like oh this risk and this risk and you're not chasing ambulances everywhere you really understand what your business is facing why specifically your business faces that problem you know what the technical controls are that are available to you are not available to you and um yeah but you can't you can't hire experts around you who can do who can do those things too like director levels or Deputy levels and I've seen plenty of people do it that way and who just focus on the leadership and culture part and there's some amazing csos who do that but I think one of most of the ones I know actually also have a technical background so I just think it's a tremendous superpower um and this is a culture war between csos actually like um like when you talk to csos they will they will be on one side of this fence and they'll be like a technical person should never be a CSO like ever they don't understand business they don't understand how to be an executive honestly I find it easier I found it easier in my career to learn executive leadership executive management how to handle my seat at the board influence and power politics I learned all that whereas I don't know many csos who could learn how to design an effective security program at a technical level so that's a great answer I I want to dive into an area that you you touched on a little bit back that's going all the way back to Shadow Labs um and it's the questions are going to evolve around building you know effective teams because this is one of the most important things that any security division you know and the ceso can can Implement is maintaining stuff in this climate finding stuff making them feel value now the reason I bring out Shadow Labs is because I sent out uh some inspectors and spies recently and I heard rumors coming back about Shadow lab tattoos you know how do you make your employees feel like such a family and such a team and want to do to such Goods that they going to get tattoos none of the company that I work for but of the division inside that company boy so okay so I did a whole talk on this at black hat last year so um so black hat has what they call the CSO Summit it happens the day before the actual um Regular events um and so I talked last year about this experience and how I've built some of the really like tight-knit teams that I did and so at HP um I mean first of all it was it was the interview process for everybody that you know we brought on board right like we wanted really passionate people and over the course of my career I've learned that um I actually don't want to over index on the most technical people I want to over index on the people who have like a passion and drive and fire and so I have some interview questions that are specifically about how you handle situations or you know how you spend your free time and I'm not into like hustle mentality where like you need to be working 15 hours a day that's not really it I just want to make sure that the thing you're doing every day you wake up to do right um like that's like if you're a marketing person I want you waking up every day like hyped about our brand and like to build our brand and like you know if you're a tester like you know I want you to believe in the thing that we're doing and so that was like the first step right and then the next step is once you can get some of those people in the doors to train them up to be the best in the world and luckily I had already some support there I worked with Daniel Mesler at shadow labs and myself and Rick Dunham and some other really really good testers and we built out our own methodology did our own research and then trained everybody up to be at our level and uh better than us honestly eventually and so we hired good people um who had you know like good traits and then we trained really really really well cross-training training all the time hanging out with each other and then it was about retaining and so that was mostly what I did the talk about and so what we did is is you know like people have to believe in your brand um wherever I don't care if you're an internal security team at Salesforce or you're a consultancy that's out in the world trying to sell your stuff or if you're a git guardian or whatever people have to believe in your brand and branding and marketing is important even to employees and so we went out and we built a custom logo that looked dope um every year we did versions of different stickers and stuff around our logo um and uh and we made it like a Hallmark piece of our identity and um we were always at conferences presenting and you know like so our branding and marketing was really really strong for our internal team and presented them as you know the experts that they were and that's really important to employees to be like presented and like that um and so you know like that on top of like you know generous packages honestly like you know if you're if you're hiring Security Professionals right like there's you know one of the things I talked about was like what can you compromise on in the offer phase I broke it down into phases right and so you're getting these employees on board usually the stuff that's sentenced down is you know base range Equity range yearly bonus and benefits usually those have a range that are set in stone but the things that are negotiable that are really powerful for testers are the hiring bonus the PTO amount the work from home Arrangement nowadays more people are talking about four day work weeks 80 20 time of 20 research where they can go off and make tools or do presentations or whatever they want the travel and training amount the mentorship you give them um you know these type of things are the intrinsic and negotiable sections of of getting employees to to come on and then once we got on we had this great brand with um you know we had like this castle-like dude who had three points on his head um with you know like mean looking eyes and he kind of looked like the autobot symbol from Transformers and so he is the thing that we would transform every year into a different thing so when we got really big into Mobile testing we had an artist do a rendition of him exploding out of an iPhone and that was our brand for that year at Defcon and we all wore that shirt and everybody was like who the hell is Shadow labs this is amazing like look at this and you know another great consultancy that did this in offensive security was spider lab spider Labs also did this with their spider and like had different spider stuff and you have to embrace that and then we did award ceremonies challenge coins we inducted people into this like kind of upper echelon of Shadow Labs who had contributed so much to the business you know we had Regional meetups you know gaming days together we would game together cross training days movie nights like book stipends I mean we really tried to do everything to make the employees feel like a family that they worked together that they were respected by us um and then also one of the things I think is actually really great that you can do as an employer of Highly technical people is is reduce friction and so like in our world was uh reporting was the worst thing in offensive security at the end is writing the report right um and so we invested a lot of money into making a platform to make reporting as easy as possible and I can't tell you how many people were like that's amazing I wouldn't go anywhere else honestly because I love my job I can focus on the testing I don't really have to worry about the reporting um which is the you know so reducing friction in certain ways is also like um you know one of the best things and you know there's there's a lot of other stuff but I had a whole presentation on it but that's kind of the the you know the blueprint I used and it seemed to work out pretty good so yeah it's The Branding that you're speaking about you know is that branding was that like when we talk about marketing and branding and you know I'm in the marketing team at get Guardian so this is kind of like what I you know think a lot about and when I'm thinking about branding it's like outward branding I'm trying to get everyone we invest in like cool t-shirts and stuff but that's all for the purpose of like ultimately that somehow some way down the track that's going to lead to some big lead whatever when you're talking about branding you're talking about branding in a way that kind of brings an internal team together and it's like it doesn't even matter what anyone else thinks is that right like you're not you it serves both purpose yeah that idea serves both purpose so like internally it was like Shadow Labs numbers had like way cooler swag than any HP person ever had they had the logo which only they could carry around and I mean we did um Timbuktu shoulder bags backpacks patches um challenge coins that only they got for big Milestones like I said we had those award ceremonies where we gave out actual physical Awards I have a couple of my wall with the you know the um the logo on them and um uh and then like the t-shirts that were only for staff we had versions that were only for staff and then other ones that we gave out to everybody and just that kind of stuff I think you know like uh I mean I guess it worked as I said basically so yeah yeah that's awesome is I think that's such a cool idea of like that internal internal marketing to make everything did you have anyone from HP try and come over to Shadow Labs do you have people being like what is it how do I get one of those coins or those bags you know it was you know we ran Shadow labs for a couple years um stealth and then big HP got kind of like angry about it they were like you can't divert from the HP branding and we were like Hey listen it's already making money and going like get out of here like um and we were outperforming many other divisions at HP by far and so um really we got to keep it but yeah I mean as far as coming over we had a couple people come over during our tenure but we tended to grow our own Talent a lot we would we would find people who we were hungry who were new um you know who wanted to be part of this family and we would we would basically mold them into these awesome testers so yeah and so I wonder was the circle back before we um completely finished talking about the CSO role and the transition over there um there's a lot of people listen to this or fresh in their uh journey into security or uh they just simply you haven't had the chance to ask like what resources would you tell them hey go investigate this or is there any Civic courses or any res anything that they should be digging into if CSO is their end goal yeah so there's been a I mean so like the CSO job is so wide right and it depends on what type of CSO you're you're heading out to be um there's a great mind map out there that I don't know I can't remember the person who made it but it breaks down all the components of basically a cso's job into sections of a security program um if you just Google see some mind map um you'll probably find it it's like a it's got it broken into quadrants with color and I think it's it's a really good visibility of all the domains you need to have some mastering so then you need to go like at least be dangerous in a couple of those domains so as far as like resources to get you there like uh I would say there's there's a couple good books they're called The csos Desk Reference um and it's a it's a couple of cisos who um basically built three books um of their experiences and chronicled their jobs for like the last 10 years um on you know different types of seesaw roles like you know when they had to implement compliance or when they were more of a cloud shop or when they did X Y and Z and so getting an exposure to you know what a day-to-day is like building that security program and everything like that but um I mean honestly like there's not a lot of resources out there for people to become a CSO right like I think I think what happens most of the time is you have an executive who's a really good executive and gets handed the Reigns because they know that that person can you know handle a seat at the table with the board and the other Executives at the company and can work well and you know can do politics and all that kind of stuff and then they take over security and they become a CSO and they stay at career CSO and um that's a thing and they move from CSO job to CSA job usually about three years at a time and then you have the people who come up through the tech side which end up like going from you know like sock manager to Sock director to operations to security operations director then to like Deputy CSO and then to see so and that's the track that they go through is one of the technical domains so you know you have security engineering that could lead there you have sock that could lead there you have offense that could lead there like red teaming and penetration testing you have um you know even I've even seen some development leaders move up to see so uh ctOS sometimes step over to cso to um like as an adjacency um and so like one of the technical domains will lead them there but uh there's not a lot honestly out there um there's a couple podcasts that exist um ciso tradecraft is one which interviews a lot of popular csos the CSO Summits exist at like black hat and stuff like that which you can get a lot of experience from but those are invite only so I wish there was more resources to help people get to the end goal but that's kind of not how it really works right now at least you know from from my view honestly uh Jason I want to dive straight into a a topic here maybe a little bit out of left field but I've heard that you've hacked into a bank with some teams yeah how do you start when your goal is to kind of penetrate penetrate a bank what is the first star and I want to kind of come with this with like the the mind of like what what the heck is thinking how do you start by eating this this elephant and you know how does it work because we understand how you you might get initial access into into a smaller company or or deliver some malware or fish how do you start with a bank yeah I mean um as far as like a red teaming or penetration testing engagement goes like it depends on the scope right like um but I mean uh a bank is no different than any other Enterprise um and in fact in some ways is is worse or better um than some other Enterprises so uh when we uh my first job was where I had the most exposure to hacking Banks and so as part of a small company called redspin which was a consultancy that I started off in I was on the pen test team and the red team and we had a whole bunch of full scope engagements which were really fun um which some of my my stories and darknet Diaries come out of and and some other ones but um that's where I did a lot of my physical stuff and so uh you know our our basic methodology was to basically profile all of the employees at the bank um and to Target for fishing right um which if it's a big bank there's a lot of opportunity there to profile the technology they use for email and you know what type of filters might be in place and what type of protections might be in place so we could try to get around that um and then send out phishing campaigns to try to get from external to internal and pivot and install like a you know back door C2 malware or something like that eventually um when that wouldn't work uh we would do physical reconnaissance on the bank to understand what their physical security controls were you know hours of operation when employees left when people were doing like the deposit drop-offs and um you know where they held their shred bins you know where their trash was um all that kind of stuff and uh and so as part of red spin we would we would do the external and try to fish and then also look for any application vulnerabilities right we found many banks you know who had web application flaws or we were able to you know like get into things we weren't you know we weren't supposed to but you know some of the funner stories I have are like actually the physical things I remember um one of the credit unions um we were part of uh uh we literally just waited for somebody to come to the back alley of this bank to open the door we walked up in a shredman uniform of the company that managed the shred bins we had found a shirt at a thrift store um that looked like the same color it was a button up and we made our own patch for it we walked in said we were here to pick up the shred bin they hadn't locked it physically to the um to the premises and we just walked out and put the shred bin in our truck and took off um and inside the shred bin hadn't been any anything shredded and had a bunch of passwords on sticky notes and all this kind of stuff and so immediately we got like more access than pretty much we had gotten through the fishing campaign and the external and stuff like that um and uh and that's you know that's kind of stuff that can happen um you know always wear hats so that the overhead cameras can't identify you um you know dress the part have confidence to social your way in there we've also you know like as as part of those I've dumpster dived you know in the rain where I'm in the trash and it's like nasty and I'm digging through trash bags to find credentials and stuff like when you're targeting like those kind of physical assessments it's um you know it's not always glamorous you're you know sometimes you're falling through things sometimes you get caught like uh but some of the better stories are you know ones like that or we we pulled off some other stuff so yeah it's it's so interesting when you talk about it because I think most people when they think about hacking into areas that is going to you're going to be sitting behind your computer um you know trying to but actually security encompasses so many different elements and then I I guess what this the the real challenge is being able to put these together in a chain of events that's going to enable you access so you know okay you found a password on a sticky note how do you then actually leverage that to be able to get into the network or or some other areas so putting that together I'll create a how-to video after this how Jason says you should become a bank hacker I mean I mean I've actually targeted a ton of banks in my in my history through through like full scope red teaming through bug Bounty through just application pen tests and stuff like that so I mean I I you know like one of the stories that I wrote about um you know I write some of my hacking exploits I obviously nuke the names of the customers and you know uh but I talk about the and like how I did them and like you know some are as easy as like you know like um you know I disassembled a mobile app and found a hard-coded credential in a mobile app for a major bank and then wrote that password down and then was doing and you know Recon on their whole web footprint which was massive found a couple sites for that credential worked logged into there pivoted from there found you know an S3 bucket I had access to managed to grab a whole bunch of pictures of checks managed to Pivot to the internal network of the bank and then it was Off to the Races from there so like um really every every test is different um but uh one of the things I tell testers is is like always write down credentials that you find because if they don't work where you are at they will work somewhere else eventually so like don't just discard them a lot of people like will find these days in bug Bounty they will find credentials on GitHub that a developer has accidentally committed to um the public instead of the private and um they're they're public repos of the company's private repo or they've like committed their you know bash or C file or some environment variable or something like that and they've accidentally committed it to um public GitHub and people will find it and then they won't be able they don't know where it's where that username or password is referenced but I always keep those in you know my notes so that where I'm testing the web applications that maybe that employee worked on I can try those credentials again and a lot of times it works I would say like 50 of the time it works we know it's a growing problem too not a not a drinking problem but uh we just put on our state of secret sprawl report for 2023. last year we found over 10 million credentials just hanging out in public GitHub reviews uh 6 million in the year before that and that's not cumulative that's new a question about something you did recently you put on Twitter back a couple months uh not a bank but the other end of the entire Spectrum I think from one way to make at a prison you had a prison um you did a great write-up so don't really want to get into the Nitty Gritty details of exactly the exploit here because people can go read that on Twitter we'll put that link um in the description but uh I'm really curious about a little like the why more than more than the how um I know it's bug mounting and you know part of your purview but it was just like the fun challenge of a prison sounded fun to hack into or like why was that a Target uh I mean the well first of all it was a software that was specifically made for prison so it was syndicated basically to you know most of the prisons in the United States um to do a thing um that's about as much as I can say about it so it wasn't like I hacked into any one specific prison the hack the software that had tendrils into all of these prisons um and so much worse than hacking into a single prisoner sorry the heck of prison guys [Laughter] um but yeah so it was uh it was a bug Bounty program I mean like bug Bounty is an opportunity cost right because their public bug Bounty isn't private bug bounties and if you're going to spend the time you only get if you only get money or you know a payout if you find something and so like there has to be this right combination of interest in the Target and an opportunity for payout in order for me to look at it because I have a nine to five job I have three kids and you know it's it's crazy on the CSO so um so that one came at the perfect time where it was like right before a weekend um where I had some time on my hands just for myself I was feeling the itch to hack um it was a private program meaning that you know I'm not facing the whole world I'm only facing maybe like a hundred other hackers who got invited to this program and it was interesting software you know belonging to this like you know company and so that combination I was like all right I'll look at it and um yeah so I mean I basically uh took a shot and I got lucky and um you know like it's also an interesting story because you know what I don't reference in my Twitter thread is actually that the the end point explicitly that I found that had all of the um flaws in it was um not in scope for that it was out of scope but I still found it through the main site that was in scope so like I kind of had to talk to the program owner through a submission and I'm like Hey listen I found this really bad flaw um it's on this endpoint um but that's not in scope do you want me to submit it and I kind of talked through what it was but I didn't give them the whole volume they were like yeah absolutely submit it and so I submitted it and I was lucky that they accepted it and expanded the scope and basically they had this chat bot on the main page and um the chat bot they were hosting the technology for the chatbot themselves was hosted off another server of theirs um and so I followed that to its endpoints and I found a directory listing I did some brute forcing and I found a directory listing and some of these paths that basically had the keys to the kingdom in um an open directory to the internet it had credentials it had TeamViewer credentials to get into the main software components to administrate some of the servers I had videos from you know a lot of the different um prisons that had uh it was crazy it was nuts they basically had used the server not only for their chat bot but also for like their total Nas for the whole software um which which blew my mind but also I was like okay and you know and then I found some other vulnerabilities too with some other sites and um yeah it was it was a good good time that uh okay that's a wild story let's pivot away from your your hacking Adventures now and I want to look at it from the exact opposite side now you've you're the CSO now you were a CSO at Ubisoft massive companies you're being in in security for a long time and one thing that's quite interesting is that we last year we followed the lapses group uh in the year before quite closely because they were doing a lot of source code leaks of of organizations you know and so we we were we were following them quite closely now they didn't well as I know they didn't leak out Ubisoft source code but you were affected by the the the lapses you know let's call it incident so I guess I'll start off with is kind of like what what actually happened how did how did what did lapses do how do they get into ubisoft's kind of environment so lapsis was they had a couple of ttps um that were pretty prevalent for them but mostly they used credentials on companies like us and Nvidia and stuff like that and so a lot of times they weren't getting those credentials from anything they were doing they weren't running phishing campaigns um they were actually buying them off of the dark web off of places like genesis market and things like that so they were they were letting other people do Mass fishing campaigns searching through the output of mass Fishers campaigns which were mostly cred Stiller malware and then they were buying them as opportunities to hack into bigger businesses for the cloud basically um and so they uh yeah they they basically found a credential that you know you know basically what happened is over covid everybody went work from home right and a lot of businesses couldn't afford immediately to provision laptops to every employee who pivoted to work from home and so what that meant is that everybody was Now using SAS software to interface with with work on their personal computers and which had no protection basically no no EDR you know there's no no nothing um and so you know like if you'd ever torrented anything you might have already had a key logger on your machine or if you know you got hit by a fish on your personal Gmail account you know now this extends the boundary of the normal security kind of perimeter to people's personal computers and so a lot of them had already been compromised or whatever but you know uh basically those credentials end up on the on the dark web forums and then lapsis would go there and buy them for you know like 20 to 50 bucks and look for domains like and video like Ubisoft like EA like whatever and that's what would be the start of their their campaign against you is gathered some credentials um and so then they would go from there and they would use the credentials on common uh well it's not just credentials too they sell cookies as part of those packages too session session and variables basically our session identifiers and so not only do they have this credential to log into your systems which normally would be kind of okay because everywhere you should have two-factor authentication but they have the cookies and the cookies bypass two-factor authentication if you don't have a bunch of security control setup for like impossible travel or all these fancy things um and so they would Target things like slack and um and Mike and Microsoft 0365 and uh try to get into those using these cookies and they would just paste the cookies into their browser and then try to log in as the person and a lot of times they would get into slack or they could get into the web portal of Office 365 of the user because the cookie was already there and then a lot of people tie their um they tie their access to their internal Network to a VPN and they tie that to the SSO that Microsoft provides which uh is your Microsoft credential and so they have that already because it's the same credential as the portal for office 365. so now what they need to do is get past a VPN two-factor authentication prompt if you have one set up many people didn't two years ago now they know better to have two-factor authentication on your VPN um and so if you didn't have that that or if you had two-factor authentication then they would do a couple things they would um they would attempt to change your two-factor authentication device in the Office 365 portal because they had the cookie and they were in there already and they could change it to their own phone and then they had access to get in if for some reason you had security controls that didn't allow them to change the two-factor authentication token they would just bomb your user with request after request after request you know at 8 pm at night and the employee would just be like what the heck is going on with my phone they were getting these push notifications for like let me in you know let me in and they would push trust eventually and they would get in and their last method um would be to Vish the person so in the case of uber they basically uh found the user's WhatsApp number called them on WhatsApp said that they were Uber security they were doing some integration testing and they needed to click the trust button on the phone prompt they were getting and then that got them into um the network um and so that's how they got into like the corporate land in many of the instances um in cases where that path didn't work they would go to slack and they would basically root around and slack as the user because they had the cookie bypassed everything and they would look for um they would look for credentials that were hard-coded in messages to other people pinned in channels uh you know posted in documents that were inside slack and invariable any company whose slack has done that before you know is pasted a credential and they would gather all these credentials and documentation until they found another Avenue into the local network it sounds like a very familiar story that you just painted because you know we broke down what happened to the Uber and all this it's that that thing you buy the credential I'm amazed at how cheap they are like these credential packs obviously they're selling it to multiple people but that's uh it's it's wild how that all works and then just kind of you know just keep trying until they till they get in what what happens you're the CSO you've got the alert okay we think that we have a security incident what do you do like what how does that day go when you're the CSO and you've just been told that a hacker could be in your system it's not a great day I would say uh yeah so I mean my specific story with that was um you know like you get a call very early in the morning and at first it's you know it started off as like we think there's an issue but it wasn't really necessarily a security issue but it was a you know a different type of issue a downtime issue and uh everybody's looking into it so as an executive you get on board you get up you get your coffee and um and then eventually you know you figure out and this is for any breach right eventually you figure out through you know collaboration with it and your sock investigating you know you know an issue of that magnitude that oh it's a security issue and then you're kind of like oh crap here we go um and so you you basically rally your team to uh you know stop working on everything that they're currently working on and we're heading into kind of Investigation incident response mode for you know this thing depending on how big it is like you know small one person you know ransomwares probably not going to Institute something like this but anything that causes downtime or significant loss or the attacker still running amok in your network or any of these other things it's kind of all hands on deck and so um you know really we had a very well uh set up um kind of cabal of the of the executive leaders in I.T and so we would get together and discuss our strategy I would kind of verse them break down what we knew was happening versus what we thought was happening you know it's it's really important that phase to understand that there are things that you're assuming that there are things that actually have taken place and so break it all down for them and then talk about what your proposed strategy is and they'll have a lot of context around you know I.T and other systems and even you know like PR be in there sometimes to like you know when you have to do a notification legal will be in there and um yeah and so we got together and then after that we basically battle plan you know what the response would be and this is in conjunction with your sock already and threat intelligence already working on the problem right going into their methodology for you know finding out if there's still exposure and things like that but um yeah I mean it's it's really different in every you know instance because in this case the lapsis prerogative was to bring down the business very few times did they ask for ransom to any of their victims that I think it was only two times out of all their victims they asked for a ransom really they were in it for the clout and to bring these companies down and so um you know like I I went on a tear after this because I was so pissed that you know like we had a great team at Ubisoft fantastic people we had a great leadership team we had um we had adequate budget for our size and we still got breached in this way and so did some really really big companies I mean even Microsoft got breached by lapses yeah yeah in Nvidia Microsoft Samsung videos these are all what companies that I would consider have great security posture yeah yeah and so I basically picked up the phone after we were out of incident response mode and we had kind of like worked to recovery and started calling the csos of all of those places that you just listed and other people I knew privately who had been affected the lapses but weren't in the news and um I think I must have called about 40 45 csos um just because I I knew them from slack and was like Hey I just want to get on the phone with you talk about this and we started sharing threat intelligence and where our gaps were for you know this particular type of threat actor and we just realized that like so there were so many blind spots that we had not been prioritizing and it's kind of changed the way I look at security Security Programs and security leadership since that incident honestly it's um it's really changed the way I look at stuff so so you're telling me there's a secret size of slack channel for all the sizes I got how do I get a there's a multiple actually yeah I want to find a cookie for that slack Channel so your boat about now um what what is that what's that all about what's the high level yeah so butterbot is an adversarial emulation company um so basically we're bringing kind of red teaming to the mid-market uh full scope red teaming um minus physical but basically what I learned from the lapsis incidents and and kind of my bug Bounty experience is that um is that the security industry is not addressing a lot of the real problems they're addressing little point in time problems and small things uh but people are still continuing to get breached and you know I'll talk later about kind of the lapsis incident and stuff like that but um we basically built a service that emulates a real attacker so we you know we do this year-long service um where my team and I go out and act like an adversary and we will go to the dark web and look for credentials and try to use them against you and pivot internally and so it's really kind of Cutting Edge red teaming stuff and um that's kind of my jam right I really love offensive security and I've built out some great methodologies and so uh we do you know full external attack surface management too we profile your whole company it's not like we're just doing a pen test against one site it's unscoped and really at the end at the end of the day it's to answer the question can you get breached that's the one thing we care about in this service and um it's a year-long service and so it's called continuous adversary emulation and so that's what we're doing right now at butterbot with um with a few customers and the government and it's really been kind of a little bit of a shift in how testing takes place in offensive security a really holistic sounding approach to actually improving your security yeah exactly yeah yeah thank you thank you I didn't want to just open the floor to you is there anything you just want to tell us in the security Community anything we should be aware of on the horizon any just general advice you think should be floating out there I promise this is not like a plug for for you guys because you work at a company that deals with Secrets but one thing I learned through lapsis um is how important a Secrets Management program is and it's one of these things like I said I've I learned a lot from that that breach um and and I feel like a lot of people are only addressing certain parts of a seek of a comprehensive Secrets management program and so what I would say if you're an organization and you want to um yeah I mean you we want to avoid you know kind of the lapse of style you know attackers and you know kind of how they pivot because I only told the external internal story there's a whole nother story about what they did when they got into the network um which involves a lot of uh Secrets management components and so uh the way I've broken down Secrets Management Programs now that I I build places and I consult my other ciso friends is is in four phases and if you're a security practitioner or you're a leader or whatever like I would suggest trying to emulate this because it's it's been kind of passed to those 40 seatas he says I talked to so in a Secrets management strategy it's not just about detecting and preventing there's four areas there's detect prevent respond and educate inside of a comprehensive Secrets management program and so the first area is detect which um is basically you're bound to at this moment in time and all of your repos and all of your slack and all of your um in all of your document education places you're bound to have hard-coded secrets no company is immune from it it happens everywhere and so you have to find a technology that can detect where they are latently are right now and so for us we used a couple tools I know that Guardian does stuff like that and so um you have to have a detection mechanism and you have to build it into we built it into our red team our red team's mandate is to help us scan for Secrets everywhere and then we built it into our build pipelines um and uh and we used you know regex for custom secrets too that were only you know um that were only ours right because like you know a lot of the tools out there today have regex for certain types of secret certificates usernames and passwords API Keys um but we had some custom stuff too that we had to build ourselves so that's part of the detect Branch then there's prevent which you don't you want you know the detectives to stop the bleeding kind of and the prevent is to you know help you build for the future and so uh really we found the most efficient thing is pre-commit hooks um pre-commit hooks in the development life cycle uh to stop developers from committing Secrets anymore and in your pre-commit hook applying a pop-up that basically says Hey either a command line warning or a web pop-up if you're using the web components um that says hey our preferred method of storing Secrets is this and so we went with you know Vault um and so that's in the respond category is you got to give them some way to do it correctly you can't just block them from doing it if you don't give them a correct way to do it so we went with Vault and then our you know our level up on top of having you know a great policy for vaulting Secrets was also um rotating them automatically with the vaulting technology and so that's in our respond branch and then the last is educate and we had to go on a tear about educating all developers and employees on not sharing Secrets verbatim in chat channels in documents and anywhere we built a custom platform to share user-based secrets we educated people on password manage and corporate password managers um and uh and those are like the four areas that we had to build a comprehensive program about and so it's going to become like it's not on that CSO mind map right now right like that I talked about earlier right but it's it's a big big thing that's coming down the pipe that people are not addressing and they need to address because when you get breached um a lot of times the ttps of the attackers especially ones like lapsis are to move around your network silently and not trip any of your EDR or any of your internal controls they are literally just collecting documentation and credentials until they feel like they have enough that they can strike so fast that your sock can't even detect them or do anything with many accounts that they've gathered and to achieve their objective and it's just not possible um do that so that the way that you know kind of the vaccine is having the secrets Management program so they can't pivot and that your other tools have a better chance of detecting them they have to resort to different things like installing software and stuff like that and you can catch them you know doing that with EDR and you know nips and stuff like that so um that was one of the things I learned at Ubisoft we built that program as very successful people you know that I used to work with are still there using that program today so there's also we couldn't have paid for a better I know I know I was thinking about it I'm like people are going to think that uh they paid me no no this is absolutely a sound sound see so advice I just you know if people are out there want to know like where you should invest in your program that says area you should invest in so well Jason I don't want to take up any of you more time it's been a really awesome episode I don't think I've laughed so much that I have with you so this is gonna be great I can't wait to get it out there so thanks for taking the time and uh I know that they're you're pretty active on Twitter if our users want to follow you is Twitter the best place where else and what's your handles yeah I'm really active on Twitter at J Haddix j h a d d i x um my account my new company that I'm building right now butterbot is at butter b u d d o b o t butterbot um and so we're on Posting and you can find like my threads there and where we're gonna be and all that kinds of stuff so great well definitely check that out and uh thanks again so much for for coming Jason