I need to bring in a team to augment what we're doing to help us understand or do something with the security alerts and things that are are being brought in by these tools that I can still get value from my tools and I'm not just stuck doing that one side the reality of security for most businesses it's a cost and it is only ever a cost in those cases it is very hard for csos for you know security managers to make a business case to folks that may not understand and may not have had to look at security in in today's world that was part of a conversation that we had with Troy Santana we sat down with Troy to discuss a really interesting topic that is Staff augmentation the idea that we can supplement our security staff by augmenting it with consultants and services to really fill the gaps that we have it's a very relevant conversation and one that's really interesting theresentada joins us as a senior consultant for critical start after spending six years as part of their 24 7 security operations center in their team as an analyst working in management and other supporting roles he's moved over to Consulting to provide technical in-depth operations experience into the buying process a veteran of the Marine Corps he obtained an M.A in criminology and criminal justice and uses that experience and education to help us team clients understand the importance of layered security and operational expertise in both tools and personnel that complete conversation is coming up but first it's time to have a look at our breach of the week this one is really interesting because we're going to look at some breaches that were reported on a bug Bounty platform by chat GTP yes the AI chat bot that we've all become so well known for is making it back into the headlines now we know when chat GTP started out it was producing some malware I think this was quite well reported on and there were some changes that well was meant to stop chat GTP from being able to create you know ransomware and malware and other malicious programs but it turns out that actually well you can still trick it into doing that with the right amount of commands specifically in this section we're going to be looking at some some malware that was written at the instruction of a user named code blue 29 and actually the use case behind this is really quite fascinating kobu 29 actually got chat GDP to write some malware to see if it could bypass some EDR solutions that it was testing now here's where today's story gets really interesting the malware that chat GDP wrote was actually being used to evaluate EDR Solutions endpoint detection and response Solutions they were able to create this malware without much knowledge in writing python or writing malware themselves and appended to actually circumvent the EDR Solutions defenses they were able to write functional malware that got passed some of the solutions of which they were evaluating so that's really interesting malware was written not for malicious purposes but actually to try and test how good certain security vendors actually are code blue 29 the user that created this was actually able to generate a bug Bounty payment for a fairly significant amount of money through one of the vendors by basically saying hey here is how I was able to circumvent some of your systems so not only did that vendor probably not get selected uh because of its flaws but also was actually paying out money to try and fix them and again all this was created from malware written under the instruction through chat GTP buy a user that doesn't have a lot of experience writing malware or using python code reading between the lines of this report I'm not going to go so far as to say as that a random person can input Specific Instructions to generate malware worthy of a bug Bounty payout it is clear that the user in this case code blue 29 did have knowledge of endpoints and security and knew what to ask it but still that's not a big step for a lot of people now what's really interesting here is to see how AI is going to change the security game yes we're already concerned about AI being able to write malware and being used by malicious actors but flipping the script can we use it ourselves to test Solutions can we use it ourselves to test our own defenses perhaps this is going to change Security in ways that we didn't quite expect or maybe you're one of the lead thinkers and this is exactly what you expected certainly I have to say I did not see this one coming and I really did not see that it would be good enough to generate a bug Bounty payout which I think is a pretty high bar for just an AI chat bot this takeaways us pretty well into our next topic of conversation staff augmentation because as we can see right now well security is going to become augmented with AI I don't think that security is going to avoid this revolution like many other Industries so how can we be sure that we're keeping up with the trend and that we're effectively using all the tools that our capabilities to ensure good defenses because we all know that the bad guys while they don't stop for nothing so this conversation is very topical and we all know we have a skill shortage here and security workers don't necessarily have ample free time to experiment in The Cutting Edge of Technology and that's where organizations like critical start can come in so with that I would like to introduce Troy Santana the senior security consultant at critical start to break us off in our conversation about staff augmentation yeah hey appreciate you guys have me on pleasure to be here and as always I am joined um by my partner in crime uh Dwayne McDaniel Dwayne welcome welcome again it's good to see you always good to see you too Mackenzie don't lie we know that that's not true but Dwayne um I know that you and Troy have met so I'm going to pass it over to you and uh you can tell us a little bit about how you guys met and maybe dive into a little bit about what we're going to be talking about today yeah um thanks very much Kenzie uh great to see you again Troy uh we first met back at the side Salt Lake City in 2022 back in the Far Far Away it passed I hope you don't mind me flattering you a little bit here but um I walked up talked to every vendor there and Troy uh you kind of just bowled me over with your sheer passion for what you were talking about um I talked to a lot of vendors who are like yeah we're here to like talk about what we do but you honestly had a desire to really get me to understand staff augmentation and and Outsourcing and things like the soc um and that comes from your years of doing this this I think you said six years um you were part of soc team I was uh so yeah I uh in particular with the the same company right so um I I spent six years with uh critical starts sock um starting as an uh L1 analyst so starting at the bottom came in there and uh had the opportunity to kind of learn the platform and the methodology that they had as a base level and just stayed with the team moving through and doing everything from the eyes on glass on screen to training management uh and then eventually operations support before I moved over to be a sales engineer and do essentially what you and I had a conversation about was hey let's bring the actual experience and the the technical side of things to the front of the conversation so that people that are interested in it can actually dive into the the nuts and bolts if that's what they really want well that's a really good segue because one of my first questions I had well I was hoping you would kind of do the same general Spiel you gave me at the at the b-sides but what is security staff augmentation and and how do those conversations even start with companies uh honestly they usually start when uh when people realize that they they've got a need that they have no idea how to fulfill right so um for for us in particular right critical start specializes in doing manage detection and response as a service for folks and um you'll you'll hear you know it's been mssp for some time uh and that's that's sort of like that halfway mark where it's a security team that's there to receive things and pass them along and where that varies I guess changes up a little bit from detection manage detection and response is actually having a set of professionals that are going to go in and do that initial triage and response for an organization as a as an extension of their security team um and really what I've seen is it lets the security teams on site and in specific companies focus on improving overall security posture for them instead of do doing all of the the tedium frankly that can come with trying to manage and understand tune configure everything that goes with the multitude of security tools that you could have pushed out in an environment right on now um I know you have questions here as well McKenzie but I did a follow-up question on that just how those conversations generally start is this something that teams are coming to you because they've see that they have a gap and they're trying to take the precautions to fill in the gaps before something bad happens or are these reactionary events and they come into the conversation because they realize he had a gap and that caused this last incident yeah um we see honestly a little bit of both right um with with the news cycle and everything being what it is today and you know being able to get your news from everywhere it is frankly commonplace these days to hear hey somebody's been hacked there was a breach there was um you know uh LastPass was the the big thing that happened over uh the last holiday right where hey you know they they expanded on what happened with their breach and oh yes in fact user accounts um so we have those folks you know that have uh security events breaches things like that that go yep no we have to do this for better security or to let's say maintain Insurance compliance something right and then you also have the folks that because of those stories uh they're they're executive teams or their shareholders whoever else it is that is looking at this going hey how do we not end up in the news like that uh and they go hey uh let me look into it you know we'll we'll do it and they go great like let's let's put invest in some some prevention here and those are also the people we see come and have the conversations with us so a little bit on on both sides we we hope that is uh learning from other people's mistakes rather than their own but you know you you you get there in your own time when you're working with a company you know what how often is it that they have a preconceived idea of what their problems are and when that aligns with what you discover you know so in other words you know how often is it that you enter into these conversations and maybe maybe it's different between the ones that have had an event and the ones that are doing being precautioned but how often is it where where what you actually discover and what they think their security needs are don't quite you know don't quite a line and there's like a process that people need to go through to understand really where they're where they're vulnerable uh it's actually quite often that people will come in and I think it's because we are Tech folks and we're a lot of what I've seen a lot of my experiences we focus on the problem solving aspect and so to come in with this idea of I need this solution and very often what I've noticed is is we sometimes lose sight of the bigger problem like uh we were talking about hey do people come in because of a security event well yeah some of them do and when you do of course your background is going to be hey I need to do this specific thing to make sure this specific event doesn't happen again and really you know we may see them come in and go hey guys you know yes you guys want to implement these tools and do you know uh have this new roll out and you know you're going to implement all these different layers of what's Happening but how are you addressing that from a policy perspective like let's let's start at the bottom right like are are we doing good security practice first good security Education First and then let's start adding stuff on so it's it I think there's almost always room for uh people to go in and go okay like let's let's start back and go what is the problem we're trying to solve first and then let's talk about what Solutions best get there for us so for the companies you're working with out there what are the the General Trends you're seeing the areas where small medium-sized companies are really struggling with are areas where you're identifying after the fact like well this is where you should be spending your focus for for small to medium businesses I think really the the big thing that they run into um is uh becomes a people problem really fast right they get uh they may have a security technology at the very least you know they say um endpoint is probably 70 80 percent of of what you're you're gonna do from a security perspective um to help out your business right because uh attackers need to either pull data off of particular endpoint servers something they have to interact with an endpoint at some point or another uh for a majority of things that are going to happen um and then what will happen with these smbs is they'll go in and they will get an endpoint tool or they'll they'll try and roll out an xdr detector uh agent uh depending on their vendor and suddenly there is you know they have all these agents on there and now they've got an influx of alerts even you know uh let's say 500 endpoints can generate uh depending on the agent and how in depth they want to turn the detections on uh hundreds or thousands of security alerts a day uh right off the bat and you have a team of you know one to three folks maybe uh for an SMB and suddenly if that's what you're doing that's all you're doing you're not now you're not moving the business forward you're constantly on the back foot trying to assess all this information that's now coming in from your security tool and that is disheartening uh for I think a lot of uh a lot of clients that first come and talk to us and they're like I I can't do this myself and you know you you have to reassure them tell hey that is not a unique problem it's not just a you thing I promise that is one of the most common immediate uh emotional responses really of rolling out new security things for a small team is it's overwhelming to take we're rolling out new tools and and things in in security yeah how do we how do you start to overcome the these problems because this is something that everyone faces right a low alert fatigue the technical barriers for doing it not understanding the tools not utilizing them correctly we can't be the only ones to just see companies buy expensive Solutions and never actually fully uh use them it's kind of the the gym card and the wallet analogy just because you have a gym card doesn't mean you're going to lose weight or get fitter I mean in an Ideal World right if you if you had all of the budget and you had all of the time then the solution the number one thing for everybody is build your own team somebody that's going to understand your business needs your security needs are gonna be you know adherent to your policies and procedures and you're going to bring them up from the inside and tailor everything that you you need to where your business is going but that's a you know that's a blank check Miracle sort of situation right and people just don't have that they don't but if they did like yeah 100 do that solution uh that puts me out of a job for sure but you know it that is we're talking you know best best of breed security of what's going on uh if we're being more realistic about it um very often you know their their first solution is tools and we've talked about the problems with that of of over inundation of alerts and now that's your only priority if you're going to keep up with it um so that alert fatigue becomes its problem and the opportunity cost of not being able to uh look at your security for your organization from a more holistic perspective because you're stuck so a lot of people we see over the last couple of years have kind of turned to that third option which is I need to bring in a team to augment what we're doing and to help us either understand or do something with the security alerts and things that are are are being brought in by these tools and maybe if we can split the workload or I can offload that piece then I can still get value from my tools and I'm not just stuck doing that one side of things right because I uh I would love to meet the uh the it manager the CSO or whoever that's only wearing one hat and they only have that one job um tell me where you're working and I will I'll look into it and refer my friends there but I've I've yet to find something like that so really I think the the business is really moving in that or I'd say the business the world is kind of moving in that direction of if you have a problem and there are professionals out there who can solve it then it's worth your time to at least look into getting somebody who focuses on doing that right um you know some people uh can do uh for you know your American listeners out there right like taxes uh you you do taxes people can do it themselves uh there's also people that are tax professionals that's what they do all the time um and I would personally much rather have somebody that knows what they're doing that's their job uh do that thing then Me Maybe do it 70 80 percent well uh it and and get it right no I mean that makes it it makes so much sense um and you know going back to like the blank check option you know that even if you have a blank check there's no guarantee in this market that you're even going to be able to find people um that have the expertise yeah a bit and hold on and hold on to them and that's the the the and the other area um especially with security wins there's going to be moments where for I think for any organization and perhaps you can maybe add some insight or to this or tell me if I'm completely wrong but I think there's going to be moments where your staffing needs just outstretch your capabilities in terms of not even just budget but just in terms of kind of going through different stages of the business and even having someone in there to help you set up expertise and to take you to that to that front foot um that front facing in that forward-facing position for security instead of I think so many companies are just reactive at the moment sure um and the reality of of security right is for most businesses it's a cost and it is only ever a cost um it security does not improve your production uh in most cases not directly right it's not a revenue generating arm of your business and in those cases it is very hard for um for csos for you know security managers to make a business case to folks that may not understand and may not have had to uh look at security in in today's world where it is that it can be an insurance requirement in some cases now to have cyber insurance you have to have a certain level of monitoring and you have to have a certain level of log retention you have to have uh some sort of demonstrable way of investigating security events that are generated by the tools that you do have so in those cases you know if you get a whole bunch of tools and then you can't monitor with them or you can't prove that you're you're investigating the alerts that the tools have then you may be putting your cyber insurance at risk and that that sort of uh that sort of business case just it it doesn't fly and so that's where people have sort of been driven to this whole how do I how do I solve that that part of this problem uh following up on something you said earlier like part of what drives people to look at staff augmentation for security in the first place is more and more becoming Executives who are how do I not end up on the news um I'm curious what data points they're looking for I know as a security manager they're probably have a different set of what success looks like metrics than someone sitting on the board of directors who only cares did we get our church renewed or not but at what level do you see that conversation happening like what what proof points are you finding along the way or um four four csos right like so the the whole idea of being able to have the security conversation at different levels um to say you're not going to talk to USA the same way you're going to talk to an analyst right um most of the csos that I've I've had the the pleasure of having conversations with will tell me hey um great your your widget your tool works this particular way um how do I prove that how do I see that how do I show it um give me a you know I've got 50 different tasks that I need to do and what I need is to understand in two minutes or less what is the state of my security for my organization and so being able to go in and show them hey here is auditable reportable material that if you don't trust the way it's being displayed you can pull that information yourself I need to understand that it's not black box so that if I've got a service or security that's happening I can go okay let me go validate and verify that so that when somebody asks them the inevitable question of Are We secure and they have to give a short answer um all of those criteria have to be met for them to just be like yes and this is how I know because yes is never good enough these days right you can't just say yes we're secure well how do you know cool if your security tools or your security provider both have to be able to prove that what they're doing is in fact what they're doing I I need you to be able to go back and and see uh what actions am I taking for you what did your tools say what did um what did this response when did it happen uh how was that conversation done what did I do in my investigation process I can tell you hey we went in and we investigated this and this is what we found okay according to whom according to what like help me follow your logic and where any of those pieces are missing each one starts having that exponential effect of I don't know that I can I can trust this or that I'm comfortable going back and saying yes we are secure yeah I can imagine the last thing they want is well here's a whole new set of alert story about implemented a solution for you to help solve this problem but all you get at the end is more noise sure um and you know you the it goes back to even just the vendor themselves right security vendors are incentivized to create more detections create more alerts um they are incentivized to miss nothing and that might mean from a at the very least a context perspective uh alerting on every minutia and Nuance of what is happening with processes and systems and in the end you know a lot of that is how a computer operates you know when you uh for a Windows system you go in you double click on Microsoft Word it launches that and you you click the help button right man if you go in and look at each little process tree dll everything else that's part of that there's a lot that goes on and if you you can have uh you know Epp or EDR tools that tell you about each piece of that but ultimately it is yeah somebody double clicked on Word well I I just got you know six notifications about processes happening are any of them malicious well if you get that you know times 500 users times you know 10 000 users that's a lot of just nothing that is taking up space uh a lot of data versus information right yeah this is a this is a problem I think throughout the industry of alert fatigue which has come up multiple times here it's one of the major problems when it comes to kind of maintaining your team and them being able to be effective you need to have processes set up in place to be able to actually understand what are the alerts that you need to pay attention to and how do you how do you find those out I think really like I guess to kind of put on my my sales guy hat right like of of what's going on that that problem that problem um is specifically what we saw when we were building out you know our our service right our sock MDR as a service um you know everybody that built the platform and built up the team that was there were were technical people a lot of them had been analysts previously and you know said hey like these are all the problems that we have with the analyst number one was trying to use sim as a security tool um you know it does it function as a security tool yes uh is that what it was actually designed and is really good at doing uh arguably no um it is great for log retention great for large conglomerates of information and being able to search those things but if you go back to the concept of of uh let's say just like antivirus right um we moved into the the idea of next gen antivirus to get away from signature based detection well a Sim if you're using it for security is a signature based detection engine right and that we said no that that doesn't work we need to know we need to know unusual things we need to know because if it's if it's doing something new and different for a computer system that's bad different is bad because they should function the same way all the time and so we went in and said hey uh going back to that early example when Microsoft Word launches it launches this way every single time so the way that we attack that alert fatigue problem while leaving tools to give us all of the information detections things that they can is say cool give me everything and when I see normal computer activity I want to record exactly what it is that's happening there and say hey when this happens that is a known Behavior or what we would call a trusted behavior and that's okay I don't need to see when a computer does computer things the way it's supposed to when my Microsoft Word you know somebody clicks on help well technically that's a Powershell script in the back end that goes out and grabs the the help page from you know Microsoft's website or if you're using Edge maybe it goes in and Does the Bing search for the Microsoft help page for that website then okay great that's normal stuff so if you show me everything over time what happens is I get this repo of expected normal trusted behaviors and we built a trusted Behavior registry and said hey any of this stuff this is good give me all that information I see all the good behavior and I want to take all that out if this is good I don't need to see it and I'm left with two things either unknown behavior that I have to investigate because I've never seen it before or known bad behavior both things are things that have to be addressed by your security team as fast as possible and by doing it that way we can have every bit of information the security tools provide to us and not not drown in the alerts or the data of being able to investigate that so I guess kind of sales guy hat off there but you know that that was the big thing that that you know over time has made me stick with it and I guess drink the Kool-Aid buy into the system right is it it just makes sense that you take out the what's good um and I I'm I have a background in in criminology criminal justice that was kind of a passion of mine that I I wanted to pursue and and have uh you know took the time to do that and one of my favorite things was doing uh was that forensics and counterfeiting and we had a particular speaker come in for one of my uh my uh postgrad classes and they were talking about their I think 20 years doing counterfeiting uh work for various three-letter agencies for the U.S government and um he said if you as a counterfeiting person you cannot study the bad you can't uh you can't study fakes uh the problem is you study a fake and next week they come out with a new fake and they come out with a new fake and they come out with a new fake and you're always on the back foot so what you have to do if you want to know what's real study the real thing there's a a specific subset of real so if we have paper money you know there are exact prints of money uh that you know all of the specific details for and if you know those things if you know what good looks like bad will stand out every time and we just we took that perspective and applied it to computer security and that that is what has rung true for me that was Troy Santana from critical start Troy is going to be joining us again next week for our episode which we will discuss the soc security operations center exactly what the soc is what it looks like the operations that it performs and how you can Implement one in your organization big or small we look forward to seeing you again on the next episode of the security repo thank you