[Music] all right today we're gonna run through how to set up a pre-commit git hook using gg shield and the pre-commit framework now detecting secrets can be quite difficult and it's very problematic when you detect them inside your git repositories because the keys have to be rotated and considered compromised it's best if we can detect these keys before they enter into your git repository so remediation is much more simple so in this video we'll run through how you can set up a git hook to be able to do just that now we're going to be using a couple of dependencies in this number one we're going to be using the pre-commit framework and we're also going to be using gg shield the open sourced tool from get guardian now the reason why we're doing this through the pre-commit framework is because this is actually quite powerful and we can create a layered git hooks this means that we can perform multiple actions in a single github so if you want to add in actions to detect other vulnerabilities or code quality or even if you have large files then we can still do this in the same git hook using the pre-commit framework and we'll run through that in a minute now the other area is that we're using gg shield and this relies on the get garden secrets detection engine it can detect over 300 specific different types of secrets but does rely on an api to be able to do this so we need an api key so let's get stuck in and set up these elements first so on my terminal i'm just going to install the pre-commit framework and also gg shield pip3 install pre-commit pip3 install ggshield of course i'm using pip but you can use brew or docker or however you are comfortable installing packages onto your machine and now we need to get an api key so we can head over to getguardian.com or go directly to dashboard.guardian.com it's free and very simple to create an account if you don't have one already and we're going to head down on the left to the api tab section here and select create new api key so i'm going to create one here we only need to scan scope we don't need to worry about the incident scopes this is for added functionality that we'll go through in different videos create new api and there we have it we have our api key right there so we're going to copy this now we need to have this installed as an environment variable so however you load in your environment variables we can do this here i'm just going to use the export command and we're going to use the the value get guardian underscore api underscore key as our variable and we are done we have set up gg shield a pre-commit frame hook and added in our api key so we're now ready to create our first pre-commit get hook so to do this i'm going to use the instructions on the ggshield github page so if we open that up we can see that there's actually a lot of functionality that gg shield can be used for but we're only interested in pre-commit hooks click on that and we hit down here and we can see that we're running through the pre-commit frame framework we've already installed our pre-commit so now we're on to the next step which is to create our pre-pre-commit config yaml file so let's do this okay so we have a yaml file here and this is where we're going to describe what we want our pre-commit hook to do so this here is the code that we need to put into a yaml file to be able to to add sequence detection into this but as i said we can add in different functionality in a layered manner into this yaml file and create a multi-tiered or multi-layered github git hook so let's open back up our yaml file and we can paste this in now i'm going to make one small revision to this code on line three we have red main rev main now this is just saying that we want the latest version of the main branch but this can actually create some errors because doing this is actually considered a bit of a security concern so depending on your setup you may get some errors so i'm actually going to put in the specific version that we're using so at the moment it's ten point seven one point ten point seven so that's the current version of gg shield and we can see this at the top uh of the github page here under releases okay we're going to save this file here now as you can see this has been added into our repository that i'm working on this is just an example python project there's nothing too special about it but we've added our yaml file directly into that repository and now we need to build our git hook so we're going to run the command pre-commit install and this is going to install our git hook based off our yaml file so we can see that it's installed at dot get hooks pre-commit and we can actually go in and see that so we can go into git hooks and then we see our pre-commit file right here so this is a binary file that was what runs when we do commit in that hook so that's what it actually looks like and that's where it's installed so now we have actually installed our first pre-commit git hook so we can check to see if this runs as expected so let's do that now so as i said we have these files here and i have one file within here called config.py and in here i have an aws key so this is kind of typical to what maybe you would find in a config file so let's add this in okay and now let's commit this now immediately once we do the commit it's going to scan these we've already have a failed under the duty shield so if you have multiple commit actions uh hook actions in here they'll list which ones have passed which ones have failed and we can see that it's detected an aws key and it's given us the key details uh with some of the details blurred out so here it has worked as we have expected and we've successfully installed our first pre-commit git hook now we can add some more functionality around this to help us use this in a more natural way that i'm going to run through so the first thing i want to run through is how to ignore these secrets that come through so let's say that in this scenario this is actually not a true secret this may be an example secret uh perhaps it's a tester or dummy credential or maybe just a general false positive so we want to ignore this but we also don't want this to keep bothering us into the future so one of the great things about gg shield is it gives us an ignore sha this is a sharp value of this secret and it enables us to store the secrets that we want to ignore without duplicating or cloning the actual secret so i can run the command gg shield ignore last found and we're going to note to say one secret has been added into our dot get guardian yaml file so it's created this file is right here dot guardian dot yaml and if we open this up we can see that this is our sha value of our secret so no way does it actually present our secret values so we can include this into our repository and we can share this without worrying about exposing the secret so a cool little feature in there so because it blocked my last commit well now that i've uh made the changes and added the guardian ignore file i should be able to commit this without being blocked again by the github so let's try it so i'm going to run the same commit command and now this time it's passed because we're ignored ignoring that secret so let's run through another scenario where we might want to ignore certain secrets i have a folder in here called examples now we often see that perhaps we might have example secrets in solder inside a folder maybe it's showing how to set something up it can be quite common so in this case i'm going to want to ignore all of the files that are in that folder so let's first add this and now let's commit this and see what happening happens all right we've found a lot of secrets in here we have an rsa key we have an alibaba cloud key we have a bit bucket key and so these are in different files as well so we don't want to have to go through and ignore these perhaps we have multiple of these types of secrets so what we can do is we can open back up our get guardian yaml file and on line 11 you'll see paths ignore so this is similar to secrets but instead of ignoring secrets we're ignoring certain file paths so we're going to go ahead and we're going to add in a file path for examples so star star file name star star we're going to save that we're going to try and commit these secrets in again and see if we get blocked by our commit hook all right it's passed we've been able to commit those secrets into our repository i do want to stress that in this scenario these are dummy secrets under no circumstances even if it's private would you want to commit secrets into your repository so please don't and then i'm just going to show you one other small little functionality here and that is if we want to ignore a specific file so perhaps you have example secrets in your readme perhaps you have a example config file that you want to have that has dummy secrets in that just show how to set up your project so we're going to do that here i have a file in here called examplesetup.py so let's add this in let's commit this to make sure that yes we do have secrets in here and again we've been blocked this time with grafener token octa token and a pay stack token so a number of different keys in here again so now we're going to use the file pass again but this time we're just going to ignore one particular file so we're going to run the command star start and then the file name which is example setup.py oh that's meant to be a slash going to save this and then we're going to try and run this again and see if it bypasses it which obviously it has because by now we should be used to that so there you have it that's some of the main functionality behind using a gt shield in a pre-commit fashion now you can also set up gg shield in lots of different ways in your ci cd pipelines you can scan docker images right after you publish them you can scan your directories or get repositories all using gg shield without leaving your command line and you can also install it into your software development life cycle in multiple different places so we're going to keep creating new videos to show how we can use this tool and so please subscribe to get notifications of those future videos and give us a thumbs up if you like the tool like the video or reach out to me in the comments or on twitter my handle is at advocate mac and ask me any questions that you may have about our tools thanks