Table of contentMy source code is private, so why is hardcoding credentials in git considered a bad practice?Why automate secrets scanning throughout the Software Development Life Cycle (SLDC)?How does secrets detection compare with Static Application Security Testing (SAST)?What are git hooks?What is a pre-commit hook?What is a pre-receive hook?What is a post-receive hook?Where in the DevOps pipeline to implement automated secrets scanning? Client-side or server-side?Should secrets detection be blocking or non-blocking in the SDLC?
While private repositories offer some level of protection for your code they still do not have adequate protection to store information as sensitive as secrets.
Imagine if there was a plain text file with all your credit card numbers within it, you hopefully wouldn’t put this into the companies git repository. Secrets are just as sensitive.
A few things to consider when storing secrets in private repositories:
While DevOps, Continuous Integration and Continuous Delivery speed up software development, they can also significantly increase security risks.
“DevOps is rapid and requires lots of small, iterative changes. But this increases complexity and opens up a new set of security problems. With DevOps, existing security vulnerabilities can be magnified and manifest themselves in new ways. The speed of software creation can mean new vulnerabilities are created unseen by developers. The solution is to build security monitoring into the DevOps process from the start.”
Greg Day, RSA Conference Organizer (source)
Security now needs to be part of the SDLC from the start, and part of every incremental change. This concept is called Shifting Left, a development principle which states that security should move from the right (or end) of the SDLC to the left (the beginning). A great deal of automation is needed in order to be able to cope with running security checks at each incremental change. Automation helps streamline the detection, alerting and remediation processes and workflows.
Secrets detection is often confused with SAST because both scan through static source code.
SAST is mostly testing control structure, input validation, error handling, etc. Such vulnerabilities like SQL injection vulnerabilities only express themselves the moment the code is deployed. Exposed secrets are unlike these vulnerabilities, because any secret reaching version control system must be considered compromised and requires immediate attention. This is true even if the code is never deployed.
Implementing secrets detection is not only about scanning the most actual version of your master branch before deployment. It is also about scanning through every single commit of your git history, covering every branch, even development or test ones.
To conclude, SAST is concerned only with the current version of a project, the version that is going to be deployed, whereas secrets detection is concerned with the entire history of the project.
Git hooks are scripts that are triggered by certain actions in the software development process, like committing or pushing. By automatically pointing out issues in code, they allow reviewers not to waste time on mistakes that can be easily diagnosed by a machine.
There are client-side hooks, that execute locally on the developers’ workstation, and server-side hooks, that execute on the centralized version control system.
Examples of client-side hooks:
Examples of server-side hooks:
"The pre-commit hook is run first when committing, before you even type in a commit message. It’s used to inspect the snapshot that’s about to be committed, to see if you’ve forgotten something, to make sure tests run, or to examine whatever you need to inspect in the code.
Exiting non-zero from this hook aborts the commit, although you can bypass it with git commit --no-verify. You can do things like check for code style (run lint or something equivalent), check for trailing whitespace, or check for appropriate documentation on new methods."
A pre-receive hook is a script that runs on the server. It performs checks on the content of the push; if it exits non-zero, the push is rejected. You can use this hook to do things like prevent a PR author from merging their own changes, or require commit messages to follow some specific guidelines.
Be careful when using pre-receive hooks as they are blocking: if the checks don’t pass, the server is not updated.
"The post-receive hook runs after the entire process of pushing code to the server is completed and can be used to update other services or notify users. Examples include emailing a list, notifying a continuous integration server, or updating a ticket-tracking system – you can even parse the commit messages to see if any tickets need to be opened, modified, or closed. This script can’t stop the push process, but the client doesn’t disconnect until it has completed, so be careful if you try to do anything that may take a long time."
Unlike the pre-receive hook, post-receive is non-blocking.
The earlier a security vulnerability is uncovered, the less costly it is to correct. Hardcoded secrets are no exceptions. If the secret is uncovered after the secret reaches centralized version control server-side, it must be considered compromised, which requires rotating (revoking and redistributing) the exposed credential. This operation can be complex and typically involves multiple stakeholders.
Client-side secrets detection early in the software development process is a nice-to-have as it will prevent secrets entering the VCS earlier.
Server-side secrets detection is a must have:
From our experience, when trying to impose rules that are too constraining, people will bend them, often in an effort to collaborate better and do their job. Security must not be a blocker. It should allow flexibility and enable information to flow, yet enable visibility and control.
On one hand, security measures will be bypassed, sometimes for the worst. But on the other hand, it is also good sometimes that the developer can take the responsibility to bypass them.
Because even the best algorithms can fail and need human judgement. Secrets detection is probabilistic: algorithms achieve a tradeoff between not raising false alerts and not missing keys.