Demystifying Security Source Code Reviews




February 14, 2024


Demystifying Security Source Code Reviews

Lately, there has been an uptrend on everything surrounding security source code reviews. There’s talk about SAST tools, new methodologies, AI-assisted magic, and all sorts of “novel” techniques to uncover all of your vulnerabilities by just peeking a few days at your source code.
In this blog series, we’ll try to shed some light on the topic, and demystify some concepts along the way. We’ll cover topics like the intricacies of what a security source code review entails, what are some of the challenges, and insights into the skills required. Additionally, we’ll cover how it can complement traditional penetration testing, share some reflections on different programming languages and introduce our approach to tackling these reviews.

First Things First: What Is a Security Source Code Review?

Similarly to the eternal discussion between what’s a pentest and what’s a vulnerability assessment, there’s still some discussion in the industry about what a security source code review really entails. 

Expectations on what is to be performed on a source code review, and what the results should look like, vary hugely depending on who’s providing the service and who’s requesting it, but overall, we may agree that a security source code review is the process of examining the source code of an application to identify security flaws. 

A few years ago, people used to run a few automated vulnerability scanners against the targets, polish the tool’s output and call it a pentest; with code reviews, a similar thing is happening, where the review is based around the few automated code scanners that exist today. Unsurprisingly, given that these tools are usually not context-aware, the results are plagued by false positives and cookie-cutter-like.

An actual code review goes beyond running some tools; it involves a manual effort by professionals who understand the nuances of the code (as it exists in the particular context of the application), and also understands the programming languages the application is written in and the applicable vulnerabilities that could be exploited by attackers.

What's beyond automated tools?

Automated tools, although very helpful, have strong limitations when reviewing source code. Even though the gap between machines and humans is decreasing (thanks GPT), reviewing source code is a task that requires a lot of thinking to understand the context and architecture of an application to find the really interesting security bugs. Therefore, conducting an effective source code review is not a trivial task and presents some challenges, such as the following:
  • 100% Coverage is (Usually) Not Possible: we are humans, and time is limited 🙂 Even if we pretended that the target source code was a sports magazine and we simply wanted to read all its pages without really understanding what was going on, we wouldn’t be able to do so in a reasonable timeframe. If you added the effort of truly understanding how it works, then the process would be even slower.
  • Complexity Introduced by Modern Architectures: old applications used to be monolithic, but today this is rarely the case. Nowadays, a lot of applications are designed with serverless or microservices architectures in mind. This complicates the review itself, as it’s hard to trace the flows of data through the different services and figuring out what component is behind a given feature. Also, even development teams sometimes struggle to determine in which repository a given source code lives. 
  • Modern Frameworks Make Things Harder: even though developers love to be on top of every framework trend, as security professionals, staying up to date with every framework is very hard if not impossible. Performing a source code review involves understanding what is being resolved by frameworks, how they do it, and what are the best security practices for each framework. All this is especially true when reviewing frontend source code where React, Vue.js and Next.js could be considered outdated by some.
  • Scoping Based on Lines of Code (LoC) Can Be Misleading: The sheer quantity of code isn’t always indicative of the real size, complexity or the security risks it may harbor. As mentioned above, covering all source code is very arduous (and expensive), so the efforts should instead be focused on the main areas of interest and concern for the particular application being reviewed. Also, frameworks usually add a lot of code that may not be particularly relevant to the features and business logic of the application.
  • Static Application Security Testing (SAST) Limitations: SAST tools can aid in the review process, especially on large code bases or as part of automated checks in a development pipeline. However, when using these tools in a dedicated code review, they can present some drawbacks. For example, it is very common for these tools to throw numerous false positives that would then need to be manually reviewed, taking valuable time from the engagement. Also, by their nature, these tools usually don’t have the capacity to identify complex or business logic bugs.
  • Promises to uncover ALL vulnerabilities cannot be met: there’s a famous quote by computer scientist Edsger W. Dijkstra that reads “program testing can be used to show the presence of bugs, but never to show their absence!”, so finding 100% of the bugs is not a really achievable goal outside of academic settings or in reviews of highly critical systems. 
Apart from the challenges that a code review itself presents, it is something that cannot be performed by just about anyone. There are some abilities and knowledge that is required to conduct an effective review.

Program testing can be used to show the presence of bugs, but never to show their absence!

Reflections on the Required Skill Set

Based on our experience, the following skills are important when performing a code review:
    • A Programming Background: even though this is not mandatory, understanding the nuances of code implementation and design is key to unveil complex bugs.
    • Knowledge of Design Patterns: familiarity with common design patterns can help in understanding the application’s architecture and potential vulnerabilities. This is also key to navigate the code smoothly.
    • Awareness of Language-Specific Particularities: usually languages are designed to solve specific problems, and therefore, each will have their own characteristics and its own set of potential issues and best practices.
    • Tracking Down Potential Vulnerabilities: as mentioned above, SAST tools normally throw an overwhelming amount of false positives. To provide valuable results, it’s important to be able to determine if a reported issue can be exploited in practice and presents a real risk. Also, after finding a vulnerability during a penetration test, being able to track it down in the source code will help in finding the root cause of the issue.
    • Planning, Patience and Resilience: reviewing source code may not be as exciting to some as compared to normal active testing, and it’s also hard to keep track of progress. Therefore, it’s important to work in an organized manner to keep grinding until a bug hopefully appears.

Complementing Traditional Black-Box Penetration Testing

When security professionals conduct assessments without access to the source code, such as in black-box testing scenarios, they are limited to observing the application’s behavior from an external viewpoint. However, it may not reveal deeper issues that are only visible by examining the application’s inner workings.

In the following paragraphs we’ll highlight the advantages that result from simultaneously testing and reviewing the source code of an application.

Efficiency Gains

Active testing with source code access dramatically increases efficiency in vulnerability assessment. Instead of blindly probing the application in hopes of uncovering vulnerabilities, testers can directly examine the code to understand its logic and flow. This direct insight allows testers to quickly determine whether a potential attack path is viable or if it leads to a dead end. By eliminating the guesswork, security teams can focus their efforts on the most promising attack vectors, saving valuable time and resources.

Enhanced Mapping of Endpoints and Attack Surface Discovery

One of the critical steps in security testing is identifying the application’s attack surface, which includes all the points where an attacker could try to enter or extract data. With source code access, testers can systematically map out all the application’s endpoints, including those not exposed through the user interface. This method is far more efficient than the traditional approach of brute forcing the application with millions of requests using generic wordlists. It also uncovers hidden, old, and/or deprecated endpoints that could be overlooked but might still pose a significant risk.

Determining the Real Impact of Vulnerabilities

Access to the source code allows for a more accurate assessment of the impact of vulnerabilities. By understanding the context in which these vulnerabilities occur, testers can more accurately predict their potential to compromise the application and how they could be leveraged by attackers. 

Also, tracking down a bug found during testing can help determine the real root cause of the vulnerability, which will provide additional value when providing recommendations for mitigation.

Quick Identification of Authorization Flaws

A common and often critical security issue in many applications is improper authorization. With direct access to the source code, testers could more easily discover operations that lack proper authorization checks. This enables testers to achieve a more comprehensive coverage of these types of checks.

Identifying Patterns of Weaknesses and Strengths

Finally, reviewing the source code allows testers to recognize patterns in the application’s security posture. For instance, by analyzing input validation strategies across different modules or features, testers can identify consistent weaknesses or strengths. This view not only helps in pinpointing systemic issues that need to be addressed but also in recognizing the application’s strengths.

Our Approach at Refactor Security

As it happens with other areas in the cyber security field, there isn’t a single widely accepted methodology to use when conducting source code reviews. Most of the related material on the subject often has a formal (or academic) approach, that may not be practical within the constraints of time-limited engagements. On the other hand, there are methodologies that are too simplistic, SAST and check-list oriented, that usually cover basic and generic issues and don’t yield great results, with a high probability that the only issues identified were hard-coded secrets and outdated dependencies.

While trying to find a middle ground between practicality and thoroughness, we felt the need to create a new tool that allowed performing real-world security source code reviews in an organized and collaborative manner. The tool, Security Notes, is a Visual Studio Code extension that allows the creation of notes in the source code itself, so any idea, comment and potential vulnerable snippet can be quickly marked without losing the train of thought.

There’s a lot of thinking going on behind the scenes when performing a source code review, and all that effort is usually lost unless it materializes in a specific finding in the report. For instance, following are some of the things you can capture during a code review that may not be necessarily related to a specific vulnerability:

  • Notes about the architecture and design decisions
  • Location of the code behind specific features and main security controls
  • Data entry points
  • Location of sensitive operations specific to the application
  • Location of cryptographic operations (encryption, hashing, etc.)
  • Smelly code
  • Integration with external systems or components

Below is a screenshot of how comments look on the actual tool:

vscode security notes extension notes example

In the follow-up to this blog post, we’ll be sharing additional insight into how to leverage our tool to conduct useful code reviews.


Source code review is an activity that is often overlooked and not given the value it deserves. This could be because of a mismatch between expectations and sometimes poor execution, leading to a belief that it is an activity that is not worth the effort or the cost. In reality, a well executed code review should help in uncovering hidden vulnerabilities, and even enhance the efficiency of a normal penetration test.

In practice, performing a source code review can be a challenging task, since there is no consensus regarding the approach and methodology in the industry. Some will take a very in-depth, academic approach, and others will just perform shallow reviews based around automated tooling. 

At Refactor Security, we developed our own methodology based around our tool Security Notes, that allows for a pragmatic approach without compromising thoroughness. We’ll expand on this topic in the upcoming second part to this blog post, so keep an eye out for any updates.