Threat modeling is a key security activity that no two security people do the same. And you read that title correctly, not "what is threat modeling," but "who is threat modeling." A great talk I attended at AppSec USA 2018 looked at “Value Driven Threat Modeling,” and there was a great line that has stuck with me since, “All threat models are wrong, some are useful.” You can watch the talk on YouTube. That encompasses a lot of what I believe about threat modeling for application security.
I’ve conducting on-the-job training about threat modeling and the one point that I have to preposition each with is, “threat modeling is not an exact science.” To an extent it’s a fairly subjective process with how you approach it and I’ve found that almost no one does it the same way. Given the same architecture or infrastructure for security review to two equally skilled security professionals, they would approach it very differently although they would both come to the same conclusions. It’s a skill that really needs to be built upon by your own personal experiences. However, there are plenty of tips and tricks to integrate into your own process, and that is one of the goals of this blog post.
Understand that there is no good way for me to explain to you my entire process for threat modeling an application. Firstly and honestly, I couldn’t accurately put it into words as it’s such an ingrained process in my brain now, but secondly and most importantly, it’s far more beneficial to build your own personal process that works best for you through experience.
Known Standards
With all that said, there are numerous standards out there for threat modeling and I will briefly go over a few with my thoughts:
Classic or Standard: This is the most straightforward process for threat modeling. It takes in the documentation, diagrams, and security requirements and applies a layer of analysis to all of the components, how they interact, and what data they are handling. This is where a lot of subjectivity on the process comes from. Stride: From Microsoft, it stands for Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, and Elevation of Privilege. The general thought process is to apply each of these threat categories to each node or component of the architecture and see where they may apply against the design of the application. Attack Trees: This is the case where you theorize possible attacks that are applicable to this application, and break them down into possible attack vectors. These attack vectors can be broken down into multiple layers to make them more precise. The application will then be compared against these attack trees to see where they may apply in the design. P.A.S.T.A.: Standing for “Process for Attack Simulation and Threat Analysis,” this is a seven step, risk-centric analysis of the application. It aligns business objectives and technical requirements with compliance issues and business analysis. This is all scored based on threat identification and enumeration. It is a very in-depth analysis of the business and what their associated risks will be.
Threat Modeling Basics
The first step even before threat modeling the application is to have built your security requirements, which I laid out in my previous blog post that you can read here. These requirements are what you will be referencing during your threat model.
Second, you need full access to the application’s documentation, diagrams, code base, and its development team for questions and answers. Understandably having access to all of this is not always feasible (in a partial or closed box scenario). However in a professional or corporate environment, this type of open box access is necessary to do any security review thoroughly.
Another nice to have are user stories. There is where the development team, as part of their documentation, have laid out the various use cases of the actors and services that will be interacting with the application. This allows you as the threat modeler to create “abuser stories.” This is essentially where you turn the user stories on their head and re-describe it from the attacker’s perspective. So a user story may state the following: “The user logs into the application to change their email address.” The related abuser story would then be, “The attacker logs into the application and changes the email address of another user,” or “The unauthenticated attacker is able to access change email functionality.” This is an example like that of attack trees but in reverse order.
Tips and tricks based on the types of issues
Again, it’s pointless for me to just go over my entire process from beginning to end, because the goal is not to have other AppSec engineers and developers emulate me, but to create their own personal process that works best for them.
Authentication and Authorization Good authentication and authorization security is extremely important, luckily it’s pretty simple to threat model for. Because you should consider each node or component in a diagram as an island that is under constant assault from other nodes or components. Just like in an operating system, every file has its own ACL (Access Control List) that states: “Who and what group owns this file, and what reading, writing, and executing permissions do they and others have?” A similar question needs to be asked for every node/component/process/API/user in the documentation and diagrams, “How does this component authenticate to that component and vice versa?”
This is especially true when communication or interactions cross a “trust boundary”. Every node in a diagram has its own implicit trust boundary around itself, but there are also greater trust boundaries that denote similar authentication and authorization for groups, classes, or subnets of components. Anywhere there isn’t a good answer for the above question for a particular interaction of components, it’s a possibility for an authentication (AuthN) or authorization issue (AuthZ).
Injection and Output Encoding Some of the worst security issues are allowing attackers to inject malicious payloads into applications for their nefarious gain. However, this is also an easy check while threat modeling, and it’s simply this: “Every node and component in the diagram needs to know how to validate its incoming data and how to output encode its outgoing data.” A simple example is that of the web application that has a database back end.
There are at least three components in this transaction: the user, the web application, and the database. Since the user is technically out of scope (not penetration testing people here), we first pay attention to the web application. The web application should be able to address how it will be validating data it’s receiving from the user. Is the data in the proper format? Were the steps completed in the proper order? Was certain business logic adhered to? Were all the numbers received in the proper range and not negative? There needs to be a validation strategy for every piece of data that is coming from outside sources.
Secondly, how does the web application output encode its outgoing data? In this example, the web application must ensure that it’s sending properly encoded and formatted data to the database server. This is to make sure that the SQL queries not only function properly, but also to prevent vulnerabilities like SQL Injection. The same thing could be said for pretty much anything you send outside of the component. Does the component ensure it’s sending properly encoded JSON to an API endpoint? Does the component ensure it’s sending properly encoded and syntactic XML to the SAML endpoint? This needs to be considered for all aspects of the application design.
Next, how does the application check for proper input when the database returns from the query. It’s a security myth and assumption that, just because you also own or trust other components interacting with your application that you can implicitly trust what they return to you. You should be checking for valid data coming from all other components, trusted or not.
Lastly, how does the web application output encode back to the user? This is a very important thing to ask, especially in this case because we are sending back data to the user in the form of an HTTP response containing HTML. The data that is dynamically inputted into that response needs to be properly encoded and formatted to ensure proper viewing of the HTML and to also prevent vulnerabilities like Cross-Site Scripting.
Cryptography This is generally also an easy check, as the application's designers should have called out somewhere in their documentation and diagrams what encryption the application will be doing, if any. It needs to call out what type of encrypting it is doing, what encryption algorithms and libraries it will be using, and how it will be handing encryption keys. This needs to be spelled out in the documentation and diagrams, and any missing specification is a possibility for security issues. Assumptions made at this early in the review can grow into unseen security issues.
Data Protection Every diagram node or component should be considered its own computer system (even if it’s just part of one, like a subroutine). Even the “End User” node on a diagram we have to consider that person on their computer using more software, like a web browser, to interact with our application. On each of those nodes or components, we have to analyze what is happening to the data while it’s there. Is it being stored like a database? Then we need to analyze the database security. Is it a user’s browser? Then we need to consider cookies, cache, and local storage security. Is it the application itself? Then how long does sensitive data remain on disk or in memory? This is essential to ensure that the data we are handling is handled according to your company’s data handling policies.
Communications So while this is a question that I use to have to ask all of the time, now-a-days the question is: Are you using secure communication everywhere in the infrastructure (HTTPS, SSH, SFTP, DNSSEC, etc.)? Pretty much there is no excuse now to not ensure secure communications from end-to-end. One important place however to still check is wherever secure communications take place over multiple trust boundary crossings. It’s vital that secure communications renegotiate and continue on even while on the internal network, then back out again.
Business Logic This is fairly straightforward to check for during threat modeling, because business logic requirements need to be explicitly called out by, surprise-surprise, the business. It’s paramount to find these issues as early as possible in the development process, because they become more and more difficult to discover as the application matures. Information security groups should be well informed of what particular business logic your applications and infrastructure must adhere, so it’s simply a case of ensuring the application design follows all relevant business logic rules. The development team should explicitly call out any new business logic rules that are being created due to this application or project.
A good example of this is a failure in the sequence of events during checkout. If you’re able to swap products in your cart during payment, perhaps you can order a more expensive item but be charged a lesser price or avoid paying taxes and shipping. Other threat modeling tips
While I didn’t cover all possible security issues or vulnerabilities in the above section, some I just wait to check for later when I have a chance to interact with the application and its code base. For example, session management is also a big deal when it’s relevant to the application. However those types of details tend to get omitted from larger diagrams because they’re generally ubiquitous to the whole design. However this type of information should have already been laid out in the documentation to ensure that it conforms to AppSec standards. Same goes for logging and error handling, configuration security, file and resource security, and uploading and downloading security.
Who should do Threat Modeling
Now here is the question for the ages regarding threat modeling: Who should be doing threat modeling? This blog post would imply that the security professional or application security engineer should be doing the threat modeling (since that is an intended audience of this blog of course). But I would agree, and may others in the InfoSec space would agree, that perhaps that is not the best person to be threat modeling (at least threat modeling everything outside very critical reviews).
As previously mentioned on this blog, AppSec needs to move past just being a Red Team and a bottle neck to being a more active builder of security tooling. This is mostly due to security not scaling with modern development organizations. It’s common that for every 100 developers there is only one application security person. That is a lot of code not being security reviewed.
Then of course that means developers, quality assurance, and the product owners should be doing the threat modeling. They know the most about the application and how it is supposed to work, so it makes sense that they should be doing the threat modeling.
However, developers may not have the same thought process as a security professional. And developers will, in most cases, be biased about their own code security as it may be a topic of pride. So assumptions may be made from the developers while threat modeling, and developers making assumptions about their code security is a good way to ensure bad security practices.
So what is the middle ground here? Well, first setting up security champions throughout the development organization is a good start. These security champions will be the first go to for AppSec to ensure projects are getting the security attention they require. Second, we need to make sure that whomever is threat modeling an application is not doing against their own projects, because that would be a clear conflict of interest. Lastly, setting up development with more tools and automation to make their lives easier will help make sure that policies and procedures are being followed.
Summary
We learned that although threat modeling is necessary when security reviewing applications, it’s not an exact science with exact instructions. You will need to rely on your experience to define your own personal process for threat modeling. And that whenever possible, attempt to automate manual processes.
When analyzing application security documentation and diagrams, be the journalist looking for the next big scoop. Ask about the who, what, where, when, why, and how for each development decision and diagram layout.
If you only have one takeaway from this blog post, remember this: the development staff should be able to describe exactly how they are planning to address security concerns for each component and component interaction. If they can’t, then we have to assume that there is a possible security issue there.
Does your organization’s AppSec do anything ambitious or unique in setting up success for code security? We would love to hear about it and discuss on Twitter (@HellaSecure).
Helpful Links
OWASP Cheat Sheet series on threat modeling: https://cheatsheetseries.owasp.org/cheatsheets/Threat_Modeling_Cheat_Sheet.html STRIDE Threat Models: https://dev.to/petermbenjamin/demystifying-stride-threat-models-230m Value Driven Threat Modeling: https://www.youtube.com/watch?v=3Fl_7FrM_gI P.A.S.T.A.: https://versprite.com/videos/process-for-attack-simulation-and-threat-analysis-pasta-risk-centric-threat-models/ Attack Trees: https://www.schneier.com/academic/archives/1999/12/attack_trees.html
Threat Modeling Guide: https://www.comparitech.com/net-admin/threat-modeling-guide/
About the Author
Aaron is an Application Security Engineer with over 10 years of experience. His unorthodox career path has led to many unique insights in the security industry.