When we set out to make our testing product at Metlo we took a look at the different types of common vulnerabilities (primarily the OWASP Top 10) and tried to find ways to automatically detect them using our testing language. However, out of all the vulnerabilities, the two that stuck out most were Authorization and Authentication vulnerabilities. This was because 3 out of the top 10 vulnerabilities were related to Auth and there were no actual solutions on the market to detect these vulns in an automated way, especially at earlier stages of the development lifecycle. This was magnified by the fact that these vulnerabilities are one of the most direct vectors to actually compromise an app.

Part of the problem is that these issues are deeply rooted in the business logic around how auth works for an individual app. This logic isn’t accessible by any scanner and in many cases isn’t formally defined at all. In this post I’ll go over how we designed a way to quickly represent how auth works for any particular app and how Metlo uses this information to automate testing for almost all AuthN and AuthZ vulnerabilities!

How Big of an Issue is Auth?

Before we dive in to how we test for vulnerabilities, let’s dig into how big of an issue they actually are. I think one of the best ways to understand the importance of a vuln is to look at the aggregate stats published by hackerone. Although the OWASP Top 10 list is great, it doesn’t give us numerical stats on the relative importance of each type of issue.

We can take a look at the top 10 vulnerability types to get a rough idea of how common Auth issues are. If we count Improper Access Control, Improper Authentication, IDOR, and Privilege Escalation all as auth related, they add up to around 27% of all vulnerabilities and 41% of all payouts.

It’s also important to note that this report was released in 2020 and auth vulnerabilities were some of the fastest growing! In the latest report (required an email to view), 45% of the payouts  were for auth vulnerabilities!

Detecting Auth Vulnerabilities Today

There’s a few great burp suite plugins you can use like AuthMatrix and autorize but they have a few issues:

• You have to go through each endpoint manually and define its permissions
• Difficult to automate in CI/CD
• Complex roles and permissions are tricky and tedious to encode

The manual work involved in testing each endpoint makes it really easy for auth vulnerabilities to slip into production and in many cases its often easiest to just let bug bounty hunters find them. From Hackerone’s own website: “These vulnerabilities are prevalent because they’re nearly impossible to detect using automated tools. Hacker-powered security provides a relatively inexpensive and extremely effective method for mitigating these vulnerabilities.”

Representing Business Logic

So why can’t we automatically detect auth vulnerabilities and how can Metlo make it possible? The broader issue is that, given an endpoint, most security teams don’t really have a structured way to define what its permissions are. Without that, its impossible to automatically build test cases! What we need is a simple to write DSL that describes how auth works for any given API.

At its core our DSL needs to describe the different Permissions (read, write, delete, …) that different Actors (Users, API Keys, etc..) have for different Resources (Products, Billing Info, Profiles, …).

Actors

Lets start with defining the different actors for a test API. This can be as simple as a list of JSON objects where each object has an auth field we can authenticate with. We can add an admin user and two regular users so our tests can cover all auth permutations that exist an our API.

actor User {
    items = [
        {
            "uuid": "fb0bd533-3d87-4e67-925d-5d4fed5ffeda",
            "name": "Admin User",
            "role": "admin",
            "auth": "<An Admin Auth Key>"
        },
        {
            "uuid": "cfe3b553-186d-4ab3-a501-babcd26b1c73",
            "name": "Regular User 1",
            "role": "regular",
            "auth": "<A Regular User Auth Key>"
        },
        {
            "uuid": "f2392cfa-666c-4a7e-b07b-1151dfffb7bb",
            "name": "Regular User 2",
            "role": "regular",
            "auth": "<Another Regular User Auth Key>"
        }
    ]
}

Resources

After we define the Actors in our API, we need to define the different Resources that they’ll interact with and the permissions these Resources have. This can be done very similarly to how we defined actors but with an additional list of permissions that are possible for a resource.

Here’s an example of a Product resource that users can read, write, and delete:

resource Product {
    permissions = ["read", "write", "delete"],

    items = [
        {
            "uuid": "96665ba0-e248-4360-83b9-cf1d6ffb727f",
            "name": "some test product"
        },
        {
            "uuid": "cbd8f3ed-12f9-4715-af91-471e063cbb55",
            "name": "another other test product"
        }
    ]
}

After we define the different permissions and some sample items for a resource, we also have to specify which endpoints deal with this resource and which permissions they need. This is the most important part! Being able to write generic rules here makes it so that we can define permissions for many endpoints all at once and catch any new endpoints that are created later without making any changes to this config. Here are some example rules you can make:

• All GET endpoints whose paths start with /product need read access to the Product resource.
• All POST and PUT endpoints that contain the Product resource in the request need write access to the Product resource.
• All DELETE endpoints whose paths start with /product need delete access to the Product resource.

resource Product {
    permissions = [ ... ],
    items = [ ... ],

    endpoints = [
        # All GET endpoints whose paths start with /product need read access to Product
        {
            "method": "GET",
            "path": "^/product/.*",
            "permissions": ["read"]
        },
        # All PUT and POST endpoints whose paths start with /product need write access to Product
        {
            "method": ["PUT", "POST"],
            "contains_resource": { "path": "req.path" },
            "permissions": ["write"]
        },
        # All DELETE endpoints whose host match ^.*\.product-service\.com need write access to product
        {
            "method": "DELETE",
            "path": "^/product/.*",
            "permissions": ["write"]
        }
    ] 
}

Relations

After we’ve defined our Actors and Resources, the next step is to define the relationships that they have between each other. For that we’ve designed the following simple, but expressive DSL:

# All Admin users have read, write and delete access to all Products
has_permission(User(role="admin"), [ "read", "write", "delete" ], Product)

# All Users have read access to all Products
has_permission(User, [ "read" ], Product)

# User with uuid cfe3b553... has write access to Product with uuid 96665ba0...
has_permission(
    User(uuid="cfe3b553-186d-4ab3-a501-babcd26b1c73"),
    [ "write" ],
    Product(uuid="96665ba0-e248-4360-83b9-cf1d6ffb727f")
)

# User with uuid f2392cfa... has write access to Product with uuid cbd8f3ed...
has_permission(
    User(uuid="f2392cfa-666c-4a7e-b07b-1151dfffb7bb"),
    [ "write" ],
    Product(uuid="cbd8f3ed-12f9-4715-af91-471e063cbb55")
)

Tagging Requests

The final step to automatically generate our auth tests is to tag which fields in our API relate to each resource. This is an example of tagging a field as a Product uuid.

This might seem like it would be a lot of work but once you tag a few fields and you send some example API traffic to Metlo using one of our integrations (including a Burp Suite Extension), Metlo will automatically learn the rest!

Running Tests

After you’ve defined all your Actors, Resources, and relations, Metlo can automatically generate a suite of tests that test for every single AuthZ and AuthN issue possible in your API. You can run tests in either the Metlo UI or the CLI (useful for your CI/CD). We have much more info in our docs here!

Conclusion

So thats about it! With our new Auth testing tool our goal is to make it so you can spend a couple hours defining the Actors and Resources for an API, and get continuous scanning for auth vulnerabilities at every stage of development. Whenever a new endpoint is created, as long as it isn’t dealing with a new Resource, Metlo will automatically generate the auth tests it needs with no config changes!

If you’re interested in tackling ~45% of your API security risk by setting up Auth tests for your API, schedule a meeting with me here and we can have most of this set up in the meeting!