Asset grouping and deleting urls

About 30% of the clients have hundreds of urls in their assets, and some even go beyond 1000. This very long, flat list is not very user friendly. So, time for some UX improvement.

I took a look at several of these long lists, and, as expected, there are many subdomains for the same pay level domains. This is good, since it allows grouping them. The new view under assets lists all your domains as clickable groups with the number of subdomains in front:

+ (3) domain.com

When you click it, it will show all subdomains:

- (3) domain.com
  a.domain.com
  b.domain.com
  c.domain.com

Much better :-) Just as before, the domains are sorted alpabetically on the pay level domain ("b.a-domain.com" will appear before "a.b-domain.com").

Another change is that you can now delete all urls, not just the ones you have added manually. I'm still experimenting with how this should be done, and it's likely to change again someday. There are urls that you legitimately want to delete (since they're no longer yours for instance), so the option should be there.

The problem is that some urls are related to you, and even if you don't like it, they will be discovered and added again. No matter how often you delete them. A delete option for these offers false hope, and I don't like the disappointed that follows later on.

Also, I think it's a good idea to keep monitoring your old urls that expired. These are the ideal candidates for setting up phishing sites. The same holds for those internal urls that should not appear on the internet. I hopw to come up with a better solution one day, but until then: be careful when deleting!

The blacklist counter from hell

The blacklisted page up until now listed all your hits on blacklists. That is, every hit is a separate entry in the table on your screen and is counted as a problem. The number of problems you have screams at you as a number in a red dot in the menu on your screen. Sounds good right?

As one client showed me, some ideas sound good in theory but turn into the blacklist counter from hell in practice. ShadowTrackr at this moment checks your ip addresses and websites against 127 blacklists. A lot of these blacklists overlap and from a security point of view that's just fine. You'd rather be notified twice than not at all.

When an ip gets listed as a source of SPAM on one blacklist, the chances are high that a couple of other blacklists will pick it up too. Since the counter counted the number of blacklist entries, 2 machines getting listed on 4 spamlists resulted in the number 8 screaming at you from the bright red dot. That is not the user experience I intended. In that case you have 2 problems, not 8. The counter is fixed now, and all blacklist entries are sorted per asset now.

What remains is the question on how to handle notifications. For the first time your asset is listed on any blacklist, everyone will want to receive a notification. But how about the second or third blacklist that same asset gets listed on? Do you want to know? I myself would like to get notified of every extra blacklist an asset appears on, so I left it on for now. But if enough users convince me otherwise I'll be happy to turn it off of course. Just let me know!

Red dots on the attack surface map

The attack surface map gives you a good overview of your assets and how they're are related. You can quickly see where most of your servers and websites are, and easily spot the outliers.Wouldn't it be great if it also showed where your problems are? Starting today, it does!

Any ip or url that is on a blacklist somewhere will turn red. Websites with troublesome certificates will be orange, and bad certificates will be red too. Of cource, there's a similar rating for servers. If a server has a troublesome port open it will be orange. The really bad ones (think pownable or DDOS amplifiers) will be red.

I'm quite happy with the result. You'll have an instant view of where most of your problems are and where you need to start improving your security. The thing that does need some work is the layout for really big (3000+ assets) organisations. It still works, but it's just not as beautiful. The attack surface map is built with D3 and it allows for very specific tweaking of the various forces in the force-layout graph that I use, so it should be solvable. I've put it on my todo list and will come back on this later. For now, have fun with the new fancy attack surface map.