HTTP/2 support and annoying bugs fixed

The number of ips and urls to monitor keeps increasing. This is good news of course, but it also means needing to pay attention to scaling properly. Last week it was finally time to drop a very inefficient JOIN statement from the frontend code. This has been in the making for more than 6 months, since the legacy code and data requiring the JOIN had to be phased out first.

Another performance boost is that since today ShadowTrackr supports HTTP/2. The new ability to process parallel requests helps speed up the loading of some of the slowest pages.

Some clients had a high volume of annoying messages on their timeline, and that should be fixed now. The most important changes:

• Clear and timely SSL certificate messages
• Websites for ignored urls are now ignored too
• The false positive for doubly issued certificates is gone

Lastly, Amazon Cloudfront users experienced some timeline spam due to a lagging ip range update. This resulted in a lot of new ip messages. As of today, Amazon and Cloudflare ip ranges are updated automatically and this problem is fixed.

If you still have annoying messages that you'd like to get rid of, let me know!

Finegrained Microsoft cloud detection

One client rightfully complained that his timeline got all clogged up with not so useful banner changes. I was fixing this by moving the details to the details section (duh) of the notifications. This way they only become visible after you explicitly click something. While working on this I noticed that the most chatty servers in terms of banner changes where Microsoft Exchange online servers. This should not be, cloud services like this are supposed to be recognized and handled differently.

What was happening? I immediately updated the Microsoft IP ranges, which up until now was done manually. But this only fixed part of the problem. There were still servers that were clearly part of the Microsoft cloud that were not recognized. Microsoft publishes the ip ranges for the services (Office 365, Exchange, Sharepoint, Skype, etc,) they offer in these big 5 clouds:

• Worldwide Public and Government Community Cloud (GCC)
• U.S. Government GCC High
• U.S. Government DoD
• Germany
• China (Vianet)

I thought that "Worldwide Public and GCC" had to be located in one place, and since most of the ips in this range where in the U.S., it probably all was in the U.S. Consequently, all these ips were labeled "US public" in ShadowTrackr.

I did also know from some Dutch government clients that their servers are located in the Netherlands in the datacentre MS calls "Europe West" (yes, it's a shame they don't call it "Netherlands Central"). After a bit of searching it turned out that there are many more of these local datacentra. Microsoft even publishes a nice map that shows them all: https://azure.microsoft.com/en-us/global-infrastructure/regions/

And there is a weekly updated XML file available on a different part of the Microsoft site. This XML file has ip ranges that only partly overlap with the big 5, which explains why ShadowTrackr did not recognize some servers as being in the Microsoft cloud. Problem solved!

So, if you have servers at Microsoft you might see them change the cloud description on your timeline. The old cloud clusters will still be available for about a week on your attack surface map in parallel to the new (actual) ones.

Oh, and cloud ip ranges are now automatically updated daily :-)

Better weekly pdf reports

The weekly pdf reports have been available for a while. You could even already subscribe and get them by email. This option wasn't really promoted because, to be honest, the reports were quite boring. The good news is, that has been fixed!

The first page of the new weekly report now has fancy colored donut charts that will give you an instant overview of how many assets you have and how big your problems are. There is also a graph showing how your events are distributed over the week. Critical alerts now have a colored icon showing how critical it is, just like on your timeline. And ip addresses and urls for blacklisted items, new assets and most active assets are now clickable. The links will take you straight to the source of the information in de webversion of ShadowTrackr.

Although this is a big improvement, it's not yet done. The last part of the report is still much like it was before, and traps and keywords are not properly reported. This is still a work in progress, I'll keep you updated.