Web 2.0

Coming Soon: Bit.ly Adding Real Time Security Scanning for All Links

By Rich

Like many of you, for a long time I really couldn't see the use of those URL shortener service thingies. Sure, when I was designing sites I tried to avoid long, ugly URLs, but I never saw slapping some random characters after a common base URL as being any more useful. I considered my awareness of the existence of these obscure services as an aberration induced by my geek genes, rather than validation of their existence or popularity.

Then came Twitter, and the world of URLs was never the same.

Twitter firmly swapped URL shorteners out of the occasionally useful into the pretty darn essential column. That magical 140 character limit, combined with the propensity of major sites to use URLs nearly as long as their software user agreements, thrust shorteners in front of millions of new eyeballs.

One issue, pointed out by more than a few security pundits and rickrolling victims, is that these shorteners completely obscure the underlying URL. It's trivial for a malicious attacker to hide a link and redirect a user to any sort of malicious site. It didn't take long for phishers and drive-by malware attacks to take advantage of the growing popularity of these obfuscation services.

Some of the more popular Twitter clients, like Tweetie, added optional URL previews to show users the full link before clicking through to the site. In part, this was enabled by shorteners like bit.ly enabling previews through their APIs. A nice feature, but it's not one that most users enable, and it isn't available in most web interfaces or even all standalone Twitter clients.

Bit.ly announced today that they are taking things one major step further and will soon be scanning all links, in real time, using multiple security services. Bit.ly will be using a collection of databases and scanning services to check both new and existing links as users access them. Websense's cloud-based scanner is one of the services (the one that pre-briefed me), and bit.ly will use at least one other commercial service as well as some free/open databases.

Update: according to the bit.ly blog, VeriSign and Sophos are the other scanning/database engines.

In the case of Websense, bit.ly will tie directly into their content scanning service to check links in real time as they are added to the bit.ly database. Websense uses a mix of real time scans (for things like malware and certain phishing techniques) and their database of known bad sites. The system won't rely only on the database of previously-detected bad sites, but will also check them at access time.

If a link is suspected of being malicious, Websense marks it and bit.ly will redirect users to a warning page instead of directly to the site. Users can still click through, and I'm sure plenty will, but at least those of us with a little common sense are less likely to be exploited.

Bit.ly won't only be scanning new links added to the database, but will be checking existing links in case they've become compromised. This also reduces the chances of the bad guys gaming the system by adding a clean version of their site for an initial scan, then sneaking in malware for future visits.

I like bit.ly's approach of checking existing links in case they get compromised, rather than only scanning new links as they are added. This will make it harder for bad guys to game the system. This solution is a lot better than the anti-phishing built into browsers and some search engines, since those rely only on databases of previously-discovered known bad sites.

It's also a two-way system, and although Websense is being paid for the scanning, they gain the additional benefit of now leveraging the results once millions of new (and old) links start flowing through their service. Every bad website Wensense finds when a user submits a link to bit.ly is added to the database used by all their other products.

Finally, there's nothing that says we're only allowed to use bit.ly for Twitter. The entire Internet now gains a real-time security scanning service... for free. Have a questionable link? Shorten it through bit.ly and it's scanned by Websense and at least one other commercial service, as well as all the free/open/cheap databases bit.ly uses (sorry, I don't know what they are).

This isn't to say that any of the individual scans, or all of them together, can identify every malicious link they encounter, but this is a significant advance in web services security. It's a perfect example of cloud computing enhancing security, rather than creating new risks. Links sent through bit.ly will now be safer than the original links viewed directly.

This isn't live yet, but should be by the end of the year.

–Rich

Control Your Identity

By Rich

One of the sessions I enjoyed at DefCon was Nathan Hamiel and Shawn Moyer's, "Satan is on My Friends List". Aside from directly hacking the security of some of these sites, they experimented with creating fake profiles of known individuals and seeing who they could fool. Notably, they created a profile (with permission) for Marcus Ranum on LinkedIn, then tried to see how many people they could fool into connecting to it. Yes, folks, I fell for it.

In my case it wasn't that big a deal- I only use LinkedIn as a rolodex, and always default to known email accounts before hopping into it. But that's not how everyone sees it, and many people use it to ask questions, connect to people they want to be associated with but aren't really connected to. Someone behind a fake profile could spoof all sorts of communications to either gather information or manipulate connections for nefarious reasons (pumping stock prices, getting fake references, disinformation campaigns, and so on). All social networks are vulnerable to manipulation, real world or virtual, but when you remove face to face interaction you eliminate the biggest barrier to spoofing.

I avoid some of this by only linking to people I know, have met, and have a reason to keep in contact with. If you've sent me a link request because you read the blog or listen to the podcast, and I haven't responded, that's why. Otherwise it loses any usefulness as a tool for me.

One of Shawn's recommendations for protecting yourself is to build a profile, even if you don't actively use it, on all the social networks. Thus I now have MySpace and Facebook pages under my real name, tied to a throwaway email account here at Securosis. WIll it help? Maybe not- it's easy for someone to create another account with my name and a different email address, but after I tie in a few friends that should reasonably draw people to the real me, whatever that's worth.

One unexpected aspect of this was a brief blast of mortality as Facebook splattered my high school graduating class on a signup page. I haven't really stayed in touch with many people from high school days; in my mind's eye they were frozen in the youth and vibrance of those few years we felt we ruled the world. Seeing them suddenly years later, long past the days of teenage hopes and dreams, was a visceral shock to the system. No, we're not all that old, but at 37 we're far past any reasonable definition of youth.

Damn you Mr. Moyer. I can forgive you for mildly pwning me in your presentation, but smashing open my vaulted teenage memories with a lance of reality? That sir, I can never forgive.

–Rich