Blog

Tech media has fallen down, and it can’t get up

By Mike Rothman

I’m going to rant a bit this morning. I’m due. Overdue, in fact. I have been far too well behaved lately. But as I mentioned in this week’s Incite, summer is over and it’s time to stir the pot a bit.

Tech media isn’t about reporting anymore. It’s about generating page views by hook or by crook, and when that doesn’t work, trying to get vendors to sponsor crappy survey-based reports that rank vendors based on … well, nothing of relevance. The page view whoring has driven quality into the ground. Those folks who used to man the beat of security reporting – giants like Brian Krebs, Ryan Naraine, George Hulme, Dennis Fisher, Paul Roberts, and Matt Hines – have moved out of mainstream media. Matt left the media business altogether (as have many other reporters). Ryan, Paul, and Dennis now work for Kaspersky with their hands in Threatpost. George is a freelance writer. And Krebs is Krebsonsecurity.com, kicking ass and taking names, all while fighting off the RBN on a daily basis.

Admittedly, this is a gross generalization. Obviously there are talented folks still covering security and doing good work. Our friends at DarkReading and TechTarget stand out as providing valuable content most of the time. They usually don’t resort to those ridiculous slideshows to bump page views and know enough to partner with external windbags like us to add a diversity of opinion to their sites.

But the more general tech media outlets should be ashamed of themselves. Far too much of their stuff isn’t worthy of a dog’s byline. No fact checking. Just come up with the most controversial headline, fill in a bunch of meaningless content, SEO optimize the entire thing to get some search engine love, and move on to the next one. Let’s go over a few examples.

A friend pointed me to this gem on ZDNet, highlighting some Webroot research about Android malware. Would you like a Coke or a side of exhaust fumes with that FUD sandwich? It seems the author (Rachel King) mischaracterized the research, didn’t find alternative or contrary opinions and sensationalized the threat in the headline. Ed Burnette picks apart the post comprehensively and calls out the reporter, which is great. But why was the piece green lighted in the first place? Hello, calling all ZDNet editors. It’s your job to make sure the stuff posted on your site isn’t crap. FAIL.

Then let’s take a look at some of the ‘reports’ distributed via InformationWeek. First check out their IDS/IPS rankings. 26 pages of meaningless drivel. The highlight is the overall performance rating, based on what, you ask? A lab test? A demo of the devices? A real world test? Market share? 3rd party customer satisfaction rankings? Of course not. They based them on a survey. Really, an online survey. Assessing performance of network security gear by asking customers if they are happy and about the features of the box they own. That’s pretty objective. I mean, come on, man!

I’d highlight the results, but in good conscience I can’t highlight results that are totally contrary to the research I actually do on a daily basis. And what’s worse is that InformationWeek claims these reports “arm business technology decision-makers with real-world perspective based on qualitative and quantitative research, business and technology assessment and planning tools, and adoption best practices gleaned from experience.” But what qualitative research wouldn’t include Sourcefire in this kind of assessment of the IDS/IPS business?

Their SIEM report is similarly offensive. These are basically blind surveys where they have contracted folks who know nothing about these technologies to compile the data and bang out some text so vendors on the wrong side of the innovation curve (but with name recognition) can sponsor the reports and crow about something. At least with a Magic Quadrant or a Wave, you know the analyst applied their own filter to the lies responses on vendor surveys.

What really hurts is that plenty of folks believe what they read in the trade press. At times I think the Borowitz Report does more fact checking on its news. Far too many unsuspecting end users make short list decisions based on a farcical research reports that don’t even meet The Onion’s editorial standards. I have been around the block a hundred times, and my BS filter is highly tuned. I know what to pay attention to and what to ignore. Everyone else deserves better.

No Related Posts
Comments

@Drew: I was and do speak from experience. I’ve read the reports, reviews and pure FUD that the executives have thrown my way. And I will keep doing so… not because I’m masochistic but since it is the only way to combat the FUD.

I have issue with with crap created and sold for >>>PROFIT<<< and not because it is a useful tool. In fact, all the most useful tools (software ones of course; hardware is a different issue) I’ve come across are readily available FREE (the paid for ones usually just have a lot of nice conveniences added… but as a coder I have no probs building my own conveniences).

The above issue with selling crap for >>>PROFIT<<< also and most definitely goes for reports and reviews. Skewed information in the hands/minds of executives is as bad (and to be honest, sometimes worse) than letting them into the data center with a pair of scissors.

You want to reach “IT pros”... then build a wiki for us to use for a product. Otherwise, please don’t add a bunch of FUD… I mean, GAS… to the fire.


Personally, when I go looking for information on a product (software or hardware) I use my Google-Fu to search for all the complaints, problems and issue that people have with the product. I find that this is more informative and provides incites into the companies response to these as well as if the product is worth getting in spite of these.


PS: not to “attack” you but I’ve always considered ‘ranking reports’ in the security field to be about as useful as trying to select a product by juggling live clawed cats. Just saying.

By Zac


Hi Mike,

I’m the editor of those reports you cited, and I wanted to respond to your comments.

Our IT Pro Ranking reports let potential customers get insight from their peers who’ve used or evaluated these products. The rankings don’t come from vendors and they don’t come from analysts paid by vendors—they come from other IT professionals. That’s a valuable source of information for someone who wants to get a sense of the market or put together a shortlist of products to review in depth.

Is it the same kind of information they might get from a lab test or demo? Of course not. But every third-party evaluation of a product has some kind of limitation. We don’t expect, nor do we suggest, that readers simply run out and purchase the product that had the highest rating.

You suggested that vendors sponsor these reports, but that’s not true. IT Pro Ranking reports aren’t sponsored by vendors. Vendors have zero input into the process. These reports are driven by the responses of IT pros with real-world experience. In the case of the IPS report, we had 458 responses. Where else can a potential customer of an IPS product get the opinion of almost 500 of their peers?

You’re entitled to critique our methodology and to point out the limitations of a survey. We know it’s an imperfect instrument. But I believe our IT-driven results provide real value in helping companies make a decision.

I invite your readers to heed your own advice: don’t just do what some tech site told you to do. Go and see the reports for yourselves, and then decide.

Thanks,
Drew

By Drew Conry-Murray


>>
Why are you surprised - this downward quality spiral is happening in *all* media.  “News” sites/shows try to make it seem as if they are objective, but clearly aren’t (pick any side), and the overwhelming population of the (1st) world appears to be more interesting in KimK’s behind than actual, make-you-mad-if-you-actually-knew information. Ad buys are much more important than the content which appears between them.
<<

Ignorance is bliss and we all want to be blissful.

Panem et circenses…

By ds


Truly the FUD in our industry makes the work we do harder (who hasn’t had an executive tell you all about the great piece of ‘security technology’ they just read about - and how it will instantly make our jobs redundant and so get it and work harder!).

To be honest I have always wanted to just DoS these sites and burn all their hard copy.

But there are some lines that I can’t in good conscience cross… besides, snake oil purveyors are like roaches, just as you kill one it turns out there are 10 more in the wainscoting.

:(

By Zac


Why are you surprised - this downward quality spiral is happening in *all* media.  “News” sites/shows try to make it seem as if they are objective, but clearly aren’t (pick any side), and the overwhelming population of the (1st) world appears to be more interesting in KimK’s behind than actual, make-you-mad-if-you-actually-knew information. Ad buys are much more important than the content which appears between them.

Even in our little niche field, majority “reporting” i see now is via 140 characters pointing to some *other* persons little snippet of “news”, which is often so shallow itself I feel cheated after doing some web spelunking.  Real honest-to-goodness, field/view changing research, is few and far between (even inside academia IMO).

Idocracy is just around the corner ;)

(sorry, my little rant is now over)

By MikeA


With good looks, you don’t need to rely on tricks.

By George Hulme


Without Tech Media, where would I get my ‘top ten security tricks’?

-Adrian

By Adrian Lane


If you like to leave comments, and aren’t a spammer, register for the site and email us at info@securosis.com and we’ll turn off moderation for your account.