Securosis

Research

Friday Summary: April 13th, 2012

Happy Friday the 13th! I was thinking about superstition and science today, so I was particularly amused to notice that it’s Friday the 13th. Rich and I are both scientists of sorts; we both eschew superstition, but we occasionally argue about science. What’s real and what’s not. What’s science, what’s pseudoscience, and what’s just plain myth. It’s interesting to discuss root causes and what forces actually alter our surroundings. Do we have enough data to make an assertion about something, or is it just a statistical anomaly? I’m far more likely to jump to conclusions about stuff based on personal experience, and he’s more rigorous with the scientific method. And that’s true for work as well as life in general. For example he still shuns my use of Vitamin C, while I’m convinced it has a positive effect. And Rich chides as I make statements about things I don’t understand, or assertions that are completely ‘pseudoscience’ in his book. I’ll make an off-handed observation and he’ll respond with “Myth Busters proved that’s wrong in last week’s show”. And he’s usually right. We still have a fundamental disagreement about the probability of self-atomizing concrete, a story I’d rather not go into – but regardless, we are both serious tech geeks and proponents of science. I regularly run across stuff that surprises me and challenges my fundamental perception of what’s possible. And I am fascinated by those things and the explanations ‘experts’ come up with for them – usually from people with a financial incentive. Hawking anything from food to electronic devices by claiming benefits we cannot measure, or for which we don’t have science which could prove or disprove their clams. To keep things from getting all political or religious, I restrict my examples to my favorite hobby: HiFi. I offer power cords as an example. I’ve switched most of the power cords to my television, iMac, and stereo to versions that run $100 to $300. Sounds deranged, I know, to spend that much on a piece of wire. But you know what? The colors on the television are deeper, more saturated, and far less visually ‘noisy’. Same for the iMac. And I’m not the only one who has witnessed this. It’s not subtle, and it’s completely repeatable. But I am at a loss to understand how the last three feet of copper between the wall socket and the computer can dramatically improve the quality of the display. Or the sound from my stereo. I can see it, and I can hear it, but I know of no test to measure it and I just don’t find the explanations of “electron alignment” plausible. Sometimes it’s simply that nobody thought to measure stuff they should have because theoretically it shouldn’t matter. In college I thought most music sounded terrible and figured I had simply outgrown the music of my childhood. Turns out that in the 80s, when CDs were born, CD players introduced several new forms of distortion, and some of them were unmeasurable. Listener fatigue became common, many people getting headaches as a result of these poorly created devices. Things like jitter, power supply noise, noise created by different types of silicon gates and capacitors, all producing sonic signatures audible to the human ear. Lots of this couldn’t be effectively measured but will send you running from the room. Fortunately over the last 12 years or so audio designers have become aware of these new forms of distortion, and they now have devices that can measure them to one degree or another. I can even hear significant differences with various analog valves (i.e. ‘tubes’) where I cannot measure electrical differences. Another oddity I have found is with vibration control devices. I went to a friend’s house and found his amplifiers and DVD players suspended high in the air on top of maple butcher blocks, which sat on top of what looked like a pair of hockey pucks separated by a ball bearing. The maple blocks are supposed to both absorb vibration and avoid electromagnetic interference between components. And we did several A/B comparisons with and without each, but it was the little bearings that made a clear and noticeable difference in sound quality. The theory is that high frequency vibrations, which shake the electronic circuits of the amps and CD players, decrease resolution and introduce some form of distortion. Is that true? I have no clue. Do they work? Hell yes they do! I know that my mountain bike’s frame was designed to alter the tube circumference and wall thicknesses as a method of dampening vibrations, and there is an improvement over previous generations of bike frames, albeit a subtle one. The reduction in vibrations on the bike can easily be measured, as can the vibrations and electromagnetic interference between A/V equipment. But the vibrational energy is so vanishingly small that it should never make a difference to audio quality. Then there are the environmental factors that alter the user’s perception of events. Yeah, drugs and alcohol would be an example, but sticking to my HiFi theme: a creme that makes your iPod sound better. Works by creating a positive impression with the user. Which again borders on the absurd. An unknown phenomena, or snake oil? Sometimes it’s tough to tell superstition from science. On to the Summary: Webcasts, Podcasts, Outside Writing, and Conferences Adrian’s Dark Reading paper on User Activity Monitoring. Rich’s excellent Macworld article on the Flashback malware. Adrian’s Dark Reading post on reverse database proxies. Favorite Securosis Posts Adrian Lane: The Myth of the Security-Smug Mac User. We get so many ‘news’ items, like how Android will capture the tablet market in 2015, or how Apple’s market share of smartphones is dwindling, or how smug Apple users will get their ‘comeuppance’ for rejecting AV solutions, that you wonder who’s coming up with this crap. Mac users may not have faith in AV to keep them secure, but they know eventually Macs will be targeted just as Windows has been. And I’m fairly certain most hackers run on

Share:
Read Post

Totally Transparent Research is the embodiment of how we work at Securosis. It’s our core operating philosophy, our research policy, and a specific process. We initially developed it to help maintain objectivity while producing licensed research, but its benefits extend to all aspects of our business.

Going beyond Open Source Research, and a far cry from the traditional syndicated research model, we think it’s the best way to produce independent, objective, quality research.

Here’s how it works:

  • Content is developed ‘live’ on the blog. Primary research is generally released in pieces, as a series of posts, so we can digest and integrate feedback, making the end results much stronger than traditional “ivory tower” research.
  • Comments are enabled for posts. All comments are kept except for spam, personal insults of a clearly inflammatory nature, and completely off-topic content that distracts from the discussion. We welcome comments critical of the work, even if somewhat insulting to the authors. Really.
  • Anyone can comment, and no registration is required. Vendors or consultants with a relevant product or offering must properly identify themselves. While their comments won’t be deleted, the writer/moderator will “call out”, identify, and possibly ridicule vendors who fail to do so.
  • Vendors considering licensing the content are welcome to provide feedback, but it must be posted in the comments - just like everyone else. There is no back channel influence on the research findings or posts.
    Analysts must reply to comments and defend the research position, or agree to modify the content.
  • At the end of the post series, the analyst compiles the posts into a paper, presentation, or other delivery vehicle. Public comments/input factors into the research, where appropriate.
  • If the research is distributed as a paper, significant commenters/contributors are acknowledged in the opening of the report. If they did not post their real names, handles used for comments are listed. Commenters do not retain any rights to the report, but their contributions will be recognized.
  • All primary research will be released under a Creative Commons license. The current license is Non-Commercial, Attribution. The analyst, at their discretion, may add a Derivative Works or Share Alike condition.
  • Securosis primary research does not discuss specific vendors or specific products/offerings, unless used to provide context, contrast or to make a point (which is very very rare).
    Although quotes from published primary research (and published primary research only) may be used in press releases, said quotes may never mention a specific vendor, even if the vendor is mentioned in the source report. Securosis must approve any quote to appear in any vendor marketing collateral.
  • Final primary research will be posted on the blog with open comments.
  • Research will be updated periodically to reflect market realities, based on the discretion of the primary analyst. Updated research will be dated and given a version number.
    For research that cannot be developed using this model, such as complex principles or models that are unsuited for a series of blog posts, the content will be chunked up and posted at or before release of the paper to solicit public feedback, and provide an open venue for comments and criticisms.
  • In rare cases Securosis may write papers outside of the primary research agenda, but only if the end result can be non-biased and valuable to the user community to supplement industry-wide efforts or advances. A “Radically Transparent Research” process will be followed in developing these papers, where absolutely all materials are public at all stages of development, including communications (email, call notes).
    Only the free primary research released on our site can be licensed. We will not accept licensing fees on research we charge users to access.
  • All licensed research will be clearly labeled with the licensees. No licensed research will be released without indicating the sources of licensing fees. Again, there will be no back channel influence. We’re open and transparent about our revenue sources.

In essence, we develop all of our research out in the open, and not only seek public comments, but keep those comments indefinitely as a record of the research creation process. If you believe we are biased or not doing our homework, you can call us out on it and it will be there in the record. Our philosophy involves cracking open the research process, and using our readers to eliminate bias and enhance the quality of the work.

On the back end, here’s how we handle this approach with licensees:

  • Licensees may propose paper topics. The topic may be accepted if it is consistent with the Securosis research agenda and goals, but only if it can be covered without bias and will be valuable to the end user community.
  • Analysts produce research according to their own research agendas, and may offer licensing under the same objectivity requirements.
  • The potential licensee will be provided an outline of our research positions and the potential research product so they can determine if it is likely to meet their objectives.
  • Once the licensee agrees, development of the primary research content begins, following the Totally Transparent Research process as outlined above. At this point, there is no money exchanged.
  • Upon completion of the paper, the licensee will receive a release candidate to determine whether the final result still meets their needs.
  • If the content does not meet their needs, the licensee is not required to pay, and the research will be released without licensing or with alternate licensees.
  • Licensees may host and reuse the content for the length of the license (typically one year). This includes placing the content behind a registration process, posting on white paper networks, or translation into other languages. The research will always be hosted at Securosis for free without registration.

Here is the language we currently place in our research project agreements:

Content will be created independently of LICENSEE with no obligations for payment. Once content is complete, LICENSEE will have a 3 day review period to determine if the content meets corporate objectives. If the content is unsuitable, LICENSEE will not be obligated for any payment and Securosis is free to distribute the whitepaper without branding or with alternate licensees, and will not complete any associated webcasts for the declining LICENSEE. Content licensing, webcasts and payment are contingent on the content being acceptable to LICENSEE. This maintains objectivity while limiting the risk to LICENSEE. Securosis maintains all rights to the content and to include Securosis branding in addition to any licensee branding.

Even this process itself is open to criticism. If you have questions or comments, you can email us or comment on the blog.