Blog

An Open Letter to Robert Carr, CEO of Heartland Payment Systems

By Rich

Mr. Carr,

I read your interview with Bill Brenner in CSO magazine today, and I sympathize with your situation. I completely agree that the current system of standards and audits contained in the Payment Card Industry Data Security Standard is flawed and unreliable as a breach-prevention mechanism. The truth is that our current transaction systems were never designed for our current threat environment, and I applaud your push to advance the processing system and transaction security. PCI is merely an attempt to extend the life of the current system, and while it is improving the state of security within the industry, no best practices standard can ever fully repair such a profoundly defective transaction mechanism as credit card numbers and magnetic stripe data.

That said, your attempts to place the blame of your security breach on your QSAs, your external auditors, are disingenuous at best.

As the CEO of a large public company you clearly understand the role of audits, assessments, and auditors. You are also fundamentally familiar with the concepts of enterprise risk management and your fiduciary responsibility as an officer of your company. Your attempts to shift responsibility to your QSA are the accounting equivalent of blaming your external auditor for failing to prevent the hijacking of an armored car.

As a public company, I have to assume your organization uses two third-party financial auditors, and internal audit and security teams. The role of your external auditor is to ensure your compliance with financial regulations and the accuracy of your public reports. This is the equivalent of a QSA, whose job isn’t to evaluate all your security defenses and controls, but to confirm that you comply with the requirements of PCI. Like your external financial auditor, this is managed through self reporting, spot checks, and a review of key areas. Just as your financial auditor doesn’t examine every financial transaction or the accuracy of each and every financial system, your PCI assessor is not responsible for evaluating every single specific security control.

You likely also use a public accounting firm to assist you in the preparation of your books and evaluation of your internal accounting practices. Where your external auditor of record’s responsibility is to confirm you comply with reporting and accounting requirements and regulations, this additional audit team is to help you prepare, as well as provide other accounting advice that your auditor of record is restricted from. You then use your internal teams to manage day to day risks and financial accountability.

PCI is no different, although QSAs lack the same conflict of interest restrictions on the services they can provide, which is a major flaw of PCI. The role of your QSA is to assure your compliance with the standard, not secure your organization from attack. Their role isn’t even to assess your security defenses overall, but to make sure you meet the minimum standards of PCI. As an experienced corporate executive, I know you are familiar with these differences and the role of assessors and auditors.

In your interview, you state:

The audits done by our QSAs (Qualified Security Assessors) were of no value whatsoever. To the extent that they were telling us we were secure beforehand, that we were PCI compliant, was a major problem. The QSAs in our shop didn’t even know this was a common attack vector being used against other companies. We learned that 300 other companies had been attacked by the same malware. I thought, ‘You’ve got to be kidding me.’ That people would know the exact attack vector and not tell major players in the industry is unthinkable to me. I still can’t reconcile that.”

There are a few problems with this statement. PCI compliance means you are compliant at a point in time, not secure for an indefinite future. Any experienced security professional understands this difference, and it was the job of your security team to communicate this to you, and for you to understand the difference. I can audit a bank one day, and someone can accidently leave the vault unlocked the next. Also, standards like PCI merely represent a baseline of controls, and as the senior risk manager for Heartland it is your responsibility to understand when these baselines are not sufficient for your specific situation.

It is unfortunate that your assessors were not up to date on the latest electronic attacks, which have been fairly well covered in the press. It is even more unfortunate that your internal security team was also unaware of these potential issues, or failed to communicate them to you (or you chose to ignore their advice). But that does not abrogate your responsibility, since it is not the job of a compliance assessor to keep you informed on the latest attack techniques and defenses, but merely to ensure your point in time compliance with the standard.

In fairness to QSAs, their job is very difficult, but up until this point, we certainly didn’t understand the limitations of PCI and the entire assessment process. PCI compliance doesn’t mean secure. We and others were declared PCI compliant shortly before the intrusions.

I agree completely that this is a problem with PCI. But what concerns me more is that the CEO of a public company would rely completely on an annual external assessment to define the whole security posture of his organization. Especially since there has long been ample public evidence that compliance is not the equivalent of security. Again, if your security team failed to make you aware of this distinction, I’m sorry.

I don’t mean this to be completely critical. I applaud your efforts to increase awareness of the problems of PCI, to fight the PCI Council and the card companies when they make false public claims regarding PCI, and to advance the state of transaction security. It’s extremely important that we, as an industry, communicate more and share information to improve our security, especially breach details. Your efforts to build an end to end encryption mechanism, and your use of Data Loss Prevention and other technologies, are an important contribution to the industry.

Unless your QSAs were also responsible for your operational security, the only ones responsible for your breach are the criminals, and Heartland itself. I cannot possibly believe that you trusted your PCI audit to determine if you were secure from attack; considering all we know, and all the information available on PCI, that would be borderline negligence. Even if your QSAs were completely negligent and falsified your compliance, that would not make them responsible for your breach.

Rather than blaming your QSAs, I hope you take this opportunity to encourage other executives to treat their PCI assessment as merely another compliance initiative – one that does not, in any way, ensure their security. As an industry professional I see all too many organizations do the minimum for PCI compliance, and ignore the other security risks their organizations face, even when properly informed by their internal security professionals. This is the single greatest problem with PCI, and one you have an opportunity to help change.

If I misread your statements or the article was inaccurate, I apologize for my criticism. If any of my prior criticisms of your organization were unfounded, I take full responsibility and also apologize for those.

But, based on your prior public statements and this interview, you appear to be shifting the blame to the card companies, your QSA, and the PCI Council. From what’s been released, your organization was breached using known attack techniques that were preventable using well-understood security controls.

As the senior corporate officer for Heartland, that responsibility was yours.

Rich Mogull,

Securosis

No Related Posts
Comments

Strong stuff, Rich. And I hope it is read and taken to heart by the people who should do so. —John (who found the post via Twitter)

By John Baxter on


Well said, Rich.

Keep in mind that he’s not saying any of this under oath and likely will never admit they did something wrong because their General Counsel would advise against it. (Lucky for them, their QSAs are probably gagged from responding under confidentiality clauses in their contract.)  Admitting any wrongdoing exposes the company (and himself) to a variety of legal scenarios, which is why companies will often “settle” with regulators and others without admitting guilt.  Corporations and individuals have both been socialized into never admitting wrongdoings as it can only make things worse from the legal standpoint they fear so much. 

I’d wager that the Security team asked for budget to shore up their defenses and, as often happens, it was rejected by someone in the chain of command as “too expensive” without even running the numbers.  Besides, as long as they had the shiny “PCI Compliant” sticker, they believed they had mitigated the risk of increased transaction costs.  So what justification is there to spend the money on security?

I believe that after the writedown they just took in their Q2 earnings, a shareholder class action lawsuit may get to the bottom of what really happened, but as long as it gets settled out of court, it would only benefit the lawyers.

Cheers,
// Chris

By Interested Reader on


Well said, Rich.  This letter is more than a response to one clueless CEO.  It should serve as the basis for any discussion on balancing compliance, operational effectiveness, and actual security.

By Tom Olzak on


Very well put Rich.  I found it laughable at first glance and severely disheartening the more I thought about what Mr. Carr had said to the public.  The thing is I think Mr. Carr knew exactly what he was doing.  If the mainstream press caught wind of the story it would help offset blame of the internal problem for public consumption.  Mr. Carr, to me, is in CYA mode right now as maybe his indefinite future is on the line.  Maybe not, but at the end of the day Mr. Carr feels as if they bought offset of risk through the actions of the QSA.  An auditors job is, at best, a spot check (as noted).  If you want a full blown penetration test / ethical hack performed against your infrastructure then Mr. Carr should have paid for it.

In all reality the director of internal security should end up with the firestorm—sure, not everything can always be prevented even in the best of designed systems (and I’m not just talking about exploit attack vectors), but that particular person is to blame—not the QSA.

If we can learn anything from this it’s that PCI, HIPAA, GLBA, SOX, etc are all minimum levels of security engagement.  IF, you don’t pass PCI with *flying* colors then you have much bigger issues than the QSA will have pointed out to you.  I find that all of my clients are only trying to meet the compliance minimums and I’m trying to tell them that every year they scramble (in some way shape or form) to get “ready” for the next audit.  If you build it in, you do it right, you proactively think about security as an operational requirement you don’t end up in that lather, rinse and repeat mode of security remediation every year.

I liked the open letter, glad someone took the time to address Mr. Carr directly.

-windexh8er

By windexh8er on


I disagree with parts of your criticism.  Heartland hired a third party firm to audit their IT controls against a standard list.  That organization did indeed fail them by either being too lenient with their assessment of those controls or failing to advise their client of known risks.  This situation is analogous to taking your auto to a mechanic for a tune up and only checking the fluids, while missing cracks in your timing belt or a leaky coolant system.  There needs to be a way for corporations to understand the expected quality of their audits and assessments.  I just had a similar conversation with a client regarding how to evaluate assessment proposals.  Maybe there needs to be a Yelp! for QSAs?

Also, there is no indication that this is the case:

“But what concerns me more is that the CEO of a public company would rely completely on an annual external assessment to define the whole security posture of his organization.”

By Robi Papp on


Robi,

Considering the complexity of IT systems I don’t think your mechanic analogy holds true. It’s more like a single mechanic evaluating an entire fleet of vehicles. There’s just too much to cover.

This isn’t an audit quality issue, it’s an audit scope issue, and I have no doubts Mr. Carr was aware of the limitations of the process.

By Rich on


I’d actually take that a step further, Rich, and say that it’s like expecting a single mechanic to evaluate the every aspect of an entire fleet of vehicles, but only hiring him to check the brakes and air bags in a couple days.  PCI’s breadth is so narrow and QSAs are hired for so little a period of time that you can’t legitimately expect it to be a full security assessment.  You didn’t pay him to do it, so blame him when one of the cars’ fuel line leaks on the muffler. 

// Chris

By Interested Reader on


Very interesting take, Rich. I heard Mr. Carr present their story at the Gartner IT Security Summit last month, and I have to say, despite everything I know about PCI, I was compelled by his argument that PCI and Heartland’s QSA let him down. I think it’s easy to get caught up in his argument when the reality is, as you point out, that this breach was outside of the scope of what the QSA was looking for in the first place.

I see the disconnect caused by the differences between two perspectives: I think it’s easy to look down from the top and say, “I don’t like spending money to comply with this reg, but at least we will know we’re secure”. Unfortunately, the folks on the ground supporting the audit are thinking something very different a lot of the time. They are thinking, “how do we get this auditor out of here as quickly as possible with as few new ‘to-do items’ at the end as possible.” With the guys in the trenches looking at pass/fail grading, it’s unlikely that they will communicate that they got a D+ (pass) on their audit. Meanwhile, the guys upstairs see “pass” and they think “we got an A”. Lots of room for holes between those two views.

Still, I really admire Carr for getting out and telling his story and for the way he’s leading his company out of this morass. Besides, how many other CEOs would agree to take the stage at that show?

By Jeff Allen on


Jeff,

I’d be more impressed if he released the actual breach details, which will be invaluable to the rest of the industry.

Any CEO knows better than to think an audit for a compliance initiative makes you secure. I think he’s just misdirecting.

By Rich on


I agree, I hope they release more details. I haven’t done extensive research on it, but I haven’t seen much explanation beyond it being a SQL injection attack in their corporate network that gave some rogue entity access to hunt around their systems for a few months until it found access to their payment network.

I certainly hope they’re more open about this than they appear because in his talk, Carr made a big deal about the idea that perhaps some of his competitors had information about the attack vector that they were not sharing because they considered it a competitive advantage. He also said that Heartland has since worked with FS-ISAC to organize the Payment Processors Information Sharing Council (PPISC) to distribute malware and attack vectors.

By Jeff Allen on


If you like to leave comments, and aren’t a spammer, register for the site and email us at info@securosis.com and we’ll turn off moderation for your account.