Use of tokenization continues to expand as customers look to simplify PCI-DSS compliance. With this increased adoption comes a lot of vendor positioning and puffery, as they attempt to differentiate their products in an increasingly competitive market. Unfortunately this competitive positioning often causes confusion among buyers, which is why I have spent the last couple mornings answering questions on FPE vs. Tokenization, and the difference between a token vault and a database. Lately most questions center on differentiating tokenization data vaults, with the expected confusion caused by vendor hyperbole. In this post I will define a token vault and shed some light on their pros and cons. My goal is to help you determine as a consumer whether vaults are something to consider when selecting a tokenization solution.
A token vault is where you store issued tokens and the credit card numbers they represent. The storage location is called a “token vault”. The vault typically contains other information, but for this discussion just think of the token vault as a long list of CC#/token pairs.
A new type of solution called ‘stateless’ or ‘vault-less’ tokenization is now available. These systems use derived tokens, which can be recalculated from some secret value, and those do not need to be stored in a database. Recent press hype claims that token vaults are bad and you should stay away from them. The primary argument is “you don’t want a relational database as a token vault” – or more specifically, “an Oracle database makes a slow and expensive token vault, and customers don’t want that”.
Not so fast! The issue is not clear-cut. It’s not that token vaults are good or bad, but of course there are tradeoffs. Token vaults are fine for many types of customers, but not suitable for others. There are three issues at the heart of this debate: cost, scale, and performance. Let’s take a closer look at each of them.
- Cost: If you are going to use an Oracle, IBM DB2, or Microsoft SQL Server database for your token vault, you will need a license for the database. And token vaults must be redundant so you will need at least a couple licenses. If you want to ensure that your tokenization system can handle large bursts of transactions – such as holiday shopping periods – you will need hefty servers. Databases are priced based on server capacity, so these licenses can get very expensive. That said, many customers running in-house tokenization systems already have database site licenses, so for many customers this is not an issue.
- Scale: If you have data processing sites where token servers are dispersed across remote data centers that cannot guarantee highly reliable communications, synchronization of token vaults is a serious issue. You need to ensure that credit cards are not misused, that you have transactional consistency across all locations, and that no token is issued twice. With ‘vault-less’ tokenization synchronization is a non-issue. If consistency across a scaled tokenization deployment is critical derived tokens are incredibly attractive. But some non-derived token systems with token vaults get around this issue by pre-allocating token sequences; this ensures tokens are unique, and synchronization latency is not a concern. This is a critical advantage for very large credit card processors and merchants but not a universal requirement.
- Performance: Some token server designs require a check inside the token vault prior to completing every transaction, in order to ensure to avoid duplicate credit cards or tokens. This is especially true when a single token is used to represent multiple transactions or merchants (multi-use tokens). Unfortunately early tokenization solutions generally had poor database architectures. They did not provide efficient mechanisms of indexing token/CC# pairs for quick lookup. This is not a flaw in the databases themselves – it was a mistake made token vault designers as they laid out their data! As the number of tokens climbs into the tens or hundreds of millions, lookup operations can become unacceptably slow. Many customers have poor impressions of token vaults because their early implementations got this wrong. So very wrong. Today lookup speed is often not a problem – even for large databases – but customers need to verify that any given solutions meets their requirements during peak loads.
For some customers a ‘vault-less’ tokenization solution is superior across all three axes. For other customers, with deep understanding of relational databases… security, performance, and scalability are just part of daily operations management. No vendor can credibly claim that databases or token vaults are universally the wrong choice, just like that nobody can claim that any non-relational solution is always the right choice. The decision comes down to the customer’s environment and IT operations.
I am willing to bet that the vendors of these solutions will have some additional comments, so as always the comments section is open to anyone who wants to contribute.
Reader interactions
9 Replies to “Token Vaults and Token Storage Tradeoffs”
Sorry for the late reply. I was finally cleaning out my inbox and found this post and I would like to add a couple thoughts. The post defines a “token vault” as basically a lookup mechanism for retrieving PAN information from a token. This is the true and original definition of tokenization — non-mathematically related value used as a reference to secured data.
When you define “stateless” of “vault-less” tokenization, the author is really talking about encryption or hashing and simply calling it a different name — encryption if it’s de-tokenizable or decryptable, hashing if it’s not de-tokenizable or decryptable. This was (and still is) one of my biggest beefs with PCI SSC when they redefined tokenization.
In my opinion, tokens comprised of encrypted or hashed data, or tokens renamed as “stateless” or “vault-less”, should be in PCI scope since the tokens are mathematically derived from the PAN. I know PCI has various clauses in their documents and FAQs that address token scope with a definitive “maybe”, then refer merchants consult with a qualified QSA for an answer. The original non-mathematically derived tokenization definition would have been so much easier for merchants, QSAs and PCI, but no one ever said PCI’s job was to make things easier for anyone!
I agree that data sync and security of the pan/token DB are important properties of “truly-random” tokenization solutions, and are ‘necessary evils’ of generating tokens randomly when a new pan arrives. It’s probably my fault, but I just fail to see how it’d be possible to generate a token that is totally unrelated to the PAN it substitutes (so it won’t be possible to recover the original PAN from the token *and* exclude the systems that store and process tokens from the PCI audit universe), without some method to sync back the information.
Thinking about it, even in an implementation that leverages ‘Pre-generated’ random lists of tokens (like a one time pad), where the tokenization servers have each their own ‘list of tokens to share out’, and which undoubtedly would be truly random, you’d still need to sync back the token-pan associations so lookups could be done on any node – in the end you still have a common DB that needs to be kept in sync… And you have to devise a method to ensure the same PAN is not assigned two different tokens in different nodes.
The only way I see that would give you the certainty that a given PAN is tokenized to a unique token regardless of which server it’s sent to, *withouth any kind of synchornization or communication between nodes* would be if there is an algorithm that takes the pan as input and gives us a token; some kind of fancy a one-way hash function. This approach would still need lookup tables (to be able to de-tokenize) and IMHO be open to brute force attacks if the relationship pan->token is unique – this is a similar problem as the one of securely storing passwords… Lastly, to be able to retrieve the token->PAN association, either you have a lookup table, or the algorithm is two-ways. That means there is a way to compute the PAN from the token, either using a key (which would be a type of Format Preserving Encryption, but you still have to manage the keys), or without it. This indeed would be a very interesting algorithm to examine, if it can provide two-way masking of the PAN in a way that is computationally not feasible to reverse and not using keys/secrets that need management.
As far as costs are concerned, the fact that some vault-based customers may already have database site licenses does not reduce any costs. It simply does not add additional costs on top of their current, very expensive license fees. This is not exactly a benefit, especially when you consider that any system containing sensitive data (i.e. the token vaults) will be in scope for PCI audits, and additional databases add to the list of systems that are in audit scope – more on this later.
In terms of scale, data collisions can be very difficult to deal with in many use cases. Pre-generation schemes address the problem in low to moderate transaction environments, but this issue with vault-based tokenization is, by definition, a problem when it comes to scale, as the problematic element is expanding the solution to greater usage, meaning high transaction rates. If your solution cannot scale up, with any growth, you will be forced to purchase more redundancy and synchronization hardware/software and almost certainly will not be able to reliably expand across remote data centers, including DR environments.
Today, performance may not suffer as much due to lookups (although inevitably, it must suffer a little). However, as Todd mentions, Vaultless Tokenization does not require any of the SSL communication over the wire to an external service or appliance that vault-based tokenization does. As with scalability, the impact can be severe when using remote token servers, or tokenizing multiple data types. On-node deployment of Vaultless Token servers can virtually eliminate network latency, which is impossible with vault-based tokenization.
One element in particular that has been left out of this analysis is the issue of security. Vault-based tokenization requires that all tokens and PANs are stored in each token server, which are open to external services and applications. It also requires key management and encryption of the lookup tables. These components create many avenues of attack, and with any breach of the token server, the entire security system will crumble. Obviously, these token servers create alluring targets for hackers and APTs, and should make any owner of such systems reconsider the value of a security solution that places all of its eggs in one fairly vulnerable basket.
@Javier – As Walt mentions, the PCI council has not released any specific guidance on derived tokens, but it is important in evaluating any tokenization solution that “Knowing only the token, the recovery of the original PAN must not be computationally feasible”. In other words, tokens mathematically derived through the use of an encryption algorithm and cryptographic key are not truly random, and are therefore still theoretically in scope for PCI compliance as they are essentially encrypted. Random tokens are created using random token tables and, unlike encrypted data, cannot be unlocked by cryptographic key.
I agree with Mark, in that buyers should definitely seek specific proof of security and real evidence that they can validate, rather than relying on unreliable vendor affirmations. The patent-pending Vaultless Tokenization scheme has been thoroughly vetted by Prof. Dr. Ir. Bart Preneel, of Katholieke University Leuven, Belgium (the birthplace of AES), among others, and Adrian himself has extolled tokenization as an important security tool, with cost and security advantages over encryption.
This is a timely post, shining a light on some of the important considerations when choosing Tokenization solutions given the innovations which are making Tokenization simpler to operate and use than first generation database oriented approaches.
One important point missing from the dialogue is the need for security proofs and independent validation that’s meaningful and valid. With the benefits of Tokenization ringing loud and clear in particular with PCI DSS use cases, there are a myriad of solutions on the market from a variety of sources of varying expertise in data protection.
As with the development of any new method of data protection, security proofs, the ability to review the method, and proof of independent validation by appropriate experts are critical in being able to achieve both the compliance objective, but to ensure that the data is truly protected to the strength claimed. This validation must show without a doubt and with measures that the tokens are randomly generated in a secure manner and indistinguishable from truly random numbers. Vendor affirmations are insufficient on this, yet prevail.
Buyers should ask for the proof of security and real evidence that they can validate – and their assessors can validate for their PCI ROC or other compliance assessment.
As with all of our data security development approaches, we have such data for our customers, assessors and analysts for our patent pending Secure Stateless Tokenization technology which is used by payment processors, airlines, global merchants and online retailers today: the proofs of security being vital for their selection, compliance and strong data security requirements, and of course peace of mind.
– Mark Bower, Voltage Security.
While the PCI SSC’s guidance doesn’t really address derived tokens as you point out, Visa does address it in its Tokenization Best Practices. Visa’s position on scoping and best practice is that “Knowing only the token, the recovery of the original PAN must not be computationally feasible.” That means, to get the tokens out of PCI scope, which is the whole idea, there should be no way to reverse engineer a PAN from a token.
Given this guidance, the question for derived tokens is whether the mathematical relationship — or the mere fact that there is a mathematical relationship — between the token and the PAN would cause a QSA to question whether the tokens were out of the merchant’s PCI scope.
Yet another vote for random tokens?
@Todd – I did not bring this issue up because it’s a benefit between on-prem and 3rd party tokenization services. But vaultless or stored tokens on-prem don’t suffer the latency issue.
@Javier, @Liz – I don’t know what the PCI council’s position on derived tokens, in fact their lack of clarification has been a topic of several posts here at Securosis. I expected a major debate again on the difference between derived and random tokens, but I’m glad it’s not gone there. One-time pad generation behaves a lot more like random numbers when compared to obfuscated credit card numbers.
-Adrian
Since there is a mathematical relationship between the token and the protected data (PCI, PHI, PII), it is possible to discover the source from the “derived” token. The chief use case for tokenization remains PCI compliance, but because of this mathematical relationship, will “state-less”or “vault-less” tokenization really reduce PCI audit scope?”
I’d be interested in knowing what the PCI council has to say on “derived” tokens, and whether you can reduce your PCI environment that way. My understanding is that tokenization enables systems to be ‘left out’ as there is no way to get the PAN from the token. On a derived situation, does that mean the systems that only store tokens are still within PCI?
Another huge plus for Vaultless Tokenization, and one that isn’t mentioned here, is that this service can be accomplished without a call to an external service or appliance, and does not incur additional network delays. The tokenization tables can be loaded into the database services address spaces/servers at initialization time and after that access to the appliance isn’t required.