How data encryption works in payment systems

Payment systems operate with the most sensitive data - PAN (card number), expiration date, CVV/CVC, 3-DS tokens, bank details, wallet identifiers. Their leak is fines, recall of merchant from banks/PSP and direct financial loss. Protection is built in layers: encryption in a channel (TLS), encryption and/or tokenization in storage, strict key management and hardware trusted modules (HSM). Below is the entire security "pipeline" in simple language.

Basic bricks

Symmetric cryptography

Algorithms: AES-GCM/CTR/CBC (in payments, the de facto standard is AES-GCM).

Pros: high speed, compact keys.

Cons: you need to safely agree on a key and IV/nonce.

Asymmetric cryptography

Algorithms: RSA-2048/3072, ECC (P-256/384, Ed25519).

Usage: key exchange/wrapping, signatures, PKI, TLS certificates.

Pros: Doesn't require a shared secret beforehand.

Cons: Slower than symmetric encryption.

Идея Perfect Forward Secrecy (PFS)

Session keys are negotiated by effemeric ECDHE. Even if the server's private key leaks sometime, past sessions will remain undeciphered.

In-transit encryption: TLS 1. 2/1. 3

1. Handshake (TLS handshake): the client and the server agree on versions/ciphers, the server presents a certificate (PKI), exchange ephemeral keys (ECDHE) → a session symmetric key is born.

2. Data: transmitted in AEAD modes (AES-GCM/ChaCha20-Poly1305) with authentication.

3. Optimizations: TLS 1. 3 cuts rounds, supports resumption; 0-RTT are used carefully (idempotent queries only).

4. Practice for payments: we prohibit SSLv3/TLS1. 0/1. 1, turn on the TLS1. 2/1. 3, OCSP stapling, HSTS, strict security headers.

💡 Internal calls (PSP → merchant, merchant → processing, webhooks) often additionally protect mTLS: both sides show mutual certificates.

Encryption "in storage": at rest

Options

Full volume/database encryption (TDE): quickly entered, protects against "cold" access to the media, but not from leakage through a compromised application.

Bitwise/field-level (FLE): individual fields (PAN, IBAN) are encrypted. Granular, but more difficult to implement and index.

Format-preserving encryption (FPE): Useful when you want 16 digits as 16 digits.

Tokenization: PAN is replaced by a token (meaningless string); this PAN is stored in the token vault under heavy protection. When paying/returning, a token is used → the merchant does not process "raw" cards.

Key idea

In storage, it is not "which algorithm" that is more important, but where the keys are and who can detokenize. Therefore...

Key management: KMS, HSM and envelopes

Key hierarchy (envelope encryption)

Root/KEK (Key Encryption Key): high protection class, stored and executed in the HSM.

DEK (Data Encryption Key): encrypts specific data/batches/tables; itself encrypted by KEK.

Rotation: regulations for scheduled and unscheduled (in case of an incident) rotation of KEK/DEK; the key version is specified in the ciphertext metadata.

HSM (Hardware Security Module)

A hardware module certified (for example, FIPS 140-2/3) that stores and performs key operations within itself.

Does not issue private keys to the outside, supports limits/use policy, audit.

Used for: key generation, DEK wrapper, server key 3-DS, EMV keys, PIN operations, message signing.

KMS

Centralizes key policy, versioning, accesses, logs, and APIs.

In conjunction with HSM, it implements envelope encryption and automatic rotation.

Card standards and industry specifics

PCI DSS (and minimization logic)

The main idea: do not store CVV, minimize the PAN processing area (scope).

Where possible - give PAN input to Hosted Fields/Iframe PSP → the merchant does not have access to raw data.

Logs, backups, dumps - the same rules as prod: masking, encryption, retention.

EMV, PIN и POS

EMV chip/contact-less: cryptograms at the card/terminal level, protection against mage strip cloning.

PIN blocks and ISO 9564: PIN is encrypted from pin pad to processing, works with HSM (pin transfers, key zones).

DUKPT (Derived Unique Key Per Transaction): On the POS, each payment is encrypted with a unique key derived from BDK → compromising one message does not drag others along.

PCI P2PE: certified "end-to-end" encryption scheme from pin pad to decryption provider.

3-D Secure (2. x)

Card holder authentication → less fraud/chargebacks.

Cryptography is used for message signatures, ACS/DS/3DS Server key exchange; private keys are usually in HSM.

Typical Data Protection Architectures

Option A (online merchant with PSP):

Browser → HTTPS → Hosted Fields PSP (PAN does not get to merchant).
PSP returns payment token.
The merchant database stores the token + the last 4 digits and BIN (for UX and rules).
Returns/repeats - token only.
Secrets/keys - in KMS, private keys TLS/3-DS - in HSM.

Option B (wallet/payment):

Application ↔ API - TLS/mTLS.
Sensitive fields - FLE/FPE or tokenization; vault is isolated.
Access to detokenization - only for service roles with "four-eyed," operations - through HSM.

Option C (offline-POS):

Pin pad → DUKPT/P2PE → processing.
Terminal boot keys - via secure key injectors/XSM.
Logging, anti-tamper protection of devices.

Rotation, auditing and incidents

Key rotation: planned (once every X months) and by event (compromise). DEK rewrap under the new KEK without decrypting user data.

Immutable logs: who and when accessed the detokenation/keys; signature of logs.

Compromise runbook: immediate revoke/rotate, reissue of certificates, API key block, partner notification, retrospective.

Common mistakes and how to avoid them

1. "We encrypt the database, so everything is OK."

No, it isn't. The compromised application reads the data openly. We need tokenization/FLE and the principle of least rights.

2. CVV storage.

You can't. CVV is never stored, even encrypted (via PCI DSS).

3. Keys next to data.

You can't. Keys - in KMS/HSM, access - by role, minimum privileges, separate accounts.

4. No rotation/versions.

Always version keys, store 'key _ version' in ciphertext metadata.

5. TLS on perimeter only.

Encrypt behind CDN/WAF and inside the data plan (servis→servis, webhooks).

6. Tokenization "for view."

If any service can detokenize, this is not protection. Narrow and audit calls.

7. Unaccounted backups/analytical uploads.

Encryption and masking should apply to backups, snapshots, BI-showcases, logs.

Implementation checklist (brief)

Channel

TLS 1. 2/1. 3, PFS, mTLS for internal and webhooks, HSTS, strict security-headers.

Storage

PAN tokenization, prohibition of CVV storage.

FLE/FPE for critical fields; TDE as base layer.

Keys

KMS + HSM, envelope encryption (KEK/DEK), rotation/versions, unchangeable logs.

Architecture

Hosted Fields/SDK PSP, PCI zone minimization.

Separation of roles/networks, zero trust, secrets - only through a secret manager.

Operations

Pentest/Red Team on perimeter and business logic.

DLP/CTI monitoring of drains, personnel training.

Runbook на compromise: revoke/rotate/notify.

Mini-FAQ

Is encryption or tokenization best for PAN?

In sales - tokenization (minimizes scope). In vault - encryption with HSM/KMS.

Do I need an EV certificate for a payment domain?

Optional. More important is the correct TLS profile, mTLS, keys in HSM and discipline.

Can I use 0-RTT in TLS 1? 3 for payments?

For idempotent GETs, yes. For POST, it is better to turn off or limit.

How to store "last 4" and BIN?

Separate from PAN; this is not sensitive data with correct isolation, but observe masking in the logs/BI.

Encryption in payment systems is not one toggle switch, but an ecosystem: TLS/PFS in a channel, tokenization and/or FLE in storage, strict key management via KMS + HSM, industry standards (PCI DSS, EMV, 3-DS), rotation and audit. Such a multi-layered architecture makes the leakage of card data extremely unlikely, simplifies the passage of audits and, most importantly, retains the trust of banks, payment partners and users.