Keeping data encrypted in memory with confidential computing

Cian Butler SRE@Evervault

Encrypt all the things.

Encryption at Rest

Encrypted Protocols

Whats missing?

Encryption in memory

Enclaves

Lots of 3 letter acronyms

A diagram showing the breakdown of tee techs and their isolation models

Enclaves break down into different varieties.

Applications Enclaves are specially designed applications that are used to secure computation. These apps use specific syscalls to move there processing to secure chips on the hardware.

We have two type of vm based enclaves. VM based enclaves are used to provide secure sidecars for doing compute. They have their own OS and will have a side channel of sorts to allow the host to communicate with them.

Arm TrustZones and similar are used in phones for mobile wallets. They have custom OSes and will talk have an API/RPC system for communicating.

The other style of VM's look like regular machines. You can run any OS or application you want on them. You'll again have a side channel it'll use to communicate with the main host.

The Problems

Observability
Userspace Networking
Time slippage

Observability

Loadtest Graphs showing performance improvements moving from k1 to r1 curve

Loadtest Graphs showing performance improvements moving from m5 to c5 nodes

In all forms of enclaves you'll have no shell, no logs or any of your standard obs tooling.

When we started to have performance issues in prod we had to resort to black box loadtesting. This means we added metrics and logs to the client that communicates to the side channel so we could get a sense of what kind of workloads produce what kind of results.

Eventually we had to actually implement a concept of traces into the enclave. The Service would create a trace for each request store the finished trace and all its meta in a buffer that would periodically be flushed to a reporter on the host node for forwarding on to the central trace infra.

Userspace Networking

Graph showing performance difference show running a network request in a vm vs an enclave. Enclave is much lower then vm

Time slippage

There is no NTP in enclaves.

Attestation

Now enclaves arnt all pain. They have some cool stuff.

You can attest the environment you are connecting to be that remotely or locally is 100% the code you deployed.

This work by at build time for your image hashing all the software you used to generate a shared hash that is signed by your private ca. that hash is given to a client. When the client connects to the enclave you have the enclave return its current hash. The client then can compare this against the Hash it expects and validates the signature with the public ca and if it fails to validate the connection is killed and data is never transferred.

This attestation can also be done by the HW To validate that the image being deployed is the expected image and by the process in the image to the HW to ensure that the hw hasnt been tampered with.

If any of these validations fail we can kill the process and stop an attacker getting access to our secure data.