Findings in SecurityTrackr are encrypted at rest, each organisation under its own key, and sealed with an authenticated cipher that binds them to that org. Those sensitive fields are stored as ciphertext, which is exactly what you want the morning a backup leaks. It's also a problem, because you can't run a LIKE query or a full-text index over bytes you can't read. So how do you search a register you've deliberately made unreadable? Here's how we do it without storing a single plaintext word, and the pattern transfers to any encrypted store.
You can't search what you can't read
An observation's title, affected systems, and your written description are all encrypted columns. To the database they're opaque blobs. Ordinary search is off the table: there's no LIKE over ciphertext, and a full-text index would need the plaintext it doesn't have.
You could decrypt every finding on each search and scan the plaintext in memory, but that doesn't scale. One query would mean pulling and decrypting the entire register, and it gets slower with every observation you add.
The obvious shortcut is to keep one plaintext column on the side, a small bag of keywords, and search that. It works, and it quietly undoes the encryption: anyone who can read the database (a leaked backup, a curious DBA) gets a clean, word-level fingerprint of every finding you took the trouble to encrypt. That's the trap to avoid. The question is what to do instead.
A blind index
The answer is a blind index. Instead of storing words, store one-way hashes of words, and arrange the hashes so a search can match them without anyone ever turning them back into text. We keep a side table of per-org token hashes. No plaintext is persisted, yet a finding stays searchable by which hashes it shares with your query.
Observation text → normalised terms → per-organisation keyed hashes
Search query → the same rules → compare stored hashesThe whole trick is in one word: deterministic. The same pipeline runs when we index a finding and when you type a query. Because both sides normalize the same way before hashing, the hash of a word in a title is byte-for-byte the hash of that word in your search box. Matching is then just set intersection over hashes. Equal tokens, equal hashes, no decryption involved.
Normalize before you hash
The same word must produce the same token at indexing and search time. We normalise case, handle common word endings and preserve useful identifiers such as hostnames and CVE references. Both paths use the same rules, so a plural or a change of case need not hide a relevant finding.
A separate search key for each organisation
A plain hash of a common word can be guessed with a dictionary. We use keyed hashes instead, with a separate search key for each organisation. The same word produces different stored values across organisations, and the search key is distinct from the key used to encrypt the fields.
Putting the best match first, blind
Matching is the easy half. Ranking what you can't read is harder, and it's where two small choices earn their keep.
- Identifiers stay in the clear. A finding's reference like Obs003 is not a secret, so it isn't hashed. A complete identifier such as Obs003 is matched exactly and ranked first. Identifier fragments do not trigger that lookup.
- Title hits beat body hits. A query word in the title means more than the same word buried in the notes. The title is encrypted, so we compute that at query time: decrypt only the few top candidates, re-tokenize their titles, and count the overlaps. No extra plaintext is stored to make ranking work.
A finding matched only in its body still surfaces, just below the ones matched in their title. Recall is preserved, precision is rewarded, and the whole ordering is computed from things we can re-derive on the fly rather than anything sitting in the clear.
Whole words only
A blind index isn't free, and the bill comes due as you type. Because hashing destroys structure, the hash of “unrestr” has nothing to do with the hash of “unrestricted”. So as-you-type prefix matching on words is gone: the dropdown finds nothing until you've typed a whole word (light stemming still covers plurals and tenses). You could win prefix matching back by storing a hash of every prefix, but those are low-entropy and would hand a database thief most of the plaintext anyway, which defeats the point.
A trade-off you can't see is a bug, so we don't hide this one. A small (i) next to every search box explains the whole-word rule, and the no-results state says “type a full word” rather than letting an empty result read as “nothing here”. A complete observation identifier query such as Obs003 is compared by exact equality and ranked first; fragments do not enter identifier lookup.
What the index still reveals
A reader with only a database copy does not get a plaintext keyword list. They can still see which rows share a hash and how frequently it appears within an organisation. That pattern leakage is a trade-off of deterministic search and can support inferences when combined with other information.
Per-organisation keys and authenticated encryption protect the content. Application authorisation still controls who can search and retrieve it. Encryption and access checks work together; neither makes the other unnecessary.