Intro  Journals  Filter  SideProto  Experiment  Writeup

Journaling Cache Control Side-protocol for 9P

Nathaniel Wesley Filardo (with David Eckhardt)

December 4, 2007

1 / 12

Intro  Journals  Filter  SideProto  Experiment  Writeup

Outline

Introduction : CFS Summarizing Changes: QID Journals Filtered QID Journals Journaling Side Protocol Questions for an Experimentalist Writeup

2 / 12

Introduction : CFS

• CFS is a single-threaded, synchronous cache.

• It’s behavior is simple:

• directories are not cached

• cache status is only validated on Topen requests

• CFS is not agressive, does not believe in readahead (I think), and all Tstat and QTDIR operations have high latency.

3 / 12

Summarizing Changes: QID Journals

• Caches morally want to know the “place in filesystem history” of their cached contents.

• Imagine a filesystem that recorded, since the dawn of time, a list of Qids which were modified.

• Modification here can be restricted to things which alter

cache’s view of the world.

• Tremove

• Twrite

• Twstat

4 / 12

Summarizing Changes: QID Journals

• A cache can read this QID Journal to discover cached content which should now be marked invalid.

• If the QID Journal is implemented as a blocking file (ala

usb(4) audio mixer) the cache can know

• When it is caught up (first read of return size zero)

• When it is no longer caught up (return from blocking

read)

• Thus the cache knows, up to latency, when it is safe to answer from the cached contents.

5 / 12

Summarizing Changes: QID Journals

• Because a position in the journal file fully describes the filesystem’s state, this technique works even after clients lose their connection and reattach.

• They simply reissue reads from the last offset they knew about.

• More on this later.

6 / 12

Filtered QID Journals

• Not all caches care about all edits: only care about their own stores.

• Tread implicitly indicates to the server that the cache now cares about some piece of data.

• Servers can create per cache journals which will lighten the processing load of the system in favor of increased server memory usage.

7 / 12

Filtered QID Journals

• Disconnects:

• If caches inform the server of an “identity” (e.g. UUID), …

• Then even the filtered caches can work across

disconnects.

• Limiting server memory consumption:

• The journal contents may be forgotten as soon as read.

• No harm (other than potential recaching) occurs if the server places any QID in the journal. This enables early reclaim of server memory.

• No harm happens if the server forgets part of the journal

before it’s been read, if it returns an error to the cache:

• the cache can catch up and fall back to cfs-like behavior.

8 / 12

Journaling Side Protocol

• Proposal: cache control metadata is a separate filesystem to be Tattach’d by the cache.

• Specifically, contained here is a directory, journals, which contains files named by the cache’s long-lived identifier (UUID).

• I have called this the “cc” file system (cache control), and for lack of a better notation I have used bang-path notation: “cc!/journals/$CACHENAME”

• Segregation into directory allows us to research alternatives.

9 / 12

Journaling Side Protocol

• A journal file read, at offset below the head, returns immediately.

• The first journal file read at the head returns zero (notifying the cache that it is caught up).

• Subsequent reads at the head block (until the journal is appended).

10 / 12

Questions for an Experimentalist

• Does it help to type the entries in the QID journal?

• That is, “this was invalidated by a Tremove” / Twstat / Twrite?

• (Feel free to create cc!/typedjournals/... to test

both at once)

• Cache eviction notifications?

• Caches write evicted Qids to a file (e.g., cc!/cacheevicted) which signals that the cache has dropped it.

• Should mean that the server has a more accurate idea of what’s in cache.

11 / 12

Questions for an Experimentalist

• Does it make sense to bundle files into logical units (e.g.

AFS volume callbacks)?

• Proposal: a separate file in the cc fs for Qid aliasing.

• Do write-back, not write-through, caches make sense?

• Probably want a server lease mechanism (yet another control file in cc).

• If yes, does it make sense to try cache-cache transfers (ala MOESI)?

12 / 12

Intro  Journals  Filter  SideProto  Experiment  Writeup

Writeup

• More details available :

https://wiki.ietfng.org/pub/Plan9/JournalCallbacks

13 / 12