203 lines
6.3 KiB
ReStructuredText
203 lines
6.3 KiB
ReStructuredText
.. SPDX-License-Identifier: GPL-2.0
|
||
|
||
=================================
|
||
dm-pcache — Persistent Cache
|
||
=================================
|
||
|
||
*Author: Dongsheng Yang <dongsheng.yang@linux.dev>*
|
||
|
||
This document describes *dm-pcache*, a Device-Mapper target that lets a
|
||
byte-addressable *DAX* (persistent-memory, “pmem”) region act as a
|
||
high-performance, crash-persistent cache in front of a slower block
|
||
device. The code lives in `drivers/md/dm-pcache/`.
|
||
|
||
Quick feature summary
|
||
=====================
|
||
|
||
* *Write-back* caching (only mode currently supported).
|
||
* *16 MiB segments* allocated on the pmem device.
|
||
* *Data CRC32* verification (optional, per cache).
|
||
* Crash-safe: every metadata structure is duplicated (`PCACHE_META_INDEX_MAX
|
||
== 2`) and protected with CRC+sequence numbers.
|
||
* *Multi-tree indexing* (indexing trees sharded by logical address) for high PMem parallelism
|
||
* Pure *DAX path* I/O – no extra BIO round-trips
|
||
* *Log-structured write-back* that preserves backend crash-consistency
|
||
|
||
|
||
Constructor
|
||
===========
|
||
|
||
::
|
||
|
||
pcache <cache_dev> <backing_dev> [<number_of_optional_arguments> <cache_mode writeback> <data_crc true|false>]
|
||
|
||
========================= ====================================================
|
||
``cache_dev`` Any DAX-capable block device (``/dev/pmem0``…).
|
||
All metadata *and* cached blocks are stored here.
|
||
|
||
``backing_dev`` The slow block device to be cached.
|
||
|
||
``cache_mode`` Optional, Only ``writeback`` is accepted at the
|
||
moment.
|
||
|
||
``data_crc`` Optional, default to ``false``
|
||
|
||
* ``true`` – store CRC32 for every cached entry
|
||
and verify on reads
|
||
* ``false`` – skip CRC (faster)
|
||
========================= ====================================================
|
||
|
||
Example
|
||
-------
|
||
|
||
.. code-block:: shell
|
||
|
||
dmsetup create pcache_sdb --table \
|
||
"0 $(blockdev --getsz /dev/sdb) pcache /dev/pmem0 /dev/sdb 4 cache_mode writeback data_crc true"
|
||
|
||
The first time a pmem device is used, dm-pcache formats it automatically
|
||
(super-block, cache_info, etc.).
|
||
|
||
|
||
Status line
|
||
===========
|
||
|
||
``dmsetup status <device>`` (``STATUSTYPE_INFO``) prints:
|
||
|
||
::
|
||
|
||
<sb_flags> <seg_total> <cache_segs> <segs_used> \
|
||
<gc_percent> <cache_flags> \
|
||
<key_head_seg>:<key_head_off> \
|
||
<dirty_tail_seg>:<dirty_tail_off> \
|
||
<key_tail_seg>:<key_tail_off>
|
||
|
||
Field meanings
|
||
--------------
|
||
|
||
=============================== =============================================
|
||
``sb_flags`` Super-block flags (e.g. endian marker).
|
||
|
||
``seg_total`` Number of physical *pmem* segments.
|
||
|
||
``cache_segs`` Number of segments used for cache.
|
||
|
||
``segs_used`` Segments currently allocated (bitmap weight).
|
||
|
||
``gc_percent`` Current GC high-water mark (0-90).
|
||
|
||
``cache_flags`` Bit 0 – DATA_CRC enabled
|
||
Bit 1 – INIT_DONE (cache initialised)
|
||
Bits 2-5 – cache mode (0 == WB).
|
||
|
||
``key_head`` Where new key-sets are being written.
|
||
|
||
``dirty_tail`` First dirty key-set that still needs
|
||
write-back to the backing device.
|
||
|
||
``key_tail`` First key-set that may be reclaimed by GC.
|
||
=============================== =============================================
|
||
|
||
|
||
Messages
|
||
========
|
||
|
||
*Change GC trigger*
|
||
|
||
::
|
||
|
||
dmsetup message <dev> 0 gc_percent <0-90>
|
||
|
||
|
||
Theory of operation
|
||
===================
|
||
|
||
Sub-devices
|
||
-----------
|
||
|
||
==================== =========================================================
|
||
backing_dev Any block device (SSD/HDD/loop/LVM, etc.).
|
||
cache_dev DAX device; must expose direct-access memory.
|
||
==================== =========================================================
|
||
|
||
Segments and key-sets
|
||
---------------------
|
||
|
||
* The pmem space is divided into *16 MiB segments*.
|
||
* Each write allocates space from a per-CPU *data_head* inside a segment.
|
||
* A *cache-key* records a logical range on the origin and where it lives
|
||
inside pmem (segment + offset + generation).
|
||
* 128 keys form a *key-set* (kset); ksets are written sequentially in pmem
|
||
and are themselves crash-safe (CRC).
|
||
* The pair *(key_tail, dirty_tail)* delimit clean/dirty and live/dead ksets.
|
||
|
||
Write-back
|
||
----------
|
||
|
||
Dirty keys are queued into a tree; a background worker copies data
|
||
back to the backing_dev and advances *dirty_tail*. A FLUSH/FUA bio from the
|
||
upper layers forces an immediate metadata commit.
|
||
|
||
Garbage collection
|
||
------------------
|
||
|
||
GC starts when ``segs_used >= seg_total * gc_percent / 100``. It walks
|
||
from *key_tail*, frees segments whose every key has been invalidated, and
|
||
advances *key_tail*.
|
||
|
||
CRC verification
|
||
----------------
|
||
|
||
If ``data_crc is enabled`` dm-pcache computes a CRC32 over every cached data
|
||
range when it is inserted and stores it in the on-media key. Reads
|
||
validate the CRC before copying to the caller.
|
||
|
||
|
||
Failure handling
|
||
================
|
||
|
||
* *pmem media errors* – all metadata copies are read with
|
||
``copy_mc_to_kernel``; an uncorrectable error logs and aborts initialisation.
|
||
* *Cache full* – if no free segment can be found, writes return ``-EBUSY``;
|
||
dm-pcache retries internally (request deferral).
|
||
* *System crash* – on attach, the driver replays ksets from *key_tail* to
|
||
rebuild the in-core trees; every segment’s generation guards against
|
||
use-after-free keys.
|
||
|
||
|
||
Limitations & TODO
|
||
==================
|
||
|
||
* Only *write-back* mode; other modes planned.
|
||
* Only FIFO cache invalidate; other (LRU, ARC...) planned.
|
||
* Table reload is not supported currently.
|
||
* Discard planned.
|
||
|
||
|
||
Example workflow
|
||
================
|
||
|
||
.. code-block:: shell
|
||
|
||
# 1. Create devices
|
||
dmsetup create pcache_sdb --table \
|
||
"0 $(blockdev --getsz /dev/sdb) pcache /dev/pmem0 /dev/sdb 4 cache_mode writeback data_crc true"
|
||
|
||
# 2. Put a filesystem on top
|
||
mkfs.ext4 /dev/mapper/pcache_sdb
|
||
mount /dev/mapper/pcache_sdb /mnt
|
||
|
||
# 3. Tune GC threshold to 80 %
|
||
dmsetup message pcache_sdb 0 gc_percent 80
|
||
|
||
# 4. Observe status
|
||
watch -n1 'dmsetup status pcache_sdb'
|
||
|
||
# 5. Shutdown
|
||
umount /mnt
|
||
dmsetup remove pcache_sdb
|
||
|
||
|
||
``dm-pcache`` is under active development; feedback, bug reports and patches
|
||
are very welcome!
|