This page contains list of patches and other relatd code as it is being developed for Compressed Caching (for 2.6.x kernels) project. You can always find most up-to-date progress of code at Project CVS. The code is presently very raw but its helping me getting more experience with VMM code and how things are to be done :)
Kernel changes to support Compressed Caching:
- [attachment:toy-cc-2.6.16-rc4.diff Toy ccache patch]: These are few lines I added to 2.6.16-rc4 while going through VMM code. Just some printk()s to simply highlight some kernel entry points for compressed caching work.
- [attachment:patch-cc-2.6.16-radix-replace-stable.diff patch-cc-2.6.16-radix-replace-stable]: Replace original page (for now, only clean page cache pages) with a 'chunk head' when it is to be freed under memory pressure and simply store original page uncompressed. When page cache lookup is performed, again replace the 'chunk head' with original page. This patch uses simplified (and inefficient) locking in page cache lookup functions to make it stable for now.
- [attachment:patch-cc-2.6.16-better-locking-unstable.diff patch-cc-2.6.16-better-locking-unstable]: This was an attempt to get a better (more efficint) locking in page cache lookup functions but it is not quite as stable as previous simplified patch. It causes apps to freeze as swap usage increase.
Compression algorithms to kernel mode:
There are basically three main algorithms that are well studied w.r.t compressed caching by previous works -- WKdm, WK4x4, LZO.
Of these, WKdm, WK4x4 are designed to handle anon pages (non filesystem pages) while LZO is more suitable for filesystem data. (Also, in general, compression speed is in order: WKdm > WK4x4 > LZO, while compression factor order is, in general, reverse) Currently, I have only ported WKdm to kernel mode: ( attachment:compress%2Dtest.tar.gz ): This is a kernel module to test port of WKdm algorithm to kernel mode. It creates two /proc entries: /proc/{compress, decompress}.
Writing a file to /proc/compress compresses the input file (input file must be <= 4K i.e. PAGE_SIZE) and stores it in an internal buffer.
Reading /proc/decompress again shows decompressed output. Reading /proc/compress stores statistics (currently only original and compressed size) like de/compression time, dictionary hits (partial matches, exact matches, missses).
This makes it easy to test algotithm by simply comparing original file with /proc/decompress output and statistics can be obtained by reading /proc/compress. This module will serve as simple test-bed when porting other important de/compression algorithms w.r.t compressed caching (WK4x4, LZO).