3138
Comment: ...now LZO too :)
|
2984
|
Deletions are marked like this. | Additions are marked like this. |
Line 5: | Line 5: |
This page contains list of patches and other relatd code as it is being developed for Compressed Caching (for 2.6.x kernels) project. You can always find most up-to-date code at Project CVS (at linuxcompressed.sourceforge.net). Code is placed here when it achieves any particular (maybe small) thing. | This page contains list of patches and other relatd code as it is being developed for Compressed Caching (for 2.6.x kernels) project. You can always find most up-to-date code at Project Git repository (git://dev.laptop.org/projects/linux-mm-cc). Code is placed here when it achieves any particular (maybe small) thing. |
Line 19: | Line 19: |
WKdm de/compression algorithm: attachment:compress-test.tar.gz | Kernel module to test de/compression algorithms (WKdm, WK4x4, LZO): attachment:compress-test.tar.gz |
Line 25: | Line 25: |
Now, two algos are ported - LZO and WKdm (WK4x4 also soon). You can test both of them using this module. It creates 3 /proc entries: /proc/compress-test/{compress, decompress, algo_idx}. | Now, all three algos are ported to kernel space - WKdm, WK4x4 and LZO. You can test them all using this module. It creates 3 /proc entries: /proc/compress-test/{compress, decompress, algo_idx} as described below: (for some detail see README with this module) |
Line 30: | Line 30: |
''Write to /proc/compress-test entries:'' | __''Write to /proc/compress-test entries:''__ |
Line 34: | Line 34: |
2. algo_idx: write index of algo you want to test (0: WKdm, 1: LZO) | 2. algo_idx: write index of algo you want to test (0: WKdm, 1: WK4x4, 2: LZO) |
Line 36: | Line 36: |
''Read from /proc/compress-test entries:'' | __''Read from /proc/compress-test entries:''__ |
Line 40: | Line 40: |
2. decompress: decompress compressed data stored in internal buffer. 3. algo_idx: shows list of algos supported with their index. | 2. decompress: decompress compressed data stored in internal buffer. 3. algo_idx: shows list of algos supported with their index. |
Line 43: | Line 45: |
Also commited these two algos to CVS in linux26/lib/{WKdm, LZO} | |
Line 45: | Line 46: |
This module makes it easy to test algotithm by simply comparing original file with /proc/decompress output and statistics can be obtained by reading /proc/compress. This module will serve as simple test-bed when porting other important de/compression algorithms w.r.t compressed caching (WKdm, WK4x4, LZO). | Also commited all these algos to CVS in linux26/lib/{WKdm, WK4x4, LZO} |
[:NitinGupta:Nitin Gupta]
MailTo(nitingupta.mail AT gmail DOT com)
This page contains list of patches and other relatd code as it is being developed for Compressed Caching (for 2.6.x kernels) project. You can always find most up-to-date code at Project Git repository (git://dev.laptop.org/projects/linux-mm-cc). Code is placed here when it achieves any particular (maybe small) thing.
The code is presently very raw but its helping me getting more experience with VMM code and how things are to be done
Kernel changes to support Compressed Caching:
- [attachment:toy-cc-2.6.16-rc4.diff Toy ccache patch]: These are few lines I added to 2.6.16-rc4 while going through VMM code. Just some printk()s to simply highlight some kernel entry points for compressed caching work.
- [attachment:patch-cc-2.6.16-radix-replace-stable.diff patch-cc-2.6.16-radix-replace-stable]: Replace original page (for now, only clean page cache pages) with a 'chunk head' when it is to be freed under memory pressure and simply store original page uncompressed. When page cache lookup is performed, again replace the 'chunk head' with original page. This patch uses simplified (and inefficient) locking in page cache lookup functions to make it stable for now.
- [attachment:patch-cc-2.6.16-better-locking-unstable.diff patch-cc-2.6.16-better-locking-unstable]: This was an attempt to get a better (more efficint) locking in page cache lookup functions but it is not quite as stable as previous simplified patch. It causes apps to freeze as swap usage increase.
Compression algorithms to kernel mode:
Kernel module to test de/compression algorithms (WKdm, WK4x4, LZO): attachment:compress-test.tar.gz
There are basically three main algorithms that are well studied w.r.t compressed caching by previous works -- WKdm, WK4x4, LZO.
Of these, WKdm, WK4x4 are designed to handle anon pages (non filesystem pages) while LZO is more suitable for filesystem data. (Also, in general, compression speed is in order: WKdm > WK4x4 > LZO, while compression factor order is, in general, reverse).
Now, all three algos are ported to kernel space - WKdm, WK4x4 and LZO. You can test them all using this module. It creates 3 /proc entries: /proc/compress-test/{compress, decompress, algo_idx} as described below: (for some detail see README with this module)
In short:
Write to /proc/compress-test entries:
1. compress: compress data witten to it and store in internal buffer.
2. algo_idx: write index of algo you want to test (0: WKdm, 1: WK4x4, 2: LZO)
Read from /proc/compress-test entries:
1. compress: show original and compressed size (TODO: add other stats like time taken too)
2. decompress: decompress compressed data stored in internal buffer.
3. algo_idx: shows list of algos supported with their index.
Also commited all these algos to CVS in linux26/lib/{WKdm, WK4x4, LZO}