Uncompressed VS Compressed File Write

I've got the first results of my "Compressed File Writing" test.

In both graphs, it's drawn the speedup (%) of compressed writing over uncompressed writing. I used [((UncompressedTime / CompressedTime) - 1) * 100] to get values under "0" when the compression+writing time is slower than uncompressed writing. The X axis has the file size, it goes from 256 KB to 128 MB.

I've compared a lot of "bloc size" for writing and graphed the best times for uncompressed writing (the less speedup)

In the first graph we can see that compressing binary data afte writing has a lot of speedup (70% to 140% faster) from 1 MB to 32 MB.

In the second graph we can see that compressing source code always get a lot of speedup, at least from 0 to 128 MB of file size.

These tests are done one time in a partition of my laptop (4200 rpm HD) mounted with "-o sync,dirsync". I want to make other tests to get more accuracy. One teacher and friend (Diego Sevilla Ruiz) pointed me that I should test random reads and writes. That was not my purpose by now. I'm trying to get the speedup limits using compression before writing to the disk. If I want to get random read/write speeds I need to simulate the actions of a FileSystem and usually it works with small block sizes that makes compression+write much faster than uncompressed writing.

00:00 | Comments | Ideas

Compressed filesystems and compressed network transmission.

Nowadays, the new processors speed has been increased a lot. It's time to compare the performance of a compressed hard drive with non-compressed one. I don't talk about more hard drive space, I want to reduce the hard drive read operations.

I think that reading, for example, 10MB of data and decompressing it in RAM to have from 20 to 100 MB of data can be quicker than reading 20 to 100 MB form the hard drive. Streaming compression can be used to reduce the response time.

The same idea can be applied to network connections. Networks use to be slow compared to hard drives (home and small office networks) and then, compression can be much more aggressive.

I plan to make a few tests and see if that "idea" is true or false.

00:00 | Comments | Ideas
Institutional web page
Mono Contributor