Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

5. Version history: We recommend the latest version SZ 1.0

  • SZ 01.2-0.4 Compression ratio is the same as SZ 0.5. The key difference is different implementation ways, such that SZ 0.5 is much faster than SZ 0.2-0.4.0 This version is coded in C programming language, unlike the previous version coded in Java. It also allows to set the endianType for the data to compress.
  • SZ 0.5.14 fixed a design bug, which improves the compression ratio further.
  • SZ 0.5.1 Support version checkingSZ 0.5.2 finer compression granularity for unpredictable data, and also remove redundant Java storage bytes13 improves compression performance, by replacing the implementation with classes by that of primitive data types.
  • SZ 0.5.3 Integrate with the dynamic segmentation support
  • SZ 0.5.4 Gzip_mode: defaut --> fast_mode ; Support reserved value
  • SZ 0.5.5 runtime memory is shrinked (by changing int xxx to byte xxx in the codes. The bug that writing decompressed data may encounter exceptions is fixed. Memory leaking bug for ppc architecture is fixed.
  • SZ 0.5.6 improve compression ratio for some cases (when the values in some segementation are always the same, this segment will be merged forward)
  • SZ 0.5.7 improve the decompression speed for some cases
  • SZ 0.5.8 Refine the leading-zero granularity (change it from byte to bits based on the distribution). For example, in SZ0.5.7, the leading-zero is always in bytes, 0, 1, 2, or 3. In SZ0.5.8, the leading-zero part could be xxxx xxxx xx xx xx xx xxxx xxxx (where each x means a bit in the leading-zero part12 allows users to set "offset" parameter in the configuration file sz.config. The value of offset is an integer in [1,7]. Generally, we recommend offset=2 or 3, while we also find that some other settings (such as offset=7) may lead to better compression ratios in some cases. How to automize/optimize the selection of offset value would be the future work. In addition, the compression speed is improved, by replacing java List by array implementation in the code.
  • SZ 0.5.11 improved SZ 0.5.10 on the level of guaranteeing user-specified error bounds. In a very few cases, SZ 0.5.10 cannot guarantee the error-bounds to a certain user-specified level. For example, when absolute error bound = 1E-6, the maximum decompression error may be 0.01(>>1E-6) because of the huge value range even in the optimized segments such that the normalized data cannot reach the required precision even storing all of the 64 or 32 mantissa bits. SZ 0.5.11 fixed the problem well, with compression ratio degraded by less than 1% in that case.
  • SZ 0.5.10 optimizes the offset by using the optimized formula of computing the median_value based on optimized right-shifting method. Anyway, this version improves compression ratio a lot for hard-to-compress datasets. (Hard-to-compress datasets refer to the cases whose compression ratios are usually very limited)
  • SZ 0.5.9 optimize the offset by using simple right-shifting method. Experiments show that this cannot improve compression ratio actually, because simple right-shifting actually make each data be multiplied by 2^{-k}, where k is # right-shifting bits. The pros is to save bits because of more leading-zero bytes, but the cons is much more required bits to save. See SZ 0.5.10 for the better solution on this issue!
  • SZ 0.5.10 optimizes the offset by using the optimized formula of computing the median_value based on optimized right-shifting method. Anyway, this version improves compression ratio a lot for hard-to-compress datasets. (Hard-to-compress datasets refer to the cases whose compression ratios are usually very limited8 Refine the leading-zero granularity (change it from byte to bits based on the distribution). For example, in SZ0.5.7, the leading-zero is always in bytes, 0, 1, 2, or 3. In SZ0.5.8, the leading-zero part could be xxxx xxxx xx xx xx xx xxxx xxxx (where each x means a bit in the leading-zero part)
  • SZ 0.5.11 improved 7 improve the decompression speed for some cases
  • SZ 0.5.10 on the level of guaranteeing user-specified error bounds. In a very few cases, 6 improve compression ratio for some cases (when the values in some segementation are always the same, this segment will be merged forward)
  • SZ 0.5.10 cannot guarantee the error-bounds to a certain user-specified level. For example, when absolute error bound = 1E-6, the maximum decompression error may be 0.01(>>1E-6) because of the huge value range even in the optimized segments such that the normalized data cannot reach the required precision even storing all of the 64 or 32 mantissa bits. SZ 0.5.11 fixed the problem well, with compression ratio degraded by less than 1% in that case. 
  • SZ 0.5.12 allows users to set "offset" parameter in the configuration file sz.config. The value of offset is an integer in [1,7]. Generally, we recommend offset=2 or 3, while we also find that some other settings (such as offset=7) may lead to better compression ratios in some cases. How to automize/optimize the selection of offset value would be the future work. In addition, the compression speed is improved, by replacing java List by array implementation in the code.
  • SZ 0.5.13 improves compression performance, by replacing the implementation with classes by that of primitive data types.
  • SZ 0.5.14 fixed a design bug, which improves the compression ratio further.
  • SZ 1.0 C version5 runtime memory is shrinked (by changing int xxx to byte xxx in the codes. The bug that writing decompressed data may encounter exceptions is fixed. Memory leaking bug for ppc architecture is fixed.
  • SZ 0.5.4 Gzip_mode: defaut --> fast_mode ; Support reserved value
  • SZ 0.5.3 Integrate with the dynamic segmentation support
  • SZ 0.5.2 finer compression granularity for unpredictable data, and also remove redundant Java storage bytes
  • SZ 0.5.1 Support version checking
  • SZ 0.2-0.4 Compression ratio is the same as SZ 0.5. The key difference is different implementation ways, such that SZ 0.5 is much faster than SZ 0.2-0.4.

6. Download

-->>> Code download <<<-- (soon, pending DoE approval of distribution license)

...