Investigate using google LZ4 compression for temporary files
Field Tab
Administrator use only
Field Tab
Administrator use only
Description
Conclusion
None
Activity
Show:
Mark Kelly February 22, 2016 at 9:42 PM
updated PR.
Mark Kelly February 22, 2016 at 9:41 PM
Closing PR and re-issuing new PR
Mark Kelly February 22, 2016 at 9:40 PM
Old PR with comments:
https://github.com/hpcc-systems/HPCC-Platform/pull/7845
Mark Kelly May 20, 2015 at 3:34 PM
There is also a compressor called density that shows real promise.
I have lz4 in a branch and will compare to density and we can decide which (or both) to add.
Jacob Cobbett-Smith October 23, 2014 at 2:24 PMEdited
@Mark Kelly - see https://hpccsystems.atlassian.net/browse/HPCC-12473#icft=HPCC-12473
Fixed
Pinned fields
Click on the next to a field label to start pinning.
Details
Details
Assignee
Mark Kelly
Mark KellyReporter
Gavin Halliday
Gavin HallidayPriority
MajorFix versions
Labels
Pull Request URL
Created August 30, 2013 at 2:31 PM
Updated October 4, 2016 at 4:38 PM
Resolved March 3, 2016 at 8:58 AM
See
http://code.google.com/p/lz4/
If it is very quick then it is probably worth using for compressing all temporary files and blocks send between nodes.