Engineering data is often compressed to reduce disk space being used, to speed up network transfers when moving result data across the network, and to reduce the rate of data growth.
There are a few different strategies to reducing the file size burden of engineering output data:
- Use a tool that removes unnecessary data in the output that is not required, leaving the data still readable by your post processing tools, but reduced in size. FEMZIP is a commercial tool from SIDACT that can do this. For LS-DYNA users, plotcprs compression utilities also perform a similar function.
- Extract only the parts you are interested in from your output, for example using Python to extract out the interesting parts of the ODB file if you are a Abaqus user.
- Use standard compression tools that just compress the output, for example gzip. To make it usable, the data must be uncompressed before being utilized. ( Though some applications like ANSYS Fluent natively can decompress gzip files )
This post is focusing on some emerging technologies for the third strategy of using standard compression tools. Historically standard compression algorithms were written to only utilize one CPU, and thus compressing/decompressing large files can take a very long time and don’t utilize the full compute horsepower.
TotalCompress is a multi-core compression tool which can compress results 120 times faster than legacy compression tools by utilizing multiple CPU cores with an optimized parallel algorithm. It achieves this speedup without sacrificing the level of compression it achieves. Below are some example real world comparison results compressing output from a variety of solvers
The most dramatic result was this 2.5GB Nastran OP2 file compressed by TotalCompress in less then 1 second, while gzip took over two minutes and resulted in a larger file.
A 1GB Abaqus ODB file also compressed more than FOUR times faster with smaller file size:
Finally, this STAR-CCM+ 6GB sim file took over 300 seconds with gzip, while TotalCompress did the same level of compression in 50 seconds.
With TotalCompress enabling super fast compression/decompression engineers can now automatically compress all of their output data, knowing they can quickly decompress it as needed without sitting around waiting for the results.
TotalCompress is included free of charge for TotalCAE customers as part of our management platform.