Skip to content

SimuPoLAS/Ofc

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

OFC

OFC is a tool for compressing OpenFOAM® files. It can compress files lossless or by rounding values to a specified amount of digits. It does compression by grouping numbers which have a similarity. They might have the same exponents, can all be expressed by a linear function, switch between other groups (ping-pong) or need the same amount of bits to write.

When an input file is compressed it is split up into two different files:

  • *.meta - contains all the metadata which is not getting compressed by the algorithm
  • *.dat - contains all the data which is compressed by the algorithm

These files are then compressed with LZMA (http://www.7-zip.org/sdk.html), which again improves compression. If the file can not be compressed by the algorithm. It might not contain any values or is not in the format we expect it to be (e.g. script files, log files, executables, ...) the files will still be compressed with LZMA. Because there is no compressable data the file is not split up in two different parts and is simply saved as *.lzma.

Command-line-interface (CLI)

A command line tool for compressing Open Foam (r) files.

Usage:
  ofc.exe [-h|--help]
  ofc.exe [--version]
  ofc.exe algorithms list
  ofc.exe compress directory <input> <output> [-f] [-r] [-p] [-s] [--algorithm=<name>] [--rounding=<digits>]
  ofc.exe compress file <input> <output> [-f] [-s] [--algorithm=<name>] [--rounding=<digits>]
  ofc.exe decompress directory <input> <output> [-f] [-r] [-p] [--algorithm=<name>]
  ofc.exe decompress file <input> <output> [data] [-f] [--algorithm=<name>]

Methods:
  algorithms list       Lists all available algorithms for compression/decompression.
  compress directory    Compress the specified directory.
  compress file         Compress the specified file.
  decompress directory  Decompress the specified compressed directory.
  decompress file       Decompress the specified compressed file or directory.

Options:
  -h --help            Display this help message.
  --version            Display the current version of the tool.
  --rounding           Enable rounding to the specified amount of digits.
  -f                   Force overriding of files.
  -r                   Enable recursive compression/decompression.
  -p                   Enable parallel compression/decompression.
  -s                   Treat anonymous lists as lists of one type.

General note: always provide absolute paths instead of relative paths when providing paths.

Usage

This section describes the general usage of the tool and the parameters shown in the #usage section, but in more detail. When providing paths always provide absolute paths and remember to put them in double-quotes " if they contain spaces.

Listing algorithms

Syntax: ofc.exe algorithms list


Lists all available algorithms which can be used for compression or decompression. The default algorithm used when no algorithm is specified is marked with an asterisk *. Example output which has the algorithms blocky and zetty available, while zetty is the default algorithm used:

   blocky
 * zetty

Using algorithms

Syntax: --algorithm=<name>


In order to use a specific algorithm when compressing/decompressing specify the option algorithm and supply the name of the algorithm which should be used. If you are not sure what algorithms are available use algorithms list as described here.


Example of a file compression using the algorithm zetty for more information on how to use the compression/decompression commands, see the chapters below:

ofc.exe compress file /usr/data /usr/out -f --algorithm=zetty

Compressing files

Syntax: ofc.exe compress file <input> <output> [-f] [-s] [--rounding=<digits>]


Compressing the file /usr/data and storing the compressed data in the file /usr/out. We are using the option -f in order to override the file if it does already exist:

ofc.exe compress file /usr/data /usr/out -f


Compressing the specified file while rounding all list values to 2 digits after the decimal point (option --rounding - who would have thought):

ofc.exe compress file /usr/data /usr/out -f --rounding=2

Rounding only affects items inside of a list. Numbers outside of a list are not rounded.


Compressing the specified file while treating anonymous lists as lists of one type (option -s - for simple-treating):

ofc.exe compress file /usr/data /usr/out -f -s

More information about the -s option can be found here.

Compressing directories

Syntax: ofc.exe compress directory <input> <output> [-f] [-r] [-p] [-s] [--rounding=<digits>]


Compressing the top level files of /usr/data/ and storing the compressed data in the directory /usr/out/. We are using -f in order to create the output directory if it does not exist:

ofc.exe compress directory /usr/data/ /usr/out/ -f


Compressing all files inside of the specified directory using the option -r (recursive):

ofc.exe compress directory /usr/data/ /usr/out/ -f -r


Compressing all files inside of the specified directory by using parallelization (option -p), which speeds up the process - especially if there are a lot of files:

ofc.exe compress directory /usr/data/ /usr/out/ -f -r -p


Compressing all files inside of the specified directory while rounding all list values to 2 digits after the decimal point (option --rounding):

ofc.exe compress directory /usr/data /usr/out -f -r --rounding=2

Rounding affects only items inside of a list. Numbers outside of a list are not rounded.


Compressing all files inside of the specified directory while treating anonymous lists as lists of one type (option -s - for simple):

ofc.exe compress directory /usr/data /usr/out -f -s

More information about the -s option can be found here.

Decompressing files

Syntax: ofc.exe decompress file <input> <output> [data] [-f]


Decompressing the file /usr/out.meta and storing the decompressed file as /usr/data. The -f (force) option is supplied in order to override the output file:

ofc.exe decompress file /usr/out.meta /usr/data -f


Decompressing the specified file while explicitly specifying a data file (the default is the input file with a .dat ending - e.g. info file: out.meta; default data file: out.dat):

ofc.exe decompress file /usr/out.meta /usr/out.dat -f

Note: In this case the specification of the data file is irrelevant as it is the default value, but in special cases it is useful to supply a data file explicit.

Decompressing directories

Syntax: ofc.exe decompress directory <input> <output> [-f] [-r] [-p]


Decompressing the top level files of /usr/out/ and storing the uncompressed data in the directory /usr/data/. We are using -f in order to create the output directory if it does not exist:

ofc.exe decompress directory /usr/out/ /usr/data/ -f


Decompressing all files inside of the specified compressed directory using the option -r (recursive):

ofc.exe decompress directory /usr/out/ /usr/data/ -f -r


Decompressing all files inside of the specified directory by using parallelization (option -p), which speeds up the process - especially if there are a lot of files:

ofc.exe decompress directory /usr/out/ /usr/data/ -f -r -p

Treating anonymous lists as lists of one type

By setting the -s option when compressing the behavior when parsing an anonymous list changes. Normally there is no way to know what type the contents of the list have. There might be a number then a string - back to number and so on. This possibility of changing types dramatically reduces the performance of the compression.

By enabling the -s option an anonymous list can only have of one type for all items. Here is an example of a list which can be optimized by the -s option as all element of the list have the type number:

10 (1 2 3 4 5 6 7 8 9 10)

If the -s option is enabled the following example would fail as the list contains a mix of numbers and characters.

10 (1 a 2 b 3 c 4 d 5 e)

Requirements

Windows: In order to run OFC on windows you need to have the .NET framwork 4.5 installed, which is preinstalled on most machines nowadays. If the .NET framework 4.5 is not installed on your machine you can download it here.

Windows, Linux, Mac & Docker: In order to run the core version of OFC you will need to install .NET core. Downloads for the different platforms and the installation instructions can be found here.

Installation

Make sure you are meeting the requirements above before continuing with the installation.

Download the latest release version for your platform from here. Choose the Win version if you are running on windows and choose the Core version if you are running on either Linux, Mac or Docker.

Extract all the files in a directory of your choosing. It might be a good idea to create a symbolic link in order to have easier access to the tool.

In order to run the tool under the Win version you simply have to run the Ofc.exe. If you want to run the Core version you have to use the dotnet toolchain and run the tool like this: dotnet Ofc.dll.

Future

While the project is on hold there are ideas that might be implemented in the future:

  • Integration into OpenFOAM®
  • Integration into ParaView
  • File specific configuration
  • More lossless compression methods
  • LZMA specific pre-processing
  • Faster processing of files

Bugs/Features & Co

If you find any bugs, have any new feature requests or something else related to the project leave an issue. Please remember that the project is currently on hold and there might not be a quick (or even any) response to the issues raised.