Welcome to Data Processing Compendium – Workflows for Knowledge Exploitation in the Process Industries!

This compendium contains a catalogue of methods describing single steps and best practice procedures in data processing, with so far a focus on data pre-processing. The Data Processing Compendium is a key result of the AiF research project Knowledge based pre-processing of process data (IGF-Vorhaben Nr. 16901 N). The project has involved over 10 industrial partners from Belgium, France, and Germany.


The catalogue of methods contains a list of single steps in the field of data pre-processing, e.g., Variance Estimation and Data Denoising. The catalogue is also augmented by important properties, such as input and output parameters (e.g., Variance as an input for Data Denoising) and available software tools that support the user in performing the corresponding pre-processing step. The steps are classified into several main categories, including but not limited to Data Cleaning, Data Integration, Data Transformation, Data Reduction, Data Discretization, and Auxiliary Steps. Established work processes in data pre-processing, which document best practice procedures from industrial partners and from literature are available in the catalogue as well.

A Catalogue of Methods in Data Pre-processing

The current version of the catalogue of methods collects a list of existing pre-processing steps (Single Steps) and some reference work processes in data pre-processing. The catalogue is being continuously extended.


Das IGF-Vorhaben Nr. 16901 N der Forschungsvereinigung DECHEMA e.V., Theodor-Heuss-Allee 25, 60486 Frankfurt am Main, wurde über die AiF im Rahmen des Programms zur Förderung der industriellen Gemeinschaftsforschung und -entwicklung (IGF) vom Bundesministerium für Wirtschaft und Energie aufgrund eines Beschlusses des Deutschen Bundestages gefördert.

The IGF project 16901 N of the research association DECHEMA e.V., Theodor-Heuss Allee 25, 60486 Frankfurt am Main, Germany, has been funded via AiF within the program for industrial joint research and development (Industrielle Gemeinschaftsforschung und -entwicklung, IGF) by the German Federal Ministry of Economics and Technology based on a resolution of the German Federal Parliament.