# File Space Accounting

# Introduction

File Space Accounting allows recursive monitoring of file usage and provides usage reports that can be grouped by user ID, file type, project, hosts, and more. This provides a clear understanding of how file space is being utilized. This will produce the following aggregated data types used for historical reporting:

(29) Filespace

The following section will guide you on configuring file space accounting.

# Requirements

An Open iT Client connected to an Open iT Server or a coexistent Open iT setup
Activated StorageAnalyzer and file space collector

# Configuring File Space Accounting

Open iT collects file space data from all clients. A central configuration file on the Open iT server specifies which file spaces to account for. The process is I/O intensive; therefore, each file space is accounted for locally. By default, subdirectories mounted over the network are not included in the accounting. However, network-mounted file systems can be included if configured to do so.

The filespace_win.conf configuration file contains a list of mappings to filter the components considered in storage monitoring.

The server distributes the filespace_win.conf configuration to the clients automatically. Any modification on the client will be lost.

The file has three main configuration parts:

Name	Description
site	This includes configuration for various sites, domains, or workgroups that will be included in storage monitoring.
exclude	This includes configuration for various files that will be excluded in storage monitoring.
map	This includes mapping configuration for specific hosts and files to a specific account.

Parts of the File Space Configuration File

# Adding Site to File Space Storage Monitoring

In the server, open filespace_win.conf in the Configuration directory, which is by default in C:\Program Files\OpeniT\Core\Configuration. Read through the written instructions for configuration.

Locate the line:

filespace_win.conf
all
{
# Pattern that must match for this site/domain/workgroup to be considered.
# Use * to match all possible sites/domains/workgroups.
# Example of using a workgroup:
# site=WORKGROUP
site=*
...

Add new entries as needed. following the format:
```
site=<site_name>
```
Example
```
site=*
site=SVG.OPENIT.LOCAL
site=LOCAL
```
The example means to include storage monitoring for the domain SVG.OPENIT.LOCAL and the workgroup LOCAL.
Save the changes.

# Excluding Files to File Space Storage Monitoring

Excluded file systems indicated in the configuration file are also excluded in file space accounting. All non-local file systems are also excluded by default.

In the server, open filespace_win.conf in the Configuration directory, which is by default in C:\Program Files\OpeniT\Core\Configuration. Read through the written instructions for configuration.

Locate the line:

filespace_win.conf
# Files to be excluded from accounting.
# Default is to exclude names enclosed in curly braces and names ending in
# .sys (system files).
# NOTE: Some of the files excluded by default may not be accessible, and
# warnings will be reported and incomplete reports generated if they are
# included.
exclude={*}
exclude=*.sys

Add new entries as needed. following the format:
```
exclude=<file_name>
```
Example
```
exclude=*tmp
exclude=*temp
```
The example will exclude temporary files (files ending with .tmp and .temp) in file space storage monitoring.
Save the changes.

# Adding Maps to File Space Storage Monitoring

In the server, open filespace_win.conf in the Configuration directory, which is by default in C:\Program Files\OpeniT\Core\Configuration. Read through the written instructions for configuration.

Locate the line:

filespace_win.conf
# Mappings.
map
{
  # Global match. Account all systems on "My Computer" if no other config is available.
  global
  {
	host=*
	match
	{
		all
		{
			dir=*
			account=UNDEF
		}
	}
}

Add new blocks under the map block, following the format:
Example
```
map
{
...
  temp
  {
    host=mnl164win
    match
    {
       temps
       {
         dir=?:\temp*
         account=TEMP
       }
    }
  }
```
The example will include temporary files in the host MNL164WIN for storage monitoring.
Save the changes.

The filespace_unix configuration file contains a list of mappings to filter the components considered in storage monitoring.

The server distributes filespace_unix configuration to the clients automatically. Any modification on the client will be lost.

In the server, open filespace_unix in the Configuration directory, which is by default in /var/opt/openit/etc. Read through the written instructions in the file.

filespace_unix
# This configuration file is line-oriented. Each line has two elements separated
# by space. 

# DOMAIN:       mycompany.com
# EXCLUDE:      \.snapshot
# EXCLUDE:      \.temp
# fs1:/testfs    TEST

# The example above triggers file space accounting in the domain named mycompany.com
# excluding the snapshots and temporary files. In host fs1, directory /testfs is
# accounted to TEST.

Lines starting with hash signs (#) are considered comments and are ignored in the processing.

Add new entries as needed. Make sure that all instructions and rules are followed to avoid malfunctions.

Entries are processed from top to bottom, and the last pattern that matches is prioritized.
Save the changes.

# Configuring File Type Classification

File space accounting collects all file data included in file systems by default. Three possible methods (levels) classify the files:

Level	Description
Level 0 Accounting	Select this as a faster alternative if the file type is of no interest. The file type will be regarded as undefined and set to undef.
Level 1 Accounting	This is the default method that uses filenames to decide on the file type.
Level 2 Accounting	This is the most advanced and slowest method that uses the file contents to decide on the file type.

File Type Levels

Open filespace.conf in the Configuration directory, which is by default in C:\Program Files\OpeniT\Core\Configuration.
Locate the filespace block.

Change the value of the filetype attribute to the desired value from 0 to 2. By default, value is set to 1.

filespace.conf
filespace()=""
{
   log()=""
    {
      debug-level(Integer)="0"
    }
    working-dir(DirName)="${OpeniT.directories.data}/FileSpace"{}
    temp-dir(DirName)="${OpeniT.directories.temp}"{}
        ...
    filename(FileName)="${OpeniT.directories.conf}/filename_win.conf"{}
    filetype(Integer)="1"{}
    temperatures(FileName)="${OpeniT.directories.conf}/file_temperature"{}
...

Save the changes.

# Configuring File Names to File Type Mapping

Configure the filename_win.conf file to map file extension to file types using the default file type Level 1.

The server distributes filename_win.conf configuration to the clients automatically. Any modification on the client will be lost.

Before mapping, verify the level and file used:

Open filespace.conf in the Configuration directory, which is by default in C:\Program Files\OpeniT\Core\Configuration.
Locate the filespace block.

Verify the values of the filename and filetype attributes.

filespace.conf
filespace()=""
{
   log()=""
    {
      debug-level(Integer)="0"
    }
    working-dir(DirName)="${OpeniT.directories.data}/FileSpace"{}
    temp-dir(DirName)="${OpeniT.directories.temp}"{}
        ...
    filespace(FileName)="${OpeniT.directories.conf}/filespace_win.conf"{}
    filename(FileName)="${OpeniT.directories.conf}/filename_win.conf"{}
    filetype(Integer)="1"{}
    temperatures(FileName)="${OpeniT.directories.conf}/file_temperature"{}
...

After verification:

In the server, open filename_win.conf in the Configuration directory, which is by default in C:\Program Files\OpeniT\Core\Configuration.
Read through and follow the written instructions for configuration.
After the modifications, save the changes.

# Configuring File Temperature

The file_temperature configuration file contains various temperature levels for the files accounted for. There are five default temperature levels, each with its own default label and upper boundary (in days).

# Temperature settings for file_space accounting.

# This file has the default settings for five
# temperature levels.
# Temperature labels  Upper boundary(days)
# ------------------  --------------------
hot                   30
warm                  90
tepid                 360
cold                  720
frozen                infinity

In this case, the lower boundary is considered as the upper boundary of the previous level. The lower boundary of the first level is assumed to be 0. The last level should have no number but infinity. This means that there will be no upper limit for this temperature level.

The server distributes file_temperature configuration to the clients automatically. Any modification on the client will be lost.

In the server, open file_temperature in the Configuration directory, which is by default in C:\Program Files\OpeniT\Core\Configuration.
Modify the configuration file as needed.

Temperature levels must be in ascending order of the upper boundary values.
Save the changes.

In the server, open file_temperature in the Configuration directory, which is by default in /var/opt/openit/etc.
Modify the configuration file as needed.

Temperature levels must be in ascending order of the upper boundary values.
Save the changes.

There may be any number of temperature levels, but there is no way of changing the levels for old data. The temperature levels of data collected earlier remains unchanged, even if the default settings are changed.

# Configuring File Space Period and Offset

Go to the scheduler directory, which is by default in C:\Program Files\OpeniT\Core\Configuration\scheduler, and open collect_storage_filespace-win.oconf.
Locate and set the value of root.scheduler.jobs.collect_filespace_win.scheduling.start-triggers.trigger#1.period. The default value is P7D.
collect_storage_filespace-win.oconf
```
66| period
67| {
68|     description=Run weekly
69|     type=timespan
70|     value=P7D
71| }
```
Locate and set the value of root.scheduler.jobs.collect_filespace_win.scheduling.start-triggers.trigger#1.offset. The default value is P-5DT2H.
collect_storage_filespace-win.oconf
```
72| offset
73| {
74|     description=Rather than running at midnight Monday, start 2 hours before midnight on Saturday
75|     type=timespan
76|     value=P-5DT2H
77| }
```
Specify the period and offset values to your desired timespan based on the ISO-8601 duration fromat PnDTnHnMn.nS.
Save the changes.

For Unix systems, you may refer to the Configuring File Space Period and Offset in Unix page.

# Sample Historical Report

File Storage Report for svg_host0045