The DIR_CONFIG file

The AMG expects a configuration by default in the file $AFD_HOME_DIR/etc/DIR_CONFIG. This default can be changed in $AFD_HOME_DIR/etc/AFD_CONFIG with the parameter DIR_CONFIG_NAME. By inserting multiple DIR_CONFIG_NAME lines in AFD_CONFIG, you split up the content of the configuration over several files. The content should conform to a certain set of rules. All entries are grouped in sections, a section begins with the line [section name] and continues until the next section starts. Supported sections names and the corresponding entries are listed below and must be set in this order:

Comments start with # and may be placed anywhere except directly behind the files and destination identifiers.

The directory entry [directory]

The directory in which the AMG has to search for files which have to be send. Each directory must begin with / or a ~ and this directory must exists or the AMG must have read and write access so it can create the directory recursively. Otherwise it ignores the complete entry for this directory. The number of directory entries in the DIR_CONFIG is unlimited. However duplicate directory entries will be ignored. If you wish to give the directory an alias name you can do so by putting the name directly behind the directory entry: [directory] <dir alias>. AFD is also able to fetch files from a remote host, the directory must then be specified in URL format as follows: <scheme>://[<user>][;fingerprint=<SSH fingerprint>][:<password>]@<host>[:<port>][/<url-path>][;type=<i|a|d>][;protocol=<protocol number>]

The items are separated by special characters :, /, @ and ;. If these characters appear within one of the items, they must be masked with a \.

See recipient for a more detailed description on how to use the fingerprint parameter and others.

There are currently four schemes for fetching files supported:

Scheme	Description	Example
ftp	Normal FTP transfers.	ftp://donald:secret@ducktown/directory
ftp[sS]	FTP transfers over secure sockets (TLS/SSL). When fpts is used only the control connection is encrypted. With ftpS both control and data connection are encrypted.	ftps://donald:secret@ducktown/directory
sftp	SFTP, file transfer via SSH.	sftp://donald:secret@ducktown/directory
http	Normal HTTP transfers.	http://donald@ducktown/directory
https	HTTP transfers over secure sockets (TLS/SSL).	https://donald@ducktown/directory

For remote FTP directories it is possible to insert a date or current hostname with the following modifiers:

Feature	Description	Example [directory] entry	Resulting directory name
%tx	Insert the actual time in any form. The character x determines the time format which can be one of the following: a - short day "Tue", A - long day "Tuesday", b - short month "Jan", B - long month "January", d - day of month [01,31], m - month [01,12], j - day of the year [001,366], y - year [01,99], Y - year 1997, H - hour [00,23], M - minute [00,59], S - second [00,60] U - Unix time, number of seconds since 00:00:00 01/01/1970 UTC	ftp://daisy@ducktown/diary/%tY/%tm/%tH	ftp://daisy@ducktown/diary/2006/04/01
*%T[+\|-\|\|/\|%]xS\|M\|H\|d**	Time modifier for the above %tx option. It adds (+), subtracts (-), multiplies (*), divides (/) or computes the modulo (%) with the value x to the current unix time. The default unit for the time modifier is seconds this can be changed to minute (M), hour (H) or day (d). Once set the modifier is used for all subsequent %t until it changed by the next modifier as shown in the example.	ftp://goofy@hollywood/%td_%T+1d%td_%T0%td	ftp://goofy@hollywood/28_29_28
%h	Insert the hostname of the current node. Depending on how the system is configured, this might return full network node hostname.	ftp://donald@daisy/archive/%h/todo	ftp://donald@daisy/archive/ducktown/todo

NOTE: The option [directory] must always start at the begining of a line. There may not be any tabs or spaces in front.

The directory options [dir options]

With the options in the table below, it is possible to specify what the AFD should for example do with files that are in the directory but are not being distributed, how files are to be retrieved from a remote hosts, etc. [dir options] may only be specified once for each [directory] option. A full list of options for this optional section and the defaults is shown in the table below:

Option

Description

Local Dir

Remote Dir

Default

delete unknown files [time in hours]

Delete all files that are older then the number of hours specified with the <old file time> option and are not to be distributed. If the optional time in hours is specified it will override the value given in "old file time".

do not delete unknown files

The opposite of the above option.

delete queued files [time in hours]

Delete all files queued on input that are older then the number of hours specified with the <old file time> option and are to be distributed. If the optional time in hours is specified it will override the value given in "old file time".

delete old locked files <time in hours>

Delete all old locked files on input that are older then the number of hours specified.

report unknown files

Any files found in this directory that are not being distributed and are older then the number of hours specified in the <old file time> option will be reported in the system log.

do not report unknown files

The opposite of the above option.

old file time <value in hours>

The age of a file in hours before a file is being reported in the system log or being deleted.

end character <decimal number>

Sometimes some application may not be able to send files with a leading dot and then rename these files after they have send all data. With this option AFD will wait until the character specified with <decimal number> will appear a the last character in the file before it will grab the file.

important dir

This marks the directory as important. This is useful when you have lots of directories that are to be monitored by the AMG and/or you have only a slow disk. The process dir_check will now check all important directories every DIR_CHECK_TIMEOUT (default: 30) second. This ensures that the important directories will at least be checked every DIR_CHECK_TIMEOUT second and not until all directories have been checked, which could take very long.

time <min hour dom month dow>

With this option it is possible at what interval the directory is to be scanned. See time under options for a more detailed description of what each field means.

do not get dir list

Setting this option will disable the directory scanning for this directory. This can be used when directory listing are not possible or disabled because of security reasons. Currently this only works for HTTP. This also means that under [files] the full filenames without wildcards must be listed, since the protocol does not resolve any wildcards.

do not remove

By default when AMG will remove all files that it finds in the directory being scanned. With this option set it will NOT remove these files.

store retrieve list [once]

When the above option <do not remove> is set, it can happen that always the same files are being collected by the AMG. To avoid this situation this option can be set. Then AMG will keep a list of files and their modification time and there size of all files found during the last scan. So only when a file has changed will it be retrieved by the AMG. When the optional parameter 'once' is appended, files will be fetched only onced, regardless if the size or modification time changes. This is can be usefull for directories with lots of files and protocols where we need to poll each individual file for size and date. For SFTP the once parameter is not needed since all needed values are retrieved during the directory scan and no additional polling is required. However when set for SFTP it will only fetch the file once.

priority <value>

With this you can give the directory a priority. This option is only useful for remote directories, ie. from where the files have to be retrieved. 0 is highest priority and 9 is the lowest.

force reread

Before AFD scans the directory for files it checks if the time has changed since the last scan. With this options set it will always read the directory. This can be useful for some broken filesystems that do not update this information correctly. Needless to say that this will create a higher system load.

max process <value>

Here you may limit the number of process that the AMG may fork to handle files for this directory. AMG will only fork a process when using the exec option, or some other option that can use a lot of time to process.

ignore size <[=|>|<] value>

If the files in the directory have the given size they will be ignored. A greater or less then sign may also be used to ignore files which have a size greater or less then the given value. The unit entered is in bytes.

ignore file time <[=|>|<] value>

If the difference of the files modification time in the directory and the current time is outside the given value, then it will be ignored. A greater or less then sign may also be used. The unit is in seconds. This option is best used together with 'force reread', because the directory is only scanned when something happens (rename, delete or added a new file) in the directory.

max files <value>

The maximum number of files that are copied before they are send to the FD. This value is set so that the AFD starts sending files as soon as possible even when there are thousands of files waiting in the source directory to be send. Otherwise it might take to long to process all the files at once and time is lost that could be used to transmit data.

100

max size <value>

The maximum size in Kilobytes that are copied before they are send to the FD.

100 MB

accept dot files

Setting this option will enable AFD to retrieve files starting with a dot from a remote host. When downloaded the dot will be autamatically removed. Use the trans_rename option if you want to keep the dot.

wait for <file pattern>

With this option it waits for the given file before it sends all the other files from this directory.

warn time <time in seconds>

When no data has arrived in the directory for the given time it will warn in the RECEIVE_LOG.

0 (disabled)

accumulate <number of files>

Waits for the given number of files before it accepts files from this directory.

accumulate size <number of bytes>

Waits for the given number of bytes before it accepts files from this directory.

dupcheck[ <timeout in secs>[ <check type>[ <action>[ <CRC type>]]]]

Performs a duplicate on files that come into this directory. The options have the following meaning:

Option

Description

Default

timeout

Time in seconds when this CRC value is to be discarded from the database.

3600

check type

What type of check is to be performed, the following values are possible:

1	On filename only.
2	File content.
3	Filename and file content.

action

What action is to be taken when we find a duplicate. The following values are possible:

24	Delete the file.
25	Store the duplicate file in the following directory: $AFD_WORK_DIR/files/store/<id>. Where <id> is the directoy id.
26	Only warn in SYSTEM_LOG.
33	Delete and warn.
34	Store and warn.

CRC type

What type of CRC check is to be performed. Currently only 16 (CRC32) is supported.

The files entry [files]

In this section it possible which files are to be distributed from the directory specified in the directory section. Each file name may contain the wild cards '*' and/or '?' in any order. One may also list characters enclosed in []. A pair of characters separated by a hyphen denotes a range expression; any character that sorts between those two characters is matched. If the first character following the [ is a ! then any character not enclosed is matched.

If the file entry starts with a '!' it means NOT this file. It is important to put these entries first otherwise the '!' option is meaningless. Here follow some more additional options that one can insert into the file filter:

Feature	Description	Example [files] entry	Resulting filter
%tx	Insert the actual time in any form. The character x determines the time format which can be one of the following: a - short day "Tue", A - long day "Tuesday", b - short month "Jan", B - long month "January", d - day of month [01,31], m - month [01,12], j - day of the year [001,366], y - year [01,99], Y - year 1997, H - hour [00,23], M - minute [00,59], S - second [00,60] U - Unix time, number of seconds since 00:00:00 01/01/1970 UTC	cloud_%tH-*-%tY.png	cloud_19-*-2006.png
*%T[+\|-\|\|/\|%]xS\|M\|H\|d**	Time modifier for the above %tx option. It adds (+), subtracts (-), multiplies (*), divides (/) or computes the modulo (%) with the value x to the current unix time. The default unit for the time modifier is seconds this can be changed to minute (M), hour (H) or day (d). Once set the modifier is used for all subsequent %t until it changed by the next modifier as shown in the example.	*.%td_%T+1d%td_%T0%td	*.28_29_28
%h	Insert the hostname of the current node. Depending on how the system is configured, this might return full network node hostname.	*.%h	*.ducktown

The buffer for storing the file names of this section is allocated as needed, so there is no restriction. The number of file groups is also unlimited.

The destination entry [destination]

This marks the beginning of the destination section which contains one recipient and one options section. There is no limit in the number of destination entries per files section.

The recipient entry [recipient]

In the recipient section you specify the user name, password, remote host and remote directory in the following URL format:

<scheme>://[<user>][;fingerprint=<SSH fingerprint>][:<password>]@<host>[:<port>][/<url-path>][;type=<i|a|d>][;server=<server name>][;protocol=<protocol number>]

The items are separated by special characters :, /, @ and ;. If these characters appear within one of the items, they must be masked with a \.

The fingerprint= is only valid for scheme scp and sftp. To get the fingerprint from a host with OpenSSH enter the following command:

                ssh-keygen -lf /etc/ssh/ssh_host_rsa_key.pub

This will generate the fingerprint as shown in the following example:

     1024 11:ff:22:ee:33:dd:44:cc:55:bb:66:aa:77:99:88:88 /etc/ssh/ssh_host_rsa_key.pub

This fingerprint is then added as follows in the recipient line:

     sftp://donald;fingerprint=11-ff-22-ee-33-dd-44-cc-55-bb-66-aa-77-99-88-88:secret@hollywood//home/user

Another valid way of writting this is as follows:

     sftp://donald;fingerprint=ssh-rsa-11-ff-22-ee-33-dd-44-cc-55-bb-66-aa-77-99-88-88:secret@hollywood//home/user

Note that the : are replaced by a minus -.

It is not necessary to specify the real hostname or IP number for host. In fact it might be much more user friendly to use an alias name here, since this will be the name displayed in the afd_ctrl dialog. This alias name is then resolved via the HOST_CONFIG file. However, the host must always be supplied.

When scheme is sftp or scp one can use the option protocol= to specify the SSH protocol version.

The server= option is only useful for the mailto scheme. With this option it is possible to set the mailserver address. If it is not set the default is localhost (SMTP_HOST_NAME in protocols/smtpdefs.h). If it is set this will be the hostname that AFD will take as alias/real hostname.

Another useful feature for the mailto scheme is the possibility to group several mail addresses into one group. Under recipient it is then only necessary to specify one group name and not each recipient each on a separate line. This can be done as follows:

         [recipient]
	 mailto://$hotels@HOTELS

You then need to create a file in $AFD_WORK_DIR/etc/group.list with a list of mail addresses as shown in the following example:

         [hotels]
	 manager1@hotel1.com
	 manager2@hotel2.com
	 manager3@hotel3.com
	 manager4@hotel4.com

In this case it is also possible to just write mailto://$hotels. In the afd_ctrl dialog it will then appear as hotels. The group name may not be longer then 40 characters.

The directory name may contain modifiers for which AFD will insert the current or a modified time or the current host name where AFD is running. The modifiers are:

Feature	Description	Example [recipient] entry	Resulting directory name
%tx	Insert the actual time in any form. The character x determines the time format which can be one of the following: a - short day "Tue", A - long day "Tuesday", b - short month "Jan", B - long month "January", d - day of month [01,31], m - month [01,12], j - day of the year [001,366], y - year [01,99], Y - year 1997, H - hour [00,23], M - minute [00,59], S - second [00,60] U - Unix time, number of seconds since 00:00:00 01/01/1970 UTC	ftp://daisy@ducktown/diary/%tY/%tm/%tH	ftp://daisy@ducktown/diary/2006/04/01
*%T[+\|-\|\|/\|%]xS\|M\|H\|d**	Time modifier for the above %tx option. It adds (+), subtracts (-), multiplies (*), divides (/) or computes the modulo (%) with the value x to the current unix time. The default unit for the time modifier is seconds this can be changed to minute (M), hour (H) or day (d). Once set the modifier is used for all subsequent %t until it changed by the next modifier as shown in the example.	ftp://goofy@hollywood/%td_%T+1d%td_%T0%td	ftp://goofy@hollywood/28_29_28
%h	Insert the hostname of the current node. Depending on how the system is configured, this might return full network node hostname.	ftp://donald@daisy/archive/%h/todo	ftp://donald@daisy/archive/ducktown/todo

There are currently seven schemes supported:

Scheme	Description	Example
ftp	Normal FTP transfers.	ftp://donald:secret@ducktown/directory
ftp[sS]	FTP transfers over secure sockets (TLS/SSL). When fpts is used only the control connection is encrypted. With ftpS both control and data connection are encrypted.	ftps://donald:secret@ducktown/directory
mailto	Sending files via SMTP.	mailto://daisy@ducktown
file	Copies files to another directory.	file://@local/directory
wmo	Sends files via TCP according to WMO regulations at port 1111.	wmo://@wmo-host:1111
sftp	Sends files via SSH using the SFTP protocol.	sftp://donald:secret@ducktown/directory
scp	Sends files via SSH using the SCP protocol.	scp://donald:secret@ducktown/directory

Each recipient section may have an unlimited number of recipients and each may not be longer then 256 (MAX_RECIPIENT_LENGTH) characters.

The type option allows three different types of transfer modes and are valied only for scheme ftp and ftp[sS]:

        a - ASCII
        i - Image (binary)
        d - DOS Image (same as i accept that all CR are converted to CRLF)

The options entry [options]

For each destination section you may specify one options section and may contain up to 10 (MAX_NO_OPTIONS) options each 256 (MAX_OPTION_LENGTH) characters long. All options specified here count for all recipients that are specified in the recipient section. Valid options can be taken from the following table:

Option name

Description

AMG

ftp

mailto

file

wmo

sftp

scp

priority x

The priority of this job. Valid entries for x are 0-9, 0 is the highest priority and 9 is the lowest. (default 9)

delete

Deletes all files.

time <min hour dom month dow>

This option allows to enter cron like time jobs in the AFD. The five time and date fields in the time option specify when files are to be distributed, that is when the minute, hour, and month of year fields match the current time, and when at least one of the two day fields (day of month, or day of week) match the current time. The time and date fields are as follows:

time and date fields	minute	hour	day of month	month	day of week
allowed values	0-59	0-23	1-31	1-12	1-7

In the <day of week> field 1 is Monday and 7 is Sunday.

A field may be an asterisk, which always stands for all values in this field.

Ranges of numbers are allowed. The ranges consist of two numbers and a hyphen. There may not be any blanks! The specified range is inclusive. Example: "19-22"

Its also possible to list a set of numbers or even ranges. Examples: "1,2,4,15", "4-8,18-20"

Step sizes can be used after an asterisk. So to specify every five minutes, just use "*/5". They can also be used in combination with a range. For example, "0-30/5" would equal "0,5,10,15,20,25,30".

There may be more then one time option. The number of time option is not limited.

time no collect <min hour dom month dow>

This option is the same as the above time option only that no files will be collected (these will be deleted) outside the given time frames. The number of time no collect option is not limited.

basename

Sends only the basename of the file. For example if you have the following file name xxxx.yyyy.zzzz, it will rename the file on the remote site to xxxx. NOTE: You may overwrite files when they have the same name.

extension

Removes only the extension from the file name. Taking the example file name from above, the file name on the remote site would be xxxx.yyyy

prefix add xx

Adds a prefix to the file name.

prefix del xx

Remove a prefix from the file name. If the AMG does not find this prefix, this option is ignored.

toupper

Convert all letters of the file name to upper case.

tolower

Convert all letters of the file name to lower case.

rename <rule> [overwrite]

Rename files as defined in the 'rule'-section of the etc/rename.rule file. If the overwrite option is NOT set, multiple files with the same name will not be overwritten. This is done by appending a ; and a number to the file name that is renamed.

trans_rename <rule> [primary_only|secondary_only]

Rename files on the remote site after the files have been transmitted, as defined in the 'rule'-section of the etc/rename.rule file. This renaming may include directory names, however the directories must exist on the remote site. If the additional option primary_only is specified and the system has a secondary host specified, the renaming will only occur on the primary system. When secondary_only is set, the renaming will only happen on the secondary node.

force copy

Force files to be copied and not linked nor memory mapped (Good for copying files to /dev/null).

chmod mode

Changes the permission of each file distributed. Only a numeric mode from one to four octal digits (0-7) is accepted. The first digit selects the set user ID (4), set group ID (2) or the save text image (1) attributes. The second digit selects permissions for the user who owns the file: read (4), write (2) and/or execute (1). The third selects permissions for other users in the files group. The fourth digit is for all other users (world).

chown [user][:.][group]

Changes the user and/or group ownership of files. If only a user name (or numeric user ID) is given, that user is made the owner of each given file, and the files group is not changed. If the user name is followed by a colon or dot and a group name (or numeric group ID), with no spaces between them, the group ownership of the files is changed as well.

create target dir

Creates the target directory recursively if it does not exist. This will not work when the option Fast cd is set, since the directory will be created during the cd command. If the option CREATE_TARGET_DIR is set to YES in AFD_CONFIG, then this option does not need to be set since then it is the default.

do not create target dir

If the option CREATE_TARGET_DIR is set to YES in AFD_CONFIG, then it can be overridden with this option.

tiff2gts

Converts T4-coded TIFF files to T4-files having a GTS header and end.

gts2tiff

Converts T4-GTS files to TIFF-files.

fax2gts

Converts FAX to a GTS T4 file. The GTS header is taken from the filename, so this must be a typical "TTAAii CCCC YYGGgg" WMO header.

grib2wmo[ CCCC]

Converts GRIB to WMO files with "TTAAii CCCC YYGGgg" headers, SOH and ETX at start and end and 10 Byte ASCII length and type indicator. You can optionally supply the CCCC if you do whish to overrule the one from the GRIB.

wmo2ascii

wmo2ascii will remove SOH, ETX and the two carriage returns from the given WMO file. It will convert the following text from:

          <SOH><CR><CR><LF>nnn<CR><CR><LF>
	  WMO header<CR><CR><LF>WMO message<CR><CR><LF><ETX>

          WMO header<LF>WMO message<LF>

in addition <CR><CR><LF> will be changed to <LF>. The file name will not be changed.

afw2wmo

afw2wmo will convert the old AFW format used at DWD to WMO bulletin format.

assemble <type>[ <file name>]

This options assembles all files to a single file. The original files will be deleted. To help extraction of the files a length indicator may be inserted. The following types can be assembled:

VAX	Two byte length indicator.
LBF	Four byte length indicator with low byte first.
HBF	Four byte length indicator with high byte first.
DWD	Four byte length indicator DWD standard, that is like HBF but with four NULL bytes at end.
MSS	Four byte length indicator MSS standard.
WMO	Eight byte ASCII length indicator plus two byte type indicator (00 - with SOH and ETX, 01 - no SOH and ETX) . This is the current WMO standard.
ASCII	No length indicator. The data of the next file is always appended, no matter if it contains binary or ascii data.

The <file name> may contain a %n to insert a unique number four characters long and/or with the help of %tx insert the actual time, where character x may be one of the following:

         a - short day "Tue",           A - long day "Tuesday",
         b - short month "Jan",         B - long month "January",
         d - day of month [01,31],      m - month [01,12],
         j - day of the year [001,366], y - year [01,99],
         Y - year 1997,                 H - hour [00,23],
         M - minute [00,59],            S - second [00,60]
         U - Unix time, number of seconds since 00:00:00 01/01/1970 UTC

If no file name is specified the default will be no_file_name.

convert <type>

This options converts all files fom one format into another. The original file will be overwritten with the new one. The following types can be converted:

sohetx	Adds <SOH><CR><CR><LF> to the beginning of the file and <CR><CR><LF><ETX> to the end of the file.
wmo	Just add WMO 8 byte ascii length and 2 bytes type indicator. If the message contains <SOH><CR><CR><LF> at start and <CR><CR><LF><ETX> at the end, these will be removed.
sohetxwmo	Adds WMO 8 byte ascii length and 2 bytes type indicator plus <SOH><CR><CR><LF> at start and <CR><CR><LF><ETX> to end if they are not there.
sohetx2wmo0	converts a file that contains many ascii bulletins starting with SOH and ending with ETX to the WMO standart (8 byte ascii length and 2 bytes type indicators). The SOH and ETX will be copied to the new file.
sohetx2wmo1	Same as sohetx2wmo0 only that the SOH and ETX will NOT be copied to the new file.
mrz2wmo	Converts GRIB, BUFR and BLOK files to files with 8 byte ascii length and 2 bytes type indicator plus <SOH><CR><CR><LF> at start and <CR><CR><LF><ETX> to the end, for the individual fields.

extract [-c|-C|-n|-N|-s|-S] <type> [<pattern>]

This options extracts files with WMO bulletins into one file per bulletin and will delete the original file. Since it is not possible to automatically detect the file type and its length encoding, you to specify what type of file it is. To indicate the length of the bulletin there is either a two or four byte length indicator at the beginning of the following format:

VAX	Two byte length indicator.
LBF	Four byte length indicator with low byte first.
HBF	Four byte length indicator with high byte first.
MSS	Four byte length indicator MSS standard.
WMO	Eight byte ASCII length indicator plus two byte type indicator. This is the current WMO standard.
ASCII	No length indicator. The start of a bulletin is marked by SOH (binary 1) and end is ETX (binary 3).
MRZ	Special four byte length indicator used in the DWD.
GRIB	Standard GRIB, however since there are so many different GRIB types and limited namespace for WMO header, many GRIB's will overwrite each other.

The file name of the new file will always be the WMO-header, that is: TTAAii_CCCC_YYGGgg[_BBB]-<CRC checksum>

The optional options have the following meaning:

      -c     adds a CRC checksum to end of filename [default]
      -C     do not add a CRC checksum to end of filename
      -n     adds a four digit unique number to end of filename
      -N     do not add a four digit unique number to end of filename [default]
      -s     add SOH and ETX if not there [default]
      -S     remove SOH and ETX

If a pattern is supplied, only those files will be extracted that match the given pattern.

exec [-d|-D|-l|-L|-t] <command>

Executes a command specified in <command> for each file to be distributed. The file name can be supplied with %s. There may be up to 10 places in the command where you can insert the file name. The optional options have the following meaning:

          -d                  Deletes the original file after executing
                              the command regardless if the command
                              was succesfull or not.
          -D                  When an error occurs during execution all
                              files will be deleted. This only works if
                              no %s is found in the command string
                              otherwise it will be ignored.
          -l                  This sets a lock so that only one process
                              can be executed at one time for each file.
                              If %s is being used the lock is only set
                              for the execution duration of that file,
                              this gives other jobs a chance but only
                              one can do the execution at one time.
          -L                  Some as above only that the lock is held
                              for the duration of all files when %s is
                              being used.
          -t <time in second> If after the given timeout the execution
                              has not finished the process will be killed.

execd <command>

Same as the exec option above, but deletes the original file after executing the command.

execD <command>

Again this is similar to the exec option and differs in that when an error occurs during execution that all files will be deleted. This only works if no %s is found in the command string, otherwise it will behave just like the exec option.

pexec [-l|-L|-t] <command>

Executes a command specified in <command> for each file after it was distributed. The file name can be supplied with %s. There may be up to 10 places in the command where you can insert the file name. The files stored in the archive will NOT be effected by this option. The optional options have the same meaning as those above in the exec option.

lock <type>

DOT	When transmitting file "xxxx.yyyy" it creates the file on the remote site as ".xxxx.yyyy". When the file has been completely transmitted it is renamed to its original name. This ensures that the remote user can read the file when it is complete.
DOT_VMS	When transmitting file "xxxx.yyyy" it creates the file on the remote site as ".xxxx.yyyy". After transmission this file is renamed to "xxxx.yyyy.". This ensures that the remote user can read the file when the transfer is complete.
OFF	No locking on the remote site.
LOCKFILE [file name]	Before starting transmission of the actual data files, the lock file "CONNECTED______.LCK" is send or if given the file name after LOCKFILE is used. When all files have been send successful this file is removed again.
xyz.	When transmitting file "xxxx.yyyy" it creates the file on the remote site as "xyz.xxxx.yyyy". This ensures that the remote user can read the file when the transfer is complete.

If this option is not set the default is lock DOT.

lockp .zz

When transmitting file "xxxx.yyyy" it creates the file on the remote site as "xxxx.yyyy.zz". This ensures that the remote user can read the file when the transfer is complete.

sequence locking

When transmitting file "xxxx.yyyy" it adds the number of retries to the end of the file name "xxxx.yyyy-<retries>" during tranmission. After the file was transmitted successful it is renamed to the original name "xxxx.yyyy". Note, when set appending will NOT be possible. This option is usefull when the remote side has problems with still having this file open, since with every try it will open a new file. It will try to delete the previous before it opens a new one. This option can be combined with the other locking options.

archive x[d|h|m|s]

Tells the AFD to archive files to this user for x days if no unit [d|h|m|s] is specified. The meaning of the units is as follows:

d	Days
h	Hours
m	Minutes
s	Seconds

When no archive option is set, the default (0 = no archiving (DEFAULT_ARCHIVE_TIME)) is set. With afdcfg the system administrator can disable archiving completely for all jobs.

age-limit x

Files older then x seconds will not get send. They will be removed.

no log output

When set each file name distributed will NOT be logged to OUTPUT_LOG.

file name is subject

The subject of the mail is the file name. This can only be used when sending mails.

file name is user[ rename rule]

The user name of the mail is the file name. This can only be used when sending mails. The optional rename rule allows to filter or create the user name from the file name. For a detailed description of the renaming procedure see rename rule.

file name is target[ rename rule]

The full recipient of the mail is the file name. This can only be used when sending mails. The optional rename rule allows to filter or create the user name from the file name. For a detailed description of the renaming procedure see rename rule.

mail header[ full path to mail header file]

Is used to insert some default information at the top of the main body of the mail. If the option is used without the optional mail header file, it will take the file MAIL-host_alias in the etc directory as the header file.

attach file[ rename rule]

With this option it is possible to attach a file to a mail. Each file will be attached to a separate mail (one file per mail). The encoding will be done in BASE64 according to RFC 1521. If a rename rule is specified the filename will be renamed in the attachment.

attach all files[ rename rule]

With this option it is possible to attach files to a mail. If more than one file is available they will all be attached to ONE mail. The encoding will be done in BASE64 according to RFC 1521. If a rename rule is specified the filename will be renamed in the attachment.

subject <full path to subject file> [rename rule]

Adds the subject in the <subject file> to the mail. It is possible to insert the current date or filename, for more details see the next option.

subject "xxx"[ rename rule]

Adds the line xxx after the subject identifier to the mails subject. Note that only 7-bit ASCII characters are allowed in the subject. It is possible to insert the current date in any form with %tx where character x determines the time format which can be one of the following:

            a - short day "Tue",           A - long day "Tuesday",
            b - short month "Jan",         B - long month "January",
            d - day of month [01,31],      m - month [01,12],
            j - day of the year [001,366], y - year [01,99],
            Y - year 1997,                 H - hour [00,23],
            M - minute [00,59],            S - second [00,60]
            U - Unix time, number of seconds since 00:00:00 01/01/1970 UTC

Additionally it is possible to add the filename with %s or parts of it when a rename rule is supplied.

from <mail address>

This allows to change the From: line of a mail. This can also be set in AFD_CONFIG with DEFAULT_SMTP_FROM globaly.

reply-to <mail address>

This adds Reply-To: header to a mail with another mail address to which the recipient should reply.

charset <character set>

This adds a MIME header, sets the transfer encoding to 8bit and specifies the character set. It does NOT check if the character set specified is a valied one. Possible values could be:

          ISO-8859-1     --> Latin 1; Western European Languages
	  ISO-8859-2     --> Latin 2; Slavic/Central European Languages
	  ISO-8859-3     --> Latin 3; Esperanto, Galician, Maltese, Turkish
	  ISO-8859-4     --> Latin 4; Estonian, Latvian, Lithuanian
	  ISO-8859-5     --> Cyrillic
	  ISO-8859-6     --> Arabic
	  ISO-8859-7     --> Modern Greek
	  ISO-8859-8     --> Hebrew
	  ISO-8859-9     --> Latin 5; Turkish
	  ISO-8859-14    --> Latin 8; Celtic
	  ISO-8859-15    --> Latin 9; Western European Languages with Euro

encode ansi

This converts ASCII to ISO8859 when mailing the content. Note not all characters are converted, mainly only the german characters are working. Please contact developer if you need more.

file name is header

The file name is send as WMO header. This can only be used when sending files via TCP according to WMO regulations. The file name must then be of the following format: TTAAii_CCCC_YYGGgg[_BBB].

sequence numbering

Puts in a three digit sequence number into the WMO header.

check reply

According to the WMO protocol messages are send as a continuous stream without any form of handshaking. When this option is set AFD waits for an acknowledge for each message its sends. The acknowledge consists of the following 10 characters: 00000000AK for a positive acknowledge or 00000000NA for a negative acknowledge.

mode <active | passive>

There two ways to communicate with another FTP server, either active or passive. Active mode is the more common one while passive is more secure. If no mode is specified the default will be active. NOTE: It's much better to use the HOST_CONFIG to set active or passive mode.

This option is used to execute the command once after login.

site <command>

This option is used to execute the command after each file is transmitted.

file busy rename <character>

Appends the given character when we fail to open the remote file due to some brocken FTP server that will not allow to open it because a hanging child still has this file open.

dupcheck[ <timeout in secs>[ <check type>[ <action>[ <CRC type>]]]]

Performs a duplicate on all files for this job. The options have the following meaning:

Option

Description

Default

timeout

Time in seconds when this CRC value is to be discarded from the database.

3600

check type

What type of check is to be performed, the following values are possible:

1	On filename only.
2	File content.
3	Filename and file content.

action

What action is to be taken when we find a duplicate. The following values are possible:

24	Delete the file.
25	Store the duplicate file in the following directory: $AFD_WORK_DIR/files/store/<id>. Where <id> is the job id.
26	Only warn in SYSTEM_LOG.
33	Delete and warn.
34	Store and warn.

CRC type

What type of CRC check is to be performed. Currently only 16 (CRC32) is supported.

socket send buffer <buffer size>

Specifies the socket send buffer size in bytes.

socket receive buffer <buffer size>

Specifies the socket receive buffer size in bytes.

eumetsat <DestEnvId>

This option creates the header and subheader for EUMETSAT according to "MSG Ground Segment Design Specification Volume F". You might also need to execute create_eumetsat_name to create the correct file name. Syntax of create_eumetsat_name is as follows:

       create_eumetsat_name <file name>[ <rename rule>]

restart <file>

Append file on remote site. This option is for internal use only!

Option name

Description

AMG

ftp

mailto

file

wmo

sftp

scp

Example 1

An example entry in the DIR_CONFIG file might look as follows:

      [directory]
      /usr/data/tmp

           [files]
           abc*

                [destination]

                     [recipient]
                     ftp://donald:secret@hollywood//home/user
                     mailto://daisy@hollywood

                     [options]
                     archive 3
                     priority 1

The result of this entry would be that all files starting with abc in the directory /usr/data/tmp would be send to the user donald via FTP with the password secret at hollywood and there stored in the directory /home/user. They will also be send to daisy via SMTP. All files will be send with the priority 1 and will be archived for three days. The ftp files will be send with a dot first and only when a file is completely transmitted will it be renamed to its original name.

Example 2

In the following example we want different options when we send mails to daisy. For this we need to split the two recipients so that each have their own options as follows:

      [directory]
      /usr/data/tmp

           [files]
           abc*

                [destination]

                     [recipient]
                     ftp://donald:secret@hollywood//home/user

                     [options]
                     archive 3
                     priority 1

                [destination]

                     [recipient]
                     mailto://daisy@hollywood

                     [options]
                     archive 3
                     attach file
                     priority 1

Now daisy gets the same files as in example 1 only this time the files are attached.

Example 3

If daisy should get more files we need to split up the above example another time:

      [directory]
      /usr/data/tmp

           [files]
           abc*

                [destination]

                     [recipient]
                     ftp://donald:secret@hollywood//home/user

                     [options]
                     archive 3
                     priority 1

           [files]
           abc*
           def*
           ghi*
           jkl*

                [destination]

                     [recipient]
                     mailto://daisy@hollywood

                     [options]
                     archive 3
                     attach file
                     priority 1

Daisy will now also get the additional files def*, ghi* and jkl*.

Example 4

The source directory for files can also be on another host. For this we just need to change the directory entry as follows:

      [directory]
      ftp://micky:mouse@ducktown//usr/data/tmp

           [files]
           abc*

                [destination]

                     [recipient]
                     ftp://donald:secret@hollywood//home/user
                     mailto://daisy@hollywood

                     [options]
                     archive 3
                     priority 1

The result of this entry will be similar to the one in example 1, only that in this example AFD will look every minute on the host ducktown in the directory /usr/data/tmp for the files.

If we do not like AFD to delete the files on the remote we can tell it via the [dir options] to not remove the files on the remote host:

      [directory]
      ftp://micky:mouse@ducktown//usr/data/tmp

           [dir options]
	   do not remove

           [files]
           abc*

                [destination]

                     [recipient]
                     ftp://donald:secret@hollywood//home/user
                     mailto://daisy@hollywood

                     [options]
                     archive 3
                     priority 1

Now AFD will collect the same files every minute and send them to donald and daisy. If one does not wish to send the same files again one can add the following option:

      [directory]
      ftp://micky:mouse@ducktown//usr/data/tmp

           [dir options]
	   do not remove
	   store retrieve list

           [files]
           abc*

                [destination]

                     [recipient]
                     ftp://donald:secret@hollywood//home/user
                     mailto://daisy@hollywood

                     [options]
                     archive 3
                     priority 1

With the option store retrieve list only new files or those where the date and/or size has changed will be collected. If one want's to increase the poll interval from 1 minute to 30 minutes, it needs to be done asfollows:

      [directory]
      ftp://micky:mouse@ducktown//usr/data/tmp

           [dir options]
	   do not remove
	   store remote list
	   time */30 * * * *

           [files]
           abc*

                [destination]

                     [recipient]
                     ftp://donald:secret@hollywood//home/user
                     mailto://daisy@hollywood

                     [options]
                     archive 3
                     priority 1

Index

Home