AFD Tools


AFD has many small programs that can be used to view, control or configure it via command line. These files are located in the directory $AFD_WORK_DIR/sbin. Here follows a description of some of the tools:

AFD commands (afdcmd)

With this program you can send most commands that you can also issue with the afd_ctrl and dir_ctrl dialog on a per host or directory basis. When you call afdcmd without any parameters it will show all commands it knows:

   SYNTAX  : afdcmd[ -w working directory][ -p <role>][ -u[ <user>]] options hostname|directory|position
       FSA options:
                  -q      start queue
                  -Q      stop queue
                  -t      start transfer
                  -T      stop transfer
                  -e      enable host
                  -E      disable host
                  -s      switch host
                  -r      retry
                  -d      enable/disable debug
                  -c      enable/disable trace
                  -C      enable/disable full trace
                  -X      toggle enable/disable host
       FRA options:
                  -b      enable directory
                  -B      disable directory
                  -R      rescan directory
                  -W      toggle enable/disable directory
   General options:
                  -f      start FD
                  -F      stop FD
                  -a      start AMG
                  -A      stop AMG
                  -Y      start/stop AMG
                  -Z      start/stop FD
                  -k      force file dir check
                  -i      reread local interface file
                  -o      show exec statistics
                  -p      force archive check
                  -v      just print Version

In conjunction with cron you can enable/disable certain hosts and control when they should receive their data. You may specify more than one hostname or position.

AFD configuration (afdcfg)

The program afdcfg allows you to enable/disable archiving and/or retrieving of files from remote hosts. The disabling of archiving can be usfull when you notice that your disk space gets to low or you want to reduce the load on the node. The other option to enable/ disable retrieving of files is good when you have two nodes with identical configured AFD's and one node is configured as hot standby. This will make sure that only the AFD on the active node will retrieve data. Otherwise both AFD's will retrieve the data and that can be quit annoying. The syntax of afdcfg is as follows:

   SYNTAX  : afdcfg [-w working directory] options
                                           -a      enable archive
                                           -A      disable archive
                                           -c      enable create target dir
                                           -C      disable create target dir
                                           -d      enable dir warn time
                                           -D      disable dir warn time
                                           -r      enable retrieving of files
                                           -R      disable retrieving of files
                                           -s      status of the above flags

View FSA (fsa_view)

To view the content of the FSA (Filetransfer Status Area) there is a program called fsa_view. An example output might look as follows:

afd@r2d2:~$ fsa_view fiau_
    Number of hosts: 34   FSA ID: 1  Struct Version: 2  Pagesize: 4096

=============================> fiau_ <=============================
Host alias CRC       : e831dc61
Real hostname 1      : 127.0.0.1
Real hostname 2      :
Hostname (display)   : >fiau_<
Host toggle          : HOST_ONE
Auto toggle          : OFF
Original toggle      : NONE
Toggle position      : 5
Protocol(   40000001): FTP active fast_move fast_cd
Direction            : SEND
Socket send buffer   : Not set
Socket rcv buffer    : Not set
Keep connected       : Not set
Proxy name           : >$Udwd@ifu;$Pgtswetter<
Debug mode           : OFF
Dupcheck timeout     : Disabled
Host status (   0)   : NORMAL_STATUS
Transfer timeout     : 120
File size offset     : -2
Successful retries   : 0
MaxSuccessful ret.   : 10
Special flag ( 64)   : HOST_IN_DIR_CONFIG
Error counter        : 0
Total errors         : 7
Max. errors          : 10
Error history        : 000 -> Transfer success
                       000 -> Transfer success
                       000 -> Transfer success
                       000 -> Transfer success
                       000 -> Transfer success
Retry interval       : 600
Transfer block size  : 4096
TTL                  : 0
Time of last retry   : Sat Apr  8 23:36:41 2006
Last connection      : Sat Apr  8 23:36:41 2006
First error time     : Not set.
Total file counter   : 0
Total file size      : 0
File counter done    : 104
Bytes send           : 3100838
Connections          : 26
MC NACK counter      : 0
Jobs queued          : 0
Active transfers     : 0
Allowed transfers    : 2
Rate limit           : 0
Rate limit per proc  : 0
MC Rate limit        : 0
MC Rate limit/proc   : 0
                    |   Job 0   |   Job 1   |   Job 2   |   Job 3   |   Job 4
--------------------+-----------+-----------+-----------+-----------+-----------
PID                 |        -1 |        -1 |        -1 |        -1 |        -1
Connect status      |DISCONNECT |DISCONNECT |DISCONNECT |DISCONNECT |DISCONNECT
Number of files     |         0 |         0 |         0 |         0 |         0
No. of files done   |         0 |         0 |         0 |         0 |         0
File size           |         0 |         0 |         0 |         0 |         0
File size done      |         0 |         0 |         0 |         0 |         0
Bytes send          |   3100838 |         0 |         0 |         0 |         0
File name in use    |           |           |           |           |
File size in use    |         0 |         0 |         0 |         0 |         0
Filesize in use done|         0 |         0 |         0 |         0 |         0
Unique name         |           |           |           |           |
Job ID              |         0 |         0 |         0 |         0 |         0

Under 'File name in use' will be shown the current file being transmitted. To view the full name use the '-l' option to show the long view:

-------- Job  0 -----+------------------------------------------------------
PID                  : -1
Connect status       : DISCONNECT
Number of files      : 0
No. of files done    : 0
File size            : 0
File size done       : 0
Bytes send           : 3100838
File name in use     : 
File size in use     : 0
File size in use done: 0
Unique name          : 
Job ID               : 0
-------- Job  1 -----+------------------------------------------------------
PID                  : -1
Connect status       : DISCONNECT
Number of files      : 0
No. of files done    : 0
File size            : 0
File size done       : 0
Bytes send           : 0
File name in use     : 
File size in use     : 0
File size in use done: 0
Unique name          : 
Job ID               : 0

View FRA (fra_view)

To view the content of the FRA (Fileretrieve Status Area), the directories monitored by AFD, there is a program called fra_view. The output looks as follows:

afd@r2d2:~$ fra_view 3
     Number of directories: 42   FRA ID: 32  Struct Version: 4

============================> testmaus (4) <============================
Directory alias      : testmaus
Directory ID         : 99ca01ad
URL                  : ftp://funk:bla@testmaus
Host alias           : testmaus
Wait for             : 
FSA position         : 25
Priority             : 9
Number of process    : 0
Max number of process: 10
Bytes received       : 0
Files received       : 0
Files in directory   : 0
Bytes in directory   : 0
Files in queue(s)    : 0
Bytes in queue(s)    : 0
Accumulate size      : 0
Accumulate           : 0
gt_lt_sign           : 0
Max errors           : 10
Error counter        : 0
Warn time            : 0
Keep connected       : 0
Ignore size          : 0
Ignore file time     : 0
Max files            : 100
Max size             : 104857600
Directory status( 12): NORMAL_STATUS
Directory flag(  8)  : LINK_NO_EXEC
In DIR_CONFIG flag   : DIR_ALIAS
Dupcheck timeout     : Disabled
Force reread         : NO
Queued               : NO
Remove files         : NO
Stupid mode          : NO
Protocol (   0)      : FTP
Delete input files   : NO
Report unknown files : YES
Important directory  : NO
End character        : NONE
Time option          : YES
Next check time      : Sun Jun 19 11:32:00 2005
Minute (uchar[8])    : 01000000 00001000 00100001 10000100 00010000 01000010 00001000 00100001
Continues (uchar[8]) : 01000000 00001000 00100001 10000100 00010000 01000010 00001000 00100001
Hour (uint)          : 00000000 11111111 11111111 11111111
Day of month (uint)  : 01111111 11111111 11111111 11111111
Month (short)        : 00001111 11111111
Day of week (uchar)  : 01111111
Minute (uchar[8])    : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Continues (uchar[8]) : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Hour (uint)          : 00000000 00000000 00000000 00000000
Day of month (uint)  : 00000000 00000000 00000000 00000000
Month (short)        : 00000000 00000000
Day of week (uchar)  : 00000000
Last retrieval       : Sun Jun 19 10:39:10 2005

AFD Status (afd_status)

The program afd_status prints the current status of the AFD that is diaplayed at the bottom row of the afd_ctrl dialog.

AMG                  : 1
AMG jobs             : 1
FD                   : 1
System log           : 1
Event log            : 1
Transfer log         : 1
Trans debug log      : 1
Archive watch        : 1
afd_stat             : 1
afdd                 : 1
input_log            : 1
output_log           : 1
delete_log           : 1
production_log       : 1
Receivelog indicator : 155 < I I W I I >
Receive log history  : ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? I E ? ? ? ? ? I I ? I I I I W W W W I
Syslog indicator     : 787 < I I I I I >
System log history   : ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? W W C I W C C W I W W W I I W W I W W
Translog indicator   : 2317 < I I I I I >
Transfer log history : ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? I I I I I I I I I I I I I I E E I E I
Number of transfers  : 0
Number of retrieves  : 0
Jobs in queue        : 0
AMG fork() counter   : 6
FD fork() counter    : 1213
Burst2 counter       : 4
Max. FD queue length : 17
Directories scanned  : 2418
AFD start time       : Thu Jan 19 20:29:37 2006

The process status of each can have the following numbers:

-3 The process has stopped normally.
0 Process is not running.
1 Process is up and running.
19 Process is currently shutting down.

Edit FSA (fsa_edit)

The program fsa_edit can be used to edit the FSA (Filetransfer Status Area). When you use this program it will always show the current value that you might wish to manipulate as shown below:





                     FSA Editor (fiau)

        +-----+------------------+----------------+
        | Key | Description      | current value  |
        +-----+------------------+----------------+
        |  1  |total_file_counter|              0 |
        |  2  |total_file_size   |              0 |
        |  3  |error counter     |              1 |
        |  4  |No. of connections|             26 |
        |  5  |host status       |              0 |
        |  6  |Max. errors       |             10 |
        |  7  |Block size        |           1024 |
        |  8  |Allowed transfers |              2 |
        |  9  |Transfer timeout  |            120 |
        |  a  |Real hostname     |      127.0.0.1 |
        |  b  |Host display name |           fiau |
        |  c  |Error offline stat|            YES |
        |  d  |Active transfers  |              0 |
        |  e  |File name         |                |
        |  f  |Jobs queued       |              0 |
        |  g  |Transferrate limit|              0 |
        +-----+------------------+----------------+


If you wish to change for example the 'error counter' just press '3' and you will be asked to enter a value:





                     FSA Editor (fiau)

        +-----+------------------+----------------+
        | Key | Description      | current value  |
        +-----+------------------+----------------+
        |  1  |total_file_counter|              0 |
        |  2  |total_file_size   |              0 |
        |  3  |error counter     |              1 |
        |  4  |No. of connections|             26 |
        |  5  |host status       |              0 |
        |  6  |Max. errors       |             10 |
        |  7  |Block size        |           1024 |
        |  8  |Allowed transfers |              2 |
        |  9  |Transfer timeout  |            120 |
        |  a  |Real hostname     |      127.0.0.1 |
        |  b  |Host display name |           fiau |
        |  c  |Error offline stat|            YES |
        |  d  |Active transfers  |              0 |
        |  e  |File name         |                |
        |  f  |Jobs queued       |              0 |
        |  g  |Transferrate limit|              0 |
        +-----+------------------+----------------+


     Enter value [3] :  _


Now enter the new value and send the new value to the FSA with the return key. If you wish to terminate the program just press the q key.

Please use this program with caution! Only use it when you know what you are doing.

Update DIR_CONFIG (udc)

When there is a change in the DIR_CONFIG and you do not want to use the afd_ctrl dialog to activate this DIR_CONFIG just use udc.

Update HOST_CONFIG (uhc)

When there is a change in the HOST_CONFIG and you do not want to use the afd_ctrl dialog to activate this HOST_CONFIG just use uhc.

Both udc and uhc return 0 when the update was performed succesfully. On error either 3 or -1 is returned. If it returns 2 the update was done but there are warnings, meaning that you have to check the SYSTEM_LOG for possible errors in DIR_CONFIG or HOST_CONFIG.

Get DIR_CONFIG data (get_dc_data)

The program get_dc_data shows what AFD has stored internally for the given host alias. The output looks as follows when a host alias is given as the first argument:

    Directory     : /home/afd/data/mfa
    Dir-alias     : MRZ
    DIR-options   : time */5 * * * *
    Filter        : G1*
                    G2*
                    G3*
                    G4*
    Recipient     : ftp://user:passwd@host//data/mfa
    Real hostname : 127.0.0.1
    AMG-options   : extract MRZ
    FD-options    : archive 1
    Priority      : 6
    Job-ID        : 1c7eafb9

If no argument is given, the whole DIR_CONFIG as stored within AFD will be printed to stdout. So with this command you can reconstruct the DIR_CONFIG if you lost it or it is damaged and AFD is still active.

Show exec statistics (show_exec_stat)

If called will cause dir_check to print out some exec statistics about CPU usage in SYSTEM_LOG. This is very usfull to find the jobs that cause most of the load. Below is an example output:

   10 14:40:06 <D> CPU clock times for exec option:
   10 14:40:06 <D> Job ID     Forks      User       System     Total
   10 14:40:06 <D> 4b5313cf   3          78         14         92

This statistic will always be printed when dir_check terminates.

Remove a job (rm_job)

To remove a job from the AFD system there is the program rm_job <full job name>. To get the full job name just look into the files directory. To delete for example all jobs with the job ID 999 the following command will do this for you: rm_job `find $AFD_WORK_DIR/files -name "*_999" -print` Currently rm_job can only delete outgoing jobs, not those that are fetching files from a remote host.

Set password (set_pw)

If you do not want to store passwords in the DIR_CONFIG you can set them with this program. The syntax is as follows:

   Usage: set_pw [-w <AFD work dir>] [--version] [-s] -i <job id>|-c <user@hostname>

AFD monitor commands (mafdcmd)

With this program you can send many commands that you can also issue with the mon_ctrl dialog on a per AFD basis. When you call mafdcmd without any parameters it will show all commands it knows:

   SYNTAX  : ./mafdcmd [-w working directory] options AFD|position
                                              -e          enable AFD
                                              -E          disable AFD
                                              -r          retry
                                              -s          switch AFD
                                              -X          toggle enable/disable AFD
                                              -u[ ] fake user
                                              -v          just print Version

You may specify more than one AFD or position.

Automatic FTP (aftp)

aftp is a tool small tool that sends or retrieve file(s) via FTP without any user interaction, so it can be used from any shell scripts or programs such as xsend_file. The syntax of aftp is as follows:

   SYNTAX: [t|r]aftp [options] [file 1 ... file n]

      When calling it with raftp files will be retrieved from the
      given host, otherwise (when using aftp) files will be send to that host.

     OPTIONS                              DESCRIPTION
     --version                          - Show current version
     -a <file size offset>              - Offset of file name when doing a LIST
                                          command on the remote side. If you
                                          specify -2 it will try to determine
                                          the size with the SIZE command.
     -b <block size>                    - FTP block size in bytes. Default 4096
                                          Bytes.
     -c <config file>                   - Configuration file holding user name,
                                          password und target directory in URL
                                          format.
     -C                                 - If target directory does not exist create
                                          it.
     -d <remote directory>              - Directory where file(s) are to be stored.
     -f <filename>                      - File containing a list of filenames
                                          that are to be send.
     -h <host name | IP number>         - Hostname or IP number to which to
                                          send the file(s).
     -k                                 - Keep FTP control connection with STAT
                                          calls alive/fresh.
     -l <DOT | DOT_VMS | OFF | xyz.>    - How to lock the file on the remote site.
     -m <A | I | D>                     - FTP transfer mode, ASCII, binary or DOS.
                                          Default is binary.
     -p <port number>                   - Remote port number of FTP-server.
     -P <proxy procedure>               - Use the given proxy procedure to
                                          login. See documentation for more
                                          details on syntax.
     -u <user> <password>               - Remote user name and password. If not
                                          supplied, it will login as anonymous.
     -r                                 - Remove transmitted file.
     -S <buffer size>                   - Socket send buffer size
                                          (in bytes).
     -t <timout>                        - FTP timeout in seconds. Default 120s.
     -v                                 - Verbose. Shows all FTP commands and
                                          the reply from the remote server.
     -x                                 - Use passive mode instead of active
                                          mode when doing the data connection.
     -X                                 - Use extended mode active or passive
                                          (-x) mode.
     -z                                 - Use SSL/TLS for control connection.
     -Z                                 - Use SSL/TLS for control and data
                                          connection.
     -?                                 - Display this help and exit.
     The following values are returned on exit:
          0 - File transmitted successfully.
          1 - Failed to connect.
          9 - SSL/TLS authentification error.
          2 - User name wrong.
          3 - Wrong password.
          4 - Failed to set ascii/binary mode.
          5 - Failed to send NLST command.
         10 - Failed to open remote file.
         11 - Error when writting into remote file.
         12 - Failed to close remote file.
         13 - Failed to rename remote file.
         14 - Remote directory could not be set.
         20 - Operation received timeout.
         28 - Connection reset by peer.
         29 - Connection refused.
         30 - Could not open source file.
         31 - Failed to read source file.
         17 - System error stat().
         34 - System error malloc().
         40 - Failed to read file name file.
         60 - Syntax wrong.

Automatic SMTP (asmtp)

asmtp is a tool small tool that sends file(s) via SMTP without any user interaction, so it can be used from any shell scripts or programs such as xsend_file. The syntax of asmtp is as follows:

   SYNTAX: asmtp [options] file(s)

     OPTIONS                      DESCRIPTION
     --version                  - Show current version
     -a <user@host>             - The address where the mail is sent to.
     -b <block size>            - Transfer block size in bytes. Default 4096
                                  Bytes.
     -c <config file>           - Configuration file holding user name,
                                  domain and SMTP server in URL format.
     -e                         - Encode files in BASE64.
     -f <filename>              - File containing a list of filenames
                                  that are to be send.
     -h <hostname | IP number>  - Recipient hostname of this mail.
     -m <mailserver-address>    - Mailserver that will send this mail.
                                  Default is localhost.
     -n                         - File name is subject.
     -o <mail address>          - Where the receiver should send is reply.
     -p <port number>           - Remote port number of SMTP-server.
                                  Default 25.
     -r                         - Remove transmitted file.
     -s <subject>               - Subject of this mail.
     -t <timout>                - SMTP timeout in seconds. Default 120s.
     -u <user>                  - The user who should get the mail.
     -v                         - Verbose. Shows all SMTP commands and
                                  the reply from the SMTP server.
     -y                         - File name is user.
     -?                         - Display this help and exit.
     The following values are returned on exit:
          0 - File transmitted successfully.
          1 - Failed to connect.
          2 - User name wrong.
          4 - Failed to set ascii/binary mode.
          5 - Failed to send NLST command.
         10 - Failed to open remote file.
         11 - Error when writting into remote file.
         12 - Failed to close remote file.
         13 - Failed to rename remote file.
         14 - Remote directory could not be set.
         20 - Operation received timeout.
         28 - Connection reset by peer.
         29 - Connection refused.
         30 - Could not open source file.
         31 - Failed to read source file.
         17 - System error stat().
         34 - System error malloc().
         40 - Failed to read file name file.
         60 - Syntax wrong.

Copyright © 2001 - 2007 by H.Kiehl
Holger.Kiehl@dwd.de
Last updated: 12.11.2007
[red dot]Index [red dot]Home SourceForge.net Logo