The HOST_CONFIG file

The second configuration file for the AFD is used to do a configuration by host name. There can be as many entries as you want and it must not always have a corresponding host name in the DIR_CONFIG file or visa versa. This file consists of a table with 24 entries for each host. See below for a description of each entry.

  Protocol options 2      <-----------------------------------------------------+
  Warn time               <-------------------------------------------------+   |
  Keep connected          <----------------------------------------------+  |   |
  Duplicate check flag    <-------------------------------------------+  |  |   |
  Duplicate check timeout <----------------------------------------+  |  |  |   |
  Socket receive buffer   <-------------------------------------+  |  |  |  |   |
  Socket send buffer      <---------------------------------+   |  |  |  |  |   |
                                                            |   |  |  |  |  |   |
  AH:HN1:HN2:HT:PXY:AT:ME:RI:TB:SR:FSO:TT:NB:HS:PO:TRL:TTL:SSB:SRB:DT:DF:KC:WT:PO2
  |   |   |   |  |  |  |  |  |  |   |  |  |  |  |   |   |
  |   |   |   |  |  |  |  |  |  |   |  |  |  |  |   |   +-> TTL
  |   |   |   |  |  |  |  |  |  |   |  |  |  |  |   +-----> Transfer rate limit
  |   |   |   |  |  |  |  |  |  |   |  |  |  |  +---------> Protocol options
  |   |   |   |  |  |  |  |  |  |   |  |  |  +------------> Host status
  |   |   |   |  |  |  |  |  |  |   |  |  +---------------> Number of no bursts (not used)
  |   |   |   |  |  |  |  |  |  |   |  +------------------> Transfer timeout
  |   |   |   |  |  |  |  |  |  |   +---------------------> File size offset
  |   |   |   |  |  |  |  |  |  +-------------------------> Successful retries
  |   |   |   |  |  |  |  |  +----------------------------> Transfer block size
  |   |   |   |  |  |  |  +-------------------------------> Retry interval
  |   |   |   |  |  |  +----------------------------------> Max. errors
  |   |   |   |  |  +-------------------------------------> Allowed transfers
  |   |   |   |  +----------------------------------------> Proxy name
  |   |   |   +-------------------------------------------> Host toggle
  |   |   +-----------------------------------------------> Real hostname 2
  |   +---------------------------------------------------> Real hostname 1
  +-------------------------------------------------------> Alias hostname

Alias hostname

The alias host name. This is the host name that is being displayed in the afd_ctrl window and is used in the log files. If a secondary host can be specified, ignore the toggling characters, ie. when the host name in the DIR_CONFIG file is specified as 'host{12}' only state 'host' as host name. The host name may only be 8 (MAX_HOSTNAME_LENGTH) characters long. If this is to small change MAX_HOSTNAME_LENGTH to a larger value and recompile AFD. Before you startup AFD with the new values you need to run 'afd -i' to initialize internal AFD structures.

Real hostname 1

The real host name or IP number of this host. If there is a secondary host this will be the primary host, ie. if auto toggle is set it will always try to switch back to this address.

Real hostname 2

The real host name or IP number of the secondary host. If there is no secondary host just don't enter anything.

Host toggle

Host switching information. This string holds the toggling character to be displayed for the primary and secondary host. The two characters must be put in either curly brackets {} for automatic host switching or square brackets [] host switching by the user.

Proxy name

If remote server is a proxy, enter the rule on how to enter this proxy here. Currently you can only specify the order of the user name and the password. To enter the user name just enter $U<user name> and the password is entered with $P<password>. You may specify as many user and password elements as you want, each one must however be separated by a ;. If you only put a $U followed directly by a ; it will take the user name from the recipient entry. The same goes for the password. If the remote server requires the ACCT (account) directive instead of the USER to enter the login name you may also use $A instead of $U.

Allowed Transfers

The maximum number of parallel transfers for this host. On very slow connection it is best to allow just one. The default here is three.

Max. Errors

If max. errors is reached the destination identifier turns 'red' and the queue of this host will be paused.

Retry Interval

If an error occurs, this is the delay (in seconds) before another transfer is initiated.

Transfer Block Size

The size of the blocks being used to send files to the remote host (in Bytes). Maximum value is 65536 and minimum is 256. Default value is 4096 (DEFAULT_TRANSFER_BLOCKSIZE).

Successful Retries

This is the number of successful retries to the secondary host before it will try to send to the original host to see if it has become available again. If a value of less then one is specified it will NOT try to go back to the original host. It will from then on always send to the secondary host.

File Size Offset

When transmitting large files and the transfer gets interrupted, the AFD can append a file on the remote site. For this it needs to know the file size on the remote site. And to get the size it does a dir 'filename' at the remote site. Due to different replies of the FTP servers, the position of the file size is needed. You can easily determine this value simply doing an FTP to the remote site and a dir and count the spaces to the file size. For example:

     -rw-r--r--   1 afd      mts-soft   14971 Jan  3 17:16
               ^^^ ^   ^^^^^^        ^^^
                |  |     |            |
                |  |     |            |
                1  2     3            4

i.e. the file size offset is 4. Or if the remote site supports the SIZE command you can select -2. Then the remote site will tell AFD what the current size is. This seems to be also much more reliable.

Transfer Timeout

The time (in seconds) how long the AFD should wait for a reply from the remote site (default 120).

Number of no bursts

This option is no longer used.

Host status

This indicates the status of the host, currently only bits number 1, 2, 3, 5, 6, 7 and 16 can be set. The meaning is as follows:

Bit number	Integer value	What it does
1	1	If set transfer is stopped for this host.
2	2	If set queue is stopped for this host.
3	4	If set host is NOT in DIR_CONFIG.
5	16	Error status offline.
6	32	If set this host is disabled.
7	64	If set and host switching is used this tells that host two is active.
16	32768	If set do not delete files due to age-limit and/or delete queued files option. This can be useful when the target host is taken down for maintenance and you do not want to loose any data.
19	262144	If set simulation mode is enabled for this host.

The default is zero.

Protocol options

To set some protocol (currently only for FTP, FTPS, HTTP, HTTPS and SFTP) specific features for this host. The following bits can be set:

Bit number	Integer value	What it does
1	1	Sets FTP passive mode.
2	2	Set FTP idle time to transfer timeout.
3	4	Send STAT command to keep control connection alive.
4	8	Combine RNFR and RNTO to one command.
5	16	Do not do a cd, always use absolute path.
6	32	Do not send TYPE I command.
7	64	Use extended active or extended passive mode.
8	128	When set bursting is disabled. AFD by default tries to append a new job to an existing sending job, to reduce the number of connections and the overall latency (this we call a burst). It can however happen that some servers have problems with this. So with this option it is possible to disable bursting.
9	256	If set FTP passive mode allows to be redirected to another address. Best is to not set this.
10	512	When set it will replace the given scheme with file if the hostname matches local hostname or one in local_interface.list.
11	1024	Set TCP keepalive.
12	2048	Set sequence locking.
13	4096	Enable compression.
14	8192	Try and preserve the time stamp if the protocol supports this.
15	16384	Sort the files by their names before they are going to be transmitted.
16	32768	When set, the priority does not decrease when it fails to transmitt the file.
17	65536	If set this will check the size on the remote host matches the size on the local host.
18	131072	This will timeout the tansfer of a single file. If the tranmission takes more then the number of seconds specified by "Transfer Timeout" above and will count this as a failure.
19	262144	If this is set it will disable keep connected the option for fetching files.
20	524288	If this is set it will disable keep connected the option for sending files.
21	1048576	FTPS Clear Control Connection. After sending the password the communication via the control connection is done without encryption.
22	2097152	When getting remote FTP directory this, when enabled, uses the LIST command and evaluates this listing (the output of the dir command of an ftp client). This can be far less accurate with regard to time and size, depending on FTP server and configuration of it. Some represent larger size values in human readable form kilobytes, megabytes, etc. Some FTP servers represent the date of a file different when the file reaches a certain date.
23	4194304	TLS uses strict verification of host. [Still untested!!!]
24	8388608	By default AFD tries to use MLST listing to get a directory listing if the remote FTP server supports this. If this causes problems you can disable it by setting this flag.
25	16777216	If keep connected is set and this flag, then it will always disconnect after the time specified in keep connected.
26	33554432	If Strict Host Key is set SSH connections will not check the remote host key.
27	67108864	If STAT is set a STAT command is send instead of LIST. The STAT command does not require a separate connection to send the data.
28	134217728	If Implicit is set implicit FTPS is used to connect.
30	536870912	Do not use HTTP expect.
31	1073741824	Bucketname is in path.
32	2147483648	TLS legacy renegotiation.

The default is 1.

Transfer rate limit

The maximum number of kilobytes that may be transferred per second. This option does nothing for the scheme file.

TTL

The time-to-live for outgoing multicast. Currently this is not used.

Socket send buffer

How large the socket send buffer should be in bytes. If this is zero it will leave it unchanged ie. it will leave the system default.

Socket receive buffer

How large the socket receive buffer should be in bytes. If this is zero it will leave it unchanged ie. it will leave the system default.

Duplicate check timeout

Check for duplicates if the value is bigger then 0. The unit is seconds and is the time how long the CRC is to be stored.

Duplicate check flag

This flag specifies how to determine the checksum, which CRC to use and what action should be taken when we find a duplicate. The bits have the following meaning:

      1 (1)          - Only do CRC checksum for filename.
      2 (2)          - Only do CRC checksum for file content.
      3 (4)          - Checksum for filename and content.
      4 (8)          - Checksum of filename without last suffix.
      5 (16)         - Checksum of filename and size.
      16(32768)      - Do a CRC32 checksum.
      17(65536)      - Do a CRC32c checksum.
      24(8388608)    - Delete the file.
      25(16777216)   - Store the duplicate file.
      26(33554432)   - Warn in SYSTEM_LOG.
      31(1073741824) - The timeout value is fixed, not cumulative.
      32(2147483648) - Use full recipient as reference instead of alias name.

Keep connected

The number of seconds that the connection should be kept open for retrieving and/or sending files.

Warn time

When the given time in seconds have elapsed with no data being send to this host, the script/program in $AFD_WORK_DIR/etc/action/target/info/ or $AFD_WORK_DIR/etc/action/target/warn/ with the host alias as filename will be executed with the parameter 'start'. As soon as data has been send successful the script/program is called again with the parameter 'stop'.

Protocol options 2

To set some protocol specific features for this host. The following bits can be set:

Bit number	Integer value	What it does
1	1	Sends OPTS UTF8 ON to the FTP server.

The default is 0.

Index

Home