There are eight different types of log files used by the AFD: system, receive, transfer, transfer debug, input, production, output and delete log. All these files are written in ASCII notation, so they can be evaluated with standard Unix tools.
This is the most important log file for the AFD system administrator. Here all errors encountered by the AFD are logged. It also gives the following information:
Here more debug infomation is shown that can be useful for developing. This is only available if you compiled AFD with the configure option --enable-maintainer_log. This is a high volume log.
All events that occur are held in this log file. A more detailed description of what events and the format is shown in this text file.
![[image of transfer_log dialog]](../images/transfer_log_small.png)
The transfer log tells how many files and bytes are being send to a particular host and the time when they where send. The general form of such an entry is as follows:
26 12:17:11 <I> ducktown[0]: 1568 Bytes send in 3 file(s). | | | | | | | | | | | | | +----> General message saying how | | | | | | many files have been send. | | | | | +----------------> Sum of the size send. | | | | +----------------------> The job number. | | | +-----------------------------> Remote Host Name | | +----------------------------------> Log level. | +-----------------------------------------> hh:mm:ss +-----------------------------------------------> Day of the month.
Currently there are nine different log levels: Fatal <F>, Error <E>, Warning <W>, Information <I>, Configuration <C>, Offline <O>, Debug <D>, Trace <T> and "other information" <#>.
More important are the warnings/errors that get logged here that may have occurred during the distribution. Below is an example of an error that occurred while distributing files via FTP:
26 07:51:09 <E> ducktown[0]: Failed to open remote file .PAXX82_EDZW_J0570749 (-1). #124 (sf_ftp.c 783) 26 07:51:09 <E> ducktown[0]: 150-Opening BINARY mode data connection for '.PAXX82_EDZW_J0570749'.
The second line is the reply that was received from the remote FTP server. This makes the finding of errors very easy, since the user immediately sees what type of error it was and what the reason was for this error. The number behind the # sign is the message number that was used by the AFD for this transmission.
![[image of transfer debug dialog]](../images/transfer_debug_log_small.png)
If the information in the transfer log is not enough, to evaluate an error the user has the possibility to enable debugging and/or tracing. Here all activities of the sending/receiving process are being logged. The form of the output is the same as above. Below is an example of a debug log output:
13 07:29:51 <I> ducktown[0]: Connected. (sf_ftp.c 204) 13 07:29:51 <I> ducktown[0]: 220-hollywood FTP proxy (Version V1.3) ready. 13 07:29:52 <I> ducktown[0]: Entered user name daisy@hollywood. (sf_ftp.c 253) 13 07:29:52 <I> ducktown[0]: 331-Password required for daisy. 13 07:29:53 <I> ducktown[0]: Logged in as daisy@hollywood. (sf_ftp.c 302) 13 07:29:53 <I> ducktown[0]: 230-User daisy logged in. 13 07:29:53 <I> ducktown[0]: Changed transfer mode to I. (sf_ftp.c 351) 13 07:29:53 <I> ducktown[0]: 200-Type set to I. 13 07:29:53 <E> ducktown[0]: Failed to change directory to /duck_tales (550). (sf_ftp.c 367) 13 07:29:53 <I> ducktown[0]: 550-/duck_tales: Not a directory.
The lines containing the numbers 220, 331, 230, 200 and 550 are always the reply from the remote FTP server. If you want to know the exact meaning of these numbers see FTP reply codes or for a more detailed description see RFC 959. In the above example we see that we have failed to change the remote directory to /duck_tales and that the remote server says that it is not a directory, but a file or link.
![[image of transfer debug dialog]](../images/receive_log_small.png)
The receive log shows the number of files received and if exec option is set what is done by the exec option in the directories monitored by AFD. The output is very similar to the transfer log. In this log all errors will be shown that the AMG encounters while it processes these files (eg. the exec option).
04 12:37:15 <I> inetkonrad: Received 6 files with 769995 Bytes. 04 12:37:15 <I> wwweamdar : Received 4 files with 1958858 Bytes. 04 12:37:15 <I> kon_vxc : Received 1 files with 1485 Bytes. 04 12:37:40 <I> swismeldwm: Received 1 files with 6328 Bytes. 04 12:37:45 <I> f80422d7 : [14328] tr -s "[\015\015]" < gts01-SMDL92_EDZW_041200-0507041200-afsv--72- > gts01-SMDL92_EDZW_041200-0507041200-afsv--72-.tmp; rm gts01-SMDL92_EDZW_041200-0507041200-afsv--72- 04 12:37:45 <I> f80422d7 : [14328] Exec time: 0.010s 04 12:37:50 <I> rad-stat : Received 2 files with 37888 Bytes. 04 12:37:50 <I> 3a310ad5 : Received 1 files with 1024 Bytes. 04 12:37:50 <I> 3a310ad5 : [25349] bzip raa00-dv_10440-0507041239-fld---bin 04 12:37:50 <I> 3a310ad5 : [25349] Exec time: 0.010s 04 12:37:55 <I> jm-swis : Received 17 files with 72033 Bytes. 04 12:37:55 <I> jm-swistxt: Received 64 files with 37007 Bytes.
The System, Receive, Transfer and Transfer Debug Log can all be viewed with the X program show_log.
![[image of show_ilog dialog]](../images/show_ilog_small.png)
Here all file names of files that are picked up by the AFD get logged in the INPUT_LOG.x file. Data for this file is collected for one day (SWITCH_FILE_TIME). This file is then kept for 7 (MAX_INPUT_LOG_FILES) days. Every time a new log file is created, the x for all old log files is incremented by one. Thus the newest log file is always INPUT_LOG.0.
An entry might look as follows:
4167b8f8 RADAR_PASW42_LSSW_J0570001|23fe|b72afd25|10f | | | | | | | | | +-> Unique number in hex | | | +--------> Directory Identifier | | +---------------> File Size in hex | +-------------------------------> File Name +---------------------------------------------------> Unix Time in hex
The time is not entered in a human readable form, so that we do not need to convert back to the UNIX time when we show the time in the Input Log Window. Unique number, Directory Identifier, File Size and Unix Time are hexa decimal values.
Here all file names of files that are picked up by the AFD and are to be distributed get logged in the DISTRIBUTION_LOG.x file. This log is rotated the same way as the INPUT_LOG.x above.
An entry might look as follows:
40bae084 0-1|dat.txt|40bae083|e8c13458|ba2366|20b50|556a367_0,4e327a5_3
| | | | | | | |
| | | | | | | +-> Jobs that
| | | | | | | received the
| | | | | | | file. The last
| | | | | | | number says how
| | | | | | | many times it
| | | | | | | appears in
| | | | | | | PRODUCTION_LOG.
| | | | | | +------> File size.
| | | | | +-------------> Unique number.
| | | | +----------------------> Directory
| | | | Identifier.
| | | +------------------------------> Pickup time.
| | +---------------------------------------> File name.
| +-------------------------------------------> First digit
| Distribution
| type:
| 0 - Normal
| 1 - Time job
| 2 - Queue
| stopped
| 3 - Disabled
| 4 - Age limit
| delete
| 5 - Dupcheck
| Second digit
| is number of
| distribution
| types used.
+----------------------------------------------------> Date when
logged.
If something is being done via one of the options it will be logged to PRODUCTION_LOG.x. This log is rotated the same way as the INPUT_LOG.x above.
A log entry of this type looks as follows:
4167b8f3 1:1|0.010.0.104a|4167b8f3_10f_0|682d6409|d1fe5d48|fiau_9605060847-814|26a0|fiau_9605060847-814.bz2|14ac1|0|bzip2 %s
| | | | | | | | | | | |
| | | | | | | | | | | +-> Command executed.
| | | | | | | | | | +----> Return code of command.
| | | | | | | | | +--------> Size of changed file.
| | | | | | | | +--------------------------> New file name. If this
| | | | | | | | is empty, then the file
| | | | | | | | has been removed.
| | | | | | | +-------------------------------------> Original file size.
| | | | | | +-------------------------------------------------> Original file name.
| | | | | +-----------------------------------------------------------------> Job Identifier.
| | | | +--------------------------------------------------------------------------> Directory ID.
| | | +-------------------------------------------------------------------------------------> Unique Identifier. The
| | | first number is the date
| | | when this file was picked
| | | up by AFD, the second is
| | | a unique number and third
| | | is the split job counter.
| | +---------------------------------------------------------------------------------------------------> Production time (decimal
| | value with a maximum of 3
| | digits behind the decimal
| | point) and CPU time (all
| | hexa values, seconds and
| | then microseconds behind
| | the last decimal point).
| +------------------------------------------------------------------------------------------------------------> Ratio relationship.
+--------------------------------------------------------------------------------------------------------------------> Unix time for this
entry.
There is currently no log file viewer for this log type.
![[image of show_olog dialog]](../images/show_olog_small.png)
All file names that have been distributed successfully are shown in this file. The log files are stored as described for the Input Log Files except that they are called OUTPUT_LOG.x. In the DIR_CONFIG the system administrator can specify with the option no log output that these files are not to be logged. This is useful when sending lots of small unimportant files, it helps to reduce the CPU and disk usage.
It has a different form then the input log file as shown below:
4167b8f9 ducktown 0 0 1|PASW42_LSSW_J0570001|/data/PASW42|23fe|0.27|0|9e78fc94|4167b8f9_38_0[|archive directory] | | | | | | | | | | | | | | | | | | | | | | | | | Directory where file | | | | | | | | | | | | is being archived. | | | | | | | | | | | +---------> Unique ID. | | | | | | | | | | +--------------------> Job Identifier | | | | | | | | | +------------------------> Number of retries. | | | | | | | | +----------------------------> Transmission time in | | | | | | | | seconds. | | | | | | | +---------------------------------> File Size | | | | | | +------------------------------------------> When trans_rename was | | | | | | specified this is the | | | | | | remote name. NOTE: | | | | | | This is optional. | | | | | +----------------------------------------------------------> File Name. | | | | +----------------------------------------------------------------------> Transfer Type | | | | 0 - FTP | | | | 1 - LOC | | | | 2 - SMTP | | | | 3 - MAP | | | | 4 - SCP | | | | 5 - WMO | | | | 6 - HTTP | | | | 7 - FTPS | | | | 8 - HTTPS | | | | 9 - SMTPS | | | | a - SFTP | | | | b - EXEC | | | | c - DFAX | | | | d - DEMAIL | | | +------------------------------------------------------------------------> Alternate Host Number. | | +--------------------------------------------------------------------------> Output Typ | | 0 - Normal (delivered) | | 1 - Age limit (del) | | 2 - Duplicate (stored) | | 3 - Duplicate (del) | | 4 - Other process (del) | | 5 - Address rejected (del) | | 6 - Host disabled (del) | | 7 - Duplicate | | 8 - Unknown | | 9 - Normal (received) | | : - Confirmation dispatch | | ; - Confirmation receipt | | < - Confirmation retrieve | | = - Confirmation time up | +-----------------------------------------------------------------------------> Remote Host Name. +-----------------------------------------------------------------------------------------> Unix Time.
To view these log files, use show_olog.
![[image of show_dlog dialog]](../images/show_dlog_small.png)
All file names that have been deleted are shown in this file. The log files are stored as described for the Input Log Files except that they are called DELETE_LOG.x. There are lots of ways how files can be deleted by the AFD. The most common way is when the administrator specifies this in the DIR_CONFIG file, by either setting the aging option for the directory entry or by setting the age-limit option in the option part. Other possibilities are that the administrator removes them from the system, or the AFD deletes them because the message for this job ID is no longer available.
An entry in the DELETE_LOG.x file is described below:
4167b8fa ducktown 001|PASW42_LSSW_J0570001|5eb7|9e78fc94|3ab56ea2|4167c70a_a230_0|sf_ftp|>13445 | | | | | | | | | | | | | | | | | | | +-> Optional additional | | | | | | | | | reason. | | | | | | | | +--------> User/process that | | | | | | | | deleted the file. | | | | | | | +-------------------> Unique ID consisting | | | | | | | of creation time, | | | | | | | a unique number and the | | | | | | | split job counter. | | | | | | +--------------------------------> Directory ID. | | | | | +-----------------------------------------> Job ID. | | | | +-----------------------------------------------> File Size. | | | +------------------------------------------------------------> File Name deleted. | | +-------------------------------------------------------------------------> Deletion Type. | | 0 - AGE_OUTPUT | | 1 - AGE_INPUT | | 2 - USER_DEL | | 3 - EXEC_FAILED_DEL | | 4 - NO_MESSAGE_FILE_DEL | | 5 - DUP_INPUT | | 6 - DUP_OUTPUT | | 7 - DEL_UNKNOWN_FILE | | 8 - JID_LOOKUP_FAILURE_DEL | | 9 - DEL_OLD_LOCKED_FILE | | a - DEL_QUEUED_FILE | | b - DELETE_OPTION | | c - DELETE_STALE_ERROR_JOBS | | d - CLEAR_STALE_MESSAGES | | e - FILE_CURRENTLY_TRANSMITTED | | f - DELETE_UNKNOWN_POOL_DIR | | 10 - EXEC_FAILED_STORED | | 11 - DELETE_HOST_DISABLED | | 12 - CONVERSION_FAILED | | 13 - RENAME_OVERWRITE | | 14 - RECIPIENT_REJECTED | | 15 - MIRROR_REMOVE | +-------------------------------------------------------------------------------> Destination Host Name. +-------------------------------------------------------------------------------------------> Unix Time when deleted.
To view these log files, use show_dlog.
|
Copyright © 1997 - 2022 by H.Kiehl Holger.Kiehl@dwd.de Last updated: 21.03.2022 |
![[red dot]](../images/reddot.gif) Index |
Home |