%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %deffont "typewriter" xfont "courier new-r" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %default 1 area 90 90, leftfill, size 2, fore "black", back "white" %default 2 size 7, vgap 10, prefix " " %default 3 size 2, bar "gray70", vgap 10 %default 4 size 5, fore "black", vgap 30, prefix " " %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %tab 1 size 5, vgap 40, prefix " ", icon box "black" 40 %tab 2 size 4, vgap 40, prefix " ", icon arc "black" 50 %tab 3 size 3, vgap 40, prefix " " %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Agenda %size 4 Introduction %size 6 Architecture of AFD %size 4 Compiling Setting up AFD Graphical User Interface Configuration by examples Evaluating Log Data Performance and other considerations %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Architecture of AFD Init Process init_afd Description of all Process Internal Directory Structure What happens when a file goes through AFD Overview of command line tools Monitoring several AFD's (afd_mon) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Init Process init_afd AFD works without any user interface and runs as a daemon process in the background. The parent of all AFD process is the process init_afd. This process ensures that all other process are started in the correct order and that the infrastructure is setup correctly for AFD to run. Once AFD is up running it monitors all process it started. If a process fails for whatever reason it notes this in a log file and will try to restart this process again. IMPORTANT: Do not try to start directly on the command line. There is a commandline tool 'afd' that does this for you. Also do not try to stop AFD by killing init_afd, especially not with the option -9! %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Description of all Process init_afd starts all process in the following order: system_log log process responsible for writing all system information to the file $AFD_WORK_DIR/log/SYSTEM_LOG.0. The log data is read via a FIFO ($AFD_WORK_DIR/fifodir/system_log.fifo). event_log, receive_log, transfer_log, trans_db_log archive_watch Process that ensures that files that have been put in the archive are deleted when their time is up. input_log Process logging each file name that enters AFD. output_log, delete_log, production_log, distribution_log %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Description of all Process (continued) amg Automatic Message Generator (AMG), this process evaluates the configuration files $AFD_WORK_DIR/etc/DIR_CONFIG and $AFD_WORK_DIR/etc/HOST_CONFIG. After successful evaluation it will start process dir_check and will monitor its status. dir_check Process that monitors all local directories for any arriving files. fd File Distributor (FD), responsible for the transmission of files. To send files it starts process starting with sf_xxx where xxx is the protocol name. For fetching it starts gf_xxx. afdd To monitor the AFD this daemon is started. Is not started automatically, you must configure it that it starts. aldad Log process that generates logs in a user specified format. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Description of all Process (continued) Process relationship %font "typewriter", size 2.5 init_afd-+-afd_stat |-afdd---3*[afdd] |-aldad |-amg---dir_check-+-dir_check---sh---sh---encbufr | `-3*[dir_check---sh---sh---bzip2] |-archive_watch |-delete_log |-distribution_lo |-event_log |-fd-+-5*[gf_ftp] | |-5*[gf_http] | |-211*[sf_ftp] | |-17*[sf_loc] | |-sf_scp---ssh | |-7*[sf_sftp---ssh] | `-2*[sf_wmo] |-input_log |-output_log |-production_log |-receive_log |-system_log |-trans_db_log `-transfer_log %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Internal Directory Structure %size 3 When AFD is started it automatically creates the following directory structure in $AFD_WORK_DIR: %font "typewriter", size 1.5 $AFD_WORK_DIR/ | +----> archive/ | | | +--> alias host name 1 | . | alias host name n +----> etc/ | | | +-> action/ | | | +--> source | +--> target +----> fifodir +----> files/ | | | +-> crc | +-> error | +-> incoming/ | | | | | +---> user@host/directory 1 | | . | | user@host/directory n | +-> outgoing/ | | | | | +---> job ID 1 (1a3d01fa) | | . | | job ID n | +-> pool/ | | | | | +-> unique name 1 (505712ae_80f1_0_f017c351) | | . | | unique name n | +-> store | +-> time/ | | | +-> job ID 1 (1e2b6029) | . | job ID n +----> log +----> messages %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page What happens when a file goes through AFD %center, newimage "../images/amg_fd_en_1.jpg" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page What happens when a file goes through AFD %center, newimage "../images/amg_fd_en_2.jpg" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page What happens when a file goes through AFD %center, newimage "../images/amg_fd_en_3.jpg" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page What happens when a file goes through AFD %center, newimage "../images/amg_fd_en_4.jpg" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page What happens when a file goes through AFD %center, newimage "../images/amg_fd_en_5.jpg" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page What happens when a file goes through AFD %center, newimage "../images/amg_fd_en_6.jpg" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Overview of command line tools afd start/stop/initialize AFD udc reread configuration file DIR_CONFIG uhc reread configuration file HOST_CONFIG afdcmd tool to send commands to the AFD afdcfg tool to configure AFD %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Overview of command line tools (continued) afd_status show status of all AFD process and some additional information such as the number of jobs queued, number of transfer jobs active, different log status, etc. fsa_view view status of the transmitting part of AFD fra_view view status of the input directories aftp send/retrieve files via FTP asmtp send files via SMTP (mail) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Monitoring several AFD's (afd_mon) In an environment where you have several AFD's it is advisable to use afd_mon. This is a program that collects all status information of all AFD's so that they can be made visible via the dialog mon_ctrl. If wanted it is also able to collect all the log files of the remote AFD's. afd_mon gets the information via TCP-socket from the process afdd mentioned above. To setup afd_mon you must create a configuration file $AFD_WORK_DIR/etc/AFD_MON_CONFIG which looks as follows: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Monitoring several AFD's (afd_mon) %font "typewriter", size 3 \# +--------------------------------------------------> Alias \# | +-------------------------------------> Hostname \# | | +-----------------------------> Port \# | | | +--------------------------> Interval \# | | | | +-----------------------> Connect time \# | | | | | +--------------------> Disconnect time \# | | | | | | +---------------> Options \# | | | | | | | +-----------> Remote cmd \# | | | | | | | | +------> Convert user name \# | | | | | | | | | Central-AFD ducktown 4444 5 30 600 1024 ssh donald->daisy %pause %center, newimage "../images/mon_ctrl.png"