AMG (Automatic Message Generator)

To send files with the AFD, it must know from which directory and what file(s) it has to transmit. This information and that about the remote site is found in the file DIR_CONFIG. There may be more then one DIR_CONFIG, this can be done by using multiple DIR_CONFIG_NAME entries in the AFD_CONFIG file. So all the AMG does is that when files show up it generates a message for the fd. The message is written in ASCII and stored in the directory 'messages'. Below is an example of what might be the contents of a message:

   [recipient]
   ftp://donald:secret@hollywood//home/user

   [options]
   archive 3
   lock DOT

The password will only be visible when it is configured with --enable-passwd_in_msg.

These messages are generated only ones in the message directory with the job ID as there name. At start the FD reads all these messages into a cache. When the AMG has created a new job it sends the job ID over the fifo to the FD which now knows which message it needs to apply.

Dir Check

This process constantly looks into the directories specified in the DIR_CONFIG to see if any files arrive. It scans all directories in that order that they have been specified in the DIR_CONFIG every 5 seconds (DEFAULT_RESCAN_TIME). If the directory is not in the same file system as the AFD it forks to copy the files to the file directory of the AFD. This is done, to reduce the time when copying files, especially when the files are mounted via NFS. When the option 'extract', 'exec', or 'gts2tiff' is set, the inst_job will fork, since the conversion time can be very long (at least on some systems). If there are lots of files in a directory it will only take the first 100 (MAX_COPIED_FILES). So it does not take too long to create a job. Distribution of files should start as early as possible.

If files are found, a directory of the following format is created in the AFD file directory:

     files/outgoing/<JID>/<counter>/<creation time>_<unique number>_<SJC>

               JID - Job ID
               SJC - Split Job Counter

The field is incremented when there are more then 1000 directories, ie. jobs queued.

All files that are found are hard linked into this directory. This procedure is continued for each recipient that wants the same files. Every time a new directory has been created and all files are linked into this directory, a message is send to the message fifo MSG_FIFO. The message in the fifo has the following format:

     <creation time><JID><SJC><dir no><unique number><Priority><Originator>
            |         |    |      |         |           |       |
            |         |    |      |         |           | +-----+
            |         |    |      |         |           | |
            |         |    |      |         |           | |
            |         |    |      |         |           | |
            |         |    |      |         |           | +-> char
            |         |    |      |         |           +---> char
            |         |    |      |         +---------------> unsigned short
            |         |    |      +-------------------------> unsigned short
            |         |    +--------------------------------> unsigned int
            |         +-------------------------------------> unsigned int
            +-----------------------------------------------> time_t

               JID - Job ID
               SJC - Split Job Counter

These messages are read by the FD which will distribute the files immediately if it still has a process slot free, or will store them into an internal queue.

Index

Home