%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%deffont "normal" xfont "times"                                        
%deffont "typewriter" xfont "courier new-r"
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%default 1 area 90 90, leftfill, size 2, fore "black", back "white"
%default 2 size 7, vgap 10, prefix " "
%default 3 size 2, bar "gray70", vgap 10
%default 4 size 5, fore "black", vgap 30, prefix " "
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%tab 1 size 5, vgap 40, prefix "  ", icon box "black" 40
%tab 2 size 4, vgap 40, prefix "      ", icon arc "black" 50
%tab 3 size 3, vgap 40, prefix "            ", icon delta3 "black" 40
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
%nodefault
%center, size 7, fore "black", back "white", vgap 20


%image "../images/afd_final.png"

Automatic File Distributor (AFD)

%center, size 4
A tool to distribute your data


Holger Kiehl <Holger.Kiehl@dwd.de>
%image "../images/dwd_logo.png"
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Agenda

%size 6

          Introduction

%size 4
          Architecture of AFD
          Compiling
          Setting up AFD
          Graphical User Interface
          Configuration by examples
          Evaluating Log Data
          Performance and other considerations
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Introduction


	Overview of AFD

	Usage at DWD

	Other usage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Overview of AFD

	Tool to collect and distribute files.
%newimage -scrzoom 90 "../images/AFD_data_distribution.jpg"
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Overview of AFD (continued)


	Do something with the file before or after distribution.

		External programs like bzip2, convert, tar, etc
		Use Internal routines as renaming, assemble, extract, etc

	Supports many protocols (uses its own implementation)

		FTP + FTPS
		SFTP + SCP
		HTTP + HTTPS
		SMTP
		WMO
		EXEC (execute an external program, eg. lpr)
		LOCAL (copy/move files locally)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Overview of AFD (continued)


	Everything done by AFD is protocolled to some log files.

	Enable different levels of tracing the network connections

		view each individual command AFD sends and receives
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Overview of AFD (continued)


	Everything can be managed and monitored via command line

	and/or graphical user interface

%center, newimage "../images/afd-half-size.png"
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Overview of AFD (continued)


	You can give files an age limit

	Time based collection + distribution (same syntax as cron)

	Configure actions (eg. send a mail if you cannot reach a certain customer)

	If you have many AFD's, there is a tool AFD Monitor, that helps you to monitor and control them
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Overview of AFD (continued)


	All the code in AFD is written in C

		All variables and comments are in English

	Extensive documentation in HTML

	AFD is Open Source (GPL v2)

	http://www.dwd.de/AFD
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Exercise


Look at http://www.dwd.de/AFD and see where one can download and read information about configuration files, etc.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Usage at DWD


	Installed on approximately 280 systems at DWD

	System size range from embedded to high end systems

	AFD distributes 17 million files with 16 TiB data daily

	DWD has one big central AFD server called AFDZ

	AFDZ distributes 8.6 million files with 6.2 TiB data daily.

	Most of this is managed by three persons
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Usage at DWD (continued)

%center, newimage "../images/afdz.jpg"
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Usage at DWD (continued)

%center
Network Load
%newimage "../images/afdz-net-usage.png"
Disk Load
%newimage "../images/afdz-disk-usage.png"
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Usage at DWD (continued)


	AFDZ Hardware + Software

%font "typewriter", size 4
   Server           : HP ProLiant DL380 G7
   CPU              : 2 x Intel Xeon X5690 @ 3.47 GHz
                      (total of 12 cores)
   RAM              : 96 GiB
   System Disk      : 2 x 200 GiB Software Raid 1 SSD's
   Data Disk        : 14 x 146 GiB Software Raid 1+0
                      1 x 1.2 TB Fusion IO ioDrive Duo
   OS               : Linux Kernel 3.5.4 from kernel.org
   Distribution     : Scientific Linux 6.3
   Cluster-software : Heartbeat
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Other usage

	Other meteorological centers such as:
		Brazil
		Bulgaria
		China
		Ireland
		Italy
		United Kingdom
		Russia
		South Korea
		South Africa
		Suriname
		etc.

	DLR Germany's national research center for aeronautics + space.
	Bundeswehr Germany's armed forces.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Other usage (continued)


	Brazilian Navy

	Private weather company's and newspapers.