Scheduled FTP Exports Fail to Transfer Data

Symptoms

Scheduled FTP exports fail to transfer data.

Issue

FTP server configuration, as well as management plane load during the scheduled export job can cause issues with a FTP export job.

Resolution

To isolate the problem to the FTP server configuration, a FTP export can be run manually via the CLI with debug output enabled.  The debug output will show all of the communication between the FTP server and the firewall and highlight any errors encountered.

To enable CLI debug log output issue the command:

debug cli on

To disable CLI debug log output issue the command:

debug cli off

To run a FTP export from the CLI use the following command format:

ftp export log <type> to username:password@host start-time equal YYYY/MM/DD@HH:MM:SS end-time equal YYYY/MM/DD@HH:MM:SS

<type> can be: traffic, threat, data, url

The scheduled FTP job when run will export the last calendar day of the logs specified in the Scheduled Log Export (Device tab).  The start-time and end-time values should define a 24 hour period during the manual FTP export to match what the scheduled job generates.

Example Output:

admin@PA> debug cli on

admin@PA> ftp export log traffic to user:pass@192.168.1.3 start-time equal 2012/07/24@00:00:00 end-time equal 2012/07/24@23:59:00

(container-tag: export container-tag: log container-tag: traffic leaf-tag: to value: user:pass@192.168.1.3 container-tag: start-time leaf-tag: equal value: 2012/07/24@00:00:00 container-tag: end-time leaf-tag: equal value: 2012/07/24@23:59:00)

((eol-matched: . #t) (cli-handler: . ftp-handler) (context-inserted-at-end-p: . #f))

/usr/local/bin/pan_log_export_ftp --interactive --type=traffic --user=user --passwd=pass --host=192.168.1.3 --verbose --start='2012/07/24@00:00:00' --end='2012/07/24@23:59:00'

/usr/local/bin/pan_logquery -bn -b  -t traffic -t1 2012\/07\/24\@00\:00\:00 -t2 2012\/07\/24\@23\:59\:00  2> /dev/null

*get* '220-FileZilla Server version 0.9.41 beta\r\n'

*get* '220-written by Tim Kosse (Tim.Kosse@gmx.de)\r\n'

*get* '220 Please visit http://sourceforge.net/projects/filezilla/\r\n'

*resp* '220-FileZilla Server version 0.9.41 beta\n220-written by Tim Kosse (Tim.Kosse@gmx.de)\n220 Please visit http://sourceforge.net/projects/filezilla/'

*cmd* 'USER user'

*put* 'USER user\r\n'

*get* '331 Password required for user\r\n'

*resp* '331 Password required for user'

*cmd* 'PASS ****'

*put* 'PASS ****\r\n'

*get* '230 Logged on\r\n'

*resp* '230 Logged on'

*cmd* 'TYPE A'

*put* 'TYPE A\r\n'

*get* '200 Type set to A\r\n'

*resp* '200 Type set to A'

*cmd* 'PORT 192,168,1,1,215,114'

*put* 'PORT 192,168,1,1,215,114\r\n'

*get* '200 Port command successful\r\n'

*resp* '200 Port command successful'

*cmd* 'STOR PA_traffic_2012_07_24_00_00_00_to_2012_07_24_23_59_00_0.csv'

*put* 'STOR PA_traffic_2012_07_24_00_00_00_to_2012_07_24_23_59_00_0.csv\r\n'

*get* '150 Connection accepted\r\n'

*resp* '150 Connection accepted'

*get* '226 Transfer OK\r\n'

*resp* '226 Transfer OK'

*cmd* 'QUIT'

*put* 'QUIT\r\n'

*get* '221 Goodbye\r\n'

*resp* '221 Goodbye'

/usr/local/bin/pan_logquery -bn -b -m -t traffic -t1 2012\/07\/24\@00\:00\:00 -t2 2012\/07\/24\@02\:59\:00  > /dev/null 2>&1

If the manual run completes successfully then further troubleshooting of the scheduled job may be required with the assistance of support.  Modifying the scheduled export to run at 30 or 45 minutes past the hour may allow the job to complete successfully.  Many reporting and log indexing jobs are run at the top of the hour and the load of all the processing at that time can affect the FTP export process.

owner: kfindlen