PA-VM Update Check Fails

[xandout]

We have recently deployed PA-VM to ESXi for testing and we have found that any attempt to upgrade the unit fails with a very vague message.

cfg.platform.serial': NO_MATCHES
'cfg.general.vm-mode-type': NO_MATCHES
2016-03-10 09:14:42.447 -0800 updater error code:-1
2016-03-10 09:14:48.140 -0800 Error:  refresh_uploaded_image_info(pan_ops_common.c:8516): Bad update information on disk2016-03-10 09:14:48.140 -0800 Error:  refresh_uploaded_image_info(pan_ops_common.c:8519): Error removing /opt/pancfg/mgmt/global/upgradeinfo.xml
2016-03-10 09:14:48.412 -0800 No update information available
2016-03-10 09:14:48.412 -0800 Error:  get_sw_version_info(pan_ops_common.c:7675): Error extracting sw version info from file upgradeinfo.xml
2016-03-10 09:14:48.412 -0800 No upload information available

admin@PA-VM> request system software check

Server error : Failed to check upgrade info due to generic communication error. Please check network connectivity and try again.
admin@PA-VM> 

I have set the update server in Device > Setup > Services to 199.167.52.141 and updates.paloaltonetworks.com.

I put in proxy information to assist in the debug but no requests are ever made.  

My assumption is that the appliance never touches the network because of some file issues.

Does anyone have any ideas on how I can go about fixing this?

[bmorris1]

Hey,

Does look like a connectivity problem.

You could try changing the service routes of the firewall so that it uses a dataplane interface rather than the management?

Device > Setup >  Services > Service Features > Service Route Configuration.

Change DNS & Updates to a dataplane interface. If you prefer to use the management then make sure your device can make DNS requests ok in order to resolve the updates.paloaltonetworks.com server and make sure that if traffic is routed through the device, the device is not blocking itself.

hope that helps,

Ben

[xandout]

I have verified that the device can resolve updates.paloaltonetworks.com. a ping host gives me the IP,

I will setup a data plane interface and see if that helps.

Should the appliance be able to use the mgmt interface for updates?

[xandout]

I went as far as doing a fresh install

admin@PA-VM> ping host updates.paloaltonetworks.com
PING updates.paloaltonetworks.com (199.167.52.141) 56(84) bytes of data.
^C
--- updates.paloaltonetworks.com ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1007ms

admin@PA-VM> request system software check

Server error : Failed to check upgrade info due to generic communication error. Please check network connectivity and try again.
admin@PA-VM> tail 
+ follow          output appended data as the file grows
+ lines           output the last N lines, instead of the last 10
> agent-log       agent-log 
> mp-log          mp-log 
> webserver-log   webserver-log 

admin@PA-VM> tail mp-log m
masterd.log         masterd_apps.log    masterd_detail.log  mgmt_fb.log         mp-monitor.log      ms.log              mprelay.log         
admin@PA-VM> tail mp-log ms.log 
ln: creating symbolic link `3a7f6b22.0' to `/opt/pancfg/certificates/predefined/Thawte Personal Basic CA.cer': File exists
ln: creating symbolic link `64d1f6f4.0' to `/opt/pancfg/certificates/predefined/Thawte Personal Freemail CA.cer': File exists
ln: creating symbolic link `09ca81a7.0' to `/opt/pancfg/certificates/predefined/Thawte Personal Premium CA d.cer': File exists
ln: creating symbolic link `98ec67f0.0' to `/opt/pancfg/certificates/predefined/Thawte_Premium_Server_CA.cer': File exists
ln: creating symbolic link `6cc3c4c3.0' to `/opt/pancfg/certificates/predefined/Thawte_Server_CA.cer': File exists
ln: creating symbolic link `415660c1.0' to `/opt/pancfg/certificates/predefined/Verisign_Class_3_Public_Primary_Certification_Authority.cer': File exists
2016-03-10 11:12:24.819 -0800 updater error code:-1
'cfg.platform.serial': NO_MATCHES
'cfg.general.vm-mode-type': NO_MATCHES
2016-03-10 11:12:49.998 -0800 updater error code:-1
admin@PA-VM> 
admin@PA-VM> 
admin@PA-VM> 
admin@PA-VM> 
admin@PA-VM> 
admin@PA-VM> request system software 
> check      Get information from PaloAlto Networks server
> download   Download software packages
> info       Show information about available software packages
> install    Install a downloaded software package

admin@PA-VM> request system software 
> check      Get information from PaloAlto Networks server
> download   Download software packages
> info       Show information about available software packages
> install    Install a downloaded software package

admin@PA-VM> request system software in
> info      Show information about available software packages
> install   Install a downloaded software package

admin@PA-VM> request system software info 

Server error : No update information available
admin@PA-VM> 
admin@PA-VM> 
admin@PA-VM> 
admin@PA-VM> 
admin@PA-VM> tail mp-log ms.log
ln: creating symbolic link `6cc3c4c3.0' to `/opt/pancfg/certificates/predefined/Thawte_Server_CA.cer': File exists
ln: creating symbolic link `415660c1.0' to `/opt/pancfg/certificates/predefined/Verisign_Class_3_Public_Primary_Certification_Authority.cer': File exists
2016-03-10 11:12:24.819 -0800 updater error code:-1
'cfg.platform.serial': NO_MATCHES
'cfg.general.vm-mode-type': NO_MATCHES
2016-03-10 11:12:49.998 -0800 updater error code:-1
2016-03-10 11:13:25.284 -0800 Error:  refresh_uploaded_image_info(pan_ops_common.c:8516): Bad update information on disk2016-03-10 11:13:25.284 -0800 Error:  refresh_uploaded_image_info(pan_ops_common.c:8519): Error removing /opt/pancfg/mgmt/global/upgradeinfo.xml
2016-03-10 11:13:25.528 -0800 No update information available
2016-03-10 11:13:25.528 -0800 Error:  get_sw_version_info(pan_ops_common.c:7675): Error extracting sw version info from file upgradeinfo.xml
2016-03-10 11:13:25.528 -0800 No upload information available

[Lucky]

Hi,

what is your OS version you are running? If you have 7.0.0 or some beta release, download 7.0.1 image, install that one and try to upgrade from it.

If not, you can do pcaps on management interface to verify what is going on with traffic because by default it does use management interface to communicate to the cloud; commands to do that would be:

tcpdump snaplen 0 filter "host 199.167.52.141"

view-pcap verbose++ yes mgmt-pcap mgmt.pcap

change 199.167.52.141 to whatever you resolve updates.paloaltonetworks.com

you can also export pcap by tftp export mgmt-pcap... or scp export mgmt-pcap

Check if you are attempting to decrypt that traffic along the way somewhere as well - that would break updates too.

Let us know if none of above helps.

Best regards,

Luciano

[tsrivastav]

Could you be able to check the Time and date on ther firewall are accurate or not

Also kindly open the cli run this command and  do a check now paste the output here 

admin@admin> tail follow yes mp-log devsrv.log    

and do you see any message in the system logs regarding to the url filtering 

[tsrivastav]

DNS resultions are working fine, that means changing service route  may not address the isse however if the traffic is passing through the firewall Mgmt port>>>firewalls data port>>>cloud  make fure you have allow rules for Managment ip address more or you can check global counters also if the traffic is passing through the firewalls data port

[pulukas]

In these situations I generally download the PanOS file to my workstation and do the upload and upgrade from there instead of from the cloud.  This will generally get around the issue of communications errors.

[schopra]

Hey 

Can you verify the content version i.e Application and Threats version.

Ideally you should have a version higher than 550.

If you are runningn on a verison less than that, then please upgrade the version to any value higher than 550.

Disable the Verify server identity and also check.

If these things do not work out, then the pcap on the management interface is the best.

[abjain]

Could you verify the licenses are proper and installed and updated in the support portal?

Also please enable debug mode on management server and collect the logs:

> debug management-server on debug

> tail follow yes mp-log ms.log

Now do a Check Now from GUI or "request content upgrade check" from another CLI to see what are the logs showing.

At the end set the management-server debug to info level:

> debug management-server on info

 If licenses are properly installed, and logs do not show enough information, kindly open a support case

[rbista]

I have resolved this kind of issues by clicking once on the "retrieve licenses link, then do check now.