Unable to pull policies from a firewall managed by Panorama

[PAN-Expert]

I am running latest version 1.0.91. I have a user at both Panorama and the firewall with the API key that is associated with Expedition. Panorama holds all of the config, but I am unable to pull it from either the firewall or Panorama. The devices are added fine, they pull all of the information about licenses and other information, but it does not pull any policies objects or network information. I am not sure if this is limitation by the tool where it is not able to handle Panorama, but I cannot import any information from them. The firewall import imports 3 default services and that is it and Panorama import says following message 

There is no configuration downloaded for the Device [Panorama]

[alestevez]

If this happens, please STOP the Task Manager from the Main Screen after Login and Start it Again. Then try to Retrieve the configuration again and try to import the device into the project again. Let us know how it goes

[PAN-Expert]

I tried this, but it did not help. 

[PAN-Expert]

I re-deployed another VM with version 1.0.84 and this one does not have the issue. 

Two points for this:

1. The latest version should be fixed

2. We should have a way of reverting to the older version 

[alestevez]

since are ubuntu packages im pretty sure you can uninstall a version and request another one, they are in the repository so can you can ask to install like . 

sudo apt-get install expedition-beta=1.0.84

[PAN-Expert]

That does help to install older versions.

The other issue that I experienced after adding Panorama to 1.0.84 and a firewall to the project is that I was unable to machine learn from a specific rule even though I have added the exported logs to the machine learning path.

What else could I try to ge this working?

[dgildelaig]

Besides adding the csv files from a Firewall, we need to preprocess the files.

Afterwards, within the Project, we need to create a log connector that specifies which sources (devices-vsys) do we want to consider for the ML purposes.

You must have imported the config from the device in order to correctly create the Log Connector. Do not fill in the fields manually to specify the Device and the VSys (or DGs), but use the options within the Menus. If the menius show empty, then the device configuration may have not been correctly imported.

[PAN-Expert]

I was able to do that part with a standalone firewall, but it does not work with Panorama owned firewall which also includes that firewall.

I exported the csv log from both Panorama and the firewall, but it does not process it, just hangs in "Pending" state for hours. It does not even show progress, so I assume it is not running as CPU and Memory untilization are not high.

Log connector will show up with Panorama, but it will not with firewall, but that is fine as I have export of logs from both. However they do not want to be processed.

Therefore my best guess is that this was fixed in version after 1.0.84, but the upgrade command that was mentioned earlier does not work. It only works if I specify latest version. However I need to upgrade one version at the time at which Policies will still import from Panorama (which is broken in newest versions), but will fix the current issue. 

Below is the log from my attempt 

root@Expedition:/# sudo apt -o "Acquire::AllowInsecureRepositories=true" -o "Acquire::AllowDowngradeToInsecureRepositories=true" update
Hit:1 http://us.archive.ubuntu.com/ubuntu xenial InRelease
Hit:2 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:3 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease
Hit:4 http://security.ubuntu.com/ubuntu xenial-security InRelease
Hit:5 http://ppa.launchpad.net/adiscon/v8-stable/ubuntu xenial InRelease
Hit:6 http://sgp1.mirrors.digitalocean.com/mariadb/repo/10.1/ubuntu xenial InRelease
Ign:7 https://conversionupdates.paloaltonetworks.com expedition-updates/ InRelease
Ign:8 https://conversionupdates.paloaltonetworks.com expedition-updates/ Release
Hit:9 http://www.rabbitmq.com/debian testing InRelease
Ign:10 https://conversionupdates.paloaltonetworks.com expedition-updates/ Packages.diff/Index
Ign:11 https://conversionupdates.paloaltonetworks.com expedition-updates/ Translation-en_US
Ign:12 https://conversionupdates.paloaltonetworks.com expedition-updates/ Translation-en
Ign:13 https://conversionupdates.paloaltonetworks.com expedition-updates/ Packages
Ign:11 https://conversionupdates.paloaltonetworks.com expedition-updates/ Translation-en_US
Ign:12 https://conversionupdates.paloaltonetworks.com expedition-updates/ Translation-en
Ign:13 https://conversionupdates.paloaltonetworks.com expedition-updates/ Packages
Ign:11 https://conversionupdates.paloaltonetworks.com expedition-updates/ Translation-en_US
Ign:12 https://conversionupdates.paloaltonetworks.com expedition-updates/ Translation-en
Ign:13 https://conversionupdates.paloaltonetworks.com expedition-updates/ Packages
Ign:11 https://conversionupdates.paloaltonetworks.com expedition-updates/ Translation-en_US
Ign:12 https://conversionupdates.paloaltonetworks.com expedition-updates/ Translation-en
Hit:13 https://conversionupdates.paloaltonetworks.com expedition-updates/ Packages
Ign:11 https://conversionupdates.paloaltonetworks.com expedition-updates/ Translation-en_US
Ign:12 https://conversionupdates.paloaltonetworks.com expedition-updates/ Translation-en
Ign:11 https://conversionupdates.paloaltonetworks.com expedition-updates/ Translation-en_US
Ign:12 https://conversionupdates.paloaltonetworks.com expedition-updates/ Translation-en
Reading package lists... Done
Building dependency tree
Reading state information... Done
34 packages can be upgraded. Run 'apt list --upgradable' to see them.
W: The repository 'https://conversionupdates.paloaltonetworks.com expedition-updates/ Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
root@Expedition:/# apt-get install expedition-beta=1.0.85
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Version '1.0.85' for 'expedition-beta' was not found
root@Expedition:/# apt-get install expedition-beta=1.0.90
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Version '1.0.90' for 'expedition-beta' was not found
root@Expedition:/# apt-get install expedition-beta=1.0.93
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  linux-headers-4.4.0-124 linux-headers-4.4.0-124-generic linux-headers-4.4.0-31 linux-headers-4.4.0-31-generic linux-image-4.4.0-124-generic linux-image-4.4.0-31-generic linux-image-extra-4.4.0-124-generic
  linux-image-extra-4.4.0-31-generic
Use 'sudo apt autoremove' to remove them.
The following packages will be upgraded:
  expedition-beta
1 upgraded, 0 newly installed, 0 to remove and 33 not upgraded.
Need to get 40.1 MB of archives.
After this operation, 0 B of additional disk space will be used.
WARNING: The following packages cannot be authenticated!
  expedition-beta
Install these packages without verification? [y/N] N
E: Some packages could not be authenticated
root@Expedition:/#

[alestevez]

By now you have to say YES to this question

Install these packages without verification? [y/N] y

Our server has invalid certificate still, as soon we change to the new package named expedition instead of expedition-beta we will add a valid certificate then. Sorry for the confusion.

[PAN-Expert]

That was not the point of my comment, check the entire log.

I attempted to do this:

apt-get install expedition-beta=1.0.85

I also attempted some other versions. However the expedition-beta=version# does not work, it only works if I put in the latest version. I need other interim versions as well, but that does not work, only latest one. 

[alestevez]

Probably is related we are not encrypting the Release file who contains the list of available version. We will request to have it when we move the packate expedition-beta to expedition. Thanks