Receiving "Internal error" on commit

[Brinkman]

Running 4.1.6 on 5050's with a Active/Passive HA.

I activated the PAN-DB filtering on my Active firewall and then rebooted it, it failed over to the passive firewall just like it should.  However, when my firewall came back up it came up as having HA not enabled, the autocommit is failing, and if I try to force a commit I get this error:

· Error: Internal Error(Module: device)

I tried to revert back to a known good config and get the same thing, even if I try a force through the command line.  Here is what I get in the devsrv.log:

Jul 25 08:05:10 Config commit phase0 starts

Jul 25 08:05:13 Config commit phase1 started

Jul 25 08:05:13 Last committed config available: no

Jul 25 08:05:13 TDB compilation started

Jul 25 08:05:13 Content Engine version: 0x4010000 APP version: 0x17e0747, Threat 0x17e0747

Jul 25 08:05:15 End of translating global

Jul 25 08:05:17 End of parsing custom threat

[TDB] Loading tdb cache  with virus loaded

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 0 aho partition 79

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 10 aho partition 65

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 12 aho partition 66

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 19 aho partition 80

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 25 aho partition 73

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 27 aho partition 67

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 29 aho partition 74

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 30 aho partition 75

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 46 aho partition 68

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 51 aho partition 81

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 59 aho partition 69

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 61 aho partition 70

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 63 aho partition 76

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 64 aho partition 71

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 67 aho partition 77

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 68 aho partition 78

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 70 aho partition 64

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 72 aho partition 72

Jul 25 08:05:17 Warning: pan_tdb_unserialize(pan_tdb_ser.c:1077): [regex group above 63 appid 76 aho partition 82

Jul 25 08:05:17 [Cache] Load /opt/pancfg/mgmt/content//cache/40100//tdb.cache.ser-0 success

Jul 25 08:05:17 [TDB] compile virus cache

Jul 25 08:05:19 TDB compilation done

Jul 25 08:05:20 sqlite3 /opt/pancfg/mgmt/content/global/threats.xml.db query SELECT id from tdb WHERE id >= 10000 AND id < 30000  AND (  severity = 5  )

Jul 25 08:05:20 sqlite3 /opt/pancfg/mgmt/content/global/threats.xml.db query SELECT id from tdb WHERE id >= 10000 AND id < 30000  AND (  severity = 4  )

Jul 25 08:05:20 sqlite3 /opt/pancfg/mgmt/content/global/threats.xml.db query SELECT id from tdb WHERE id >= 10000 AND id < 30000  AND (  severity = 3  )

Jul 25 08:05:20 sqlite3 /opt/pancfg/mgmt/content/global/threats.xml.db query SELECT id from tdb WHERE id >= 10000 AND id < 30000  AND (  severity = 2  )

Jul 25 08:05:20 sqlite3 /opt/pancfg/mgmt/content/global/threats.xml.db query SELECT id from tdb WHERE id >= 30000 AND id < 45000  AND (  severity = 5 OR  severity = 4 OR  severity = 3  )

Jul 25 08:05:20 Error: pan_global_from_obj(pan_config_parser.c:13281): pan_url_category_from_obj failed

Jul 25 08:05:20 Error: pan_config_from_obj(pan_config_parser.c:15141): pan_global_from_obj failed

Jul 25 08:05:20 Error: pan_ctrl_save_config(pan_config_handler_sysd.c:1484): Error compiling config

Jul 25 08:05:20 Error: pan_ctrl_compile_cfg(pan_config_handler_sysd.c:1640): pan_ctrl_save_config() failed

Jul 25 08:05:20 Error: pan_config_handler_sysd(pan_config_handler_sysd.c:1714): pan_ctrl_compile_cfg() failed

Jul 25 08:05:20 Error: pan_ctrl_proc_handle_config(pan_controller_proc.c:1793): pan_config_handler_sysd() failed

Jul 25 08:05:20 Config commit phase1 failed

Jul 25 08:05:20 Error: cfgagent_modify_callback(pan_cfgagent.c:79): Modify string (sw.mgmt.runtime.clients.device.err) error: Unknown error code (1)

Jul 25 08:06:12 Warning: pan_cloud_agent_cat_req_handler(pan_cloud_agent_connect.c:2175): Cloud is not ready, There was no update from the cloud in the last 1130 seconds.

Jul 25 08:09:25 path : https://s0000.urlcloud.paloaltonetworks.com/urlcloud_list, path

Jul 25 08:10:49 path : https://s0000.urlcloud.paloaltonetworks.com/urlcloud_list, path

Any suggestion or someone else seen this before.

Thanks

Kris

[kprakash · Accepted solution]

It looks like we are running into a limitation of a memory buffer leak when parsing the custom signatures ( vulnerability, data filtering, spyware ). Do you have custom signatures that we can disable/delete and see if it makes a difference?

Moreover PAN-DB is not supported on 4.1.6 and is supported on 5.0 and above. Can you revert it back to Brightcloud and see if that was the problem.

BR,

Karthik

[Brinkman]

That was it, once I changed it back to brightcloud it fixed everything.  Is there a way to remove the PAN-DB and Brightcloud because I've determined we don't actually need it.

Thanks

Kris

[kprakash]

You can delete the license key for both PAN-DB and Brightcloud.

Issue the below command, followed by the "tab" key to look at what keys are available, and then delete the PAN_DB_URL_FIltering and Brightcloud

>delete license key

Also remove any URL profiles that are part of the policies, after deleting the keys.

BR,

Karthik