In this case, "Internet Security" is my Rulebase to export for migration. Based on what we have seen from the Checkpoint API, in case we have more than 400 rules, we have to use an offload to generate exportations in chunks of 400 rules, which means if we have 650 security rules, we will have to do this:
mgmt_cli show access-rulebase offset 0 limit 400 name "Internet Security" details-level "full" use-object-dictionary true --format json > RuleSet_0_400.json mgmt_cli show access-rulebase offset 401 limit 650 name "Internet Security" details-level "full" use-object-dictionary true --format json > RuleSet_401_650.json
After we created all the json files, we will need to create a new file, and we will name it "order" to put inside the filenames in the order they need to be read, like in this example:
Edit "order" file
RuleSet_0_400.json RuleSet_401_650.json
After that, we have to ZIP it. To do it right, all these three files need to be under the same folder, and from within the folder, we can run the command from the CLI:
zip Rules.zip *
This will create a new Rules.zip only contaning the three files without any other folder inside. It's important to remember when you create the ZIP file to avoid having any folder inside the ZIP file—just the json and "order" files.
If the Security rules are less than 400, we have to just export to a single json file. There is no need to ZIP it then.
Exporting Nat Rules
For Nat rules, we have found the limitation is for 500 Nat Rules per export, so we can apply the samething we did with the Security Rules. For example, we are going to sat we have 600 Nat Rules.
Notice the name of the Ruleset, in this case, is Internet without Security at the end. You have to check from the SmartCenter to see the Nat ruleset Name.
After we created all the json files, we will need to create a new file, and we will name it "order" to put inside the filenames in the order they need to be read, like in this example:
Edit "order" file
NATRuleSet_0_500.json NATRuleSet_501_600.json
After that, we have to ZIP it. To do it right, all these three files need to be under the same folder, and from within the folder, we can run the command from the CLI:
zip NatRules.zip *
If the Nat rules are less than 500, we have to just export to a single json file. There is no need to ZIP it then.
Exporting Routing and interfaces
From the Firewall CLI, you can run the following:
netstat -nr > routes.txt
Export the routes.txt to where you store the NatRules.zip and Rules.zip
With all this information, we can go to Expedition, Create a new Project, enter the Project, and go to IMPORT > CHECKPOINT > VERSION R80.
Assign a name to your configuration such as "MyInternetGW"
I ran the command in expert mode and the file generated correctly, however I have tried to import into the Expedition tool and each time it states There are not Objects defined. Import Failed!
Completed all the config associated with that link How to Upload conf files with no luck. I have attempted to complete this instructions in this thread EXACTLY as perscribed with the most progress being the Import progress status flapping between Initializing and Reading Config.... of which it continues this back and forth for quite sometime.
Next I attempted this option outlined in this thread
RESOLVED: The issue was with the .json files and piping the output to a seperate file. The commands above are correct, however the output files were adding the following "Username" at the TOP of the file:
This appears to be an issue/bug within the R80.10 code when doing outputs to a file as this "Username" is coming from the prompt before launching the api call.
Once this "Username:" line was removed from the json file, the import worked as expected.
I have confirmed there is a limitation on the Checkpoints of 500 rules when exporting through the API. With our environment, I had to break it up into groups of 200 due to time out and RAM issues.
I have yet to get the zip file with the orders.txt file to work as perscribed above. I am on the latest version of the expedition tool and when I upload it, it will transistion from initilizing to reading config files, but stalls from there.
Anyone else ran into issues like this or did something change with the new version?
Just want to ask you one question that suppose security and NAT policies count around 1500 rules then what is the best way to export policy in one file. Otherwise. I have to export the security policy in 400 rules count like below and zip it one folder.
mgmt_cli show access-rulebase offset 0 limit 400 name "Internet Security" details-level "full" use-object-dictionary true --format json > RuleSet_0_400.json
mgmt_cli show access-rulebase offset 401 limit 800 name "Internet Security" details-level "full" use-object-dictionary true --format json > RuleSet_401_800.json
mgmt_cli show access-rulebase offset 801 limit 1200 name "Internet Security" details-level "full" use-object-dictionary true --format json > RuleSet_801_1200.json
mgmt_cli show access-rulebase offset 1201 limit 1500 name "Internet Security" details-level "full" use-object-dictionary true --format json > RuleSet_1201_1500.json
It is my understanding that the limitations are 500 for any of the rule sets according to Checkpoint. NAT and Security are treated as 2 SEPERATE groups. For example if you have 1000 security rules and 200 NATs; you will need 2 seperate security rules files of 500 each and 1 NAT rule file.
This being said I have yet to get the zip file proceedure to work and actually had to export out my security policies in seperate files as perscribed, but then manually merge the files into 1 .json file using notepad++.
Won't lie, it would be great if the zip proceedure worked
There are a few misconceptions of what each command does in your procedure, first of all "limit" is the number of results after the specified offset, for instance you set offset to 0 and limit to 400 you get rules 1 to 400.
Now the next set of rules you should consider offset 400 instead of 401, if 0=1 then 400=401 in the offset counting.
Also it's a pain to always have to request the customer for username and password and the command you wrote "hides" username request, my suggestion to update and works like a charm, do has you like:
Step1
#mgmt_cli login (when asked the customer should type his username and password here)
Step2
Copy output of the result login to a file, for the sake of my example I used id.txt
This id lets you reuse the logged in session id, with an idle timeout of 10 minutes, more then enough to run all queries, even for quite big rulebases like in my test scenario 2569 rules (don't ask...)
Step 3 run your instructions with slit differences...limit as I explained should be set to 400, you are right 400 is good number, although it allows up to 500, in rulebase it normally crashes the api or it times out, so:
Also it's good idea to set limit and the default value is 50 so you don't want to think you have it all, but the end file will tell you a tip.
NOTE: See "total" in the end of file? 2569 is the total number of rules, so has a tip you will know if you have reached "to" the last rule when "total" and "to" matches.
I am trying to run below command on checkpoint management center for security policy contains around 1500 security policies. So I am runing the below command to fecth the policy in 400 offset limit however i have not received output in the file and size 1KB size.
mgmt_cli show access-rulebase offset 0 limit 400 name "Internet Security" details-level "full" use-object-dictionary true -u username --format -json > RuleSet_0_400.json.
When I opend the 1 KB size file. The content shows timeout reached.
Check Point api can expire during your query hence the timeout, reducing the limit might help, but there are other issues to look at.
details-level "full" calls in new queries, like listing "hosts, networks, groups, services tcp/udp/other, and again service-groups, etc" unfortunately there are also limits to this new queries, and in some cases only running rule by rule will do, or even worse big rule can even crash the api, I noticed that a rule with a big list of hosts will crash the api, grouping them in a group however helps it get fixed, but other issues show up, like unexported objects within the show rulebase command, producing UID network objects with 1.1.1.1, I do have to say it's a nice trick from Albert's team at PaloAlto, but never the less this needs to get fixed by exporting smaller and smaller data through the api and compile this for Expedition.
Best regards, I hope reducing the limit like Albert says works for you, it's not working for me with some rules anyway, but I hope I'll get it fixed using the api and will share it here once I figure it out.
It worked,. Started to get output when I set limit to 100 so each file contains 100 rules. The same limit applied for NAT rules as well.
However I would like to know that how to combine all files into one file to process on Expedition tool . As I have done the same way what mentioned in the above article but not worked.
After we created all the json files we will need to create a new file and we will name it "order" to put inside the filenames in the order they need to be read, like in this example:
Edit "order" file
NATRuleSet_0_500.json NATRuleSet_501_600.json
After that we have to ZIP it, to do it right all these 3 files needs to be under the same folder and from within the folder we can run the command from the CLI:
Sorry for not replying, you cannot combine the files, just add them to a the zip, along with "order" file, the content of the order file should be the list of the rules by order top down, that's it.
I wrote a script for exporting policies on our Provider-1 system. It takes 2 variables, one is the CMA IP and the other is the Policy name. It will export the files in chunks of 50 rules and create a zip file containing them all with the order file as well. It will do this for both the access policy and the nat policy. We had performance issues on our MDS for exports so we have it set to 50. The big issue we still run into due to performance is if it fails to properly get the data back. You never get a warning, you just end up with a mostly empty file. So look for any files in your zips that are oddly small.
#!/bin/bash
#
# This script exports the given access and nat policy for the CMA given and
# puts it into the proper format for importing into Expedition
#
#
VERSION=1.0.21
. /etc/profile.d/CP.sh
cd /var/log/tmp
ID="id-$$.txt"
USAGE='Usage: export-policy.sh (CMA IP) (Policy)\t\n '
if [ $# == 0 ] ; then
echo -e $USAGE
exit 1;
fi
CMAIP="$1"
if [ $2 ] ; then
POLICY="$2"
else
echo -e $USAGE
exit 1;
fi
if ! mdsenv $CMAIP >& /dev/null ;
then
if [ ! $CMAIP = "Global" ] ; then
echo "That is not a proper CMA IP"
exit 1
fi
fi
mgmt_cli login -r true -d $CMAIP > $ID
TOP_LIMIT=`mgmt_cli show access-rulebase -d $CMAIP offset 50000 limit 50 name "$POLICY Security" details-level "standard" use-object-dictionary true --format json -s $ID | grep total | awk -F " " '{print $3}'`
OFFSET="0"
FILENAME="0_50"
echo "Total Number of Rules: $TOP_LIMIT"
echo -n "Exporting Rules."
while [ "$OFFSET" -lt "$TOP_LIMIT" ] ;
do
mgmt_cli show access-rulebase -d $CMAIP offset $OFFSET limit 50 name "$POLICY Security" details-level "full" use-object-dictionary true --format json --conn-timeout 3600 -s $ID >> RuleSet_$FILENAME.json
OFFSET=$((OFFSET+50));
FILENAME="$((OFFSET+1))_$((OFFSET+50))"
echo -n ".$OFFSET"
done
echo " "
echo "Packing up files"
ls -rt RuleSet*.json >order
/opt/CPsuite-R80/fw1/Python/bin/python -m zipfile -c $POLICY-Rules.zip order RuleSet*.json >/dev/null 2>&1
rm RuleSet*.json
echo " "
if [ ! $CMAIP = "Global" ] ; then
TOP_LIMIT=`mgmt_cli show nat-rulebase -d $CMAIP offset 50000 limit 50 package $POLICY details-level "standard" use-object-dictionary true --format json -s $ID | grep total | awk -F " " '{print $3}'`
OFFSET="0"
FILENAME="0_500"
echo "Total Number of NAT Rules: $TOP_LIMIT"
echo -n "Exporting NAT Rules."
while [ "$OFFSET" -lt "$TOP_LIMIT" ] ;
do
mgmt_cli show nat-rulebase -d $CMAIP offset $OFFSET limit 500 package "$POLICY" details-level "full" use-object-dictionary true --format json --conn-timeout 3600 -s $ID >> NATRuleSet_$FILENAME.json
OFFSET=$((OFFSET+500));
FILENAME="$((OFFSET+1))_$((OFFSET+500))"
echo -n ".$OFFSET"
done
echo " "
echo "Packing up files"
ls -rt NATRuleSet*.json >order
/opt/CPsuite-R80/fw1/Python/bin/python -m zipfile -c $POLICY-NatRules.zip order NATRuleSet*.json >/dev/null 2>&1
rm NATRuleSet*.json
echo " "
echo "Output found in $POLICY-Rules.zip and $POLICY-NatRules.zip."
else
echo "Output found in $POLICY-Rules.zip"
fi
echo " "
mgmt_cli logout -s $ID
rm $ID >/dev/null 2>&1
echo "All Done. Don't forget to grab a copy of the routing table from the gateway"
