Expedition 2.0 release date: Postponed

[dgildelaig]

Palo Alto Networks,
August 2, 2021

Dear Expedition Users,

During the years, we have evidenced a sustained and increased usage of the Expedition 1.0 tool, earlier known as the Migration Tool. As many of you know, we want to increase the number of functionalities in our tools, to enhance existing functionalities and to improve their quality.

To achieve these improvements, we have decided to join efforts with Professional Services. A dedicated team in Professional Services will take ownership of the code used for configuration translations from third party vendors to PANOS.

This strategy will improve the migrations that we have been offering during the years with the Expedition tool, 

• increasing the resources in the team dedicated to the translations, 
• improving the quality assurance with fewer bugs,
• having a closer the relationship with the Professional Services consultants that consume the translation functionalities on a daily basis, 
• increasing the number of Use Cases that are supported and
• decreasing the response time to support new functionalities ,

making the migrations in your projects more pleasant and efficient.

As a consequence, we have taken the decision to postpone the launch of Expedition 2.0 until April 2022, to guarantee the quality of the release and to extend the functionalities that the tool will provide. During this period, Expedition 1.0 will continue to be supported by the Expedition team, and we are working on updating our code and installation process to make it available for Ubuntu Server 20.04.

We would like to remark that Expedition 2.0 will continue being offered free of charge, as well as the translations from third party vendors to PANOS.

The Expedition Team

[martinsmid]

Personally I am always in favour of postponed launches (regardless of what it is - new OS patch, new game, new software, you name it) if the aim is to make sure the product is indeed ready. I am sure many people have seen rushed product launches in their professional as well as personal lives where it took months for the devs to actually make the final product stable. I am glad Expedition team is not following this trend.

[utahman3431]

I've been able to get the script to work on Ubuntu 20.04.  It took a bit of time to update the modules for newer software.  To use the script make sure you have a fresh install of 20.04.  Download the current install file and untar it in a folder.  Rename/delete/move the initSetup.sh file and create a new one with the below code.  Make sure you make the file executable as well!

Spoiler

#!/bin/bash

currentwd="$(pwd)"
interactive=

# Configure variables
declare_variables() {
#user=$(echo "$USER")
#sourcePath=/PALogs/PaloAltoSC2
#TrafficRotatorPath=/var/www/html/OS/trafficRotator/prepareTrafficLog.sh
#deviceDeclarationPath=/var/www/html/OS/trafficRotator/devices.txt

bold=$(tput bold)
normal=$(tput sgr0)
#BLACK=$(tput setaf 0)
RED=$(tput setaf 1)
GREEN=$(tput setaf 2)
#YELLOW=$(tput setaf 3)
BLUE=$(tput setaf 4)
#MAGENTA=$(tput setaf 5)
#CYAN=$(tput setaf 6)
#WHITE=$(tput setaf 7)
}

printBanner(){
echo ""
echo "${GREEN}${bold}************************************************************"
echo "$1"
echo "************************************************************${normal}"
}

printTitleWait(){
if [[ $interactive -eq 1 ]]; then
echo ""
echo "${GREEN}"
echo "$1"
read -p -r "${BLUE}Press enter to continue${normal}"

else
echo ""
echo "${GREEN}"
echo "$1"
echo "${normal}"
fi
}

printTitle(){
echo "${GREEN}"
echo "$1"
echo "${normal}"
}

printTitleFailed(){
echo "${RED}"
echo "$1"
echo "${normal}"
}

updateRepositories(){
printTitle "Updating APT"
apt-get update
apt-get install -y software-properties-common
printTitle "Installing Expect"
apt-get install -y expect
printTitle "Installing RSyslog debian repository"
expect -c "
set timeout 60
spawn add-apt-repository ppa:adiscon/v8-stable
expect -re \"Press *\" {
send -- \"\r\"
exp_continue
}
"

printTitle "Installing Expedition debian repository"
# wget https://conversionupdates.paloaltonetworks.com/ex-repo.gpg > /etc/apt/trusted.gpg.d/ex-repo.gpg
echo 'deb [trusted=y] https://conversionupdates.paloaltonetworks.com/ expedition-updates/' > /etc/apt/sources.list.d/ex-repo.list

printTitle "Installing RabbitMQ debian repository"
echo 'deb [trusted=y] http://www.rabbitmq.com/debian/ testing main' | tee /etc/apt/sources.list.d/rabbitmq.list
wget -O- https://www.rabbitmq.com/rabbitmq-release-signing-key.asc | sudo apt-key add -

printTitle "Installing MariaDB debian repository"
# (more info: https://www.linuxbabe.com/mariadb/install-mariadb-10-1-ubuntu14-04-15-10)
apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xF1656F24C74CD1D8
#add-apt-repository 'deb [arch=amd64,i386] http://sgp1.mirrors.digitalocean.com/mariadb/repo/10.1/ubuntu xenial main'

printTitle "Updating keystore files"
gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv F1656F24C74CD1D8
gpg --export --armor F1656F24C74CD1D8 | sudo apt-key add
add-apt-repository ppa:deadsnakes/ppa

apt-get update
}

prepareSystemService(){
#sudo vi /etc/ssh/sshd_config

printTitleWait "Changing CLI root password to 'paloalto'"
echo -e "paloalto\npaloalto" | passwd root

printTitleWait "Installing SSHD service"
sudo apt-get install -y openssh-server

printTitle "Enabling ROOT ssh access"
filePath=/etc/ssh/sshd_config
lineToChange=$(grep -n "PermitRootLogin prohibit-password" $filePath | awk -F ':' '{print $1}')
sed -i "${lineToChange}s/.*/ PermitRootLogin yes/" $filePath;
service sshd restart

# Add ZIP and Zlib
printTitle "Installing ZIP libraries"
apt-get install -y zip
apt-get install -y zlib1g-dev

# Rsyslog
printTitleWait "Installing Rsyslog for syslog Firewall traffic logs"
apt-get install -y rsyslog

systemctl disable syslog.service

cp /lib/systemd/system/rsyslog.service /etc/systemd/system/rsyslog.service
# vi /lib/systemd/system/rsyslog.service
# [Service]

# Type=notify

# ExecStart=/usr/sbin/rsyslogd -n
# 
StandardOutput=null
# 
Restart=on-failure

#update-rc.d rsyslog enable
#systemctl enable rsyslog.service
}

installLAMP(){
# Install all Apache required modules
printTitleWait "Installing Apache service and dependencies for PHP"
apt-get install -y apache2 \
php7.4 libapache2-mod-php7.4 \
php7.4-bcmath php7.4-mbstring php7.4-gd php7.4-soap php7.4-zip php7.4-xml php7.4-opcache php7.4-curl php7.4-bz2 \
php7.4-ldap \
php7.4-mysql

# Install openssl for https
printTitle "Activating SSL on Apache"
apt-get install -y openssl
# Enable SSL for the Web Server
a2ensite default-ssl; a2enmod ssl; systemctl restart apache2

# Database Server
printTitleWait "Installing the DB server. " # Please, do not enter a password for root. We will automatically update it later to 'paloalto'.Remember: DO NOT ENTER A PASSWORD"
# printTitleWait "Let us emphasize it: DO NOT ENTER A PASSWORD"
expect -c "
set timeout 600
spawn apt-get install -y mariadb-server mariadb-client
expect -re \"New password for the MariaDB *\" {
send \"\r\"
exp_continue
}
"

echo 'update mysql.user set plugin="" where User="root"; flush privileges; ' | mysql -uroot

# Install the secure controls for MySQL
# Make sure that NOBODY can access the server without a password. Password changes to "paloalto"
mysql -e "UPDATE mysql.user SET Password = PASSWORD('paloalto') WHERE User = 'root'"
# Kill the anonymous users
#mysql -e "DROP USER ''@'localhost'"
# Because our hostname varies we'll use some Bash magic here.
#mysql -e "DROP USER ''@'$(hostname)'"
# Kill off the demo database
#mysql -e "DROP DATABASE test"
# Make our changes take effect
mysql -e "FLUSH PRIVILEGES"
# Any subsequent tries to run queries this way will get access denied because lack of usr/pwd param

filePath=/etc/mysql/my.cnf
sed -i 's/max_allowed_packet\t= 16M/max_allowed_packet\t= 64M/g' $filePath
sed -i 's/bind-address\t\t= 127.0.0.1/#bind-address\t\t= 127.0.0.1/g' $filePath
sed -i 's/#binlog_format=row/binlog_format=mixed/g' $filePath
service mysql restart

# Create Databases
printTitle "Creating initial Databases"
mysqladmin -uroot -ppaloalto create pandb
mysqladmin -uroot -ppaloalto create pandbRBAC
mysqladmin -uroot -ppaloalto create BestPractices
mysqladmin -uroot -ppaloalto create RealTimeUpdates

# PERL
printTitleWait "Installing Perl"
apt-get install -y perl
apt-get install -y liblist-moreutils-perl

printTitleWait "Installing Python dependencies for BPA modules"
apt-get install -y python
apt-get install -y python3
apt-get install -y python3-pip
pip install lxml
pip install --upgrade pip
pip install unidecode
pip install pandas
pip install six
pip install sqlalchemy

# RabbitMQ
printTitleWait "Installing Messaging system for background tasks"
apt-get install -y rabbitmq-server
update-rc.d rabbitmq-server defaults
apt-get install -y policycoreutils
/usr/sbin/setsebool httpd_can_network_connect=1

#Add www-data to expedition group
usermod -a -G expedition www-data
}

installExpeditionPackages(){
# apt-get Repository
printTitleWait "Installing Expedition packages"

printTitle "Updating databases"
cd "$currentwd" || exit
tar -zxvf databases.tgz
mysql -uroot -ppaloalto pandb < databases/pandb.sql
mysql -uroot -ppaloalto pandbRBAC < databases/pandbRBAC.sql
mysql -uroot -ppaloalto BestPractices < databases/BestPractices.sql
mysql -uroot -ppaloalto RealTimeUpdates < databases/RealTimeUpdates.sql

#Get the GPG key:
cd "/etc/apt/trusted.gpg.d/" || exit
# wget https://conversionupdates.paloaltonetworks.com/ex-repo.gpg
# Installing Expedition package
# apt-get install -y --allow-unauthenticated expedition-beta
expect -c "
set timeout 600
spawn apt-get install -y --allow-unauthenticated expedition-beta
expect -re \"Do you want to *\" {
send \"Y\r\"
exp_continue
}
"

printTitle "Tunning some Expedition parameters"
filePath=/etc/mysql/my.cnf
sed -i 's/log_bin/skip-log_bin/g' $filePath

filePath=/etc/php/7.4/apache2/php.ini
sed -i 's/mysqli.reconnect = Off/mysqli.reconnect = On/g' $filePath
# sed -i 's/mysqli.reconnect = Off/mysqli.reconnect = On/g' /etc/php/7.4/apache2/php.ini

filePath=/etc/php/7.4/cli/php.ini
sed -i 's/mysqli.reconnect = Off/mysqli.reconnect = On/g' $filePath
# sed -i 's/mysqli.reconnect = Off/mysqli.reconnect = On/g' /etc/php/7.4/cli/php.ini
sudo service apache2 restart

printTitle "Updating Python modules"
expect -c "
set timeout 600
spawn bash /var/www/html/OS/BPA/updateBPA306.sh
expect -re \"Do you want to *\" {
send \"Y\r\"
exp_continue
}
"

printTitle "Installing Spark dependencies"
apt-get install -y openjdk-8-jre-headless
apt-get install -y --allow-unauthenticated expeditionml-dependencies-beta

cp /var/www/html/OS/spark/config/log4j.properties /opt/Spark/
rm -f /home/userSpace/environmentParameters.php

}

settingUpFirewallSettings(){
printTitle "Installing Firewall service"
apt-get install -y firewalld

printTitle "Firewall rules for Web-browsing"
#APACHE2
firewall-cmd --add-port=443/tcp
firewall-cmd --permanent --add-port=443/tcp

printTitle "Firewall rules for Database (skipped)"
#MySQL/MariaDB (optional)
#firewall-cmd --add-port=3306/tcp
#firewall-cmd --permanent --add-port=3306/tcp

#RabbitMQ

#SPARK
printTitle "Firewall rules for ML Web-Interfaces"
firewall-cmd --add-port=4050-4070/tcp
firewall-cmd --permanent --add-port=4050-4070/tcp

firewall-cmd --add-port=5050-5070/tcp
firewall-cmd --permanent --add-port=5050-5070/tcp
}

createExpeditionUser(){
exists=$(id -u expedition | wc -l)
if [ "$exists" -eq 1 ]; then
printTitle "Expedition user already exists"
else
printTitleFailed "expedition user does not exist"
printTitleFailed "Create expedition user via \"sudo adduser --gecos '' expedition\""
printTitleFailed "Execute this installer again afterwards"
exit 1
fi
}

createPanReadOrdersService(){
cp /var/www/html/OS/startup/panReadOrdersStarter /etc/init.d/panReadOrders
chmod 755 /etc/init.d/panReadOrders
chown root:root /etc/init.d/panReadOrders
ln -s /etc/init.d/panReadOrders /etc/rc2.d/S99panReadOrders
ln -s /etc/init.d/panReadOrders /etc/rc3.d/S99panReadOrders
ln -s /etc/init.d/panReadOrders /etc/rc4.d/S99panReadOrders
ln -s /etc/init.d/panReadOrders /etc/rc5.d/S99panReadOrders

systemctl daemon-reload
service panReadOrders start
}

controlVersion(){
ubuntuVersion=$(lsb_release -a | grep Release | awk '{print $2}')
if [ "$ubuntuVersion" == "20.04" ]; then
printTitle "Correct Ubuntu Server 20.04 version"
else
printTitleFailed "This script has been prepared for Ubuntu Server 20.04"
printTitleFailed "Current version: "
echo "$ubuntuVersion"
exit 1
fi

# Check if some packages has already been installed
expeditionAlreadyInstalled=$(apt-get list --installed | grep -c expedition-beta)
if [ "$expeditionAlreadyInstalled" -ne 0 ]; then
printTitleFailed "This script has been prepared to install Expedition from scratch"
printTitleFailed "Expedition package is already present"
exit 1
else
printTitle "This machine does not have Expedition installed"
fi;

}

updateSettings(){
# myIP=$(ifconfig | grep -Eo 'inet (addr:)?([0-9]*\.){3}[0-9]*' | grep -Eo '([0-9]*\.){3}[0-9]*' | grep -v '127.0.0.1')
myIP=$(hostname -I)
echo "INSERT INTO ml_settings (server) VALUES ('${myIP}') " | mysql -uroot -ppaloalto pandbRBAC
}

introduction(){
echo
echo "${GREEN} ****************************************************************************************"
echo " * *"
echo " * WELCOME TO EXPEDITION ASSISTED INSTALLER v.0.3 (10/24/2019) *"
echo " * *"
echo " * This script will download and install required packages to prepare Expedition on *"
echo " * Ubuntu server 16.04. A ${bold}NEW image${normal}${GREEN} is expected for this installer to take effect. *"
echo " * This installer requires ${bold}Internet Connection${normal}${GREEN} *"
echo " * *"
echo " * *"
echo " * We do not take any responsibility and we are not liable for any damage caused *"
echo " * through use of this tool, be it indirect, special, incidental or consequential *"
echo " * damages (including but not limited to damages for loss of business, loss of pro- *"
echo " * fits, interruption or the like). If you have any questions regarding the terms of *"
echo " * use outlined here, please do not hesitate to contact us at *"
echo " * fwmigrate@paloaltonetworks.com *"
echo " * *"
echo " * If you continue with this installation you acknowledge having read the above lines *"
echo " * *"
echo " ****************************************************************************************${normal}"
printTitleWait ""

}

usage()
{
echo "usage: initSetup [-i] | [-h]"
}

# Establish run order
main() {

while [[ $# -gt 0 ]]; do
key="$1"
case $key in
-i | --interactive ) interactive=1
;;
-h | --help ) usage
exit
;;
* ) usage
exit 1
esac
shift
done

declare_variables

introduction

controlVersion

createExpeditionUser

printBanner "Updating Debian Repositories"
#apt-get -y install expect
updateRepositories # Update Debian repositories

# Prepare userSpace for Expedition data storage
printTitle "Preparing the /home/userSpace Space for data storage"
mkdir /home/userSpace; chown www-data:www-data -R /home/userSpace
mkdir /data; chown www-data:www-data -R /data

printBanner "Installing System Services"
prepareSystemService # Allow remote root ssh access. Change PermitRootLogin prohibit-password to PermitRootLogin yes

printBanner "Installing LAMP Services"
installLAMP

settingUpFirewallSettings

printBanner "Installing Expedition packages"
installExpeditionPackages

printBanner "Starting Task Manager"
createPanReadOrdersService

updateSettings

}

main "$@"

[JustinWalker]

I recently went to install expedition, so I googled to find the installer guide.   The guide that's posted on the pan.dev site (which looks great by the way) is for version 2.0.   You may want to post something at the top of the installer guide to indicate that the version is not yet ready for release.  

[lychiang]

Yes, we will update the pan-os website to indicate the delay . thank you!

[AlexNC]

Hi,

What is the planned release date of the Expedition 1.0 for Ubuntu 20.04 LTS?

Thank you and best regards

Alex

[justamoment]

@utahman3431 Hi - is that script for v1 on Ubuntu 20.04 or for v2 beta?  Thanks!

[lychiang]

V1

[justamoment]

Thanks!

[summands]

Any update on the release of Expedition 2.0?

[lychiang]

Scheduled release will be sometime in May . 

[summands]

Any updated release timeline? Still happening in May?

[lychiang]

Yes , the last week of the May

[Alabrir]

Hello @lychiang,

Do you have any update on the release date ? Maybe a little bit more precise one ?

Since we're one day away from the deadline.*

Regards, 

Alabrir

[summands]

Do we know when it was pushed back to? We perform tons of migrations and depend on Expedition greatly. We are eager to get in front of the new product as soon as possible to get cycles on it before we have to leverage it for a customer migration. 

[Alabrir]

Hi @lychiang,

Your last answer have disapear from the thread..

So, same question than @summands, do you have the new release date ?

Regards,

Alabrir

[lychiang]

There will be an announcement soon. We want Expedition 2 to offer security and reliability in the functionalities it exposes, for this reason the tool is now going through an internal threat analysis. We will make it available for download once we get the positive verdict on the studies.  

[Alabrir]

Hello @lychiang !

Since we're already at the end of this month (almost) is it ok to say that Expedition 2 has been postponed for July ?

Can you give us some news about the actual verdict on the studies and if it needs more time to be tested ?

Regards,

Alabrir

[lychiang]

Hi @Alabrir , thanks for your follow-up on Expedition2 launching.
Right now we could not establish an ETA.
Our main goal is to make sure our tool is properly secured following PAN development
and delivery standards. For that reason we are going through an internal threat analysis.
We will keep you posted.
Thanks, ”

[justamoment]

I would much rather have a fully polished product than something a bit rushed out the door.  Please, take your time 🙂

Thanks!

[Alabrir]

Hi @lychiang,

Sorry for being a little bit insistent but I'm racing against time actually that's why I ask for some news so often ^^' 

I'm grateful for what you've done !

@justamoment, I totally agree with you that I prefer a fully supported software and well optimised😶

Regards,

Alabrir

[Will-78]

Is there anywhere we can subscribe to be notified when it is released? I'm in agreement with others, better delayed and a proper release.

[C.Pfitzer]

After this long postponement you might as well switch to Ubuntu 22.04. 🙂

[summands]

Not to beat a dead horse here, but is there any update on the expected release time?

[Alabrir]

Hello @summands,

As far as I know, PaloAlto dev team will not communicate anymore on it until the release since it's been security tested.

They know that they're late....

Let's tag @lychiang so he could see that we're still very interested and if he has more news than me I think that we will be glad to know them 🙂

Regards,

Alabrir

[lychiang]

Hello @Alabrir @summands , Thank you for your interests on expedition 2.x , currently the product is still under security review, no ETA at this point.  I will update the article once the product is ready.  Please stay tuned. 

[Alabrir]

Hello @lychiang !

It's been 2 month since the last update on this thread.

Do you have some new to share about the product ?

Is it still under security review ? Or something else went wrong ?

Best regards,

Alabrir

[lychiang]

@Alabrir The security review has been done, and we are working on the product, we will have announcement when the product is ready. Thank you for your patience. 

[AlexNC]

@lychiang 

Any update on a Q1-4 for 2023? or maybe even a x-mas present for 2022 still?

Will there be a Beta to join?

Best regards

Alex

[lychiang]

@AlexNC It won't be released by end of this year, no ETA at this point . Thank you for your patience. 

[jasoncooper1]

It's 2023 now. Any update on a release date? 

[Alabrir]

Hello @lychiang,

First of all, happy new year to you and the Palo Alto team.

As @jasoncooper1 say, it's now 2023 and I know that some products have been left behind since Palo Alto is focusing on other projects more important.

Does Expedition one of them ? (Project that was left behind) 

Actually, even if it is, I think that most of us just wanted to have some news about it (ETA, Testing Phase, annulation ?)

Regards, 

Alabrir

[AlexNC]

@Alabrir

I sure hope not that Expedition 2.0 or for that matter MineMeld (local install, or they give us a SaaS EDL Manager that covers MineMeld) is one of them.
I see many competitors beefing up their migration tool efforts in a way to gain customers and make the transition as easy as possible. Now not all migrations are good migrations and a **bleep** ASA config will end up a **bleep** PAN config. 
But if done right tools like Expedition can be a huge selling point to get a customer to go for Palo and not just lifecycle up with his existing vendor.

Not to speak of doing house cleaning of existing or old PAN installations out there. Specially since PAN-OS and Panorama are lacking a HUUUGE amount of security-rule manageability.
I remember how last year the Expedition 2.0 pan.dev page even had the instructions how to install it and use it with what was working back then. Now it is dead silent and no work about it anymore. 
It does worry one a bit.

Anyhow I do appreciate the effort that's been put into 1.0 and 2.0 so far and I greatly enjoyed using the 1.x version so far.
Palo Alto needs to make it an official and fully supported tool in their arsenal or tools. This way it might also get you guys the necessary time and funding to move forward more efficiently.

best regards

Alex

[summands]

Hi there, as we near a year past the already postponed release date, I was curious if we have any updates? Even some information on what to expect, e.g. what firewall platforms will be supported for migration. 

[Alabrir]

Hello  Everyone,

As  @summands mentionned it, it's been almost a year past the release date and we have no information about it except that the security test went well.

@lychiang, since it's becoming kind of worrying, don't you have any information about Expedition 2.0 ? It Really seems that this product is being left behind...

@AlexNC Since you seems to be really interested too 😉

Regards,

Alabrir

[AlexNC]

@Alabrir  Indeed, we heavily use Expedition for migrating from other vendors and also for M Learning and other Tasks.
There are a few things desired in Expedition but we understand that Exp. 1.0 won't get much love in regards to new features and functions. So yes! Expedition 2.0 is highly anticipated by us.

Not getting any info is indeed worrying, specially since there once was the download and install available a longish time ago and then got pulled.

Cheers
Alex

[lychiang]

@AlexNC @Alabrir , please write an email to fwmigrate@paloaltonetworks.com with your use cases, let the team know the reason you need to use Expedition 2.x instead of Expedition 1.x , the team will evaluate  and see if you can be part of the early beta testers. 

[PeterSouthwick]

Any updates to when we can expect the launch of 2.0?

[lychiang]

Hi @PeterSouthwick , please refer the details about how to get access to beta release here : https://live.paloaltonetworks.com/t5/expedition-articles/introducing-expedition-2-beta/ta-p/542787