on 09-19-2019 01:45 AM - edited on 09-26-2019 10:29 AM by Retired Member
An easy and powerful way of installing MineMeld is using MineMeld docker image. A docker-based installation of MineMeld can run on any Linux distribution supported by Docker and it is extremely easy to upgrade and maintain.
The procedure to use MineMeld is pretty simple:
The rest of the article will guide you thru installing Docker CE on RHEL 7 and run MineMeld on top of it. Only the first part, the one related to installing Docker on RHEL, is RHEL specific. The second part, the one related to MineMeld itself, is distribution independent.
sudo yum update -y
sudo yum remove docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-engine
sudo yum install -y http://mirror.centos.org/centos/7/extras/x86_64/Packages/container-selinux-2.107-1.el7_6.noarch.rpm
sudo yum install -y yum-utils device-mapper-persistent-data lvm2 container-selinux
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum update -y && sudo yum makecache fast
sudo yum install docker-ce docker-ce-cli containerd.io
sudo systemctl start docker
sudo docker run hello-world
sudo docker pull paloaltonetworks/minemeld
sudo docker volume create minemeld-logs sudo docker volume create minemeld-local
sudo docker run -dit --name minemeld --restart unless-stopped --tmpfs /run -v minemeld-local:/opt/minemeld/local -v minemeld-logs:/opt/minemeld/log -p 443:443 -p 80:80 paloaltonetworks/minemeld
[ec2-user@minemeld ~]$ sudo docker logs minemeld
*** Running /etc/rc.local...
*** Booting runit daemon...
*** Runit started as PID 7
minemeld: checking if dependencies are running...
run: redis: (pid 18) 0s
run: collectd: (pid 19) 0s
Copying constraints
Starting redis-server...
Regenarating CA bundle
Sep 18 14:07:31 d0b5d1fbc102 syslog-ng[20]: syslog-ng starting up; version='3.5.6'
2019-09-18T14:07:31 (35)cacert_merge.main INFO: config: {'cafile': ['/opt/minemeld/local/certs/site/'], 'dst': '/opt/minemeld/local/certs/bundle.crt', 'config': '/opt/minemeld/local/certs/cacert-merge-config.yml', 'no_merge_certifi': False}
(integer) 0
Starting minemeld...
/opt/minemeld/engine/0.9.64/local/lib/python2.7/site-packages/supervisor/options.py:383: PkgResourcesDeprecationWarning: Parameters to load are deprecated. Call .resolve and .require separately.
return pkg_resources.EntryPoint.parse("x="+spec).load(False)
2019-09-18 14:07:32,153 CRIT Set uid to user 106
2019-09-18 14:07:32,154 WARN Included extra file "/opt/minemeld/supervisor/config/conf.d/minemeld-engine.conf" during parsing
2019-09-18 14:07:32,154 WARN Included extra file "/opt/minemeld/supervisor/config/conf.d/minemeld-supervisord-listener.conf" during parsing
2019-09-18 14:07:32,154 WARN Included extra file "/opt/minemeld/supervisor/config/conf.d/minemeld-traced.conf" during parsing
2019-09-18 14:07:32,154 WARN Included extra file "/opt/minemeld/supervisor/config/conf.d/minemeld-web.conf" during parsing
2019-09-18 14:07:32,164 INFO RPC interface 'supervisor' initialized
2019-09-18 14:07:32,164 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2019-09-18 14:07:32,164 INFO supervisord started with pid 21
2019-09-18 14:07:33,167 INFO spawned: 'minemeld-supervisord-listener' with pid 60
2019-09-18 14:07:33,168 INFO spawned: 'minemeld-engine' with pid 61
2019-09-18 14:07:33,170 INFO spawned: 'minemeld-traced' with pid 62
2019-09-18 14:07:33,172 INFO spawned: 'minemeld-web' with pid 63
2019-09-18 14:07:34,322 INFO success: minemeld-supervisord-listener entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-09-18 14:07:34,322 INFO success: minemeld-traced entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-09-18 14:07:34,322 INFO success: minemeld-web entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-09-18 14:08:03,191 INFO success: minemeld-engine entered RUNNING state, process has stayed up for > than 30 seconds (startsecs)
You can check MineMeld engine and MineMeld audit logs from outside the container. Inspect the minemeld-logs volume to grab the directory used by the Docker engine to store volume files. Your logs will be there.
[ec2-user@minemeld ~]$ sudo docker inspect minemeld-logs
[
{
"CreatedAt": "2019-09-18T10:07:32-04:00",
"Driver": "local",
"Labels": {},
"Mountpoint": "/var/lib/docker/volumes/minemeld-logs/_data",
"Name": "minemeld-logs",
"Options": {},
"Scope": "local"
}
]
[ec2-user@minemeld ~]$ sudo ls /var/lib/docker/volumes/minemeld-logs/_data
minemeld-engine-stderr---supervisor-PV9ZPJ.log minemeld-supervisord-listener.log minemeld-web-stderr---supervisor-cXauy4.log
minemeld-engine.log minemeld-traced-stderr---supervisor-4V3T4E.log minemeld-web.log
minemeld-supervisord-listener-stdout---supervisor-T3Wfwi.log minemeld-traced.log supervisord.log
The config directory in the minemeld-local volume contains all the file needed to restore the MineMeld installation. You can backup the files by inspecting the volume and perform a backup of the config directory.
[ec2-user@minemeld ~]$ sudo docker inspect minemeld-local
[
{
"CreatedAt": "2019-09-18T10:03:12-04:00",
"Driver": "local",
"Labels": {},
"Mountpoint": "/var/lib/docker/volumes/minemeld-local/_data",
"Name": "minemeld-local",
"Options": {},
"Scope": "local"
}
]
[ec2-user@minemeld ~]$ sudo ls /var/lib/docker/volumes/minemeld-local/_data
certs config data library prototypes redis supervisor trace
To upgrade to the latest and greatest release:
sudo docker pull paloaltonetworks/minemeld
sudo docker stop minemeld sudo docker rm minemeld
sudo docker run -dit --name minemeld --restart unless-stopped --tmpfs /run -v minemeld-local:/opt/minemeld/local -v minemeld-logs:/opt/minemeld/log -p 443:443 -p 80:80 paloaltonetworks/minemeld
The default certificate used by MineMeld is a self-signed certificate. You should change it when running in production. To do it just map your certificate and private key on the default certificate:
sudo docker run -dit \
--name minemeld \
--restart unless-stopped \
--tmpfs /run \
-v minemeld-local:/opt/minemeld/local \
-v minemeld-logs:/opt/minemeld/log \
-v /var/lib/minemeld/real-cert.crt:/etc/nginx/minemeld.cer:ro \
-v /var/lib/minemeld/real-cert.pem:/etc/nginx/minemeld.pem:ro \
-p 443:443 -p 80:80 \
paloaltonetworks/minemeld
Thanks @lmori this looks great I may give it a go. I am at that point where I'm about to build our Minemeld solution. I have successfully created a 16.04 Ubuntu build using anisible in our Dev environment. All other distro are listed as "Still experimental" on the minemeld site.
How stable is the Docker install? I'd be interested in what you would recommend.
I can confirm this will work on Ubuntu as well. Get docker running on Ubuntu using the instructions below. Then follow the instructions from Install & Run MineMeld.
https://docs.docker.com/install/linux/docker-ce/ubuntu/
Great article @lmori
Thank you to Ksampson and Lmori. Following the provided instructions I was able to install MimeMeld within Ubuntu 18.04.1 LTS and have it properly function. I have been attempting to replace my Ubuntu 14.x MineMeld install and these were the only instructions which actually worked.
It doesn't look like port 13514 is being exposed for the syslog miner. Does the docker container support the syslog miner in this article?
Hello 🙂 can anyone help?
I am running Unbuntu 18.04.3 LTS and have Minemeld installed and running in docker, however I can't not get my new certificate into the docker and replace the the self-signed certificates for Minemeld. I have tried SCP/WinSCP/SFTP I just can't seem to get my new certificates in the correct location. #nearlythere! Can anyone point me in the right direction I am a total novice with Docker please bear with! 😞
Hello all,
So I am able to get the cert there now 🙂 as above ^
However I have an issue now that I can't restart the NGINX service 😞
root@4416471727a8:/etc/nginx# sudo /etc/init.d/nginx restart
* Restarting nginx nginx [fail]
This happens even if I do a fresh rebuild of the docker. Has anyone else had this issue? I can't see any issues?
Any help or ideas would be great 🙂
I have a strange issue with this. I run throught the setup guide and it works great, minemeld docker instance starts-up ok, I login, change admin password, add another user, logout and back in again without issue, however, as soon as I reboot the VM (it on ESXi) when I go to login I get "Error checking credentials: Timeout" - any help appreciated on getting this fixed is appreciated.
Update, if I login as the new user I created it works great.
So, to re-cap.
I wish this was well documented for macOS users running MineMeld on Docker. "tmpfs" is Linux only so in order to use persistent memory, I created RAM-only volumes and symlink'd each to the respective minemeld volumes; and then modified the syntax to start the container without the tmpfs argument. Would be great if there was proper documentation for this; spent a lot of productive hours just to get this working. Thanks though for all the work gone into MineMeld.
per @jdanjuma's note, tmps wont work on mac, follow steps 1 & 2 and then use the following:
sudo docker run -dit --name minemeld --restart unless-stopped -v minemeld-local:/opt/minemeld/local -v minemeld-logs:/opt/minemeld/log -p 443:443 -p 80:80 paloaltonetworks/minemeld
launch your browser to localhost, default creds & you're in.
@John_Merry , we're having a bit of trouble with the certificate commands. Can you elaborate on your procedure for uploading and replacing the default certificates?
After getting the docker image running I am seeing an error with the default miners. I'm not sure how to troubleshoot it as the container doesn't seem to have any network tools to verify access.
I've reviewed the resolv.conf file and it has the correct dns servers.
I've also added the docker host to the an ssl bypass to ensure it isn't getting an ssl certificate verification problem when accessing external sources.
Any ideas?
I am able to successfully perform the wget from the docker host but not from the cli of the container. It seems to be a name lookup issue:
wget https://www.dshield.org/block.txt
--2020-05-19 11:18:54-- https://www.dshield.org/block.txt
Resolving www.dshield.org (www.dshield.org)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘www.dshield.org’
I've checked /etc/resolv.conf which has the same dns servers as the docker host.
The issue was the host firewall running on a standard Centos 8 install and blocking outbound connections on the docker interface.
I used the busybox image to troubleshoot connectivity until I was satisfied it was working correctly. Hope this helps someone else.
@lmori Thank you for putting this together. It made getting Minemeld running very simple and painless, right up to the point of replacing the certificates. Would you mind expanding a bit on how to replace the certificate so that the outputs can be used as EDLs? Thank you in advance.
Do you know how to re install minemeld using docker.
I am facing some Engine Fatal Error, and I want to reset everything and install it again from beginning.
For that I need to remove current details and then have to follow above mention steps.
So if you could help me with that.
So, I'm using ports 80 and 443 on my docker box, so I re-mapped those ports on the host side to 8080 and 8443, as one does. Imagine my surprise when I looked at the feed URLs and they had the correct re-mapped ports! Great job guys!
excuse a docker noob....but is there anyway to patch some of the vulnerabilities in this container like the linux host?
Looks like we have 17 vulns listed with potential patches?
Also wondering how work is going migrating to a python3 stack? Love the product....but moved to docker due to ansible build having dependancy hell breaks relating to the python. I imagine that fixes probably won't be too available until a migration to Python3 happens.
In meantime i have been checking and upgrading docker image as per instructions above.
Screengrab of the Vulnerabilities listed:
Hi @Paul_Stinson,
we are working on the Python3 release. In the meantime you can download the latest docker image with all the Ubuntu security updates applied (we just pushed it). You can also build the image directly from the Dockerfile (https://github.com/PaloAltoNetworks/minemeld-docker), the script will automatically apply all the security updates.
@lmori this appears to have cleared up the vulnerabilities being reported. Awesome.
I had just tried the pull of latest build just before i posted so amazing timing on getting update out after my post! universe in sync, check!
Am looking forward to python 3 upgrade for Minemeld and any future enhancements planned. Thanks for all the good work the team put into the product! 🙂
@lmori when will the image be updated so that it does not void all security best practices detected by Twistlock? (not running as root being the simplest one...)
@JohanBogema working a new version of MineMeld based on python3 and new docker mechanism. Please note that while the init process inside the current docker image is running as root, MineMeld and all the exposed services are running as limited users.
Hi team,
Love the work you have been doing with this tool, just wondering with Docker Image about plans around a move from base Ubuntu 16.04LTS (to say 18 or 20) and move to python3 timeline?
( as support for Ubuntu patching this version is EOL for patching come end of April this year).
Also wondering would you have any instructions to build from a container image direct rather than the following command if you wanted to muck around building from source and try to make own container?
sudo docker pull paloaltonetworks/minemeld
** sorry if last question a noob question but not really a dev mindset and just learning docker ropes.
Would be nice to include in doco above for fellow noobs around cleaning up old containers etc ( as our vuln scanning tools pick these up it is good to clean up the old containers)
i.e sudo docker images ( to list images on your docker instance ) and sudo docker rmi [image name] (to remove the old image)
In your docker or kubernetes (or whatever orchestration you're using) just run -
FROM paloaltonetworks/minemeld:latest
This will pull the most up to date stable build they have. If you have a devops team running kubernetes, they should be able to help you with some things that will get it into your actual environment. Most companies who are doing devops will have a methodology to safely pull a public container in this way and deploy it internally.
Hi,
In the section on updating the UI certs can you clarify whether /etc/nginx/minemeld.cer is the ssl private key and .pem file is a certificate chain? Or is it the other way around?
Many thanks.
Tony.
@Tony-Arnold .crt (.cer) is the certificate chain and .pem is the private key.
@stangri-la Thanks. I have it the right way round.
However, the docker logs are giving the error message:
nginx: [emerg] PEM_read_bio_X509_AUX("/etc/nginx/minemeld.cer") failed (SSL: error:0906D06C:PEM routines:PEM_read_bio:no start line:Expecting: TRUSTED CERTIFICATE)
My .cer file has my domain cert followed by the CA cert and is base64 encoded. Is nginx expecting a different format for the certificate chain?
All, thanks for the info on the thread, it has been very helpful. Like others, I am stuck on replacing the certificates in a docker installation. Where do I copy the "real certs" and how do I update the nginx CA chain to include my internal CA (Docker appears to be using an internal nginx server)?
Based on the sample command, I thought the real certs would go in /var/lib/minemeld. I have to create the /var/lib/minemeld directory and when I do and run the command above, after putting the real certs in /var/lib/minemeld, I receive this error:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: rootfs_linux.go:76: mounting "/var/lib/minemeld/real-cert.pem" to rootfs at "/etc/nginx/minemeld.pem" caused: mount through procfd: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type.
Thanks,
Mike
I've made some progress. I can get Minemeld to start with, supposedly, my certs. The issue was the -v option, which apparently, at this point is only used for volumes/directories. For files, --mount is needed instead (https://docs.docker.com/storage/bind-mounts/). My run command now looks like this:
sudo docker run -dit \
--name minemeld \
--restart unless-stopped \
--tmpfs /run \
-v minemeld-local:/opt/minemeld/local \
-v minemeld-logs:/opt/minemeld/log \
--mount type=bind,source=/var/lib/minemeld/real-cert.crt,target=/etc/nginx/minemeld.cer,readonly \
--mount type=bind,source=/var/lib/minemeld/real-cert.pem,target=/etc/nginx/minemeld.pem,readonly \
-p 443:443 -p 80:80 \
paloaltonetworks/minemeld
Docker starts Minemeld without any errors returned, but the site is still not accessible. I first assumed that it did not like my root CA, which is a local CA from my PAN firewall. I added the CA to the file
/var/lib/docker/volumes/minemeld-local/_data/certs/bundle.crt (I also tried simply copying the root CA file into that directory and restarting docker).
Unfortunately, although it appears that Minemeld is running, the internal nginx server resets my HTTPS connection attempt and I cannot load the site.
Still searching ......
How do you get the certs going?
Had minemeld running bare bones Ubuntu 16.04 LTS. A lot of vulnerabilities were detected so we tried a dist-upgrade to Ubuntu 18.04 LTS. That cleared many of the vulnerabilities but broke minemeld. Posted about it and was told, hey run it in docker. Ok great! Found this thread, took Ubuntu all the way up to 20.04 LTS to get even more vulns to go away, installed it, it runs, set a password, great. But now I want to continue on with your instructions to run it with my certs. Our internal CA signed our minemeld.domain.com and its already sitting in /home/minemeld so I tried this but not knowing anything about how this all works is confusing....
root@minemeld:/home/minemeld# sudo docker run -dit --name minemeld --restart unless-stopped --tmpfs /run -v minemeld-local:/opt/minemeld/local -v minemeld-logs:/opt/minemeld/log -v /var/lib/minemeld/real-cert.crt:/home/minemeld/minemeld.cer:ro -v /var/lib/minemeld/real-cert.pem:/home/minemeld/minemeld.pem:ro -p 443:443 -p 80:80 paloaltonetworks/minemeld
docker: Error response from daemon: Conflict. The container name "/minemeld" is already in use by container "e37c49d9b85a8bb6b47d248666bb8265bccca6fb59dece63316d8aabefc35376". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
There's a ton of confusion around getting the certs to work. The documentation isn't very clear but this is what worked for me (I have a .pem wildcard certificate from a CA).
1. Rename the .pem certificate with a .crt extension. Open the file with a text editor and copy everything starting with
-----BEGIN PRIVATE KEY-----
and ending with
-----END PRIVATE KEY-----
and paste into a new file and save it with the same name and a .pem extension. There should now be two files on your local machine: <your_cert>.crt and <your_cert>.pem.
2. Transfer both files from your local machine to /var/lib/minemeld
3. Stop and remove the current container
[stang@docker ~]$ sudo docker stop minemeld [stang@docker ~]$ sudo docker rm minemeld |
4. Start the container
[stang@docker ~]$ sudo docker run -dit \ > --name minemeld \ > --restart unless-stopped \ > --tmpfs /run \ > -v minemeld-local:/opt/minemeld/local \ > -v minemeld-logs:/opt/minemeld/log \ > -v /var/lib/minemeld/<your_cert>.crt:/etc/nginx/minemeld.cer:ro \ > -v /var/lib/minemeld/<your_cert>.pem:/etc/nginx/minemeld.pem:ro \ > -p 443:443 -p 80:80 \ > paloaltonetworks/minemeld |
This post falls under the category of "I'm a dork" or "DUH". First, thanks to @stangri-la for the post and reminding me I wanted to get back to this issue. I've been running my EDLs without a certificate profile because I could not get my Docker MM instance to take the certificate I generated from my PAN 220, with the PAN as the root CA. I had exported and re-exported the cert from the PAN multiple times, tried various command line switches, all to no avail.
This week, coming back to this, instead of simply stripping out the text between ---- BEGIN PRIVATE KEY ---- and the start of the key, I actually looked at it. And, DUH, the PAN exports the private key in encrypted form! Using openssl (openssl rsa -in <source .key file> -out <outfile>), I stripped the password from the key file, copied it and the cert file, without adding the PAN root cert to the cert file, stopped, deleted and restarted MM per above (thanks again!) and it worked like a charm.
Morale of the story, slow down and look at what you are doing instead of just doing what you have always done.
Thank you for this article. I was able to get this to run on Ubuntu 21.04 with no issues. Except, I can't figure out how to do the certificates. From the other posts, looks like the spot to put the certificates is in /var/lib/minemeld folder. I don't have that.
Everything looks like it's in the /var/lib/docker/overlay2/ directory with 2 crazy long numbers with either merge or diff in the directory path, then you get /var/lib/minemeld and /etc/nginx at the end of the path string.
Has anyone ran into this?
I do not recall, but I think I created the directory after the fact, in order to use custom certs. I suggest creating the directory and changing the permissions, if not already set, to:
drwxr-xr-x 3 root root 4096 Dec 21 08:47 minemeld/
The other directories, with the "crazy long numbers" are, on the system, the physical location of the container parts (best I can explain, still a beginner in containers). From there, the rest of the content above should fall into place. If not, I'm happy to help in this conversation or you can send me a DM.
From th
Finally got this working. My certificate issue was related to using a Windows CA. This article help me resolve the certificate problem (https://live.paloaltonetworks.com/t5/general-articles/how-to-generate-new-minemeld-https-cert/ta-p/1...).
Everything is working now as expected.
