topic Re: Ansible playbook using panos_op "Missing required library." in Automation/API Discussions

Ansible playbook using panos_op "Missing required library."

rbolze — Mon, 14 Nov 2022 23:29:11 GMT

Trying the run an existing playbook via Github Actions and the error message is "Missing required library."

I cannot figure out which library could be missing.

The execution of the playbook is returning me (with -vvvv option in command line)

TASK [show list of all interfaces] ********************************************* task path: /opt/actions-runner/_work/playbooks/get_interface_logical.yaml:15 <XX.XX.XX.XX> ESTABLISH LOCAL CONNECTION FOR USER: runners <XX.XX.XX.XX> EXEC /bin/sh -c 'echo ~runners && sleep 0' <XX.XX.XX.XX> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/runners/.ansible/tmp `"&& mkdir "` echo /home/runners/.ansible/tmp/ansible-tmp-1668465228.5914114-3653-188195413439237 `" && echo ansible-tmp-1668465228.5914114-3653-188195413439237="` echo /home/runners/.ansible/tmp/ansible-tmp-1668465228.5914114-3653-188195413439237 `" ) && sleep 0' <FW> Attempting python interpreter discovery <XX.XX.XX.XX> EXEC /bin/sh -c 'echo PLATFORM; uname; echo FOUND; command -v '"'"'python3.10'"'"'; command -v '"'"'python3.9'"'"'; command -v '"'"'python3.8'"'"'; command -v '"'"'python3.7'"'"'; command -v '"'"'python3.6'"'"'; command -v '"'"'python3.5'"'"'; command -v '"'"'/usr/bin/python3'"'"'; command -v '"'"'/usr/libexec/platform-python'"'"'; command -v '"'"'python2.7'"'"'; command -v '"'"'/usr/bin/python'"'"'; command -v '"'"'python'"'"'; echo ENDFOUND && sleep 0' <XX.XX.XX.XX> EXEC /bin/sh -c '/opt/actions-runner/_work/_tool/Python/3.10.8/x64/bin/python3.10 && sleep 0' Using module file /home/runners/.ansible/collections/ansible_collections/paloaltonetworks/panos/plugins/modules/panos_op.py <XX.XX.XX.XX> PUT /home/runners/.ansible/tmp/ansible-local-3649no1gltbg/tmpduhjtz2w TO /home/runners/.ansible/tmp/ansible-tmp-1668465228.5914114-3653-188195413439237/AnsiballZ_panos_op.py <XX.XX.XX.XX> EXEC /bin/sh -c 'chmod u+x /home/runners/.ansible/tmp/ansible-tmp-1668465228.5914114-3653-188195413439237/ /home/runners/.ansible/tmp/ansible-tmp-1668465228.5914114-3653-188195413439237/AnsiballZ_panos_op.py && sleep 0' <XX.XX.XX.XX> EXEC /bin/sh -c '/usr/bin/python3 /home/runners/.ansible/tmp/ansible-tmp-1668465228.5914114-3653-188195413439237/AnsiballZ_panos_op.py && sleep 0' <XX.XX.XX.XX> EXEC /bin/sh -c 'rm -f -r /home/runners/.ansible/tmp/ansible-tmp-1668465228.5914114-3653-188195413439237/ > /dev/null 2>&1 && sleep 0' fatal: [FW]: FAILED! => { "ansible_facts": { "discovered_interpreter_python": "/usr/bin/python3" }, "changed": false, "invocation": { "module_args": { "api_key": null, "cmd": "show interface logical", "cmd_is_xml": false, "ignore_disconnect": null, "ip_address": null, "password": null, "port": 443, "provider": { "api_key": null, "ip_address": "XX.XX.XX.XX", "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", "port": 443, "serial_number": null, "username": "***" }, "username": "admin", "vsys": "vsys1" } }, "msg": "Missing required libraries." }

the exact definition of the playbook task from where the failure is coming:

- name: show list of all interfaces paloaltonetworks.panos.panos_op: provider: '{{ palo_provider }}' cmd: "show interface logical" register: show_interface_logical

of course, as specify in the docs of panos_op module , the 3 python librairies are installed on the Github runner via "pip install"

pan-python pandevice xmltodict

Extra info :

#> ansible --version

ansible-playbook [core 2.13.6] config file = None configured module search path = ['/home/runners/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /opt/actions-runner/_work/_tool/Python/3.10.8/x64/lib/python3.10/site-packages/ansible ansible collection location = /home/runners/.ansible/collections:/usr/share/ansible/collections executable location = /opt/actions-runner/_work/_tool/Python/3.10.8/x64/bin/ansible-playbook python version = 3.10.8 (main, Oct 18 2022, 06:43:21) [GCC 9.4.0] jinja version = 3.1.2 libyaml = True

#> pip list

Package Version ------------------ --------- ansible 6.6.0 ansible-core 2.13.6 ansible-pylibssh 1.0.0 ansicolors 1.1.8 bcrypt 4.0.1 certifi 2022.9.24 cffi 1.15.1 charset-normalizer 2.1.1 cryptography 38.0.3 future 0.18.2 idna 3.4 Jinja2 3.1.2 MarkupSafe 2.1.1 ntc-templates 3.1.0 numpy 1.23.4 packaging 21.3 pan-os-python 1.7.3 pan-python 0.17.0 pandas 1.5.1 pandevice 0.14.0 paramiko 2.12.0 pip 22.3.1 pycparser 2.21 PyNaCl 1.5.0 pyparsing 3.0.9 python-dateutil 2.8.2 python-dotenv 0.21.0 pytz 2022.6 PyYAML 6.0 requests 2.28.1 resolvelib 0.8.1 setuptools 63.2.0 six 1.16.0 termcolor 2.1.0 textfsm 1.1.3 urllib3 1.26.12 xmltodict 0.13.0

#> ansible-galaxy collection list

# /home/runners/.ansible/collections/ansible_collections Collection Version ---------------------- ------- ansible.netcommon 4.1.0 ansible.utils 2.7.0 cisco.ios 4.0.0 paloaltonetworks.panos 3.0.0 # /opt/actions-runner/_work/_tool/Python/3.10.8/x64/lib/python3.10/site-packages/ansible_collections Collection Version ----------------------------- ------- amazon.aws 3.5.0 ansible.netcommon 3.1.3 ansible.posix 1.4.0 ansible.utils 2.7.0 ansible.windows 1.12.0 arista.eos 5.0.1 awx.awx 21.8.0 azure.azcollection 1.14.0 check_point.mgmt 2.3.0 chocolatey.chocolatey 1.3.1 cisco.aci 2.3.0 cisco.asa 3.1.0 cisco.dnac 6.6.0 cisco.intersight 1.0.20 cisco.ios 3.3.2 cisco.iosxr 3.3.1 cisco.ise 2.5.8 cisco.meraki 2.11.0 cisco.mso 2.1.0 cisco.nso 1.0.3 cisco.nxos 3.2.0 cisco.ucs 1.8.0 cloud.common 2.1.2 cloudscale_ch.cloud 2.2.2 community.aws 3.6.0 community.azure 1.1.0 community.ciscosmb 1.0.5 community.crypto 2.8.1 community.digitalocean 1.22.0 community.dns 2.4.0 community.docker 2.7.1 community.fortios 1.0.0 community.general 5.8.0 community.google 1.0.0 community.grafana 1.5.3 community.hashi_vault 3.4.0 community.hrobot 1.6.0 community.libvirt 1.2.0 community.mongodb 1.4.2 community.mysql 3.5.1 community.network 4.0.1 community.okd 2.2.0 community.postgresql 2.3.0 community.proxysql 1.4.0 community.rabbitmq 1.2.3 community.routeros 2.3.1 community.sap 1.0.0 community.sap_libs 1.3.0 community.skydive 1.0.0 community.sops 1.4.1 community.vmware 2.10.1 community.windows 1.11.1 community.zabbix 1.8.0 containers.podman 1.9.4 cyberark.conjur 1.2.0 cyberark.pas 1.0.14 dellemc.enterprise_sonic 1.1.2 dellemc.openmanage 5.5.0 dellemc.os10 1.1.1 dellemc.os6 1.0.7 dellemc.os9 1.0.4 f5networks.f5_modules 1.20.0 fortinet.fortimanager 2.1.6 fortinet.fortios 2.1.7 frr.frr 2.0.0 gluster.gluster 1.0.2 google.cloud 1.0.2 hetzner.hcloud 1.8.2 hpe.nimble 1.1.4 ibm.qradar 2.1.0 ibm.spectrum_virtualize 1.10.0 infinidat.infinibox 1.3.7 infoblox.nios_modules 1.4.0 inspur.ispim 1.2.0 inspur.sm 2.3.0 junipernetworks.junos 3.1.0 kubernetes.core 2.3.2 lowlydba.sqlserver 1.0.4 mellanox.onyx 1.0.0 netapp.aws 21.7.0 netapp.azure 21.10.0 netapp.cloudmanager 21.21.0 netapp.elementsw 21.7.0 netapp.ontap 21.24.1 netapp.storagegrid 21.11.1 netapp.um_info 21.8.0 netapp_eseries.santricity 1.3.1 netbox.netbox 3.8.1 ngine_io.cloudstack 2.2.4 ngine_io.exoscale 1.0.0 ngine_io.vultr 1.1.2 openstack.cloud 1.10.0 openvswitch.openvswitch 2.1.0 ovirt.ovirt 2.3.1 purestorage.flasharray 1.14.0 purestorage.flashblade 1.10.0 purestorage.fusion 1.1.1 sensu.sensu_go 1.13.1 servicenow.servicenow 1.0.6 splunk.es 2.1.0 t_systems_mms.icinga_director 1.31.4 theforeman.foreman 3.7.0 vmware.vmware_rest 2.2.0 vultr.cloud 1.3.0 vyos.vyos 3.0.1 wti.remote 1.0.4

Re: Ansible playbook using panos_op "Missing required library."

gfreeman — Wed, 16 Nov 2022 02:25:57 GMT

As improbable as it seems, you're missing xmltodict, as the other module that panos_op loads is json which is a standard Python library. You are also potentially missing pan-os-python, but as the check for xmltodict happens before the check for pan-os-python does, it is being hidden and thus not showing you the sys.path that your Ansible environment is searching for installed libraries.

I recommend using almost any other module and seeing if it works, as if it fails, the panos collection will at least tell you what the sys.path is that your Ansible environment is looking for installed dependencies in:

- name: Check env panos_address_object: provider: '{{ provider }}' state: 'gathered' gathered_filter: '*'

Assuming that fails, one of the things it outputs is the sys.path. Use that and verify it is as expected, and that both xmltodict and pan-os-python are installed inside one of those directories.

Re: Ansible playbook using panos_op "Missing required library."

cremsburg — Wed, 16 Nov 2022 17:58:14 GMT

Hello @rbolze , Ansible struggles to execute the correct Python executable when you're working outside of the standard install path.

Your output shows a step where Ansible is looking for the Python path

<FW> Attempting python interpreter discovery

And ultimately settles on `/usr/bin/python3`

"ansible_facts": { "discovered_interpreter_python": "/usr/bin/python3" },

But I am willing to bet my lunch that you're actually wanting Ansible to look here for your Python environment:

/opt/actions-runner/_work/_tool/Python/3.10.8/x64/bin/python3.10

Quick way to validate the Python environment packages is to include a short task above your actual work:

tasks: - name: debug Python packages installed ansible.builtin.shell: pip freeze

If you find that you're expected packages aren't there (xmltodict, pan-os-python, etc.), then Ansible's Python discovery process failed you. A couple quick fixes available, but I recommend the following:

1. Create a file to store variables within, I typically default to `group_vars/all/python.yaml`

2. Store the following in the file and save:

ansible_python_interpreter: "{{ ansible_playbook_python }}"

When Ansible runs next, it will find this variable and skip any Python environment discovery process.

Please let us know the results,

Calvin

Re: Ansible playbook using panos_op "Missing required library."

rbolze — Wed, 16 Nov 2022 21:59:39 GMT

Thank you both @cremsburg and @gfreeman,
Indeed that was an issue with the python interpreter path.
so what I did to confirm this.
I have added in the ansible playbook the following tasks for debugging:

- name: Check env panos_facts: provider: '{{ palo_provider }}' gather_subset: ['config'] register: panfacts

the task "Check env" have provided me this message :

"msg": "Missing required library \"pan-python\".", "pypi": "https://pypi.org/project/pan-python", "syspath": [ "/tmp/ansible_panos_facts_payload_np_e2_4_/ansible_panos_facts_payload.zip", "/usr/lib/python38.zip", "/usr/lib/python3.8", "/usr/lib/python3.8/lib-dynload", "/usr/local/lib/python3.8/dist-packages", "/usr/lib/python3/dist-packages" ]

so, what is the solution I have chosen to fix this.

while executing the Github actions, I have set the environement variable :

ANSIBLE_PYTHON_INTERPRETER: /opt/actions-runner/_work/_tool/Python/3.10.8/x64/bin/python

another option is to set it via "ansible.cfg" file

[defaults] interpreter_python = /opt/actions-runner/_work/_tool/Python/3.10.8/x64/bin/python

Re: Ansible playbook using panos_op "Missing required library."

cremsburg — Wed, 16 Nov 2022 22:04:02 GMT

This is great, glad you got it sorted out.