Ansible playbook using panos_op "Missing required library."

[rbolze]

Trying the run an existing playbook via Github Actions and the error message is "Missing required library."

I cannot figure out which library could be missing.

The execution of the playbook  is returning me (with -vvvv option in command line)

TASK [show list of all interfaces] *********************************************
task path: /opt/actions-runner/_work/playbooks/get_interface_logical.yaml:15
<XX.XX.XX.XX> ESTABLISH LOCAL CONNECTION FOR USER: runners
<XX.XX.XX.XX> EXEC /bin/sh -c 'echo ~runners && sleep 0'
<XX.XX.XX.XX> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/runners/.ansible/tmp `"&& mkdir "` echo /home/runners/.ansible/tmp/ansible-tmp-1668465228.5914114-3653-188195413439237 `" && echo ansible-tmp-1668465228.5914114-3653-188195413439237="` echo /home/runners/.ansible/tmp/ansible-tmp-1668465228.5914114-3653-188195413439237 `" ) && sleep 0'
<FW> Attempting python interpreter discovery
<XX.XX.XX.XX> EXEC /bin/sh -c 'echo PLATFORM; uname; echo FOUND; command -v '"'"'python3.10'"'"'; command -v '"'"'python3.9'"'"'; command -v '"'"'python3.8'"'"'; command -v '"'"'python3.7'"'"'; command -v '"'"'python3.6'"'"'; command -v '"'"'python3.5'"'"'; command -v '"'"'/usr/bin/python3'"'"'; command -v '"'"'/usr/libexec/platform-python'"'"'; command -v '"'"'python2.7'"'"'; command -v '"'"'/usr/bin/python'"'"'; command -v '"'"'python'"'"'; echo ENDFOUND && sleep 0'
<XX.XX.XX.XX> EXEC /bin/sh -c '/opt/actions-runner/_work/_tool/Python/3.10.8/x64/bin/python3.10 && sleep 0'
Using module file /home/runners/.ansible/collections/ansible_collections/paloaltonetworks/panos/plugins/modules/panos_op.py
<XX.XX.XX.XX> PUT /home/runners/.ansible/tmp/ansible-local-3649no1gltbg/tmpduhjtz2w TO /home/runners/.ansible/tmp/ansible-tmp-1668465228.5914114-3653-188195413439237/AnsiballZ_panos_op.py
<XX.XX.XX.XX> EXEC /bin/sh -c 'chmod u+x /home/runners/.ansible/tmp/ansible-tmp-1668465228.5914114-3653-188195413439237/ /home/runners/.ansible/tmp/ansible-tmp-1668465228.5914114-3653-188195413439237/AnsiballZ_panos_op.py && sleep 0'
<XX.XX.XX.XX> EXEC /bin/sh -c '/usr/bin/python3 /home/runners/.ansible/tmp/ansible-tmp-1668465228.5914114-3653-188195413439237/AnsiballZ_panos_op.py && sleep 0'
<XX.XX.XX.XX> EXEC /bin/sh -c 'rm -f -r /home/runners/.ansible/tmp/ansible-tmp-1668465228.5914114-3653-188195413439237/ > /dev/null 2>&1 && sleep 0'
fatal: [FW]: FAILED! => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "invocation": {
        "module_args": {
            "api_key": null,
            "cmd": "show interface logical",
            "cmd_is_xml": false,
            "ignore_disconnect": null,
            "ip_address": null,
            "password": null,
            "port": 443,
            "provider": {
                "api_key": null,
                "ip_address": "XX.XX.XX.XX",
                "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
                "port": 443,
                "serial_number": null,
                "username": "***"
            },
            "username": "admin",
            "vsys": "vsys1"
        }
    },
    "msg": "Missing required libraries."
}

the exact definition of the playbook task from where the failure is coming: 

- name: show list of all interfaces
  paloaltonetworks.panos.panos_op:
     provider: '{{ palo_provider }}'
     cmd: "show interface logical"
  register: show_interface_logical

of course, as specify in the docs of panos_op module , the 3 python librairies are installed on the Github runner via "pip install"

pan-python
pandevice
xmltodict

Extra info :

 #> ansible --version

ansible-playbook [core 2.13.6]
  config file = None
  configured module search path = ['/home/runners/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/actions-runner/_work/_tool/Python/3.10.8/x64/lib/python3.10/site-packages/ansible
  ansible collection location = /home/runners/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/actions-runner/_work/_tool/Python/3.10.8/x64/bin/ansible-playbook
  python version = 3.10.8 (main, Oct 18 2022, 06:43:21) [GCC 9.4.0]
  jinja version = 3.1.2
  libyaml = True

#> pip list

Package            Version
------------------ ---------
ansible            6.6.0
ansible-core       2.13.6
ansible-pylibssh   1.0.0
ansicolors         1.1.8
bcrypt             4.0.1
certifi            2022.9.24
cffi               1.15.1
charset-normalizer 2.1.1
cryptography       38.0.3
future             0.18.2
idna               3.4
Jinja2             3.1.2
MarkupSafe         2.1.1
ntc-templates      3.1.0
numpy              1.23.4
packaging          21.3
pan-os-python      1.7.3
pan-python         0.17.0
pandas             1.5.1
pandevice          0.14.0
paramiko           2.12.0
pip                22.3.1
pycparser          2.21
PyNaCl             1.5.0
pyparsing          3.0.9
python-dateutil    2.8.2
python-dotenv      0.21.0
pytz               2022.6
PyYAML             6.0
requests           2.28.1
resolvelib         0.8.1
setuptools         63.2.0
six                1.16.0
termcolor          2.1.0
textfsm            1.1.3
urllib3            1.26.12
xmltodict          0.13.0

 #> ansible-galaxy collection list

# /home/runners/.ansible/collections/ansible_collections
Collection             Version
---------------------- -------
ansible.netcommon      4.1.0  
ansible.utils          2.7.0  
cisco.ios              4.0.0  
paloaltonetworks.panos 3.0.0  

# /opt/actions-runner/_work/_tool/Python/3.10.8/x64/lib/python3.10/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    3.5.0  
ansible.netcommon             3.1.3  
ansible.posix                 1.4.0  
ansible.utils                 2.7.0  
ansible.windows               1.12.0 
arista.eos                    5.0.1  
awx.awx                       21.8.0 
azure.azcollection            1.14.0 
check_point.mgmt              2.3.0  
chocolatey.chocolatey         1.3.1  
cisco.aci                     2.3.0  
cisco.asa                     3.1.0  
cisco.dnac                    6.6.0  
cisco.intersight              1.0.20 
cisco.ios                     3.3.2  
cisco.iosxr                   3.3.1  
cisco.ise                     2.5.8  
cisco.meraki                  2.11.0 
cisco.mso                     2.1.0  
cisco.nso                     1.0.3  
cisco.nxos                    3.2.0  
cisco.ucs                     1.8.0  
cloud.common                  2.1.2  
cloudscale_ch.cloud           2.2.2  
community.aws                 3.6.0  
community.azure               1.1.0  
community.ciscosmb            1.0.5  
community.crypto              2.8.1  
community.digitalocean        1.22.0 
community.dns                 2.4.0  
community.docker              2.7.1  
community.fortios             1.0.0  
community.general             5.8.0  
community.google              1.0.0  
community.grafana             1.5.3  
community.hashi_vault         3.4.0  
community.hrobot              1.6.0  
community.libvirt             1.2.0  
community.mongodb             1.4.2  
community.mysql               3.5.1  
community.network             4.0.1  
community.okd                 2.2.0  
community.postgresql          2.3.0  
community.proxysql            1.4.0  
community.rabbitmq            1.2.3  
community.routeros            2.3.1  
community.sap                 1.0.0  
community.sap_libs            1.3.0  
community.skydive             1.0.0  
community.sops                1.4.1  
community.vmware              2.10.1 
community.windows             1.11.1 
community.zabbix              1.8.0  
containers.podman             1.9.4  
cyberark.conjur               1.2.0  
cyberark.pas                  1.0.14 
dellemc.enterprise_sonic      1.1.2  
dellemc.openmanage            5.5.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.20.0 
fortinet.fortimanager         2.1.6  
fortinet.fortios              2.1.7  
frr.frr                       2.0.0  
gluster.gluster               1.0.2  
google.cloud                  1.0.2  
hetzner.hcloud                1.8.2  
hpe.nimble                    1.1.4  
ibm.qradar                    2.1.0  
ibm.spectrum_virtualize       1.10.0 
infinidat.infinibox           1.3.7  
infoblox.nios_modules         1.4.0  
inspur.ispim                  1.2.0  
inspur.sm                     2.3.0  
junipernetworks.junos         3.1.0  
kubernetes.core               2.3.2  
lowlydba.sqlserver            1.0.4  
mellanox.onyx                 1.0.0  
netapp.aws                    21.7.0 
netapp.azure                  21.10.0
netapp.cloudmanager           21.21.0
netapp.elementsw              21.7.0 
netapp.ontap                  21.24.1
netapp.storagegrid            21.11.1
netapp.um_info                21.8.0 
netapp_eseries.santricity     1.3.1  
netbox.netbox                 3.8.1  
ngine_io.cloudstack           2.2.4  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.2  
openstack.cloud               1.10.0 
openvswitch.openvswitch       2.1.0  
ovirt.ovirt                   2.3.1  
purestorage.flasharray        1.14.0 
purestorage.flashblade        1.10.0 
purestorage.fusion            1.1.1  
sensu.sensu_go                1.13.1 
servicenow.servicenow         1.0.6  
splunk.es                     2.1.0  
t_systems_mms.icinga_director 1.31.4 
theforeman.foreman            3.7.0  
vmware.vmware_rest            2.2.0  
vultr.cloud                   1.3.0  
vyos.vyos                     3.0.1  
wti.remote                    1.0.4  

[gfreeman · Accepted solution]

As improbable as it seems, you're missing xmltodict, as the other module that panos_op loads is json which is a standard Python library.  You are also potentially missing pan-os-python, but as the check for xmltodict happens before the check for pan-os-python does, it is being hidden and thus not showing you the sys.path that your Ansible environment is searching for installed libraries.

I recommend using almost any other module and seeing if it works, as if it fails, the panos collection will at least tell you what the sys.path is that your Ansible environment is looking for installed dependencies in:

- name: Check env
  panos_address_object:
    provider: '{{ provider }}'
    state: 'gathered'
    gathered_filter: '*'

Assuming that fails, one of the things it outputs is the sys.path.  Use that and verify it is as expected, and that both xmltodict and pan-os-python are installed inside one of those directories.

[cremsburg]

Hello @rbolze , Ansible struggles to execute the correct Python executable when you're working outside of the standard install path.

Your output shows a step where Ansible is looking for the Python path

<FW> Attempting python interpreter discovery

 And ultimately settles on `/usr/bin/python3`

    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },

But I am willing to bet my lunch that you're actually wanting Ansible to look here for your Python environment:

/opt/actions-runner/_work/_tool/Python/3.10.8/x64/bin/python3.10

Quick way to validate the Python environment packages is to include a short task above your actual work:

  tasks:
    - name: debug Python packages installed
      ansible.builtin.shell: pip freeze

If you find that you're expected packages aren't there (xmltodict, pan-os-python, etc.), then Ansible's Python discovery process failed you. A couple quick fixes available, but I recommend the following:

1. Create a file to store variables within, I typically default to `group_vars/all/python.yaml`

2. Store the following in the file and save:

ansible_python_interpreter: "{{ ansible_playbook_python }}"

When Ansible runs next, it will find this variable and skip any Python environment discovery process.

Please let us know the results,

Calvin

[rbolze · Accepted solution]

Thank you both @cremsburg and @gfreeman,
Indeed that was an issue with the python interpreter path.
so what I did to confirm this.
I have added in the ansible playbook the following tasks for debugging:

    - name: Check env
      panos_facts:
        provider: '{{ palo_provider }}'
        gather_subset: ['config']
      register: panfacts

the task "Check env" have provided me this message :

 "msg": "Missing required library \"pan-python\".",
    "pypi": "https://pypi.org/project/pan-python",
    "syspath": [
        "/tmp/ansible_panos_facts_payload_np_e2_4_/ansible_panos_facts_payload.zip",
        "/usr/lib/python38.zip",
        "/usr/lib/python3.8",
        "/usr/lib/python3.8/lib-dynload",
        "/usr/local/lib/python3.8/dist-packages",
        "/usr/lib/python3/dist-packages"
    ]

 so, what is the solution I have chosen to fix this.

while executing the Github actions, I have set the environement variable :

ANSIBLE_PYTHON_INTERPRETER: /opt/actions-runner/_work/_tool/Python/3.10.8/x64/bin/python

another option is to set it via "ansible.cfg" file

[defaults]
interpreter_python = /opt/actions-runner/_work/_tool/Python/3.10.8/x64/bin/python

[cremsburg]

This is great, glad you got it sorted out.