'ascii' codec can't encode characters

L2 Linker

'ascii' codec can't encode characters



0.9.26 is a great update - thanks for the hard work. Already using the CIF miner and enjoying the live UI updates.


I'm seeing a a bunch of these errors in my engine log:



2016-10-28T21:31:15 (30348)amqp._callback ERROR: Exception in handling update on topic proc-URL-name-HighConfidence with params {u'source': u'proc-URL-name-HighConfidence', u'indicator': u'http://steponnopets.com.sg/acct/identity.php/?amp=&processing_transaction_ids\xc3\xa3\xc2\xa4a0bf64c8ba839a7a7015=&sessargs=g_5rxbrvaxu_bmhpswnni8z2g3zho1ox', u'value': {u'confidence': 85, u'cif_group': [u'general.name'], u'cif_description': u'wells fargo personal banking', u'cif_tlp': u'privileged', u'share_level': u'green', u'cif_lasttime': u'2016-10-22T22:20:30Z', u'sources': [u'bl-URL-name-Phishing'], u'cif_provider': u'apwg.org', u'cif_altid_tlp': u'restricted', u'cif_tags': [u'phishing'], u'first_seen': 1477704643118L, u'type': u'URL', u'cif_reporttime': u'2016-10-22T22:24:07Z', u'last_seen': 1477704643118L}}
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.26/local/lib/python2.7/site-packages/minemeld/comm/amqp.py", line 358, in _callback
File "/opt/minemeld/engine/0.9.26/local/lib/python2.7/site-packages/minemeld/ft/base.py", line 121, in _counter
f(self, *args, **kwargs)
File "/opt/minemeld/engine/0.9.26/local/lib/python2.7/site-packages/minemeld/ft/base.py", line 494, in update
File "/opt/minemeld/engine/0.9.26/local/lib/python2.7/site-packages/minemeld/ft/base.py", line 121, in _counter
f(self, *args, **kwargs)
File "/opt/minemeld/engine/0.9.26/local/lib/python2.7/site-packages/minemeld/ft/taxii.py", line 920, in filtered_update
self._add_indicator(now, indicator, value)
File "/opt/minemeld/engine/0.9.26/local/lib/python2.7/site-packages/minemeld/ft/taxii.py", line 842, in _add_indicator
UnicodeEncodeError: 'ascii' codec can't encode characters in position 77-80: ordinal not in range(128)



The Processor seems to be working and just skipping over these invalid entries, so that's good. This is a new CIF added feed.


Just wanted to report it.


Thanks again,


L7 Applicator

Re: 'ascii' codec can't encode characters

Hi @DanWoodruff,

thanks for reporting this, I think the problem is related to the non-ascii character in the URL ('\xc3...'). I have opened bug minemeld-core#77 (https://github.com/PaloAltoNetworks/minemeld-core/issues/77) to track this issue.


I am planning to have it fixed by the next release.


Thanks !


Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!