topic Re: Device Log Processing Issues in Expedition Discussions

Device Log Processing Issues

Michael-Tatum — Wed, 18 Sep 2019 13:31:52 GMT

I am having an issue with M. Learning processing the logs files that I have uploaded for a specific device.

Looking through the forum, I have tried everything that seems to apply, but the logs are still not processing, it just sits there.

Is there a known limitation to the total size or number of logs it can process?

In this instance, I am processing roughly 1700 files equaling about 40 GB in size.

Re: Device Log Processing Issues

Michael-Tatum — Wed, 18 Sep 2019 19:33:49 GMT

I must have hit some kind of threshold - I reduced the number of files from 1700 to 500 and the machine learning process is now starting.

Re: Device Log Processing Issues

dgildelaig — Thu, 19 Sep 2019 09:51:54 GMT

It depends on the settings of your Expedition.

The process should be able to allocate a certain number of logs in batches, and this number of logs-per-batch is based on the amount of RAM and the total size of the logs.

This applies to the recent version of Expedition.

However, in your case, you may be facing problems in the number of logs, more than their size, to process at the same time. To be honest, we have not tested with such amount of files.