Logs not appearing in WebUI (likely nginx configuration fault)

[TickBoxSec]

Let me preface this by saying that I'm awre i've introduced this fault through my own modifications (and lack of experience with nginx). That being said I'd appreciate and insight into how I've broken this function.

Symptom is that the 'LOGS' tab on the webUI displays nothing inside the frame other than the MM logo and "Loading...", likewise the "Logs" button in the upper right of any given node's detail exhibits this behaviour. 

And as to how I've introduced the issue -

I've deployed the jtschichold/minemeld-docker docker container with the following modification - 

The nginx runnning config /etc/nginx/sites-enabled/minemeld-web was modified to the following:

upstream app_server {
server 127.0.0.1:5000 fail_timeout=0;
}

server {
listen 4000;
server_name _;
keepalive_timeout 5;
# path for static files
root /opt/minemeld/www/current;
location = / {
index index.html;
}
location ~* \.html$ {
try_files $uri @proxy_to_app;
        expires -1;
    }

    # first files then proxy to flask app
    location / {
        try_files $uri @proxy_to_app;
        expires off;
    }

    # for SSE
    location /status/events {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header Connection '';
        proxy_http_version 1.1;
        proxy_redirect off;
        proxy_buffering off;
       chunked_transfer_encoding off;
        proxy_cache off;
        proxy_read_timeout 120s;

        expires -1;

        proxy_pass   http://app_server;
    }

    # for content that should be handled by mw flask app
    location @proxy_to_app {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;

        proxy_pass   http://app_server;
    }

the end result being that the WebUI would be exposed on port 4000 without TLS.

It's important to note at this point that if I'm to access the service at http://<minemeldserver>:4000 the log displays are all completely functional (i.e this issue is not present)

What introduces the fault is this - I have a second docker container which runs only nginx and embeded within that service's configuration/active-sites I have the following - 

server {
    listen 443 ssl;
    server_name minemeld.example.com;

    ssl on;
    ssl_certificate /cert/cert.pem;
    ssl_certificate_key /cert/key-no-password.pem;
    ssl_session_timeout 5m;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers HIGH:MEDIUM:!SSLv2:!PSK:!SRP:!ADH:!AECDH;
    ssl_prefer_server_ciphers on;

    location / {
        gzip off;
        proxy_set_header X-Forwarded-Ssl on;

        client_max_body_size 50M;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $proxy_x_forwarded_proto;
        proxy_set_header X-Frame-Options SAMEORIGIN;
        proxy_pass http://minemeld:4000;
    }
}

among other (unrelated) servers.

Presently my two ideas about why the logs break is that there's either a conflict with housing that second nginx instance within the minemeld container in addition to the seperate front-end nginx instance or that the issue is as a result of a self-signed certificate currently in use in this development environment.

I'd originally intended to competely disable the nginx instance inside the jtschichold/minemeld-docker container however this broke integration with flask and leaving there with TLS disabled seemed like a viable solution which has otherwise worked fine.

Any troubleshooting suggestions welcome thank you

[lmori]

Hi @TickBoxSec,

I think the problem could be related to LOGS being served to the client using server-sent-events.

A location stanza in the original config takes care of setting the correct parameters for the connection:

    # for SSE
    location /status/events {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header Connection '';
        proxy_http_version 1.1;
        proxy_redirect off;
        proxy_buffering off;
       chunked_transfer_encoding off;
        proxy_cache off;
        proxy_read_timeout 120s;

        expires -1;

        proxy_pass   http://app_server;
    }

Could you try replicating this location with these parameters inside the second nginx container ?

Thanks,

luigi

[TickBoxSecc]

I've made that change and it now shows "Scroll up for latest entries. Or click here No more lines available" on all the log screens, so seems to be *almost* working, Not sure which logs to check for the server events now.

[lmori]

Server events are passed to the WebUI using a long standing connection (server-sent events). You should check:

- the nginx access log

- the developer tools of the browser to see the flow of events coming into the WebUI. In Chrome: More Tools > Developer Tools - Network pane 

[TickBoxSecc]

There's something strange going on that may or may not be expected. I get a timeout on the local browser for a request eg. https://stix.example.com/status/events/query/954cb7be-9000-4a47-9f75-4fb37a2d88f7 is showin the developer console on network, then the first nginx on the perimeter shows 172.31.0.5 - - [17/Jul/2017:15:29:16 +0930] "GET /status/events/query/fb74749d-cdcb-4809-9563-021068b33700 HTTP/1.0" 200 22 "https://stix.example.com/" "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36" and the minemeld container gets this 10.34.3.34 - - [17/Jul/2017:15:29:16 +0930] "GET /status/events/query/fb74749d-cdcb-4809-9563-021068b33700 HTTP/1.1" 200 0 "https://stix.exmaple.com/" "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36" so for some reason the query ID is changing ?

[lmori]

Hi @TickBoxSecc,

you don't see any logs for 954cb7be-9000-4a47-9f75-4fb37a2d88f7  ?

luigi