David Montgomery | 20 Jun 22:46 2016
Picon

uwsgi python module - now to get fixed id for logging per proccess



Hi,

I am using both python logging and set_logvar to log extra vars.

So  for the filename, I would one process per file to be consistent on restarts

Kinda like this -> upload-8030-stdout---supervisor-H0_GkI.log

In python..how so I get "H0_GkI" portion so I can use it too?

then handler = RotatingFileHandler(filename='/tmp/upload_server_bottle-H0_GkI.log', maxBytes=1024 * 1024 * 10,backupCount=1)

Thanks
_______________________________________________
uWSGI mailing list
uWSGI@...
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Alexander Demenshin | 15 Jun 19:51 2016
Picon

Filtering/preprocessing requests to workers before they are dispatched

Hi,

I am looking for a way to filter requests in master process,
or anywhere elese *before* the request is dispatched to any
worker.

One application for this - throttling of request rate based
ony various criterias (controlled by request handlers),
to avoid consuming workers when request must be delayed/rejected.

Something like uwsgi::preprocess(\&filter) where filter()
receives regular PSGI environment and has a chance to either
pass a request to worker (probably modified) or respond on its own.

Is there anything in uwsgi that could be used for this purpose?

--

-- 
Best regards,
Alexander.
Alexander Demenshin | 13 Jun 18:16 2016
Picon

cache_set/cache_update behavior with existing keys

Hi,

Intuitively, "set" must always set the value, while "update" - only 
updates existing
entry (if it exists), but current behavior is exactly the opposite.

Currently, cache_set() in psgi just calls uwsgi_cache_set2() with no 
flags set,
and this leads to counter-intuitive result - if key is already in the 
cache,
it will be untouched.

On the other hand, cache_update() (if it would exist in psgi and set 
update flag),
will always set the key - regardless of its existence (assuming that 
flag is set).

Is this intentional behavior?

--

-- 
Best regards,
Alexander.
Alexander Demenshin | 11 Jun 19:29 2016
Picon

Question about serving static files (how to return 404 code instead of passing request to application)

Hi,

Perhaps, my question is a little bit stupid, but I could not find an 
easy way to send 404 response when serving static files.

Using static-map hands over request to application if file does not 
exist, while using route add a bit of overhead due to URI matching.

What I basically want is something like:

static-map-only = /s=/static/files

which will *only* serve static files from defined path and return 404 
when file is not there and 403 on permission denied.

So far, I use this super-inelegant solution:

static-dir = /home/www/static
route = ^/s/ goto:static

route-label = static
route = ^/s/(.*) addvar:FILE=$1
route-if = isfile:%(static-dir)/${FILE} static:%(static-dir)/${FILE}
route = . return:404

but I refuse to believe that there is no other (easy) way :)

Any suggestions?

--

-- 
Best regards,
Alexander.
Alexander Demenshin | 10 Jun 15:44 2016
Picon

cache LRU eviction and expires parameter

Hi,

Is there any specific reason to ignore expires parameter with enabled 
LRU eviction?

IMO, it still may be useful - to catch the case when cache becomes 
(suddenly) full.

--

-- 
Best regards,
Alexander.
Alexander Demenshin | 6 Jun 20:51 2016
Picon

couple issues with mules & psgi

Hi,

I have an psgi app and I want to have a mule which will perform periodic 
housekeeping.

Straightforward approach, like:

$ uwsgi --http-socket :8080 -M --mule=mule.pl --psgi=psgi.pl

in general does what I want, but... the problem is that 
postfork()/atexit() hooks in psgi.pl are executed also for mule.pl:

spawned uWSGI master process (pid: 10371)
spawned uWSGI worker 1 (pid: 10372, cores: 1)
spawned uWSGI mule 1 (pid: 10373)
[10372]: postfork(1)
[10373]: postfork(0)
10373: Mule loaded
...
^CSIGINT/SIGQUIT received...killing workers...
Signal SIGINT received, but no signal handler set.
[10373]: atexit(0)
[10372]: atexit(1)

uwsgi::postfork()/atexit() are called only in psgi.pl, but both are 
executed also in mule.pl (value in parens is worker_id)

Is this "behavior by design" or it should not be like this (I use uwsgi 
2.0.13.1 on CentOS 7)?

I would expect that mule.pl (and any mule which is explicitly named) 
would be forked/started independently from workers.

Besides, when starting a (perl) mule, Devel::StackTrace is referenced 
but not use(ed), thus preventing stack trace generation on exceptions:

[10456]: postfork(1)
[10457]: postfork(0)
10457: Mule loaded
Can't locate object method "new" via package "Devel::StackTrace" 
(perhaps you forgot to load "Devel::StackTrace"?) at (eval 14) line 1.
[10457]: atexit(0)

this happens when mule.pl dies on any error - 1/0 in this case - which 
is invisible unless I "use Devel::StackTrace" in mule.pl

Thank you for listening :)

--

-- 
Best regards,
Alexander.
Larry Martell | 6 Jun 19:01 2016
Picon
Gravatar

django1.6 with uwsgi and nginx on RHEL 7

I have successfully deployed django1.6 with uwsgi and nginx on RHEL6
but I cannot seem to get it working on RHEL7.

I get Internal Server Error in the browser, and this in the uwsgi log:

--- no python application found, check your startup logs for errors ---
[pid: 10582|app: -1|req: -1/2] xx.xx.xx.xx () {46 vars in 803 bytes}
[Mon Jun  6 11:58:50 2016] GET /favicon.ico => generated 21 bytes in 0
msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)

I have tried everything I can think of. Where am I going wrong here?

Here are my files:

nginx config:

upstream django {
    server unix:///projects/elucid/elucid.sock; # for a file socket
}

# configuration of the server
server {
    # the port your site will be served on
    listen      9004;
    # the domain name it will serve for
    server_name foo.bar.com;
    charset     utf-8;

    # max upload size
    client_max_body_size 75M;   # adjust to taste

    # Django media
    location /media  {
        alias /projects/elucid/elucid/media;
    }

    location /static {
        alias /projects/elucid/static;
    }

    # Finally, send all non-media requests to the Django server.
    location / {
        uwsgi_pass  django;
        include     /projects/elucid/elucid/uwsgi_params;
    }
}

/etc/uwsgi/sites/elucid_uwsgi.ini:

# mysite_uwsgi.ini file
[uwsgi]

# Django-related settings
# the base directory (full path)
chdir           = /projects/elucid
# Django's wsgi file
module          = elucid.wsgi
# the virtualenv (full path)
# home            = /

# process-related settings
# master
master          = true
# maximum number of worker processes
processes       = 10
# the socket (use the full path to be safe
socket          = /projects/elucid/elucid.sock
# ... with appropriate permissions - may be needed
chmod-socket    = 666
# clear environment on exit
vacuum          = true
logto           = /var/log/uwsgi/error.log

/etc/systemd/system/uwsgi.service:

[Unit]
Description=uWSGI Emperor service

[Service]
ExecStartPre=/usr/bin/bash -c 'mkdir -p /run/uwsgi; chown nginx /run/uwsgi'
ExecStart=/usr/bin/uwsgi --emperor /etc/uwsgi/sites
Restart=always
KillSignal=SIGQUIT
Type=notify
NotifyAccess=all

[Install]
WantedBy=multi-user.target

wsgi.py:

import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "elucid.settings")
application = get_wsgi_application()
Aleks | 2 Jun 09:32 2016
Picon

Configuration over etcd/console/.. similar to LDAP

Hi all.

are there any plans to have the option to configure uwsgi with some of 
the key/value server out there?

Something similar like ldap.

http://uwsgi-docs.readthedocs.io/en/latest/LDAP.html

I have seen this for example in traefik
https://github.com/containous/traefik

Best regards
Aleks
Brian Zambrano | 1 Jun 21:15 2016
Picon
Gravatar

Routing to multiple apps on same host

I'm looking at setting up several Docker containers running uwsgi on a single host (development systems).  My idea is to have the http-router handle all incoming requests and then use the subscription server for routing.  The actual containers are running as RPC-style microservices.

The http-router is really simple:

[uwsgi]
http-subscription-server=0.0.0.0:2626
master=true

Each service then subscribes with a unique key:

subscribe2 = server=127.0.0.1:2626,key=cc_auth_service

I can get this working by sending in the HOST header to the name of my service/key above. That works but seems a little hacky.  I don't understand how to match on anything other than the $HOST for "key".  It is possible for the services to match on a special header rather than the host header..for example, X-CC-SERVICE-NAME?

Thanks,
BZ
_______________________________________________
uWSGI mailing list
uWSGI@...
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Alex Hall | 1 Jun 19:06 2016

database query errors not being logged?

Hello list,
I have a Flask app being hosted by Nginx through UWSGI. Usually, UWSGI does the error handling, printing exceptions to the app log with no problem. Recently, though, I've noticed that my webpage's ajax call will go unanswered, almost certainly due to an error in an SQL query (run through pyodbc). I see the print statements from my app in the UWSGI log, but no longer the exceptions. The page sits there and does nothing, the log has nothing out of the ordinary, the Nginx error log has nothing, and, in short, it's as though no problem happened at all. I know one did, though, because my page has no response. What might I be missing? Thanks.

--
Alex Hall
Automatic Distributors, IT department
ahall-vRNBg8LEEi9Wk0Htik3J/w@public.gmane.org
_______________________________________________
uWSGI mailing list
uWSGI@...
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Harry Percival | 25 May 14:09 2016

vassal config: set per-worker cgroups?

Hi there,

We're trying to apply some memory limitations to apps that may contain a variable number of workers.

We know we can set

  cgroup = /mnt/cgroups/memory/domains/vassal-domain-name
  cgroup-opt = memory.limit_in_bytes=1800M
  cgroup-opt = memory.memsw.limit_in_bytes=1800M

And that's fine for limiting a user with one worker to 1.8GB of RAM, but the problem is that if a user has 10 workers, the cgroup limits the *total* memory usage to 1.8GB, which is now down to less than 200Mb per worker, and that's too little.   We also don't want to just bump that user up to an 18GB limit, since that's potentially higher than the system ram, and besides, this is mainly about catching accidental memory leaks in individual processes rather than limiting total consumption.  So the question is:

* Is there any way to put each worker in its own cgroup? *

I could imagine it working if uwsgi had some magical template variable syntax, a bit like nginx?  eg:

  cgroup = /mnt/cgroups/memory/domains/vassal-domain-name/worker-$WORKER_ID

Is there any such thing?

Or perhaps there would be a way of running a script, pre-jail put post-fork, that ran as root/privileged user, that would be able to identify the spawned worker processes and manually put them into their own cgroups?   Could exec-post-jail or exec-pre-app work for this?  Would there be any way for such a job to identify the worker process ids?

thanks,
Harry

-- Harry Percival Developer harry-7DvliJ76plxWUWZDIUQIQ9BPR1lH4CV8@public.gmane.org PythonAnywhere - a fully browser-based Python development and hosting environment <http://www.pythonanywhere.com/> PythonAnywhere LLP 17a Clerkenwell Road, London EC1M 5RD, UK VAT No.: GB 893 5643 79 Registered in England and Wales as company number OC378414. Registered address: 28 Ely Place, 3rd Floor, London EC1N 6TD, UK
_______________________________________________
uWSGI mailing list
uWSGI@...
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi

Gmane