headers
C Anthony Risinger | 21 May 2013 05:59
Gravatar

Re: Access to the config options at the global scope in uWSGI app

On Mon, May 20, 2013 at 1:26 PM, dracek mracek <dracekm <at> gmail.com> wrote:
Hello,

I would like to ask if it's possible to access to config values defined in application config (ie .ini). I want mysql db connection was specified in the config but didn't find how to access to them in Python (using uWSGI 1.9.10, Debian)

So far I've gone through dozens of values but nothing seems to fit configuration.
import uwsgi
len(uwsgi)

you can use `--get` for external access, but within python you can access from the `uwsgi` module... i forget the exact attr off the top of my head, but try:

>>> import uwsgi
>>> help(uwsgi)

...instead of `len` and methinks it will all pan out from there ;)

--

C Anthony 
_______________________________________________
uWSGI mailing list
uWSGI@...
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Permalink | Reply | 1 comment |
headers
Jon Chappell | 21 May 2013 01:16
Picon
Gravatar

uwsgi emperor with multiple python versions

Hi,

We are trying to run uwsgi with multiple python versions and are currently compiling from source, but have also tried installation via pip.  We're on Ubuntu 10.04 which has Python 2.6 as the default Python version, but need to run a mix of 2.6 and 2.7 applications that should be managed by the emperor.  The 2.6 application works fine, but the 2.7 application gets an error when the emperor tries to start the workers.  The Python 2.6 and 2.7 plugins were both compiled also.

The init script being used for uwsgi is:
description "uWSGI"
start on runlevel [2345]
stop on runlevel [06]

respawn

exec uwsgi --logto /var/log/uwsgi/emperor.log --emperor /etc/uwsgi/conf.d/

The ini file in question:

[uwsgi]

socket=/tmp/uwsgi.sock
chmod-socket=660
pythonpath=/var/www/deployment
module=wsgi:application
master=True
pidfile=/tmp/deployment-master.pid
uid=uwsgi
gid=fm
harakiri=30
max-requests=5000
post-buffering=4096
logto=/var/log/uwsgi/deployment.log
harakiri-verbose=false
buffer-size=32768
virtualenv=/var/virtualenvs/deployment
plugin=python27
stats=:1717
processes=4



Here is the error:

Python version: 2.6.5 (r265:79063, Oct  1 2012, 22:16:31)  [GCC 4.4.3]
Set PythonHome to /var/virtualenvs/deployment
'import site' failed; use -v for traceback
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0xc55c50
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 547800 bytes (534 KB) for 4 cores
*** Operational MODE: preforking ***
added /var/www/deployment/ to pythonpath.
Traceback (most recent call last):
  File "/var/www/deployment/wsgi.py", line 2, in <module>
    import os
ImportError: No module named os
unable to load app 0 (mountpoint='') (callable not found or import error)

The virtualenv path has the correct python 2.7 binary installed in it, and has all of the appropriate directory structure that a properly configured virtualenv requires.  Indeed, if the virtualenv is changed to use python 2.6 (also removing the plugin from the ini file), uwsgi serves it up correctly as well.

--

Jon Chappell 
_______________________________________________
uWSGI mailing list
uWSGI@...
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Permalink | Reply | 1 comment |
headers
Andrew Knapp | 20 May 2013 23:51
Gravatar

Problem with the listen queue size

Hello,

During some recent digging around in our uwsgi logs, I noticed that we often get a few of these messages when we rollout code and restart the workers:

"*** uWSGI listen queue of socket 3 full !!! (101/100) ***"

Looking around, it became clear that we needed to up the listen queue size in our config, and I did, setting it to 1024 (and also upping the somaxconn sysctl variable to 5120 on our machines). Now, when the workers start up, they say they have a queue of 1024:

"your server socket listen backlog is limited to 1024 connections"

Unfortunately, even though it prints that, it doesn't seem to be taking that value correctly. We still get error messages exactly like the one above:

"*** uWSGI listen queue of socket 3 full !!! (101/100) ***"

I'm guessing the second part of "101/100" should be 1024, but it's not showing up. I should also note that we are running this app in a emperor setup. Is there something else we need to tune here?

Version of uWSGI: 1.2
Config file (I've hidden some variables, but it's mostly unchanged):
[uwsgi]
    uid = user
    gid = group
    socket = x.x.x.x:xxxx
    listen = 1024
    workers = 32
    max-requests = 20000
    harakiri = 60
    master = true
    single-interpreter = true
    virtualenv = env
    pythonpath = pythonpath
    plugin = python
    module = wsgi
    pidfile2 = /uwsgi.pid
    logto2 = /uwsgi.log
    logfile-chmod = 644
    stats = x.x.x.x:xxxxx
    post-buffering = 65536
    buffer-size = 32768
    disable-logging = true
    chdir = /builds

Let me know if anyone needs anymore info. Thanks for your help!

-Andy
_______________________________________________
uWSGI mailing list
uWSGI@...
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Permalink | Reply | 0 comments |
headers
Roberto De Ioris | 20 May 2013 20:35
Picon

Re: Access to the config options at the global scope in uWSGI app


> Hello,
>
> I would like to ask if it's possible to access to config values defined in
> application config (ie .ini). I want mysql db connection was specified in
> the config but didn't find how to access to them in Python (using uWSGI
> 1.9.10, Debian)
>
> So far I've gone through dozens of values but nothing seems to fit
> configuration.
> import uwsgi
> len(uwsgi)

uwsgi.opt is a dictionary exporting all of the parsed options

--

-- 
Roberto De Ioris
http://unbit.it
Permalink | Reply | 1 comment |
headers
Łukasz Mierzwa | 20 May 2013 20:31
Picon
Gravatar

Re: Access to the config options at the global scope in uWSGI app

can't you use --env for that? seems to be The Most Obvious Way for this kind of things


2013/5/20 dracek mracek <dracekm-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hello,

I would like to ask if it's possible to access to config values defined in application config (ie .ini). I want mysql db connection was specified in the config but didn't find how to access to them in Python (using uWSGI 1.9.10, Debian)

So far I've gone through dozens of values but nothing seems to fit configuration.
import uwsgi
len(uwsgi)

I also tried to build the uWSGI binary with using option embed_config in buildconf/base.ini but building didn't succeed.

Before using ConfigParser in Python I'd like to check if there is an easier solution in uWSGI :)

Thanks for any advice,
-- Tom

_______________________________________________
uWSGI mailing list
uWSGI-FfzAktRlpg7/cILp9QSAqw@public.gmane.org
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi




--
Łukasz Mierzwa 
_______________________________________________
uWSGI mailing list
uWSGI@...
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Permalink | Reply | 0 comments |
headers
dracek mracek | 20 May 2013 20:26
Picon

Access to the config options at the global scope in uWSGI app

Hello,

I would like to ask if it's possible to access to config values defined in application config (ie .ini). I want mysql db connection was specified in the config but didn't find how to access to them in Python (using uWSGI 1.9.10, Debian)

So far I've gone through dozens of values but nothing seems to fit configuration.
import uwsgi
len(uwsgi)

I also tried to build the uWSGI binary with using option embed_config in buildconf/base.ini but building didn't succeed.

Before using ConfigParser in Python I'd like to check if there is an easier solution in uWSGI :)

Thanks for any advice,
-- Tom
_______________________________________________
uWSGI mailing list
uWSGI@...
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Permalink | Reply | 0 comments |
headers
Daniel Kopka | 20 May 2013 19:02
Picon
Gravatar

Django - SMTP

Hi,

When I try to send email from Django 1.2 via uWSGI in log I receives (uwsgi 1.9.10):

uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 248]
IOError: write error
*** HARAKIRI ON WORKER 6 (pid: 30863, try: 1) ***
HARAKIRI: -- syscall> 42 0x8 0x7fff3d23f710 0x10 0xe 0x1 0x100 0x7fff3d23f6b0 0x7fe481bcb8ad
HARAKIRI: -- wchan> inet_stream_connect
*** backtrace of 30863 ***
xx_uWSGI worker 6(uwsgi_backtrace+0x29) [0x451b49]
xx_uWSGI worker 6(what_i_am_doing+0x17) [0x451fc7]
/lib64/libc.so.6(+0x32920) [0x7fe480095920]
/lib64/libpthread.so.0(__connect+0x2b) [0x7fe481bcb8ab]
/srv/venvs/xx/lib64/python2.6/lib-dynload/_socketmodule.so(+0x6018) [0x7fe47de2c018]
/srv/venvs/xx/lib64/python2.6/lib-dynload/_socketmodule.so(+0x809c) [0x7fe47de2e09c]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x4e26) [0x7fe4807025c6]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927) [0x7fe480704657]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304) [0x7fe480702aa4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927) [0x7fe480704657]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304) [0x7fe480702aa4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef) [0x7fe480703b8f]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927) [0x7fe480704657]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304) [0x7fe480702aa4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927) [0x7fe480704657]
/usr/lib64/libpython2.6.so.1.0(+0x6adad) [0x7fe480697dad]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53) [0x7fe480670c63]
/usr/lib64/libpython2.6.so.1.0(+0x566af) [0x7fe4806836af]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53) [0x7fe480670c63]
/usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43) [0x7fe4806fcc93]
/usr/lib64/libpython2.6.so.1.0(PyInstance_New+0x76) [0x7fe480682e06]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53) [0x7fe480670c63]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x47d4) [0x7fe480701f74]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef) [0x7fe480703b8f]
*** end of backtrace ***
HARAKIRI: --- uWSGI worker 6 (pid: 30863) WAS managing request /accounts/register/ since Mon May 20 18:53:54 2013 ---
*** HARAKIRI ON WORKER 6 (pid: 30863, try: 2) ***
HARAKIRI: -- wchan> do_exit
DAMN ! worker 6 (pid: 30863) died, killed by signal 9 :( trying respawn ...
Respawned uWSGI worker 6 (new pid: 30999)


Any ideas what is wrong or where to look for the error?



--
Pozdrawiam,
Daniel Kopka
Tel: 502276071
_______________________________________________
uWSGI mailing list
uWSGI@...
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Permalink | Reply | 4 comments |
headers
Riccardo Magliocchetti | 20 May 2013 12:34
Picon

handling timers / cron jobs dynamically

Hello,

i need to handle cron jobs dynamically which means adding cron jobs but 
also removing them. Since the uwsgi python plugin api does not expose a 
way to remove them the suggested solution would be to just add a timer 
every minute or so and check the cron jobs from within the app? Another 
option is calling uwsgi.reload() everytime a job change and add all the 
cron jobs i need but i don't like that much.

thanks in advance,
riccardo
Permalink | Reply | 2 comments |
headers
Tony Lambropoulos | 17 May 2013 20:57
Picon

sharing objects amongst workers

We are using the pymongo driver with mongodb and as it stands, it looks like each worker gets its own connection.  Is there any way to share a database connection in memory among the workers?  I've read about ksm on the readthedocs page, is that applicable here?  

We also have a bunch of python objects being stored in memory, would KSM be helpful in sharing memory among the workers for this?

For example:
Currently we have an identical list being stored on each worker (we keep the list stored in a python variable because its large and we need to access it quickly so storing in the uwsgi cache and pickling and unpickling (the only way I can think of to store a python object in the cache) each request, would be too slow) and we have 6 workers.  This means we are using several times the memory we actually need to.   Is there any way to share a global python object among all of the workers?  All we do is read from it, we only modify it when we are completely refreshing it periodically.

The overarching question is, how do i share db connections and python objects globally, so each worker doesn't need to have their own copy of the python objects and we aren't keeping so many db connections open?
_______________________________________________
uWSGI mailing list
uWSGI@...
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
Permalink | Reply | 10 comments |
headers
André Cruz | 17 May 2013 18:00
Picon
Gravatar

HTTP router and http-keepalive

Hello.

I'm trying to setup HTTP keepalive and, although my application appears to send all the right headers, the
connection is always closed when the client issues requests with a body, like POSTs or PUTs. GETs keep the
connection open.

Example (even with a content-length of 0 this breaks the connection):

Connected to localhost.
Escape character is '^]'.
POST /XXX HTTP/1.1
User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
Host: localhost:8000
Accept: */*
Content-Length: 0
Expect: 100-continue

HTTP/1.1 200 OK
Date: Fri, 17 May 2013 15:52:23 GMT
Vary: Accept-Language, Cookie
Content-Length: 413
Content-Type: application/json
Content-Language: pt
Connection: Keep-Alive

{"bytes": 0, "thumb_exists": false, "rev": "c3dcd9eb-bf09-11e2-9267-406c8f001c8a", "modified":
"Fri, 17 May 2013 15:52:23 +0000", "cursor":
"action_upload,event_c3dc7f2e-bf09-11e2-95e7-406c8f001c8a,d1508fef-7bec-4641-a2ed-571abf52606b_c3dcfef3-bf09-11e2-8c83-406c8f001c8a",
"path": "/11232", "is_dir": false, "icon": "page_white.gif", "root": "dropbox", "mime_type":
"application/octet-stream", "size": "0 bytes"}Connection closed by foreign host.  <-----------

I have added to my config:

    add-header: Connection: Keep-Alive
    http-keepalive: 1
    http-auto-chunked: 1

On another note, I've noticed that by connecting to the http socket and sending just carriage returns, my
WSGI application is invoked with invalid arguments. The connection is broken after the first corrupted
request. In the case of Django, it blows up with:

Traceback (most recent call last):
  File
"/Users/andre/work/penv/discosite/lib/python2.7/site-packages/django/core/handlers/wsgi.py",
line 245, in __call__
    request = self.request_class(environ)
  File
"/Users/andre/work/penv/discosite/lib/python2.7/site-packages/django/core/handlers/wsgi.py",
line 147, in __init__
    self.method = environ['REQUEST_METHOD'].upper()
KeyError: u'REQUEST_METHOD'

Is this permitted by the WSGI spec? If I send a GET first, and the connection is kept alive, I'm allowed to just
keep sending carriage returns, generating errors, and the connection is not broken.

Best regards,
André
Permalink | Reply | 18 comments |
headers
Adroaldo de Andrade | 17 May 2013 14:23
Picon
Favicon
Gravatar

Mulefunc decorator

Hi

I'm using uWSGI 1.4.9 and Python 2.7.2, for Mac (64bit), and I made a several consecutive calls to a function
that is using  <at> mulefunc decorator to make some asynchronous tasks. My setup is for 3 workers and 1 mule.

Well, it should have execute the function a bunch of times, but it is executing just twice, not the amount of
times the function was called.

First I thought that I ran out mule queue space (64Kb), but each call is only using 114 bytes, so that is not the issue.

Code example:

# config file

[uwsgi]
master = true
chdir = %d/../server
home = %d/../environment
env = LANG=en_US.UTF-8
env = LC_ALL=en_US.UTF-8
env = LC_CTYPE=en_US.UTF-8
env = DJANGO_SETTINGS_MODULE=production_settings
module = wsgi_production:application
socket = 127.0.0.1:3035
processes = 3
mule = 1
uid = 1000
gid = 1000
enable-threads = true
single-interpreter = true
die-on-term = true
harakiri = 60
harakiri-verbose = true
post-buffering = 8192
max-requests = 1000
vacuum = true
logto = /var/log/uwsgi/server.log
# Import mule funcs
import =  foobar.bar

# foo.py

from foobar.bar import bar_work

def async_barr(request):
   for i in range(0, 10):
       bar_work(i)

# bar.py

from uwsgidecorators import mulefunc

 <at> mulefunc
def bar_work(wait):
   print "** Calling mule! %d" % int(wait)
   time.sleep(int(wait))

In the case it should call bar_work ten times, but it call just twice. Adding more mules dint solve the problem.

Thank you.
Permalink | Reply | 1 comment |

Gmane