Saturday, February 17, 2007

memcached ssslllooowww on Mac OS X

The purpose of memcached is .. to be fast .. . Unfortunately the default port (Darwinports) is really slow (slow as in 3 seconds of doing nothing on a 1.8GHz PowerPC G5), and somebody figured out a fix.

Now here are few notes on how to recompile it (quick and dirty):

  • find the source /opt/local/var/db/dports/distfiles/memcached/memcached-1.1.11.tar.gz . Unpack it (ex: on Desktop)
  • as suggested in fix., edit the memcached.c file, adding #undef TCP_NOPSH on line 105
  • find the location of libevent (which came as part of port install)
  • in the folder with patched memcached run
    • ./configure --with-libevent=/opt/local/var/db/dports/software/libevent/1.1a_0/opt/local/
    • make
    • take a look at the original permissions
    • copy new file into it's original location by: sudo mv memcached /opt/local/bin/memcached
    • restore original permissions : sudo chown root:admin /opt/local/bin/memcached
Now start memcached frm command line (memcached -vv for verbose logging), and test as it says in the README (from python):

import memcache
mc = memcache.Client(['127.0.0.1:11211'], debug=0)

mc.set("some_key", "Some value")
value = mc.get("some_key")

Now it will not wait 3 seconds
Posted by at 2 comments:

threadsafe CheryPy

Using CherryPy is so easy, like everything in Python. But when running some pages with heavy SQL processing time, there are a lot of funny errors (malloc.. double free). Ooops, looks as something is not theadsafe. mySQLdb is not necessarily threadsafe, but cherrypy.engine.on_start_thread_list should handle this as suggested . Instead of digging deep into debugging, here is an alternative fix, using DBUtils.

Yipee, my code got posted into CherryPy:

import cherrypy
import MySQLdb

from DBUtils.PersistentDB import PersistentDB

class HomePage:
   @cherrypy.expose

   def index(self):
       sql = "select max(whatever) from bigtable;" #long running query

       c = persDb.connection().cursor()
       c.execute(sql)

       res = c.fetchone()
       c.close()
       return ' '.join( ['<html>', repr(res), '</html>'] )


cherrypy.tree.mount(HomePage())

if __name__ == '__main__': 
   persDb=PersistentDB(MySQLdb, 10000, host='localhost', db='db', passwd='****')

   cherrypy.config.update({'server.thread_pool': 10})
   cherrypy.server.quickstart()

   cherrypy.engine.start()
Posted by at No comments:
Subscribe to: Posts (Atom)