[ACCEPTED]-Python multiprocessing and database access with pyodbc "is not safe"?-pyodbc
Multiprocessing relies on pickling to communicate 5 objects between processes. The pyodbc connection 4 and cursor objects can not be pickled.
>>> cPickle.dumps(aCursor)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.5/copy_reg.py", line 69, in _reduce_ex
raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle Cursor objects
>>> cPickle.dumps(dbHandle)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.5/copy_reg.py", line 69, in _reduce_ex
raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle Connection objects
"It 3 puts items in the work_queue", what items? Is 2 it possible the cursor object is getting 1 passed as well?
The error is raised within the pickle module, so 7 somewhere your DB-Cursor object gets pickled 6 and unpickled (serialized to storage and 5 unserialized to the Python object again).
I 4 guess that pyodbc.Cursor does not support pickling. Why 3 should you try to persist the cursor object 2 anyway?
Check if you use pickle somewhere in your 1 work chain or if it is used implicitely.
pyodbc has Python DB-API threadsafety level 1. This means threads 5 cannot share connections, and it's not threadsafe 4 at all.
I don't think underlying thread-safe 3 ODBC drivers make a difference. It's in 2 the Python code as noted by the Pickling 1 error.
More Related questions
We use cookies to improve the performance of the site. By staying on our site, you agree to the terms of use of cookies.