BTQueue provides some functions regarding remote access, e.g., wget, wpost and scrape. Anyway, some urls are temporary or even permanent unavailable due to unpredictable reasons. It is necessary to set reasonable timeout for all operations including connect, send and receive.

For Python 2.3 and later, they offers timeout built-in in socket module so we can specify default timeout.

import socket
print socket.getdefaulttimeout()
socket.setdefaulttimeout(60)

In addition, we may specify timeout for individual connection.

import socket
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
print s.gettimeout()
socket.settimeout(60)

For Python 2.2, we have timeoutsocket.py.

Tags: python, socket