Tutorial: consuming Twitter’s real-time stream API in Python with PyCurl

Consuming the streaming API with PyCurl

The easiest way to handle HTTP streaming in Python is to use PyCurl, the Python bindings for the well-known Curl network library. PyCurl allows you to provide a callback function that will be executed every time a new block of data is available. The following code is a simple demonstration of HTTP streaming with PyCurl:

import pycurl, json

STREAM_URL = “http://chirpstream.twitter.com/2b/user.json”

USER = “segphault”


def on_receive(data):

print data

conn = pycurl.Curl()

conn.setopt(pycurl.USERPWD, “%s:%s” % (USER, PASS))

conn.setopt(pycurl.URL, STREAM_URL)

conn.setopt(pycurl.WRITEFUNCTION, on_receive)


The code example above shows how to instantiate a Curl object, set the URL, provide login credentials, and send the data to a callback function. The callback function in the example simply echoes the received data to the terminal. If you put in your own Twitter credentials and run that code in a Python script at the command line, you will see the stream of JSON data transmitted by the Twitter service.

When the connection is idle and there is no other data to send, the streaming API will emit an empty line every 30 seconds. The empty line is a keep-alive signal that is intended to prevent client applications from timing out and dropping the connection. PyCurl doesn’t require any special configuration, but other network libraries might require the user to set a custom timeout duration for idle connections. You should make sure that it is set to something that is higher than the streaming API’s 30-second keep-alive interval so that the connection isn’t dropped.

vía Tutorial: consuming Twitter’s real-time stream API in Python | Ars Technica.



Introduce tus datos o haz clic en un icono para iniciar sesión:

Logo de WordPress.com

Estás comentando usando tu cuenta de WordPress.com. Cerrar sesión /  Cambiar )

Google+ photo

Estás comentando usando tu cuenta de Google+. Cerrar sesión /  Cambiar )

Imagen de Twitter

Estás comentando usando tu cuenta de Twitter. Cerrar sesión /  Cambiar )

Foto de Facebook

Estás comentando usando tu cuenta de Facebook. Cerrar sesión /  Cambiar )


Conectando a %s

A %d blogueros les gusta esto: