Tutorial: consuming Twitter’s real-time stream API in Python with PyCurl

julio 2, 2012

Consuming the streaming API with PyCurl

The easiest way to handle HTTP streaming in Python is to use PyCurl, the Python bindings for the well-known Curl network library. PyCurl allows you to provide a callback function that will be executed every time a new block of data is available. The following code is a simple demonstration of HTTP streaming with PyCurl:

import pycurl, json

STREAM_URL = “http://chirpstream.twitter.com/2b/user.json”

USER = “segphault”


def on_receive(data):

print data

conn = pycurl.Curl()

conn.setopt(pycurl.USERPWD, “%s:%s” % (USER, PASS))

conn.setopt(pycurl.URL, STREAM_URL)

conn.setopt(pycurl.WRITEFUNCTION, on_receive)


The code example above shows how to instantiate a Curl object, set the URL, provide login credentials, and send the data to a callback function. The callback function in the example simply echoes the received data to the terminal. If you put in your own Twitter credentials and run that code in a Python script at the command line, you will see the stream of JSON data transmitted by the Twitter service.

When the connection is idle and there is no other data to send, the streaming API will emit an empty line every 30 seconds. The empty line is a keep-alive signal that is intended to prevent client applications from timing out and dropping the connection. PyCurl doesn’t require any special configuration, but other network libraries might require the user to set a custom timeout duration for idle connections. You should make sure that it is set to something that is higher than the streaming API’s 30-second keep-alive interval so that the connection isn’t dropped.

vía Tutorial: consuming Twitter’s real-time stream API in Python | Ars Technica.

Los 10 ‘hashtags’ de la jornada electoral – Aristegui Noticias

julio 2, 2012

Las siguientes son los hashtags más utilizados en el día 1 de julio 2012:

Hashtags Menciones Contexto.

#HoyVotoporAMLO 220,188 La etiqueta ha sido utilizada por usuarios partidistas para difundir mensajes de apoyo al candidato de las izquierdas al momento de depositar su voto por la campaña

#Elecciones2012 94,473 Los usuarios comparten a través de esta etiqueta lo que sucede en las casillas de votación. Algunos reportan normalidad, mientras muchos reportan todo tipo de irregularidades.

#MiVoto2012 79,482 Los usuarios compartes a través de esta etiqueta lo que sucede en las casillas de votación y su intención de participar en la jornada electoral.

#OpPRI 35,430 El grupo de ciberactivistas Anonymous lanzó la Operación PRI para atacar durante el día los principales sitios del partido durante toda la jornada. Los usuarios de twitter comparten la etiqueta para denunciar irregularidades.

#JosefinaYAGano 34,012 Partidistas difunde con esta etiqueta su intención de voto y confían en el triunfo de la candidata del PAN

#PorUnasEleccionesenPaz 20,224 Etiqueta creada por partidarios del PRI después de la votación del candidato Peña Nieto invitando a participar en la jornada sin caer en provocaciones ni en actos violentos.

#TodosaVotar 19,735 Usuarios difunden a través de esta etiqueta invitaciones a toda la comunidad para que acuda a las urnas.

#Huixquilucan 10,391 Esta etiqueta se convirtió en Trending Topic después de que Josefina Vázquez Mota acudiera a ese municipio a votar a las 9:40 a.m.

#Margarita Zavala 7,853 La comunidad twittera compartió a través de esta etiqueta el momento en donde, en compañía del Presidente de la República, asistió a votar pasadas las 13 hrs.

#Atlacomulco 6,737 La etiqueta se volvió Trending Topic después de que el candidato Enrique Peña Nieto acudiera a ese municipio a depositar su voto.

vía Los 10 ‘hashtags’ de la jornada electoral – Aristegui Noticias.

ThatCamp Paris 2010 | Non-conférence sur les Digital humanities Paris, EHESS, 18 et 19 mai 2010

marzo 20, 2011

Manifesto for the Digital Humanities

agosto 26, 2010

Por Marin Dacos

Manifesto for the Digital Humanities


We, professionals or observers of the digital humanities (humanités numériques) came together in Paris for THATCamp on May 18th and 19th, 2010.

Over the course of these two days, we discussed, exchanged, and collectively reflected upon what the digital humanities are, and tried to imagine and invent what they could become.

At the close of the camp – which represents but a first step – we propose to the research communities, and to all those involved in the creation, publication, valorization or preservation of knowledge, a manifesto for the digital humanities.

I. Definition

1. Society’s digital turn changes and calls into question the conditions of knowledge production and distribution.

2. For us, the digital humanities concern the totality of the social sciences and humanities. The digital humanities are not tabula rasa. On the contrary, they rely on all the paradigms, savoir-faire and knowledge specific to these disciplines, while mobilizing the tools and unique perspectives enabled by digital technology.

3. The digital humanities designate a “transdiscipline”, embodying all the methods, systems and heuristic perspectives linked to the digital within the fields of humanities and the social sciences.

II. Situation

4. We observe:

– that experiments in the digital domain of the social sciences and humanities have multiplied in the last half century. What have emerged most recently are centers for digital humanities – which at the moment are themselves only protoypes or areas of application specific to the approach of digital humanities;

– that computational and digital approaches have greater technical, and therefore economic, research constraints; that these constraints provide an opportunity to foster collaborative work;

– that while a certain number of proven methods exist, they are not equally known or shared;

– that there are many communities deriving from shared interests in practices, tools, and various interdisciplinary goals – encoding textual sources, geographic information systems, lexicometry, digitization of cultural, scientific and technical heritage, web cartography, datamining, 3D, oral archives, digital arts and hypermedia literatures, etc. – and that these communities are converging to form the field of digital humanities.

III. Declaration

5. We, professionals of the digital humanities, are building a community of practice that is solidary, open, welcoming and freely accessible.

6. We are a community without borders. We are a multilingual and multidisciplinary community.

7. Our objectives are the advancement of knowledge, the improvement of research quality in our disciplines, the enrichment of knowledge and of collective patrimony, in the academic sphere and beyond it.

8. We call for the integration of digital culture in the definition of the general culture of the twenty-first century.

IV. Guidelines

9. We call for open access to data and metadata, which must be documented and interoperable, both technically and conceptually.

10. We support the dissemination, exchange and free modification of methods, code, formats and research findings.

11. We call for the integration of digital humanities education within social science and humanities curricula. We also wish to see the creation of diplomas specific to the digital humanities, and the development of dedicated professional education. Finally, we want such expertise to be considered in recruitment and career development.

12. We commit to building a collective expertise based upon a common vocabulary, a collective expertise proceeding from the work of all the actors involved. This collective expertise is to become a common good. It is a scientific opportunity, but also an opportunity for professional insertion in all sectors.

13. We want to help define and propagate best practices, corresponding to needs identified within or across disciplines, which should derive and evolve from debate and consensus within the communities concerned. The fundamental openness of the digital humanities nevertheless assures a pragmatic approach to protocols and visions, which maintains the right to coexistence of different and competing methods, to the benefit of both thought and practice.

14. We call for the creation of scalable digital infrastructures responding to real needs. These digital infrastructures will be built iteratively, based upon methods and approaches that prove successful in research communities.

vía ThatCamp Paris 2010 | Non-conférence sur les Digital humanities Paris, EHESS, 18 et 19 mai 2010.