Tools and Libs for Scraping

So what am I using to scrap the site? Well, first let me tell you that i’m mostly a Microsoft Developer. I develop web applications with Visual Basic and (less) C# (using Visual Studio) and also MS SQLServer but I’ve also played and did a few projects with PHP and MySQL. Since the Hacklaviva […]


Because scraping is the boring part of exploring data I tried to find the easiest, less time-consuming, yet easy to maintain set of tools to do it. Since the URLs are (somewhat) well known, I was able to skip the crawler, url discovery part of this projects, I just curl(ed) the page that I wanted […]

Portuguese Opendata Projects

One of my “low-intensity” projects is the exploration of portuguese open-data, mostly governmental data. I’ve written some things about it (in portuguese) and also developed some (very few) applications but have never done anything really interesting, mostly because of the lack of time. Fortunately this year I came across a some interesting folks who also share […]

Transparência HackDay #2

A próxima sessão do transparência hackday realiza-se no próximo sábado à tarde no hacklaviva. Para quem não conhece, o objectivo deste projecto é explorar a informação pública e criar mecanismos para que os cidadãos consigam criar algum conhecimento a partir do grande volume de informação que alguns sites governamentais disponibilizam. Neste momento temos 2 projectos […]