brain of mat kelcey

what does it mean for a topic to be 'trending'? consider the following time series (430e3 tweets containing cheese collected over a month period bucketed into hourly timeslots)without a formal definition we can just look at this and say that the series was trending where there was a spike just before time 600. as a start then let's just define a trend as a value that was greater than was 'expected'.one really nice simple algorithm for detecting a trend is to say a value, v, is trending if v > mean + 3 * standard deviation of the data seen...

people tweet about all sorts of stuff.sometimes it's really important ground breaking world changing stuff...but most of the time it's ridiculous waste of time stuff like 'i ate some cheese'in fact how much do people actually tweet about cheese?and when they do, what are the most important cheese related topics?lets gather some data...bash> curl -s -u user:pasword http://stream.twitter.com/1/statuses/filter.json?track=cheeselet's poke around, but first some l33t hax0r bash aliases for the sake of brevityalias t='tail'alias h='head'alias s='sort'alias u='uniq'alias g='grep'let's start with a sample, the first 10 tweets...bash> ./parse_cheese_out.rb < cheese.out | hPasta with pesto and cheese. Some watermelon but alas did not...