Update: I have now released the tool.
As part of developing the Twitter Stats application, I created a standalone script that will dump a user’s entire Twitter timeline to a CSV file (comma separated value, readable by spreadsheet applications such as Excel), including the tweet text and the post time.
Initially, it was my intention to release this script to the public. I had several requests from people that wanted to have a record of all their tweets, which I kindly provided for them, and in my opinion the tool would prove quite useful.
After mentioning this to a couple of very smart people, they raised privacy concerns and suggested I keep the code to myself, which I have done thus far. These concerns stem from the fact that the tool can dump any user’s entire non-protected timeline, not just your own. Personally, I don’t really think this is a huge problem - if you have an unprotected timeline, all your tweets are public record anyway, the tool just makes it easier to extract and save these tweets. On the other hand, someone having a local copy of your stream does sound like a worrying proposition.
Clearly, there are far larger privacy issues associated with all of this, but I wanted to open up the floor and find out what other people think of the possible release of this tool. Should I put it out there, or keep it to myself?
so just protect it by making people have to log-on using the Twitter API?!
2 points -
1) Now there’s an API, is it really that hard for anyone to do what you (and Dave Winer) have done ?
2. There’s plenty of tools that will let you manually reconstruct this information. And depending on the user tweetscan can go back in time to varying degrees.
3. (i work in support and i’m used to people not being able tocunt) Does anyone seriously expect data named ‘Public Timeline’ to be private ?
Martin: The API only lets you retreive 20 tweets of any kind, with no paging. The most you can pull with it is 20 back in your timeline. This gets around that. TweetScan only lets you go back if they are following you, and as such, caching your tweets.
m1k3y: Does that help at all? Most people using this will be twitter members anyway, and it is easy enough to go in and remove that line from the code.
only if you publish the script, rather than just host it with your own front-end
m1k3y: That’s true, I could do it that way. It is a command line script at the moment, but I could make it work like that.
I understand the concerns, however a user’s entire history is already cached at Google if they’re public.
In addition, a python(?) script is only going to be usable by those who already have the skill to write something like this anyway. You’re saving someone, what 20-30 minutes? There’s plenty of other things out there which will do this for you anyway, with a little prodding (wget, anyone?), but just not as cleanly.
Brad, if it’s already publicly available then I don’t think anyone can cry foul about any privacy issues.
Anyone with privacy concerns obviously doesn’t understand how the inter-tubes work - it’s like getting upset because people can see your face while you walk down a street.
As already mentioned, if somebodies timeline is public already, then their timeline has no privacy, nor can they have any expectation that their timeline can have any privacy.
What about their followers who receive tweets via IM, who have IM clients which log every conversation?
I say release the script if you want to, and tell anyone who complains not to publish things to the internets if they want it to be private. Personally I don’t care if somebody chooses to read my tweets via the web interface, via IM, via SMS, or via a CSV on their hard drive.
If anything, I think it’s worth publishing to highlight the lack of privacy. If anyone has any illusions of their public feed being in some vague way protected, it’s important to correct that notion.
Thanks for the opinions guys, I really appreciate them. I will be releasing a web version of the tool later today that will authenticate to Twitter before doing it’s thing to only let you export your own timeline.
Those that I have spoken to believe that this is a good compromise.
Sweet, thanks for this! It seems almost impossible to get your full Twitter stream exported in any useful way.
I’m actually looking to switch back to WordPress from Twitter, and was wondering…is it at all possible to have this export to RSS instead of CSV? If that were the case, a WP install would be able to import the Twitter entries.
Am poking around with the XML/JSON exports Twitter allows as per the API, but this is way out of my league. :-/
[...] general response to my hesitation on the release of my Twitter timeline export tool was that I should, indeed, release it. So I [...]
Tweets are public — I wouldn’t worry. Please dump all my tweets to a CSV for me!