Streaming Twitter into MongoDB

curl http://stream.twitter.com/1/statuses/sample.json -u<user>:<pass> | mongoimport -c twitter_live One thing that you can do with mongodb is have 1 streaming master and 1 read/write master server A: ./mongod —master —dbpath /tmp/a server B: ./mongod —dbpath /tmp/b —master —slave —source localhost:27017 —port 9999 You can then pipe the stream into server A, and it will only process the live stream. Server B will replicate all changes. You can also write to it, query on it, etc… This way you can do operations that block writing on server B, but server A will never backlog.