More feedster.com sillyness

11 Nov 2004

Somehow, more than any other online service, Feedster really annoys me. For the last few days I noticed some 404 error on their feed of the day feed. I really should have unsubscribed from it when I said I would, but then they fixed it.

So this is what happens, they seem to be returning error codes based on user agent:

Connected to feedster.com.
Escape character is '^]'.
GET /search.php?q=f-botd&limit=15&type=rss HTTP/1.1
Host: feedster.com
User-Agent: NewzCrawler/1.7 (compatible; MSIE 6.00; Newz Crawler 1.7; http://www.newzcrawler.com/ )

HTTP/1.1 404 Not Found

compare that 404 error with this:

Connected to feedster.com.
Escape character is '^]'.
GET /search.php?q=f-botd&limit=15&type=rss HTTP/1.1
Host: feedster.com
User-Agent: test

HTTP/1.1 200 OK

wtf? One less feed to worry about.

Posted by R.I.Pienaar on Thursday, November 11, 2004 at 4:58 PM

Filed in

1 Comment

Hi,

Sorry about that. We've had major trouble lately w/ being overly crawled by major crawlers and its costing us tons of money in bandwidth so we're fairly aggressivley banning bots and I must have mistaken NewzCrawler as a bot not an aggregator due to the name. I'll take it out.

Sorry we annoy you.

S

Scott Johnson of Feedster | November 11, 2004 6:15 PM | Reply

Leave a comment

Recent Entries

  • flashpolicyd 2.0

    I wrote a multi threaded server for Adobe Flash Policy requests, some background from Adobe:Since policy files were first introduced, Flash Player has recognized /crossdomain.xml...

  • Adventures with Ruby

    Some more about my continuing experiences with ruby, in my last post I saidthe language does what you'd expect and as you'll see in my...

  • New programming language of choice - Ruby

    I have fallen out of love with Perl some time ago, I cannot point to one specific thing about it that put me off, I...

  • On working from home

    I've not been posting much here, work has been incredibly manic the last while, especially I need to still finish off my SSO posts with...

  • Rework of puppet facts for /etc/facts.txt

    Previously I blogged a custom fact that reads /etc/facts.txt to build up some custom facts for use in Puppet manifests, well I've since learned a...

Close