by R.I. Pienaar | Dec 6, 2006 | Usefull Things
I’ve previously written that I am trying out lighttpd for serving up my static files, I’ve now been running lighttpd and apache in parallel for a while and must say the results are very good in favour of lighttpd.
First a graphic to show the change:
This is a capture out of cacti showing the requests per second for some servers. Look at the yellow line, till about 12 it was running Apache 1.x, then I took that server out, around 12:30 I put in a lighttpd server on the same box and enabled stats from it around 13:00. This is on the same hardware, same files etc same IP address and you can clearly see in terms of requests per second lighttpd totally flies compared to Apache on the same box.
The Apache is a stock Debian Apache 1.3.33, I could probably have sped it up by some tuning, but installing lighttpd is much less work and much less painstaking monitoring, tuning, monitoring, tuning.
by R.I. Pienaar | Nov 29, 2006 | Code
GMapEZ has supported drawing lines using the Google Maps API for a while now, but I’ve always put off implimenting this since I thought it would be quite a bit of work and involve lots of changes in my XML format. I finally decided to impliment it today and I found a very easy way to assign points to lines.
First an example map so you can see what you can achieve:
The line thickness, color and opacity is configurable on a per-line basis and you can assign any point in your existing data file to a line on the map, to achieve this I extended the XML data using a <linemember> element, here is an example point:
<point>
<lat>51.393522</lat>
<long>0.527054</long>
<type>visit</type>
<title>Chatham Historic Dockyard</title>
<country>England</country>
<href>http://www.chdt.org.uk/</href>
<linktext>Home Page</linktext>
<linemember>1</linemember>
</point>
The points on the line get connected in order that they appear in your data file.
To configure the line created above you can add a [line1] section to your configuration file, simply match up the linemember numbers with the lineX section in your config, a sample:
[line1]
color = #00ff00
width = 2
opacity = 0.6
This will draw the line in the image above, I think it’s very simple and it should be 100% backward compatible with your exiting data should you choose not to use this feature.
As always you can get the code at http://www.devco.net/code/gmapsphp-current.tgz the documentation at http://www.devco.net/pubwiki/GMapsPHP has also been updated to reflect these changes.
by R.I. Pienaar | Nov 15, 2006 | Usefull Things
I’ve been struggling through some web development, it really is the worst kind of development possible. I’m developing an application to track my films, chemicals, cameras and development methods, so it needs a lot of stupid little buttons.
Today I cam across Buttonator.com, the site basically has a number of templates for the kind of button people use these days and let you put your own text in any color on them, simple stuff but a real time saver. If you open an account you can upload your own template buttons and have it do the text for you.
by R.I. Pienaar | Nov 2, 2006 | Usefull Things
Further to my previous post about webservers for data delivery I’ve been investigating some content delivery networks.
One that’s been getting a bit of bloggers love recently is Cachefly, on paper and on their pretty flash movies it seems pretty awesome. However when it comes time to deliver it falls pretty short.
From their site a few bits from their blurbs:
While traditional (or ‘first-generation’) CDN’s have used ‘DNS tricks’ to circumvent BGP (the internet’s core protocol), BestHop™ instead harnesses the power of BGP to determine how content can be delivered in the most efficient manner over CacheFly’s global CDN footprint. BestHop™ uses Anycast to instruct carriers routers to make connections to the best available point-of-presence for the end user. By combining anycast with our unprecidented international footprint, CacheFly has built the next-generation in Content Delivery Network.
So by using anycast they route you to a static ip – always the same one – and your ISPs routing tables figure out the nearest POP where your files live, this sounds great I had some doubts about Anycast but I’ve been informed that it does in fact work well even for TCP traffic.
Their network seems pretty extensive at the moment, they have a link on their site called “Out Network” that opens a popup, this shows an impressive world view of servers in San Jose, Phoenix, Chicago, Toronto, Ashburn, London, Amsterdam, Stockholm and finally Tokyo. These locations are a good footprint for my needs.
Some more blurps from their site:
Speed counts on the web, and by using CacheFly for your website you can deliver your website up to 10x faster than traditional hosting.
CacheFly makes it faster:
- CacheFly’s BestHop™ traffic management system delivers your content at blazing speeds by delivering your content from the ‘edge’ of the internet, placing it closer to your visitors.
- Your content is always available. Guaranteed. So there is no need to worry about losing customers because of an ineffective server or network issues.
- CacheFly provides you with instant scalability – Our technology means you’re website’s effectiveness will not be impaired by increased traffic. *Embrace* flash crowds. They mean more $$ now, not more headaches.
The load-time of your website affects your bottom line, period.
This sounds great, so I want to host my site furniture on it, buttons, logos and also user profile photos etc. I want the furnishings to load as fast as possible so that there are no perceived delay in the page drawing due to photos loading slowly one by one, and this seems to match my needs perfectly.
Finally some other claims from their site:
- No changes to your existing hosting solution
- Easy to implement, be live less than 10 minutes
So what’s the problem then? I signed up for their biggest account since I have about 1GB of jpg’s to host, the account lists 24 hour support, SCP, SFTP and rsync uploads as features along with some other stuff.
Once I opened the account and logged into the control panel I was immediately presented with only FTP upload details, no rsync or ssh based system. Turns out for this you have to open a support ticket which the auto reply claims will be actioned within a day, thats great, 10 minutes indeed.
As for delivery to nearest to your clients, well thats just bullshit to say the least, the way they do it is by hosting all your files in the states by default. If a file gets a lot of hits from a specific region they’ll then wait for some threshold to be reached then they’ll propagate your files internationally, in the words of their support people:
“in order for files to cached internationally, they need to be considered ‘hot’; which will take more than several requests”
I’ve now hit a specific file over 150 times from the UK and it is still being served from the states. So how do they serve it from a specific location given its anycast? Well I hit a webserver in the netherlands, this server then responds with a 302 and redirects me to another server – a non Anycast IP – in the states. So in effect every hit I take on my servers gets translated into 2 hits until they decide a file has hit some magic threshold, and its a varying threshold since a large file (512Kb) hit it early on after only 20 or so hits, a much smaller test file is still being served from the states after 300 UK based hits.
As for their network map showing servers in the UK? Not true at all. From their sales people:
Also, you should note that we are turning up our London POP next week, this will significantly increase the already superb performance of our product.
And another response after I queried some more:
We have routers in london now, however, they deliver traffic to our amsterdam location. Next week we’ll be adding delivery infrastructure, as well as deploying our 10G port to the LINX peering fabric.
So not even close to as advertised, I’ve had hits to files from South Africa, China, Korea, UK, US and AU and only 5 out of 351 were served from Amsterdam, UK traffic though accounted for 337 of the 351 hits.
Cachefly looks good on paper, this is only day 2 of my trial with them and even if they get my rsync access sorted out I can’t see how I can possibly go to phase 2 – actually sending traffic their way – if they can’t get my files actually served globally. I’ve asked them about this and am waiting on a response, it might still turn around and be a winner for now though I think Cachefly is ok for a reasonably popular podcast or a big RSS feed, but not for serving 6 million files per day as quick as possible which is what I need it for.
UPDATE: After waiting 2.5 days for their support to even respond to my request to enable rsync, I’m now closing my account. My recommendation is to avoid Cachefly as far as possible.
by R.I. Pienaar | Oct 31, 2006 | Usefull Things
I look after a site that serves up a lot of static content, early on already I ran into issues with Apache coping on one machine, also my bandwidth at my main site is pretty expensive so I started farming off my static content to a number of machines hosted at other ISPs, typically paying around 100 pounds per machine per month, as long as I push out less than around 100Gb/month it’s a pretty good deal.
The only problem I have is I get tons of SYN_RECV connections on each of my machines, around 300 of them at any given time. Typically these indicate a lot of connections waiting to be served but the servers handle new requests immediately, there are no time spent waiting for IO on the servers, in fact the CPU’s are 98% idle always.
In an effort to try and resolve this (after much tuning of server sysctl’s) I asked Jaco if he’s seen it before and he suggested giving Lighttpd a try.
I installed it on one of my 3 static servers and ran it for a few days now and the results are encouraging. I still have 300 SYN_RECV’s but the machine is performing much faster than its siblings. On average before Lighttpd I was getting 30 requests/sec out of each of my machines now this one is doing around 50/sec, it is also pushing out about 30Kb/sec more than the other two. Comparing the Lighttpd machine to 2 apache machines on a graph shows it consistently out performs the others by about 20%.
Uptime |
4 days 21 hours 19 min 20 s |
Requests |
18 Mreq |
Traffic |
20.77 Gbyte |
Requests |
105 req/s |
Traffic |
89.99 kbyte/s |
There has been some discussion about Lighttpd and Apache benchmarks, one Apache user has written a debunk of the benchmarks. This is linked too from the Lighttpd home page so might be worth investigating some more, I’ve done a lot of the typical things that Apache people recommend but they didn’t help much. I think I’ll try and tune my one Apache server to hit the same performance as the Lighttpd and see if it’s possible, for now though I’m quite happy with the results of a quick 30 minutes spent upgrading to Lighttpd.