Ready, Set, FoWA!
Posted by rmp at 22:05 2nd Oct 2007
Really can't wait to get to the Future_of_Web_Apps tomorrow (nice and early of course, though I'll be crossing Londinium on the tube during rush hour - joy).
I've just been circling sessions on the schedule and find myself mostly in the developer track on day 1 but with an even split between developer and business tracks on day 2. I'm almost wishing Carson hadn't packed so much goodness in so tightly.
I have to say I'm a little torn between developer sessions which are appropriate for work and business sessions which are slightly more useful for the special things I do out of hours but almost justifiable under the 'strategy' heading...
Anyway, email screwups aside it's shaping up to be a great show. More to come...
(0 comments)
Psyphi Blog v5
Steve Souders - Yahoo Performance - High Performance websites
Posted by rmp at 12:13 3rd Oct 2007
The gist of this presentation was to perform accurate, scientific benchmarking of front- vs. back-end overheads up-front as a developer. This should be common-sense but I don't know *anyone* amongst my colleagues who does this proactively. It's almost always reactive as a result of performance problems post-release.
Steve presented a series of guidelines on streamlining content with the aim of improving performance. Some of these conflict a little in terms of common design practices (e.g. use of javascript/css frameworks). These guidelines are more-or-less what the YSlow Firebug plugin incorporates and in a nutshell look something like this:
fewer http requests
use a cdn
add an expires header
gzip components
put stylesheets at the top
move scripts to the bottom
avoid css expressions
make js and css external
reduce dns lookups
minify js
avoid redirects
remove duplicate scripts
configure etags
make ajax cacheable
split static content across multiple domains
reduce the size of cookies
host static content on a different domain
minify css
avoid iframes
Ref: YSlow firebug plugin! Book: High Performance Websites Blog: YUIBlog, YDNBlog (0 comments)
Steve presented a series of guidelines on streamlining content with the aim of improving performance. Some of these conflict a little in terms of common design practices (e.g. use of javascript/css frameworks). These guidelines are more-or-less what the YSlow Firebug plugin incorporates and in a nutshell look something like this:
fewer http requests
use a cdn
add an expires header
gzip components
put stylesheets at the top
move scripts to the bottom
avoid css expressions
make js and css external
reduce dns lookups
minify js
avoid redirects
remove duplicate scripts
configure etags
make ajax cacheable
split static content across multiple domains
reduce the size of cookies
host static content on a different domain
minify css
avoid iframes
Ref: YSlow firebug plugin! Book: High Performance Websites Blog: YUIBlog, YDNBlog (0 comments)
Matt Mullenweg (WordPress) - The Architecture Behind WordPress.com
Posted by rmp at 15:18 3rd Oct 2007
Scaling platform:
88m global uniques 1.5m blogs 215m pageviews/day
- 7 boxes, $1500/mo.
- 2 balancers, 2Gb memory, any disk, pound+wackamole+spread.
- 2 databases, 4Gb+ memory, fast disk (RAID), master+slave mysql, split read/write.
- 3 webs, fast cpu, 2Gb memory, litespeed or well-configured apache.
- Everything in subversion.
- Be stateless (shared nothing)
- Memcached
geo-targetting of dns, might as well use a CDN
single box 300 req/sec 29.5m/day
Scaling community:
Ref: sxsw presentation
Scaling business:
Tie revenue streams to cost scaling, e.g. pageviews rather than bloggers. Ads scale revenues with pageviews, effective CPM, low outlay. VIP class of users with advanced features, customisation etc.
Scaling people:
Hire people as good as, or better than you. Great people = Rich environment + worthwhile problems. 5 things to look for:
- Personality fit (when the shit hits the fan)
- Ability to learn (curiosity)
- Taste (can't be taught)
- Passion for the space
- Familiarity with current technologies
Don't put out too easily - don't hire if doubts exists
It's fun to know that the folks at wordpress manage deployment with ssh scripts running svn up.
I disagree with Matt's QA response suggesting that you should never do your own hosting. In principle that's ok for the large faceless mass of current web sites & traffic but under special circumstances it's the only way. I build web apps on top of bioinformatics pipelines supporting (today) 320Tb data and there's no way on Earth this would be possible to outsource. (0 comments)
88m global uniques 1.5m blogs 215m pageviews/day
- 7 boxes, $1500/mo.
- 2 balancers, 2Gb memory, any disk, pound+wackamole+spread.
- 2 databases, 4Gb+ memory, fast disk (RAID), master+slave mysql, split read/write.
- 3 webs, fast cpu, 2Gb memory, litespeed or well-configured apache.
- Everything in subversion.
- Be stateless (shared nothing)
- Memcached
geo-targetting of dns, might as well use a CDN
single box 300 req/sec 29.5m/day
Scaling community:
Ref: sxsw presentation
Scaling business:
Tie revenue streams to cost scaling, e.g. pageviews rather than bloggers. Ads scale revenues with pageviews, effective CPM, low outlay. VIP class of users with advanced features, customisation etc.
Scaling people:
Hire people as good as, or better than you. Great people = Rich environment + worthwhile problems. 5 things to look for:
- Personality fit (when the shit hits the fan)
- Ability to learn (curiosity)
- Taste (can't be taught)
- Passion for the space
- Familiarity with current technologies
Don't put out too easily - don't hire if doubts exists
It's fun to know that the folks at wordpress manage deployment with ssh scripts running svn up.
I disagree with Matt's QA response suggesting that you should never do your own hosting. In principle that's ok for the large faceless mass of current web sites & traffic but under special circumstances it's the only way. I build web apps on top of bioinformatics pipelines supporting (today) 320Tb data and there's no way on Earth this would be possible to outsource. (0 comments)
What Can Bioinformaticians Learn from YouTube?
Posted by rmp at 22:46 6th Nov 2007
Caught Matt's talk this morning at the weekly informatics group meetings -
There were general murmurings of agreement amongst the audience but nobody asking the probing questions I'd hope for as a measure of interestedness.
Matt touched upon microformats in all but name - I was really expecting a sell of http://bioformats.org/ , websites as APIs and RESTful web services in particular.
Whilst I'm inclined to agree that standardised, discoverable, reusable web services are largely the way forward (especially as it keeps me in work) I'm not wholly convinced they remove the problems associated with, for example, database connections, database-engine specific SQL, hostnames, ports, accounts etc.
My feeling is that all the problems associated with keeping track of your database credentials are replaced by a different set of problems, albeit more standardised in terms of network protocols in HTTP and REST/CRUD. We now run the risk that what's fixed in terms of network protocols is pushed higher up the stack and manifests as myriad web services, all different. All these new websites and services use different XML structures and different URL schemes. The XML structures are analogous to database table schema and the URL schemes akin to table or object names.
At least these entities are now discoverable by the end user/developer simply by using the web application - and there's the big win - transparency and discoverability. There's also the whole microformat affair - once these really start to take off there'll be all sorts of arguments about what goes into them, especially in domains like Bio and Chem, not covered by core formats like hCard. But that's something for another day.
More over at Green_Is_Good (0 comments)
There were general murmurings of agreement amongst the audience but nobody asking the probing questions I'd hope for as a measure of interestedness.
Matt touched upon microformats in all but name - I was really expecting a sell of http://bioformats.org/ , websites as APIs and RESTful web services in particular.
Whilst I'm inclined to agree that standardised, discoverable, reusable web services are largely the way forward (especially as it keeps me in work) I'm not wholly convinced they remove the problems associated with, for example, database connections, database-engine specific SQL, hostnames, ports, accounts etc.
My feeling is that all the problems associated with keeping track of your database credentials are replaced by a different set of problems, albeit more standardised in terms of network protocols in HTTP and REST/CRUD. We now run the risk that what's fixed in terms of network protocols is pushed higher up the stack and manifests as myriad web services, all different. All these new websites and services use different XML structures and different URL schemes. The XML structures are analogous to database table schema and the URL schemes akin to table or object names.
At least these entities are now discoverable by the end user/developer simply by using the web application - and there's the big win - transparency and discoverability. There's also the whole microformat affair - once these really start to take off there'll be all sorts of arguments about what goes into them, especially in domains like Bio and Chem, not covered by core formats like hCard. But that's something for another day.
More over at Green_Is_Good (0 comments)
Web frameworking
Posted by rmp at 23:47 31st Mar 2008
It seems to be the wrong time to be reading such things, but over on InfoQ there's a nice_article introducing web development of RESTful_services using Erlang and the Yaws high performance web server.
I say "the wrong time" as this week has kicked off the "Advancing with Rails" course by David_A._Black of Ruby_Power_and_Light fame. The course is fairly advanced in terms of required rails knowledge so it's a bit of a baptism by fire for me and a few others having never written any Ruby before.
Rails is proving moderately easy to pick up but as I've remarked to a couple of people, it doesn't seem any easier coding with Rails than with Perl. Perhaps it's because I've never done it before but I reckon it's a lot harder spending my time figuring out what the heck DHH meant something to do than it is doing it myself.
Even though it's nowhere near as mature, I do reckon my ClearPress framework has a lot going for it - it's pretty feature-complete in terms of ORM, views and templating ( TT2 ). It has similar convention over configuration features meaning it's not designed for plugging in other alternative layers but it is absolutely possible to do (and I suspect without as much effort as is required in Rails). I still need to iron out some wrinkles in the autogenerated code from the application builder and provide some default authorisation and authentication mechanisms, some of which may come in the next release. But in the meantime it's easy to add these features, which is exactly what we've done for the new sequencing run tracking app, NPG to tie it to the WTSI website single sign on (MySQL and LDAP under the hood).
(0 comments)
I say "the wrong time" as this week has kicked off the "Advancing with Rails" course by David_A._Black of Ruby_Power_and_Light fame. The course is fairly advanced in terms of required rails knowledge so it's a bit of a baptism by fire for me and a few others having never written any Ruby before.
Rails is proving moderately easy to pick up but as I've remarked to a couple of people, it doesn't seem any easier coding with Rails than with Perl. Perhaps it's because I've never done it before but I reckon it's a lot harder spending my time figuring out what the heck DHH meant something to do than it is doing it myself.
Even though it's nowhere near as mature, I do reckon my ClearPress framework has a lot going for it - it's pretty feature-complete in terms of ORM, views and templating ( TT2 ). It has similar convention over configuration features meaning it's not designed for plugging in other alternative layers but it is absolutely possible to do (and I suspect without as much effort as is required in Rails). I still need to iron out some wrinkles in the autogenerated code from the application builder and provide some default authorisation and authentication mechanisms, some of which may come in the next release. But in the meantime it's easy to add these features, which is exactly what we've done for the new sequencing run tracking app, NPG to tie it to the WTSI website single sign on (MySQL and LDAP under the hood).
(0 comments)
ClearPress-146
Posted by rmp at 23:15 29th Apr 2008
Latest release of ClearPress (v146) out to the CPAN yesterday. The ClearPress data model now implements belongs_to_through, belongs_to, has_many and has_many_through entity relationships for all you ActiveRecord lovers.
Two ClearPress-derived projects are using a half-decent test fixture system. It's really making a big difference to the development of both DECIPHER and NPG so I'm planning to bundle what can be bundled with an upcoming release. (0 comments)
Two ClearPress-derived projects are using a half-decent test fixture system. It's really making a big difference to the development of both DECIPHER and NPG so I'm planning to bundle what can be bundled with an upcoming release. (0 comments)
Atom
