Hiring Perl Developers - how hard can it be?
Posted by rmp at 21:27 28th Sep 2007
All the roles I've had during my time at Sanger have more or less required the development of production quality Perl code, usually OO and increasingly using MVC patterns. Why is it then that very nearly every Perl developer I've interviewed in the past 8 years is woefully lacking, specifically in OO Perl but more generally in half-decent programming skills?
It's been astonishing, not in a good way, how many have been unable to demonstrate use of hashes. Some have been too scared of them (their words, not mine) and some have never felt the need. For those of you who aren't Perl programmers, hashes (aka associative arrays) are a pretty crucial feature of the language and fundamental to its OO implementation.
Now I program in Perl sometimes more than 7-8 hours a day. For many years this also involved reworking other people's code. I can very easily say that if you claim to be a Perl programmer and have never used hashes then you're not going to get a Perl-related job because of your technical skills. With a good, interactive and engaging personality and a desire for self-improvement you might get away with it, but certainly not on technical merit.
It's also quite worrying how many of these interviewees are unable to describe the basics of object-oriented programming yet have, for example, developed and sold a commercial ERP system, presumably for big bucks. Man, these people must have awesome marketing!
Frankly a number of the bioinformaticians already working there have similar skills to the interviewees and often worse communication skills, so maybe I'm simply setting my standards too high.
I really hope this situation improves when Perl 6 goes public though I'm sure it'll take longer to become common parlance. As long as it happens before those smug RoR types take over the world I'll be happy ;)
(0 comments)
Psyphi Blog v5
The Importance of Profiling
Posted by rmp at 21:23 10th Feb 2008
I've worked as a software developer and worked with teams of software developers for around 10 years now, Many of those whom I've worked with have earned my trust and respect in relation to development and testing techniques. Frustratingly however it's still with irritating regularity that I hear throw-away comments bourne of uncertainty and ignorance.
A couple of times now I've specifically been told that "GD makes my code go slow". Now for those of you not in the know GD (actually specifically Lincoln Stein's GD.pm in perl) is a wrapper around Tom Boutell's most marvellous libgd graphics library. The combination of these two has always performed excellently for me and never been the bottleneck in any of my applications. The applications in question are usually database-backed web applications with graphics components for plotting genomic features or charts of one sort or another.
As any database-application developer will tell you, the database, or network connection to the database is almost always the bottleneck in an application or service. Great efforts are made to ensure database services scale well and perform as efficiently as possible, but even after these improvements are made they usually simply delay the inevitable.
Hence my frustration when I hear that "GD is making my (database) application go slow". How? Where? Why? Where's the proof? It's no use blaming something, a library in this case, that's out of your control. It's hard to believe a claim like that without some sort of measurement.
So.. before pointing the finger, profile the code and make an effort to understand what the profiler is doing. In database applications profile your queries - use EXPLAIN, add indices, record SQL transcripts and time the results. Then profile the code which is manipulating those results.
Once the results are in of course, concentrate in the first instance on the parts with the most impact (e.g. 0.1 second off each iteration of a 1000x loop rather than 1 second from /int main/ ) - the low hanging fruit. Good programmers should be relatively lazy and speeding up code with the least amount of effort should be commonsense. (0 comments)
A couple of times now I've specifically been told that "GD makes my code go slow". Now for those of you not in the know GD (actually specifically Lincoln Stein's GD.pm in perl) is a wrapper around Tom Boutell's most marvellous libgd graphics library. The combination of these two has always performed excellently for me and never been the bottleneck in any of my applications. The applications in question are usually database-backed web applications with graphics components for plotting genomic features or charts of one sort or another.
As any database-application developer will tell you, the database, or network connection to the database is almost always the bottleneck in an application or service. Great efforts are made to ensure database services scale well and perform as efficiently as possible, but even after these improvements are made they usually simply delay the inevitable.
Hence my frustration when I hear that "GD is making my (database) application go slow". How? Where? Why? Where's the proof? It's no use blaming something, a library in this case, that's out of your control. It's hard to believe a claim like that without some sort of measurement.
So.. before pointing the finger, profile the code and make an effort to understand what the profiler is doing. In database applications profile your queries - use EXPLAIN, add indices, record SQL transcripts and time the results. Then profile the code which is manipulating those results.
Once the results are in of course, concentrate in the first instance on the parts with the most impact (e.g. 0.1 second off each iteration of a 1000x loop rather than 1 second from /int main/ ) - the low hanging fruit. Good programmers should be relatively lazy and speeding up code with the least amount of effort should be commonsense. (0 comments)
ClearPress-99
Posted by rmp at 22:10 3rd Mar 2008
Last week saw the latest release of ClearPress, http://search.cpan.org/~rpettett/ClearPress/ . ClearPress is a basic, RESTful, MVC Perl application framework I've developed in tandem with my work at the Sanger Institute http://www.sanger.ac.uk/ .
The original aim of ClearPress was to provide a RESTful MVC framework which integrated with the Sanger's website single sign on. Having proved its usefulness with the first release of the tracking system I developed, ClearPress was spun off into a project of its own together with dependencies abstracted out of the Sanger-specific environment.
ClearPress sports a MySQL-backed ORM, automatic, extensible content-negotiation and easily-templated HTML, XML, Atom, RSS, JSON, iCal, YAML, PNG and other format views. It can run standalone, as CGI or under ModPerl::Registry.
I'm using ClearPress in most of my projects these days, both work and non-work. Blogs, document management, laboratory tracking and various other standalone apps. Hopefully soon there'll even be a dedicated site together with examples. For now you can check out the application-builder and example distributed with the package. (0 comments)
The original aim of ClearPress was to provide a RESTful MVC framework which integrated with the Sanger's website single sign on. Having proved its usefulness with the first release of the tracking system I developed, ClearPress was spun off into a project of its own together with dependencies abstracted out of the Sanger-specific environment.
ClearPress sports a MySQL-backed ORM, automatic, extensible content-negotiation and easily-templated HTML, XML, Atom, RSS, JSON, iCal, YAML, PNG and other format views. It can run standalone, as CGI or under ModPerl::Registry.
I'm using ClearPress in most of my projects these days, both work and non-work. Blogs, document management, laboratory tracking and various other standalone apps. Hopefully soon there'll even be a dedicated site together with examples. For now you can check out the application-builder and example distributed with the package. (0 comments)
Web frameworking
Posted by rmp at 23:47 31st Mar 2008
It seems to be the wrong time to be reading such things, but over on InfoQ there's a nice_article introducing web development of RESTful_services using Erlang and the Yaws high performance web server.
I say "the wrong time" as this week has kicked off the "Advancing with Rails" course by David_A._Black of Ruby_Power_and_Light fame. The course is fairly advanced in terms of required rails knowledge so it's a bit of a baptism by fire for me and a few others having never written any Ruby before.
Rails is proving moderately easy to pick up but as I've remarked to a couple of people, it doesn't seem any easier coding with Rails than with Perl. Perhaps it's because I've never done it before but I reckon it's a lot harder spending my time figuring out what the heck DHH meant something to do than it is doing it myself.
Even though it's nowhere near as mature, I do reckon my ClearPress framework has a lot going for it - it's pretty feature-complete in terms of ORM, views and templating ( TT2 ). It has similar convention over configuration features meaning it's not designed for plugging in other alternative layers but it is absolutely possible to do (and I suspect without as much effort as is required in Rails). I still need to iron out some wrinkles in the autogenerated code from the application builder and provide some default authorisation and authentication mechanisms, some of which may come in the next release. But in the meantime it's easy to add these features, which is exactly what we've done for the new sequencing run tracking app, NPG to tie it to the WTSI website single sign on (MySQL and LDAP under the hood).
(0 comments)
I say "the wrong time" as this week has kicked off the "Advancing with Rails" course by David_A._Black of Ruby_Power_and_Light fame. The course is fairly advanced in terms of required rails knowledge so it's a bit of a baptism by fire for me and a few others having never written any Ruby before.
Rails is proving moderately easy to pick up but as I've remarked to a couple of people, it doesn't seem any easier coding with Rails than with Perl. Perhaps it's because I've never done it before but I reckon it's a lot harder spending my time figuring out what the heck DHH meant something to do than it is doing it myself.
Even though it's nowhere near as mature, I do reckon my ClearPress framework has a lot going for it - it's pretty feature-complete in terms of ORM, views and templating ( TT2 ). It has similar convention over configuration features meaning it's not designed for plugging in other alternative layers but it is absolutely possible to do (and I suspect without as much effort as is required in Rails). I still need to iron out some wrinkles in the autogenerated code from the application builder and provide some default authorisation and authentication mechanisms, some of which may come in the next release. But in the meantime it's easy to add these features, which is exactly what we've done for the new sequencing run tracking app, NPG to tie it to the WTSI website single sign on (MySQL and LDAP under the hood).
(0 comments)
ClearPress-146
Posted by rmp at 23:15 29th Apr 2008
Latest release of ClearPress (v146) out to the CPAN yesterday. The ClearPress data model now implements belongs_to_through, belongs_to, has_many and has_many_through entity relationships for all you ActiveRecord lovers.
Two ClearPress-derived projects are using a half-decent test fixture system. It's really making a big difference to the development of both DECIPHER and NPG so I'm planning to bundle what can be bundled with an upcoming release. (0 comments)
Two ClearPress-derived projects are using a half-decent test fixture system. It's really making a big difference to the development of both DECIPHER and NPG so I'm planning to bundle what can be bundled with an upcoming release. (0 comments)
Atom
