For many years the favored editor for Perl developers fell under the same umbrella as the favored editor in *nix. A major fight between Vi and Emacs, and all the other options trailing behind somewhere.

Over the past year, a team of developers has been working to change that. The result is the Padre editor, or the Perl Application Development and Refactoring Environment. The Padre team recently released their 0.5 revision and let me tell you, it is fantastic.

With the 0.5 release there are two particularly interesting developments. They are a Mac standalone package and a self-contained Windows package. This makes Padre extremely easy to install on the two most common desktop systems and puts it on level ground with all your other editing options. If you don't read another word, grab a copy and try it. * (... screenshot after the break)*

I've never really liked dealing with creating forms. You have to write a butt load of tedious HTML, validate it, and then return error messages as needed. If you want it to be fancy and pretty, you have to write a bunch of CSS to put your error messages where you want and make them shiny. Luckily, there are things to handle this.

Being a Moose fanatic, I am most partial to the latter, HTML::FormHandler. It's extremely easy to use, and requires zero configuration file crap. You get validation as Moose types/constraints, it's very cool.

So, back to me hating having to build forms. This includes building a form for something that already has enough information to build a form from, say, a database schema. My dream has been to be able to read a database schema (preferably DBIx::Class), and generate a form based on the data types specified in said schema. This is completely doable! Heck, even I drafted up some code to get some preliminary form fields from a database. The following is from the CMS I'm working on, using HTML::FormHandler and DBIx::Class to get things done. This is extremely proof of concept code, and doesn't actually have any logic to determine what kind of form element a given schema entity should be. That, however, should be fairly trivial and I will undoubtedly be writing a follow up to that effect. Go here to check out the source: http://gist.github.com/230652. Enjoy.

So I recently decided to convert the config file for a new app I'm working on to JSON. Why? Well for one, it's a lot easier to express perl data structures in, and it's also extremely easy to read. Check it out!

{
    "name": "Fnargh",
    "Model::Database": {
        "connect_info": [
            "dbi:SQLite:fnargh.db",
            "",
            "" 
        ] 
    } ,
    "session": {
        "flash_to_stash": "1" 
    }
}

Started off a bit rocky, but thanks to http://www.jsonlint.com I was able to pick at my config file until it was valid. It formats it nicely for you as well!

The best part? You can take the perl data that you use in your Catalyst app, and do something like this:

squishface:~ dhoss$ perl -e "use JSON; print to_json({ my_data => [ 'thing1', 'thing2', 'thing3' ] });"
{"my_data":["thing1","thing2","thing3"]}

JSON formatted for your config!

Recently while working on getting tests done for Catalyst::Devel so it could be shipped, and I absolutely could not figure out how to get the test for the development server to work. So, I queried #catalyst-dev about it, and my good friend mst told me to use strace to figure out what was going on behind the scenes. In this case, I used dtruss since I'm a Mac user and for some reason the usual *nix tool doesn't work on the Mac platform.

Basically, what went down was this:

1. sudo dtruss -f prove -l t/author/optional_http-server.t
2. a whole load of rather cryptic output (see below for a snippet)
3. analysis and correction

What I found was that since the Catalyst development server is chock full of system calls that are chock full of information on what's going on (and going wrong) behind the scenes, dtruss was the perfect tool for grabbing messages being relayed between processes and not necessarily being spewed out to my terminal. Here's a small snippet of the problem area I (well, mst actually) discovered:

46665/0x117e04: stat64("/opt/local/lib/perl5/vendorperl/Catalyst/ScriptRunner.pm\0", 0x7FFF5FBFEE90, 0xFFFFFFFFFFFFFFFF) = -1 Err#2 46665/0x117e04: stat64("/opt/local/lib/perl5/5.8.9/darwin-2level/Catalyst/ScriptRunner.pmc\0", 0x7FFF5FBFEF20, 0x0) = -1 Err#2 46665/0x117e04: stat64("/opt/local/lib/perl5/5.8.9/darwin-2level/Catalyst/ScriptRunner.pm\0", 0x7FFF5FBFEE90, 0xFFFFFFFFFFFFFFFF) = -1 Err#2 46665/0x117e04: stat64("/opt/local/lib/perl5/5.8.9/Catalyst/ScriptRunner.pmc\0", 0x7FFF5FBFEF20, 0x0) = -1 Err#2 46665/0x117e04: stat64("/opt/local/lib/perl5/5.8.9/Catalyst/ScriptRunner.pm\0", 0x7FFF5FBFEE90, 0xFFFFFFFFFFFFFFFF) = -1 Err#2 46665/0x117e04: stat64("./Catalyst/ScriptRunner.pmc\0", 0x7FFF5FBFEF20, 0x0) = -1 Err#2 46665/0x117e04: stat64("./Catalyst/ScriptRunner.pm\0", 0x7FFF5FBFEE90, 0xFFFFFFFFFFFFFFFF) = -1 Err#2 46665/0x117e04: writenocancel(0x2, "Can't locate Catalyst/ScriptRunner.pm in @INC (@INC contains: /Users/dhoss/web-devel/Cat-Runtime-5.8/t/author/../../lib /Users/dhoss/web-devel/Cat-Runtime-5.8/lib /opt/local/lib/perl5/siteperl/5.8.9/darwin-2level /opt/local/lib/perl5/siteperl/5.8.9 /opt/", 0x2D0)

That's just a snippet. Somewhat cryptic, but it's all right there. Check out this line: 46665/0x117e04: writenocancel(0x2, "Can't locate Catalyst/ScriptRunner.pm in @INC (@INC contains: /Users/dhoss/web-devel/Cat-Runtime-5.8/t/author/../../lib /Users/dhoss/web-devel/Cat-Runtime-5.8/lib /opt/local/lib/perl5/siteperl/5.8.9/darwin-2level /opt/local/lib/perl5/site_perl/5.8.9 /opt/", 0x2D0). Derp! I was running this test in Catalyst::Runtime trunk/ that doesn't have my ScriptRunner classes merged yet. So, I reran this test (using dtruss -f again) under my branch, found out that the options being passed to the development server were incorrect, corrected, svn committed, and can now take another step (finally) toward shipping Catalyst::Devel!

There's a new kid on the block for web application development, and it's called Plack. I'm not going to go into depth on explaining it here, as the website listed previously does a stellar job of that, but I'm going to explain and talk about what got me started on it, and what made me love it.

I've been a pretty avid MovableType fan since I first got it running. It Just Works(tm). I've always ran it under Apache, however, so when I made the big and permanent transition to nginx, I was befuddled as to how to get it to run under my new toy. A kind sir by the name of Jay Shirley recommended I use Plack to get my old love running under nginx. I talked to the creator of this mystical new being, Tatsuhiko Miyagawa on its IRC channel, #plack. We went back and forth for a few iterations until w(h)e got his mt.psgi script (listed here) working so that MT would be okay with new installs.

The result: A mostly working MovableType installation! The only problem is, I have a MT database dump from MySQL that I either A) need to convert the data portion of the dump to something PostgreSQL (what I've converted to for life) will like, or B) suck it up and set up a MySQL instance for this one app. MT itself runs just fine, it's completely my fault that anything on the database side doesn't work. Miyagawa reports that previous installations work just fine with this, so, as I said, I will need to investigate which route I want to take to get this data into a database for MT.

Either way, my experience was fantastic. Plack is really easy to set up. All the work is done for you, and you can still run things under FastCGI (if you're so inclined). It's a great bridge between server software and a web application, so we're not stranded somewhere in between holding on to the connecting ends of a bridge screaming "HOLY FUCK THIS HURTS AND GOD I HOPE IT HOLDS TOGETHER IF WE HAVE TO CHANGE ANYTHING".

Kudos to you miyagawa, I'm pretty sold on using this from here on out for web apps I write in the future (mostly Catalyst based, for which we have Catalyst::Engine::PSGI!)

UPDATE: Here's the configuration I'm using for nginx: http://wiki.catalyzed.org/codesamples/dhoss/placknginxconf

Scenario

You're trying to send an email (via Catalyst::Plugin::Email, MIME::Lite, Email::Sender::Simple, your choice) and you would like to be able to stuff your canned email messages into their respective Template::Toolkit templates. One way to do this is use Catalyst::View::Email::Template, which is pretty cool, but forces you to use Catalyst::View::Email, which some people aren't too keen on. So, you can do something like this:

sub email_stuff : Local {
    my ($self, $c, $review_id) = @_;

    $c->stash(
        review_url    => $c->uri_for("/thingy/${review_id}"),
        status_msg => "Thank you for your feedback!",
        template      => 'view_all.tt2',
    );

    $c->email(
        header => [
            From    => $c->config->{'from'},
            To      => $c->config->{'to'},
            Subject => "Emaily stuff!",
        ],
        body => $c->view('TT')->render($c, 'review.tt2'),
    );

    return; 
}

The line $c->view('TT')->render($c, 'review.tt2') allows you to have your view render your template for you using all the things you put into $c->stash. Pretty cool eh?

I've had a need for some time now to have an ability to make recursive calls to an SQL database with out completely melting down my database server using literal recursion for SQL calls (namely, a for loop that does a select for EACH record I need to retrieve, easy to see how things would go up in flames.), and have played around with a few ideas.

There are nested sets, which are kind of cool, because they make a nice bridge between programming's want for trees/graphs, and SQLs stickler for their organized sets. It's a neat concept, having the immediate knowledge of what is to the left and right of a given "node", but this makes it KILLER to make inserts into. You have a whole ton of knowledge to gather before you even insert any data for a new record.

Adjacency lists are a bit like nested sets, but they pretty much keep track of the parent id as opposed to what's on each side of the node. They're fast, but once again, munging must be done before you can get correct/normalized results and good inserts.

Materialized paths are currently my favorite, but by no means are they the be all end all solution. Especially since different RDBMSes are better optimized for other such methods (Oracle's CONNECT BY, for instance). Materialized paths are pretty much just a path up to the node you are selecting of its parents. For instance:

1.2.1.1.3.5

would be the 5th child of the 3rd child of the 1st child of the 1st child of the 2nd child of the parent whose primary key is 1. It's pretty simple. Selecting from and inserting into with this is pretty easy as well.

DBIC::T::Recursive is probably going to, in one way or another, make use of each of these (maybe more, given there is likely going to be a -handle => $methodgoeshere where $methodgoeshere could be matpath, adjacency list, nested set, connect by, etc.), and will eventually be optimized per DBMS to suit the best type of recursion.

The best part, it will all be abstracted away and will Just Work. It's going to be a mighty project, but I'm looking forward to making more progress on it.

So, I recently finished (for certain values of "finished") my Google Summer of Code grant. It was very fun, stressful, and eye opening. I learned a LOT. A lot more than I could possibly encompass in a single (or infinite, really) blog post, and way more than the realm of computer hackery involves.

I remember starting the project, getting my SVN commit bit, svn co'ing the code, and going "Holy shit. I am in WAY over my head." But I kept at it. I was stubborn, and MOST important, asked incessant questions, probably much to the chagrin of my mentor and the very kind souls that held my hand through all of this.

I finally got my head wrapped around most of the code, and went, "hey! I kick ass! I'm contributing, I'm smart, I'm getting things done, I'm good enough, and gosh darn it people like me!." Then we got into stage two. The process preeeetty much started all over again. I had no idea where I was, I was writings tests for things I didn't quite understand, and did a great deal of staring at my computer screen. I wasn't, however, committing to the wrong places or fucking up the repo (as much) anymore! Progress was being made. Slow, but there.

Software developers often like to think of themselves as artists. We like being unquantifiable and somewhat mysterious. This behavior has roots in the fact that humans generally like to be thought of as unique and special. It also has roots in the all-too-common 'Now I got you you SOB' that many development managers / executives like to play with technical staff (which usually plays out as 'I know I changed the requirements 5 times since then, but you said it'd be done by Tuesday!! You bastard!') The natural result of which is a desire not to give hard estimates so you can avoid being slapped with them later.

The fact of the matter, though, is that applying numbers or at least equations to the process of software development is not that hard. There are a number of concepts that are already out there and are fairly commonly accepted that can be applied toward that goal. Today I want to introduce some of them. I want to take some of these equations and break them down so that we can understand how they operate, and how we operate through them.

I'm a jerk. I failed to give credit to Kieren Diment who helped me with some of the finer points of this script. Thanks Kieren!

Web spiders are not a new idea.

A well written spider can gather a whole lot of data without any labor beyond actually writing it, and in a speedy manner.

I've put together a spider for a recent project I've been working on, using Web::Scraper.

So here we go!