Trying to make sense of theme inheritance in Magento, I have started to put a simple script together to help navigate the themes installed on a shop. When I say simple, I mean very simple. Jump straight in and have a browse:

http://magento.consil.co.uk/themes.php

This script is installed in the root directory of our 1.4 test installation.

What it gives you is a layered approach. First you select the interface, then the theme within that interface (try default->modern for example).

The script will then list all the layout files that make up that theme, and show you where those layout files come from, i.e. which parent themes they are inherited from. A complex theme may consist mostly of its own layout files (the iPhone theme for example). A simpler theme may override just a few layout files, choosing to fall back to the default theme, or the new “base” theme in Magento 1.4.

For each layout file, the handles are listed. Each handle provides some unit of functionality in the theme – usually a block or a widget – some simple, some very complex. The next step will be to document what all these handles do, and how they get called up. Some are invoked from the Magento core, some from modules, and some from other templates in the theme, so it can all get very complex very quickly, leaving the average themer lost when they only wanted to change a few aspects of a theme.

It is the content of the handles that are the really important part of a theme. The layout files just provide a convenient way to package together a number of handles in one place. The same handle can appear in multiple layout files, and they are simply chained together in alphabetic order (of the name of the layout file).

What the layout files do is allow a module to keep all its handle-based theme functionality in one place. A theme developer could also use it to keep all their theme additions in one place too – there is no point spreading that functionality all over a custom theme by overriding dozens of layout files if all you want to do is tweak menus and blocks.

By clicking on the “handle” link in the script, the table gets inverted – now you get to see all the layout files that each handle is included in.

Selecting a handle will list the contents of each instance of those handles – the XML instructions to the theme engine.

To take this script further, it would make sense to interpret the content of the handles. Ultimately it should then be possible to do searches on those interpreted instructions to find, say, where a particular block or image is displayed from.

Feedback is welcomed – is this any use? Should I publish the script? Does anyone want to expand on it, perhaps packaging it into a module to help theme developers?

Download the script here: themes.zip (contains theme.php, to be copied to your Magento root directory).

Follow me on Twitter

Introducing Ionic, a CSS framework put together to specifically handle layouts with more than 2 columns in CSS.

The framework, originally written by Jason under the name ‘fluid1′, has been used on our client’s website, IEMA.net, since 2005. The aim of the project was to make it accessible to all the common browsers, whilst keeping the HTML markup as semantic as possible.

Currently the script only supports a maximum of 3 columns, but there is no reason why the algorithm can’t be extended to support any number of columns.

Ionic has its own microsite under development where you’ll be able to grab the latest releases and find out more about the framework. The link is:
www.consil.co.uk/ionic.

As a part of my integration tasks, I had the following sequence of systems to deal with:

Sequence of systems handling login

This is the chain of applications that handle a single-signin for a user. At one end we have the Member Management system, written in ASP Classic, that acts as the main authentication system. For various reasons we had to use that system as the main login system.

Once a user logs in, a HTTP request is sent to the Xaraya CMS, along with user details, and a custom CMS module handles the creation of a new user, or the update of an existing user, and logs that user in.

Further down the line is a Vanilla forum application. When the user logs in or out of Xaraya, an event trigger will then send user details to Vanilla (where the user is created or updated) and then logged in.

Logging out works in just the same way, except a different message is passed down the chain.

The result of a user logging in at one end, is that they will automatically be logged into all the applications in the chain. Given a choice, I would have used something else that all the applications authenticate against, but we were tied to various technical limitations.

One specific action that needed to be carried out, was the passing of cookies down the chain, from one application to another. Each application needed to accept cookies from the browser (where available) and also needed to pass new or updated cookies back to the browser. The browser only interacted with the Member Management System, and so the cookies needed to be passed directly between the applications.

To complicate things more, we have a mix of ASP and PHP, and two separate servers. The servers shared a single domain though, so we were easily able to share the cookies across the servers.

One particular step is the passing of cookies across the ASP system. If we ignore the cookies consumed and generated directly by the MMS, then we have cookies that pass through in both directions.

To handle the flow from left-to-right, the following code is used. First initialise an object to send the HTTP message to the CMS (the message to say “log me in”). We are using SOAP to communicate with the CMS.

  ' XMLHTTP object used to send the outgoing message (to the CMS)
  set xmlhttp = server.CreateObject("MSXML2.ServerXMLHTTP")

  ' Set up the destination and headers; open the connection
  xmlhttp.open "POST", "http://example.com/soap/login", false

Now look through all the cookies that have been sent to *this* page, and pass forward any that are of interest further down the line.

  ' Work through a list of cookies that need to be passed in.
  ' Other cookies (e.g. for forum sessions)
  ' may also be available here and may be useful in logging in and out,
  ' helping to retain sessions across logins.
  ' Loop through all cookies supplied by the browser.
  cookieCount = Request.Cookies.Count
  If cookieCount > 0 Then
    For kk = 1 To Request.Cookies.Count
      cookieName = Request.Cookies.Key(kk)
      cookieValue = Request.Cookies.Item(kk)
      If instr("XARAYASID|lussumocookieone|lussumocookietwo|LussumoUserID|vanillasession",
          cookieName) then
        ' If the cookie is in the list to pass on, then do so.
        xmlhttp.setRequestHeader "Cookie", cookieName & "=" & cookieValue
      End If
    Next
  End If

Finally, after creating the SOAP payload, send the request to log in.

  xmlhttp.setRequestHeader "MessageType", "CALL"
  xmlhttp.setRequestHeader "Content-Type", "text/xml"

  ' Send the message and wait for the response
  xmlhttp.send(SOAPpayloadBody)

  ' wait for response
  ' xmlhttp.waitForResponse(3)
  While xmlhttp.readyState <> 4
    xmlhttp.waitForResponse 200
  Wend

That gets the local browser cookies for this domain, passed on to the CMS. Note we only pass on those cookies that will be of interest, i.e. Xaraya and Vanilla Forum cookies in this case.

Now we need to handle the cookies coming back from he right, as we need to pass them back to the browser. After waiting for the response, it is handled as an XML SOAP message.

  ' Anything other than 200 means error.
  If err.number = 0 And xmlhttp.Status = 200 Then
    ' Collect the data returned.
    Set xmldom = xmlhttp.responseXML
    ' Process the returned data...

Now we are able to look through the cookies that have been returned to us.

    '  We now want to sift through any cookies sent back and pass them on to the
    ' current browser. This means extracting the cookies from the response to the
    ' Xaraya login request, and inserting those same cookies into the current page.

    ' Get the response headers.
    ' This is where we would find the returned cookies.
    strHeaders = xmlhttp.getAllResponseHeaders()
    ' Extract the cookies from the headers
    hArr = split(strHeaders, vbCrLf)
    for kk = 1 to ubound(hArr)
      if instr(hArr(kk), ":") > 0 then
        if left(hArr(kk), instr(hArr(kk), ":")-1) = "Set-Cookie" then
          ' Get the cookie name and value (we will include the path and domain
          ' as part of the 'value').
          ' The name is between the first ':' and the first '='. The value is
          ' everything after the first '='.
          cookieName = trim(mid(hArr(kk), instr(hArr(kk), ":")+1, instr(hArr(kk), "=")-instr(hArr(kk), ":")-1))
          cookieValue = trim(mid(hArr(kk), instr(hArr(kk), "=")+1))

          '  Include the list of cookies that you want to be passed on here.
          if instr("lussumocookieone|lussumocookietwo|LussumoUserID|vanillasession", cookieName) then
            ' Send the cookie on to the current browser (unchanged, i.e. same path and domain)
            Response.AddHeader "Set-Cookie", cookieName & "=" & cookieValue
          end if
        end if
      end if
    next

That’s it. All cookies in that list will be returned to the browser, so if any have been created in the process of logging into the CMS or the Vanilla forums, then they will be passed back and upon visiting those applications the user will find they are logged in.

In a similar way, when the user logs off, sessions are cleared (or flags are set in the sessions to indicate the user has logged off) and some of the cookies are cleared by the user of dates in the past.

I have omitted some of the code for clarity, specifically the declarations of the variables, but the guts of the process is here, and I hope it will prove useful to others. If you need any of this explained or expaned, please ask in the comments section.

Follow me on Twitter

The Final Four

Down to last four; CakePHP, CodeIgnitor, Kohana and Zend. The test is to get each framework to print ‘Hello World’ on its own ‘/hello/’ page. For each test I will record the time taken to:

1. Work out how to complete the task, which will indicate how simple it is to use, and how good the documentation is.
2. How long the script takes to execute, to measure the performance. This will be averaged over 5 attempts

CodeIgniter

Test completed in 15 minutes. I only had to use two pages from the documentation, One to print, one to remove index.php from the url.

Execution took 0.031 seconds.

CakePHP

Took an 1 hour. Far too much made of the installation. File permissions to deal with, database configuration, why?! Did I say i wanted caching enabled and connect to a database? Then when I finally got to writing a controller, it didn’t let me do a raw echo. So I create a view, and find it comes with a default layout?! Takes me another 15 minutes to look through the manual to find how to switch it off. Too much crap to get through.

Execution took 1.388 seconds.

Kohana

It was difficult to accurately analyse this one because it’s so similar to CodeIgniter, and I think it’s assumed that people using it are familiar with CI too. The documentation isn’t at the level of CIs, but it’s still very clear. Took me slightly longer to find how to print ‘Hello World’ and a lot more pages, but it came with an example .htaccess file to speed up removing the index.php in the URL. 15 minutes.

Execution took 0.039 seconds.

Zend

Zend took 30 minutes to set up correctly. It’s very much start from scratch with Zend, nothing is there for you. You need to create the directory hierarchy, and all the files within it outside of the library itself. However the documentation was excellent, and I knew where I was going very quickly, it just involved a lot of ‘admin’ to get there.

Execution took 0.078 seconds.

The Result

I had planned to do some more tests for the final four, but it wouldn’t be justified as I’ve already made up my mind!

4th place – CakePHP

Cake is a good framework no doubt about it. One of it’s main perks that everyone raves about is its automatic code generation dubbed ‘automagic’. I could see this being an advantage for a rapidly expanding website, and for experienced PHP developers, but for me though it’s like learning to fly a plane with autopilot.

Another problem for me is the documentation. For argueably the most complex framework of the lot, it’s poor.

Finally the features, or classes, that it comes aren’t as extensive as the other three frameworks. There are many third party plugins to be found in ‘the bakery‘ but they should be integrating some of the more useful ones into the framework release so you don’t have to go looking for them.

3rd place – Zend

Zend is a damn nice library of code, and has some great components like lucene search. In terms of flexibility in the applications it can produce I’d probably put it in the number 1 slot. The documentation is also brilliant, especially the reference guide and the quick start tutorials. At this level, it’s ultimately down to preference.

2nd place – CodeIgniter

Sooo we have a winner, but why did it pip CodeIgniter? CodeIgniter is brilliantly simple, flexible, but best of all it’s the fastest framework on the market. The documentation is good, especially the screencasts. It also boasts the advantage of working with both PHP4 and PHP5.

1st place – Kohana

As this is just an evolution of CodeIgniter you can get started using both its own and the CI documentation. Why is it better than CodeIgniter? Two reasons:

• It’s strict PHP5 and takes advantage of all its new features. This makes it more future-proof and robust for building object-oriented applications.
• It’s ran by a community as oppose to an organisation.

And that’s it! Enough writing, more programming!

Read part 1

This search has gone on for years. I am looking for a newsletter system that is easy to install (to most likely LAMP-based), allows sign-up to multiple newsletters, and is reliable and flexible.

The most important thing is that once configired, I need to be able to hand over the keys to a client so that they can manage it from that point on. That is the hardest part, because no matter what I try, there are always highly technical aspects to sending out a newsletter that really stump the non-technical clients.

This is what I have come up with (and tried) so far:

PHPlist (Tincan)

PHPlist Logo

This has some lovely features, but some pretty dire ‘gotchas’ that can trap an unwary user. For a start the admin screens are as ugly as hell, and the terminology can make it hard to fathom out what each option does (for example, to create a draft newsletter, you need to use the ‘send a message’ option). It is just not intuitive.

Problems for end users involve its uncanny ability to drop stylesheets when editing templates. Another is signing lists of imported users up to text-only e-mails even when ‘HTML’ is selected. Yes these can be worked around, but only if you know the problems exist.

On the plus side, the e-mail sending system is brilliant. You queue messages to send, and leave it to do its job. If it fails (and anything that involves a browser window staying open will fail) then you just start it again and it carries on from where it left off. It keeps track of who a newsletter has been sent to, so you can requeue a newsletter every day until the next newsletter is ready, and it will send just to new subscribers.

The ability to pick of bounces from an IMAP or POP3 mail box is also great. Each bounce will increment a counter, and subscribers can be disabled when a preset number of bounces is reached.

This system also supports tracking, counting each individual e-mail as it is opened, using an image embedded in HTML e-mails.

Another downside is the lack of APIs, making integration difficult. Signing users up to a list automatically, while they are signing up to a CMS, is impossible without some major hacking.

The problems with PHPlist stem from it being around a long time. It has grown and grown over the years, but is creaking at the seams. Everything it does is great, but the way it does it is not so hot. It is time, IMO, to throw away the old code and start again, with all the same features, an established framework (e.g. Zend or CakePHP) and it will be a killer application.

I must also add that I believe PHPlist got the data schema right from the start. Most other newsletter systems I have looked at have gone for a much more simplistic approach, that makes it much harder for them to advance their products beyond very simple functionality.

With all that it is, I do use it, and find it great at what it does. What I can’t do, howver, is roll it out to clients, because it results in nothing but hassle from the non-technical users who just can’t get to grips with its mix of high-level features, and low-level ‘black art’ knowledge that is needed.

poMMo

poMMo logo

This one has to get the award for ‘hardest to find mailing list manager’. It was formally the bMail project and is hosted on SourceForge. Trying to find it when you can’t quite remember its ‘web 2.0′ name is quite an effort, but the result is well worth it.

Checking the project’s subversion repository, it does not seem there has been any activity on this project since August 2008, which is not very encouraging.

First of all, the admin screens are lovely. They are smooth, AJAX/jQuery-based, clear and well designed. The whole thing feels smooth – it works with you.

The first thing to note is that it does not support multiple mailing lists. In fact, it does not support any ‘mailing lists’ at all. However, what it does provide is a means to mail out to groups of users depending on attributes of those users.

For example, you could provide a series of checkboxes for a subscriber to select what subjects they are interested in. When sending out a newsletter, the author would send it to all subscribers who have expressed an interest in the subject of that newsletter. Now, that does sound rather like a multiple newslist system, but the subtle difference is that there is no ‘newslist’ object in the database – anything that looks like a newslist subscription is specified by the administrator, and not enforced as any kind of fundamental part of the system’s structure.

I think that approach works well. The system is simpler, and the flexibility is increased. There are a few downsides though.

The first downside is that without a newslist for a user to subscribe to, there is no place in which you can find out exactly when a user subscribed to that newslist. That means sending out additional copies of a newsletter to late subscribers simply cannot be done. Remember PHPlist knows who subscribed to what and when, and so is able to send out ‘catchup’ newsletters so a subscriber will always get the latest newsletter very soon after subscribing.

The other downside is in organisation. Without being able to group newsletters into newslists, it is much harder to provide archives organised by subject. On the other hand, the archives are there, with very easy access to the newsletters from a web-based front end.

The e-mail sending backend is both genius and a little frightening for a control-freak such as myself. It also misses a few tricks, I think.

In order to send e-mails, the system first creates a list of who to send to (in a text file, I believe). It then spawns a process through the HTTP protocal to send as many e-mails as it can in the PHP timeout period (you would set this to as long as possible). Before that script times out, it spawns another process to carry on where it left off, and so on until all the e-mails have been sent, whereupon a lock file created right at the start is released.

This is genious in that the developers have worked out a way to run a background script that can keep going even when the browser window is closed. It is frightening for much the same reason. I just don’t feel comfortable letting something like that loose on my server. It could be running all night in some kind of endless loop, and I would never know until I get irate e-mails from subscribers saying that I have filled up their inboxes.

Another minor flaw in the e-mail sending is that no record is kept of who each newsletter has been sent to. Doing that would make it very easy to sort out who has received a copy and who needs a catch-up copy sent. Following from that, without records of e-mails sent, there is no place to hang any flags to say whether that e-mail has been opened and read, so e-mail tracking is out. Clients need to know these things: how well read are those newsletters? Everyone has a master up the chain to report to, and someone up that line, often holding the purse-strings, likes simple measures of performance.

My ideal newsletter system would contain the database and functional features of PHPlist, with the administration front-end of poMMo.

Other Systems

I’ll add a few more when I get the time. I’m mainly looking at Open Source newsletter systems, aimed at sending to self-subscribed users (i.e. not bulk spam systems). Integration with an existing site or CMS is high on the priority list, along with ease of use for non-technical users. Other systems we will be evaluating are:

• ListMessenger – Free light version, but dirt cheap Pro version with all the features you would want. This one seems to include captchas for registration.
• Dada Mail – Old, well known, Perl-based, and it looks like a pain to install.
• Sympa – Again, Perl. This one seems to have been designed by engineers. That is to say the system looks very robust and complete, but there is little gloss to the system. Like Data Mail, you need root access to install it, so you have to be careful about dependances with other modules on your server.

Follow me on Twitter