PracticalWeb Ltd - websites that work for you

Drupal file_copy won't work outside of the Drupal instal

sean — Tue, 11 Nov 2008 20:52:04 +0000

The selected file could not be uploaded, because the destination is not properly configured

This "feature" has bitten me before. It's one of those annoying Drupal things which would be fine if the error message wasn't so misleading.

What's happening is that file_copy() calls file_create_path()

<?php
function file_copy(&$source, $dest = 0, $replace = FILE_EXISTS_RENAME) {
  $dest = file_create_path($dest);
?>

While file_copy() doesn't declare that it only works in the Drupal folders - file_create_path is clear about this.

<?php
/**
 * Make sure the destination is a complete path and resides in the file system
 * directory, if it is not prepend the file system directory.
 *
 * @param $dest A string containing the path to verify. If this value is
 *   omitted, Drupal's 'files' directory will be used.
 * @return A string containing the path to file, with file system directory
 *   appended if necessary, or FALSE if the path is invalid (i.e. outside the
 *   configured 'files' or temp directories).
 */
function file_create_path($dest = 0) {
?>

I need to copy files outside of the DocumentRoot to keep them private (yes I could put them in the DocumentRoot and restrict access to the folder but that is just perverse - and easy to get wrong).

So I need to avoid these functions.

Eclipse PDT and Subversion

sean — Sun, 02 Nov 2008 17:21:34 +0000

Version conflicts to be aware of

Ubuntu 8.10 (Intrepid Ibex) now ships with Subversion 1.5

You can't use a svn 1.4 client and a svn 1.5 client on the same working copy.

While the subversion integration in Eclipse is great I still prefer to do some things on the command line and some with kdesvn.

Then you need to upgrade your subclipse plugin to subclipse 1.4

If you can - then upgrade to Eclipse 3.4

But Eclipse 3.4 doesn't run the latest stable version of PDT - and last time I tried to use an Integration build it was unusable - I may try again sometime but right now I want a small upgrade.

I followed this instructions here http://subclipse.tigris.org/install.html

But Eclipse reported a "Network problem"

So I downloaded the zip file
http://subclipse.tigris.org/files/documents/906/43792/site-1.4.5.zip

Unpacked it in a folder on my local webserver, pointed Eclipse at this location and the installation worked!

Phew - I have a working Eclipse for PHP development on Ubuntu 8.10

IE6 on VirtualBox - installing the network driver

sean — Sun, 02 Nov 2008 12:03:49 +0000

One of the trial of web development is ensuring that your site works across a range of browsers.

Standards compliance is an important part of this - but testing cannot be avoided.

I work on Ubuntu, and use VirtualBox together with a licensed copy of Windows XP to test in a Windows environment, which also allows me to at least test the Sindows version of Safari.

Unfortunately Microsoft's decision to tie Internet Explorer closely with the operating system means that it isn't possible to install IE6 alongside IE7.

(Multiple IEs gets close but doesn't exactly replicate each version, and isn't compatible with the IE developer toolbar)

Microsoft have recognised the problem and do provide an Internet Explorer Application Compatibility VPC Image which allows you to create a virtual Windows with IE6 installed

It is possible to convert this image for use in VirtualBox

But I couldn't initially get the network working. What I had missed is that you have to

go into the VirtualBox settings for this virtual machine
in the to select the network section
specify the "PCnet-PCI II (am79C970A)" network adpator
boot up the vitual machine
Windows will try and install the Driver
Specify a location manually
Browse to D:\32Bit\Windows\3rdParty\AMD_PCnet

The above assumes you have already installed the Guest Additions.

This should succesfuly install the network driver - it worked for me.

I now have a true copy of IE6 on Ubuntu and can test away.

Ubuntu 8.10

sean — Sat, 01 Nov 2008 23:01:43 +0000

Having just upgraded to Ubuntu 8.10 my key observation is that it's important to read the release notes first.

http://www.ubuntu.com/getubuntu/releasenotes/810

There are a few bugs which have affected me

there is no bluetooth support in KDE
my wifi card only works in the US
(there's a workaround and I'm back online)
Dual head display with xrandr is causing problems
(my screen flashes and KDE is putting widgets in parts of the screen I can't see)
Subversion has moved to version 1.5 and getting Eclipse to use a compatible svn library is proving tricky

On the plus side KDE 4.1 is looking very pretty, and the move to subversion 1.5 will be very useful in the long run.

I had hoped to get my 3G connection working as a previous update seems to have broken it - but so far still no joy.

update:
There's a solution for the Kubuntu Intrepid Dual Monitor Blinking
issue here: http://blog.nixternal.com/2008.10.22/kubuntu-intrepid-dual-monitor-blink...

How to read compressed Apache docs

sean — Sat, 18 Oct 2008 10:26:38 +0000

One of the great things about Debian/Ubuntu is the consistency with which system documentation is placed in /usr/share/doc/

Assuming you have a web server installed this documentation is then available at http://localhost/doc/

But to keep filesize down - many files are compressed and have to be decompressed before you can read them.

The browser can actually do that compression on the fly - if it receives the right headers - what it needs to be told is that the content is compressed plain text.

To achieve this add the following lines to your default apache config for the docs directory (there should already be a docs section)

<Directory "/usr/share/doc/">
AddEncoding x-gzip gz tgz
AddType text/plain .gz
</Direcory>

svnversion

sean — Thu, 16 Oct 2008 10:25:40 +0000

I just discovered a nifty little utility

Often working copies are only partially updated, this command gives a very quick command-line summary of the state of the working copy.

It's not as rich an output as kdesvn which will colour code all the updateable files, but its a standard part of svn so should always be available.

Name

svnversion — Summarize the local revision(s) of a working copy.

Synopsis

svnversion [OPTIONS] WC_PATH [TRAIL_URL]

Description

svnversion is a program for
summarizing the revision mixture of a working copy. The
resultant revision number, or revision range, is written to
standard output.

TRAIL_URL, if present, is the trailing portion of the
URL used to determine if WC_PATH itself is switched
(detection of switches within WC_PATH does not rely on
TRAIL_URL).

Examples

If the working copy is all at the same revision (for
example, immediately after an update), then that revision is
printed out:

$ svnversion .
4168

For a mixed-revision working copy, the range of
revisions present is printed:

$ svnversion .
4123:4168

Number Crunching : Database vs OOP

sean — Sun, 05 Oct 2008 09:50:21 +0000

For the last few months I've been working on a project that is part website and part data processing.

In theory the website is the bulk of the project, but in practice processing the data has taken a disproportionate amount of resources.

Data exchange with another organisation is always tricky - and in this case we have incoming data that doesn't conform to specification, business rules that change, and a daily data import to run.

The code I've been working on has been through a couple of versions, and now I have in mind a third

process data from file a line at a time, comparing each record with the database individually
load all data from file into a temporary table, apply all rules as SQL updates to the whole table - joined to other tables where comparisons are needed.
load all data from file into a temporary table, retrieving data joined with the comparison data - process in batches say 1000 lines at a time.

Version 1 was impossibly slow, and it was bad code we inherited so it had to go.

Version 2 is much faster, but as the business rules are documented per record and we're performing set based operations it has proved very hard to verify that the code matches the required logic. It's also very hard to unit test as all the logic is performed in the database.

Version 3 probably won't get written now - as is the way of programming - you always learn how you'd do it next time.

I have one colleague in particular who thinks that the database should do the heavy lifting as that is what it is optimised for, and to an extent I agree. But the more I get into unit testing the more I want to write code that neatly packages up logic in a testable way.

It has also become apparent just how important it is to have code that follows business rules in a clear way. It's not enough that the output should be right. I recently handed this code to a very capable developer - he's normally so polite - but trying to reconcile a flowchart with a long series of SQL queries tested his patience.

In the end leaning so hard on the database has made the system much harder to read, test, change, and maintain.

I wonder if version 3 would be any slower anyway - it might even be faster. In fact version 3 could be run multithreaded - or split into multiple processes....

Software Estimation

sean — Sun, 05 Oct 2008 09:14:13 +0000

This is another great book by Steve McConnell.

His empahsis is on large projects with thourough development processes; he gives many examples of rigorous studes that back up his advice.

If you follow all the advice in this book - you'll be able to make very accurate estimates with a clear idea of the risks involved.

Even if you don't need so much detail - the book provides many tips to greatly improve the accuracy of even rough calculations.

Tip #1 Distinguish between estimates, targets and commitments.

Tip #31 Look for something you can count that is a meaningful measure of the scope of work in your environment.

Drupal multisite with shared tables

sean — Thu, 07 Aug 2008 11:42:21 +0000

Drupal allows you to create multiple sites sharing some or all of the same code and optionally sharing some database tables.

The documentation for the Drupal settings file
http://api.drupal.org/api/file/sites/default/settings.php/5/source

includes the following

<?php
/*
 * To have all database names prefixed, set $db_prefix as a string:
 *
 *   $db_prefix = 'main_';
 *
 * To provide prefixes for specific tables, set $db_prefix as an array.
 * The array's keys are the table names and the values are the prefixes.
 * The 'default' element holds the prefix for any tables not specified
 * elsewhere in the array. Example:
 *
 *   $db_prefix = array(
 *     'default'   => 'main_',
 *     'users'     => 'shared_',
 *     'sessions'  => 'shared_',
 *     'role'      => 'shared_',
 *     'authmap'   => 'shared_',
 *     'sequences' => 'shared_',
 *   );
 */
?>

This allows you to have several related sites which might (for example) share user tables so that users have single sign on, while having different tables for content.

Note that to get this functionality the sites have to share a single database.

I've never quite seen the point of multi site Drupal installs for unrelated projects, in theory this could save on upgrade work - but in practice I'd rather have the greater control that separate install gives you.

For a family of websites I can see a multi site install with some shared data being a really powerful and effective solution.

Don't make me think!

sean — Wed, 06 Aug 2008 08:43:44 +0000

This book is well written, short, and clear.

It has reassured me about much of what I do - and given me some pointers for areas I can improve on.

One of the great things about this book is the way it addresses the combination of technical and political issues that web projects can get bogged down in - it has a nice section called "Help my boss wants me to ___" with some good sample letters to the boss.

It really brings home that conventions are useful - and you need to test the site on users - little and often.

I'm a bit surprised that the second edition doesn't deal with web 2.0 issues. There is a lot of (good) discussion on navigation hierarchies - but no acknowledgement of the more free flowing practices like content tagging.

There is useful guidance on forms - but nothing on the newer issues of AJAX feedback mechanisms.

But the book can be forgiven these faults because its just so easy to read and absorb, other issues can be left to further reading.