If we move forward to the present day, then in Web 2.0 the idea of bringing in content from other sources is the distinct flavour of the month. In some cases this is still just images (for badges), but the norm is either to have RSS feeds (essentially bringing in HTML code, usually with no filtering of what is displayed) or JavaScript files (running programs in the end-user’s browser).

So you can imagine the following fictitious scenario. A well-known charity runs an extremely successful campaign using all the tricks of the social web trade. They’ve had fund-raising plugins downloaded & added to thousands of WordPress blogs; widgets on MySpace to email the key people concerned; badges appearing left, right & centre and RSS feeds being devoured. All being served from www.examplecharitycampign.org, which once the campaign is over is no longer required and the domain registration lapses.

In those circumstances it could be very easy for someone else to step in, and cause havoc – unsuspecting people could find themselves on what looks like a legitimate donation page, email addresses could be harvested if those were in forms, JavaScript programs not adding visible content to a page but still active (waiting for a browser security loophole, analysing what else is on the page, etc.), hidden links in RSS feeds to help with search engine ranking schemes, etc.

So it is important to plan what happens at the end of a campaign right from the start, rather than casually abandon it or start the process once it’s up and running. Amongst other things you need to decide whether you’re going to keep the domain name for the long term (in which case it needs to be treated like your organisation’s domain), where content is going to be made available (e.g. it could come from your organisation’s domain or from another ‘permanent’ address such as YouTube) and develop methods to try to keep in touch with your supporters/activists (not the easiest thing with viral marketing).

The typical patterns for badges/widgets are either to provide ongoing functionality (e.g. Flickr badges or Google Adsense) or to offer something time-limited for a specific campaign. For someone like the DEC where the timescales are compressed the second option isn’t really worth pursuing. They do have a Rapid Response Network for more traditional media outlets, e.g the main television and radio networks, the national newspapers, etc.

I think they could benefit enormously from providing a badge/widget that is available all the time. If there’s an appeal happening then content connected with that is delivered, otherwise it’s empty. This would allow the ongoing development of a network that could be brought into play (more or less) instantly that a new appeal is launched.

In the main this would mean that it would need to be designed to appear along with other small elements of a page (e.g. in the sidebar of a blog). It would be difficult to make allowances for a large banner ad to appear & disappear in the look and feel of most sites. However this obstacle could be overcome with a simple API (e.g. providing people with a web address that either said “yes, there’s an appeal on now” or “no, there isn’t”) so that with some additional programming the DEC banner could be added onto the page or something else put in its place.

Inevitably new technologies and ideas will spring up over time, which means that you will have different versions operating at the same time. So a few years down the road you will have meetings to decide what content/functionality to deliver to everyone left on version 1 compared to the whizzy new version 2, etc.

At some stage that will become too complicated to deal with effectively, so thinking about a support structure would be very beneficial. This could be as simple as expanding the web content from a single page of buttons/widgets to keep your network informed, along with an RSS feed so that you can push updates out to them.

Much of the focus has been about on what the functionality of each service offers and how that can be used by third parties. Recently though there have been reminders about the limits  from a business perspective. Amazon has moved against Alexaholic/Statsaholic and Zlio as it felt that in both cases they broke the terms of the agreement. I’m sure that some in the American Congress are regretting how some of the government data is now being brought together. It’s that much harder to prevent it though when the ‘public interest’ is involved.

Dear PayPal customer!

As part of our security measures, we regularly screen activity in the
PayPal system. We recently contacted you after noticing an issue on your
account.We requested information from you for the following reason: …

Well apart from not having a PayPal account, I’m certainly suspicious of anyone asking me to go to something other than the expected url and even if it did I would be very wary. Unfortunately you’re probably familiar with this type of scam and have a similar careful response.

And as if on cue I’ve just received a Security Bulletin from Microsoft which contains a digital signature so that I can verify that it was sent by them.

Phishing has effectively ruled out the use of emails to customers in the financial sector for anything other than promotional marketing. I suspect that we’re going to find similar roadblocks with widgets, badges, or whatever term is used to describe those bits of code that we drop so willingly into our blogs or other Web 2.0 applications.

This all stems from some research I was undertaking into Netvibes, and in particular their ‘Universal Widget API‘. This holds out the possibility of creating widgets that could be used on a number of websites (e.g. Netvibes and Google IG) and desktops (Mac Dashboard and Vista Sidebar) without any changes to the programming.

An exciting prospect. Unfortunately the technology that underlies the promise looks also to be responsible for restricting its use. A French security blogger created a widget that allowed him to read the rest of the contents on a user’s Netvibes page, and in this case it also contained access details to one of their servers.

If you’re interested in the details then you can find the nitty gritty on Niall Kennedy’s blog. The basics though lie in the way that the widgets are combined on a page & their reliance on JavaScript. There are some safeguards built in to browsers, but they couldn’t stop a program in one widget wandering through the rest of the data on a page. So if there’s a Gmail widget then that could include your latest emails and possibly even the login credentials.

Other forms of add-ons using different technologies aren’t exempt from problems either. In WordPress they’re known as plugins – and they have full access to the database. So here there’s a possibility that code could be surreptitiously hidden to transmit user names & passwords to another server. In fact there was malicious code added in to the core of WordPress by a hacker at the beginning of last month (so if you’re still using version 2.1.1 you need to upgrade now – the WordPress blog has more details).

So all in all it’s probably time for us to become more cautious and in all the buzz & excitement of Web 2.0 applications to heed some of the warnings of the security community. Somehow that has to be achieved without halting the innovation and momentum produced by a burgeoning development community.

As the name suggests it is more to do with the plumbing behind the scenes than enhancing the user experience. However there are many exciting possibilities that it opens up – bringing content together from disparate websites and then combining them until you have something that could be feed into your own site.

The idea is borrowed from the pipes and filters functionality offered in Unix – take the information coming out of one process and feed it into another. Yahoo! Pipes applies that paradigm to the Internet. You start off with an information source (or a range of them), chain together a number of processes (combining, filtering, sorting, etc.) and then pump out an RSS feed at the other end. (There are options for entering data through the Yahoo! Pipes website & also viewing the results there, but at the moment I’m just concentrating on providing content to your website.)

An example Pipe that they provide is the Aggregated News Alerts. This allows you to search for information from a range of sources (Bloglines, Google Blog Search, Technorati, etc.). The results are combined, sorted in date order & de-duplicated. It would be very easy to duplicate this Pipe, amend it so that the search term was already entered and therefore require no user interaction. If I wanted to include the results of that Pipe in this blog then I could use a WordPress plugin such as FeedWordPress or inlineRSS.

Now none of that is rocket science, it’s all something that a web developer could provide for you. However they’re not going to be able to do it with so little effort, adjust it so quickly to your changing requirements, nor offer an interface that’s so easy to use. Aggregating content has just become a commodity.

There are a few downsides to it all. The most notable at the moment is that it’s still in beta – and this is a real beta phase as I’ve had a few errors as I’ve been experimenting with creating new Pipes. In the longer term it could be the hardware and network resources made available to the service that could cause problems – the more successful it is the more data it needs to bring in, process & send out.

I’m not entirely clear where it fits into the business plan for Yahoo!, but I’m quite content to operate in ignorance on this one. Maybe it will end up mirroring Flickr with free and subscription accounts.

I would suspect that if it proves to be unsustainable then the code could easily be made available as open-source – a route that other corporates such as IBM have taken in the past. There doesn’t seem to be much that would be commercially sensitive, for example at the moment I would think there’s only the content analysis and location extraction that would use some of their core search technology.

The skills involved reduce the number of people who are likely to create their own Pipe, but it wouldn’t take a lot of effort to show someone with normal IT skills around an existing one in the visual editor so that they could make amendments on their own (e.g. adding another RSS feed, changing the number of items to include, the order they are sorted in, etc.).

I’m off to explore more …

Given the recent announcements about Google’s customised search offering I was anticipating that it would be built on the back of that technology. Instead it’s a service that has been developed over the last 3 years by FAST – an enterprise search company.

The timing of the two announcements though has the perennial development debate whirring away in the back of my mind – when is it better to use a ’shrink-wrapped’ solution or to ‘roll your own’?

In this case there’s an interesting fact tucked away in the Hurisearch documentation – they need $187,000 a year to keep the service going. Given the prominence of the FAST logo on the home page I’d suspect that there’s been a deal made, so the costs could be higher than that. How the overall price tag actually breaks down is another matter, but you’d expect that the development, maintenance and hosting costs to make up the majority of it.

Using Google Custom Search wouldn’t involve any monetary costs, but there would be a different set of issues to take on board:

• tucked away in the logo for the service is the word ‘beta’ – no service level guarantees at this stage (and it’s worth noting that their email service – Gmail – has been in beta testing since April 2004) and even when it is finally live you would expect it to be at a lower level than for a system that you are paying for
• no clear indication of how the search engine works – the natural assumption is that this sits on top of the normal Google search, but details about how this works aren’t generally available as, in common with other search engines, they’re fighting a constant battle against spam; the hope is that this extra dimension won’t detract from creating a good specialised search tool
• compromises on features – e.g. in addition to the content you see on a page Hurisearch will also look for metadata using a common standard (Dublin Core), currently this is not available in the Google offering

The issue of functionality available can be the seductive siren song of creating your own solution. All those extra bells and whistles that will make your site stand above the crowd. But it’s also worth taking a minute to reflect on the content in the Google Custom Search blog – the core development is ongoing and you’re not paying for it – and the activity of the development community in the discussion group.

The promise of a customised set of features can also be a distraction from the real work that is required – in the case of a search it might be less about the search page and more about creating the content and making sure that it’s all properly tagged & organised so that it’s easily found.

In the case of Hurisearch this functionality wasn’t on offer back in 2003, so their business case has a strong argument. It might still do so now if the additional features and service level are providing a real benefit to the intended audience – that’s not a judgement I can make. If you do decide to take the ‘roll your own’ route then at some stage you might need to concede that your trail-blazing development has been steamrollered by a major entity – and on the web that can be within a few years, or months if you’re unlucky.

If you’re thinking about different search options then you might want to check out the following resources to see what’s available:

• Make your own search engine with Google Custom Search
• Your Search, Your Way (Part 1)
• Your Search, Your Way (Part 2)