Drobe :: The archives
About Drobe | Contact | RSS | Twitter | Tech docs | Downloads | BBC Micro

Behind the scenes at Drobe

By Ian Hawkins. Published: 31st Aug 2004, 15:56:55 | Permalink | Printable

The history of AcornSearch and more

Whilst desperately trying to avoid the old 'blowing your own trumpet' routine, we let overworked Drobe webmaster Ian Hawkins out into the sunlight, and the quickly back inside again to pen this article on what the heck's been going on inside Drobe for the past fortnight: here's the low-down on the history of AcornSearch and the cool new stuff available to you, our users.

For a couple of evenings I have been busily bashing out improvements to Drobe - mainly new features. Most of these (and one feature in particular) were inspired by the recent unfortunate iconbar downtime, as well as others inspired from the good old 'skool' days of acornsearch.com.

Improved websearchEnhanced web searching (and a little history)
Back in November 1997, a month after I started playing with Linux, I created 'AcornSearch', a search engine that indexed only RISC OS and Acorn related websites. I was impressed at the time that something I cobbled together with an extremely rudimentary knowledge of Perl and Bash actually worked, and with the reaction from the RISC OS newsgroups and people on IRC being favourable I decided to register the acornsearch.com domain, and thus 'AcornSearch' was born.

After a month or two it was becoming painfully obvious that my existing method of indexing websites (at the time a wget --mirror, no less) left a lot to be desired in terms of automation. The actual way of performing the searches at the time was by way of grep, and that too was causing problems by way of becoming increasingly slower, and slower as the sites list grew and more people searched for things at the same time. I had to do something.

So I set about sorting the immediate problem, speed. Armed with my complete lack of knowledge of Linux, Perl, web serving and anything else remotely useful, I did the first thing that came to my mind: I searched for something on AltaVista (at that time the best search engine) to help me.

The solution I found was a program called 'Isearch' and 'Iindex', utilities that enabled me to pre-index all the text documents (web pages) I had from the Internet and store locally, then Isearch could be used from the crude web interface to query that index to retrieve the website that the user was hopefully looking for. This worked great with only minor alterations to my search script for around six months, which had given me valuable time to learn more about Linux and more importantly Perl scripting skills. The engine still had the downside of a manually maintained index, which was about to be fixed in the next version.

Now, I had no intention of spending my life in front of a computer maintaining a big list of links, so the next version would be made to make my life easier in the long run, and this is where all that learning basic Perl came in handy. To cut a long story short, I was able to, through several more revisions, make AcornSearch completely autonomous in terms of file indexing and webpage indexing.

You can access AcornSearch right now, from the front page and every article, by using the search form at the top of the page.

Back to the point
Since those good old days, AcornSearch has been merged into Drobe Launchpad and although through some minor hiccups (like a bug in Linux 2.4 and SiS IDE drivers causing extensive filesystem damage upon shutdown), it has now been improved to a point where I hope you'll agree that it's become quite a useful little tool - especially with the latest modifications.

Improved websearchThese latest upgrades are to do with the page scoring system and how the results are displayed to you. Results have an image of the page next to them (where available) and a selection of maybe relevant files along the top if any were found. The results ranking code has also been improved a little and one of the main features that it now provides is a Google style globbing of results per website. One of the more interesting things you can do is ask the search engine a question, and it will try to answer it. Whilst the search may take a little longer to complete (I don't have the resources of Google yet, but keep clicking on those ads) you should find that it works really well. One of my favourite searches that best demonstrates this point is this one.

Some of you have asked how you add your site to the search engine, given that it lacks an 'Add URL' link. The answer is that AcornSearch looks at various key places on the Internet for people talking about RISC OS code, programs and websites and if anyone mentions your content then it has a look and sees if it can index it. Basically it uses the rule: 'if people are talking about it, then it is worth indexing'. There will be a faster way to get your site listed on AcornSearch, and updated on a daily basis and that would be to use the web hosting facility offered in the last part of this article.

Improved file searchA file search that works
I've overhauled the file search engine to provide better search capabilities, so it no longer just searches on just filename, and to return you nice icons back in any results found.

When the file searcher finds an archive online, the software actually unpacks it, has a look at the applications inside, decides which one is important and proceeds to index it. Details like the program name and the version number are selected using a piece of 'fuzzy logic' code - so it might not always get things right, but 99 percent of the time, it does. However, when it doesn't, it's usually down to poor organisation inside the archive file. The engine also makes a note of the version numer of the program if it is present, and this is particularly useful in trying to ensure that you always get the latest version of the program in the search results that is known to the search engine. You can also dig through previous versions of a program by clicking on the version link to list them (if there are any): here's an example that illustrates some of my points.

At some point in the future we may provide a filebase-esque admin interface whereby people can vote for, and also take responsibility of programs indexed from the local filebase and off-site, possibly something along the lines of freshmeat.net, but for the moment the search interface is usable and reasonably accurate.

Lots of votes!Polls galore
By now you've seen the polls on the right side of the main Drobe Launchpad page when you visit the site. Yes, I know, we said we wouldn't do polls, but, well, now we do. Consistency, we've heard of it, etc.

The polls are divided into two result sections, one for registered users and one for guest users and both are completely anonymous (we don't store who voted what, only that you have voted), so you can easily see the spread of votes for each poll option. We plan to have fresh polls more or less on a weekly basis.

If you can't vote in the poll then someone on your IP has either beaten you to it, or there is something strange about the browser you are using. If this is the case then unfortunately it is tough luck, and you will either have to wait until the next round of polls to be able to vote and enjoy the rest of the site, or try voting after logging in.


Ooo, Drobe hosting!The big one: hosting
This is what I was refering to in the beginning of the article: web hosting. We decided a week or two ago that far too many RISC OS sites are hinging on just one server or two servers, and when those computers goes down, they all disappear (which hardly looks good for the platform), so we decided to create a facility on drobe.co.uk for people to host their programs with us as well.

Here's what you get:
  • A whole 100MB for your RISC OS related web pages

  • Your own sub-domain name: http://{username}.drobe.co.uk/

  • A web based interface and FTP access to administer your page/

  • We don't supply PHP or any kind of serverside processing, at least not at the moment


There are some further guidelines that we'd like you to keep in mind from this page when using the service. If you look on the left sidebar you will see a 'Your webspace' link, where you can administer your website.

Anyway, we hope you enjoy and use the new features that we've have put in to make Drobe an even more useful resource to our users.

Links


Back to the front page Comments, suggestions?

Previous: Video playback on RISC OS
Next: VirtualAcorn moves "further forward"

Discussion

Viewing threaded comments | View comments unthreaded, listed by date | Skip to the end

Are there any plans for people who ave spaces in their username like me? ATM I can't use hosting and webmail, but I also like my username with the space. Can it not be removed for the benefit or email/hosting?

 is a RISC OS UserAndrewDuffell on 31/8/04 4:13PM
[ Reply | Permalink | Report ]

Done.

 is a RISC OS Userpiemmm on 31/8/04 4:31PM
[ Reply | Permalink | Report ]

A big thanks for all these new features. It's looking really excellent.

 is a RISC OS Userfylfot on 31/8/04 4:49PM
[ Reply | Permalink | Report ]

Ewwwww, SiS in a server? Ick ick ick!

 is a RISC OS Usernunfetishist on 31/8/04 5:33PM
[ Reply | Permalink | Report ]

piemmm:thanks :)

 is a RISC OS UserAndrewDuffell on 31/8/04 5:37PM
[ Reply | Permalink | Report ]

I can't seem to be able to get !FTPc to work with VirtualRPC. Anyone got any ideas? Netsurf works ok.

 is a RISC OS Userhutchies on 31/8/04 5:59PM
[ Reply | Permalink | Report ]

I notice acornsearch.com is now up for sale...

 is a RISC OS UserEddie on 31/8/04 6:24PM
[ Reply | Permalink | Report ]

Well done with all the improvements and good luck with anymore you may have planned.

 is a RISC OS UserRevin Kevin on 31/8/04 7:06PM
[ Reply | Permalink | Report ]

hutchies: no idea :-( Try asking on comp.sys.acorn.apps.

Ian: How does the file search choose what icon to display? For CaretChase it comes up with the low res (!Sprites) icon. Also, how do the polls deal with transparent proxies and the like? (e.g. ntl)

 is a RISC OS Useradamr on 31/8/04 8:40PM
[ Reply | Permalink | Report ]

Very nice :) Looking forward to the possibie addition of PHP.

 is a RISC OS Userjohn on 31/8/04 10:21PM
[ Reply | Permalink | Report ]

Yes, Drobe has been busy and thanks for offering the recent new features. I do like Google and use it most of the times. A web page I do not have, thanks. Drobe's new feature packed site may be coming useful and attractive as a User's browser "Home Page" !! ;-) Regards, Steve

 is a RISC OS UserSawadee on 31/8/04 11:36PM
[ Reply | Permalink | Report ]

The file searcher is very nice but at the moment it seems to be throwing up some very old versions of popular programs for instance: searching for "BeebIt" gives Beebit 0.32 searching for "Zap" gives Zap 1.30

 is a RISC OS UserIvanDobski on 1/9/04 2:36PM
[ Reply | Permalink | Report ]

I would be very interested in the long story about how you "were able to, through several more revisions, make AcornSearch completely autonomous in terms of file indexing and webpage indexing."

 is a RISC OS UserCJE on 1/9/04 4:14PM
[ Reply | Permalink | Report ]

With the new User Gallery - has anyone been able to use it via the RISC OS side?

If so which browser did you use please?

 is a RISC OS Userrmac on 2/9/04 4:04PM
[ Reply | Permalink | Report ]

"WebSpace" is nice and simple... just two hiccups, I couldn't find <= Back or an exit to take me out & back to the Drobe Home page? :blush: The other is some directories I created in WebSpace worked, a few jpgs loaded into one directory loaded and Click 'N Show image worked the first day, but now the after the first try the images links are dead and won't Click 'N Show on RO4 or the Windows XP...??? I tried to Delete and reload images (no worries) but they still now won't show...? (Is this feature disabled while Ian H is doing some finishing touches to the site?) :blush:

*rmac* Everything in the (New) User Gallery works on XP Windows. I have RO4.2 with Oregano2 & I loaded jpg photos and could edit text no problems. The only thing that will NOT work in RO Oregano2 I found was the [Slideshow] feature, the Slideshow did not work in !NetSurf either. Why??? :unhappy: Regards, Steve.

 is a RISC OS UserSawadee on 2/9/04 9:39PM
[ Reply | Permalink | Report ]

Ah, I remember, randomly typing in www.AcornSearch.co.uk and .com at school.

Was very welcoming to see such a thing, without it, i wouldnt have seen this place!

 is a RISC OS Userem2ac on 7/9/04 9:26AM
[ Reply | Permalink | Report ]

Please login before posting a comment. Use the form on the right to do so or create a free account.

Search the archives

Today's featured article

  • A9home first impressions review
    Getting up close and personal with an A9home
     13 comments, latest by RickCB on 7/11/06 9:25AM. Published: 5 Nov 2006

  • Random article

  • PCI TV card progress
    Where the heck is it?
     19 comments, latest by quatermass on 24/2/04 3:28PM. Published: 20 Feb 2004

  • Useful links

    News and media:
    IconbarMyRISCOSArcSiteRISCOScodeANSC.S.A.AnnounceArchiveQercusRiscWorldDrag'n'DropGAG-News

    Top developers:
    RISCOS LtdRISC OS OpenMW SoftwareR-CompAdvantage SixVirtualAcorn

    Dealers:
    CJE MicrosAPDLCastlea4X-AmpleLiquid SiliconWebmonster

    Usergroups:
    WROCCRONENKACCIRUGSASAUGROUGOLRONWUGMUGWAUGGAGRISCOS.be

    Useful:
    RISCOS.org.ukRISCOS.orgRISCOS.infoFilebaseChris Why's Acorn/RISC OS collectionNetSurf

    Non-RISC OS:
    The RegisterThe InquirerApple InsiderBBC NewsSky NewsGoogle Newsxkcddiodesign


    © 1999-2009 The Drobe Team. Some rights reserved, click here for more information
    Powered by MiniDrobeCMS, based on J4U | Statistics
    "Your fellow contributor to Drobe seems to have a personal dislike of anything that does not come from Castle"
    Page generated in 0.1322 seconds.