RISC OS news and information from Drobe


Beta! \| About us \| Contact \| RSS	Webspace \| Tech docs \| Downloads \| BBC Micro \| Gallery \| Wallpaper

RISC OS Package management

By Peter Naulls. Published: 25th Dec 2003, 15:34:01.

Xmas presents aren't the only packages to open this festive season [Updated]

After you've finished opening your presents, we've found there's one more for RISC OS users sent to us early this morning by Graham Shaw.

Graham has released RiscPkg, a RISC OS package manager, a project Graham had hinted to drobe a few months ago. The idea behind package management is relatively simple, but it can hide suprisingly complex details in its aim to ensure a system has the latest versions of software, along with making sure dependent programs are installed, and downloading these easily with an automated installation system.

Package managers have existed on other systems for some time - Microsoft Update embodies the same principles for OS updates (although invididual programs in Windows often have their own independent methods of checking for updates) - and most Linux systems have comprehensive package management, such as RedHat's RPM format, and the much praised (and sometimes critisised) apt/dpkg system used by Debian.

Package management and autoupdating isn't an entirely new idea to RISC OS either with Castle's Iyonix Update Watcher being the most prominent example. The version checker in Thomas Olsson's IRC client, LIRC, is another example. Although all these efforts have been important, none has fully addressed all the significant issues needed for a RISC OS package manager.

It's easy to suggest that something like apt/dpkg should be ported to RISC OS (which is certainly possible), but even a cursory examination of the issues quickly reveals that something which is suitable for Linux is not really appropriate for RISC OS - fixed paths being one of several items. With this in mind, Graham based many of his concepts and file formats from the Debian system, but the implementation is entirely new. Graham also exchanged ideas with Andrew Sidwell, who had previously conducted his own invesitgation into RISC OS packaging issues.

RiscPkg

RiscPkg in action

Unfortunately, this very early 0.01 version is a little bit unstable, and I wasn't able to fully evaluate RiscPkg, but I was nevertheless impressed by what I did see of its functionality. The initial installation is a little complex, which can hopefully be improved in future, requiring a two-step bootstrap whereby RiscPkg initially updates itself.

Graham also provides a comprehensive manual and the source, so you can poke around with it yourself. It will be interesting to see how this idea develops.

Update 1/1/2004 16:25

Graham today updated his site, outlining more fully the aims of the project. He also crucially made a call for people to package RISC OS programs; a task that doesn't generally require programming skill, just a little knowledge of obey scripts.

If you've been looking for a chance to contribue to RISC OS in a practical way, but don't have programming savvy or large amounts of time to spare, this could be an excellent choice of activity. Decisions made and implemented by such people at this early stage of implementation will be important to the success of the project.

Graham makes it clear that he's not trying to impinge on the upstream authors (meaning the actual programmer or porter of a program), but rather this system may make it easier to distribute their software and make sure available versions are up to date.

Time to get involved.

Links
RiscPkg
[20:48 26/12/2003: Edited out reference to a defunct package project]

Discussion

Viewing threaded comments | View comments unthreaded, listed by date

Doesn't Chocky have Christmas dinner to eat ot presents to unwrap instead of doing articles?

-- Spriteman - Off to have some turkey ;-)

Spriteman on 25/12/03 5:44PM [ Reply | Report ]

What happened to ROUpdate?

piemmm on 25/12/03 10:33PM [ Reply | Report ]

This sounds promising, I have thought that something like this was needed for a long time. Here is wishing Graham good luck with this project.

vshears on 26/12/03 12:13PM [ Reply | Report ]

I'd prefer peoppe to strive for drag and drop installation wherever humanly possible. At the moment we have one or two apps that incorrectly require you to put stuff in Choices (Zap, POPStar, maybe more).

I think it'd be useful in the right situation, but I have trouble thinking of any right situations. Boot already has the BootMerge tool. Anyone?

john on 26/12/03 2:00PM [ Reply | Report ]

Manual drag-and-drop does not scale well (a) with the total number of packages to be kept up-to-date, or (b) with the number of dependencies per package.

Dependencies are not bad - they are inevitible whenever resources are shared between applications (and will become much more common if we ever settle on a shared library system). A package manager encourages sharing by ensuring that dependencies do not complicate the installation process.

(That said, I am all in favour of keeping non-shared resources within the relevant application directory. I'm not trying to encourage packages that scatter files throughout the filesystem.)

gdshaw on 26/12/03 6:21PM [ Reply | Report ]

This is an excellent idea.

Hopefully the initial install of programs will still be drag and drop, but programs will call the packager when the correct modules etc aren't present.

jess on 28/12/03 7:39PM [ Reply | Report ]

jess:

Indeed it is a very nice idea

I would imagine that when you request a package (that isn't something like !System modules, things to go in !Boot.Resources etc..) the program can ask you where you would like to install the package.

Andy. At home.

andypoole on 28/12/03 11:47PM [ Reply | Report ]

My ultimate aim is this:

You are /effectively/ able to install applications anywhere you wish, but this is done using symbolic links. The 'real' copies are stashed away in a fixed heirarchy, by default rooted in your Apps directory.

This seemed like a good compromise between the needs of the package manager (which benefits from knowing where the files are and having control over part of the filesystem) and the needs of the user (who will often wish to put applications and data close together).

An additional benefit is that you can have multiple copies of the application if you wish. (For example, you may have several users who each want a copy of something in their own working directory.)

RiscPkg will probably provide an easy mechanism for creating these links at some point in the future, once I have worked out a suitable user interface and settled on which type of symbolic link to use.

gdshaw on 29/12/03 8:11AM [ Reply | Report ]

I don't keep applications and data close together. I can't say I like the idea of having all my apps in a fixed place, either, no matter how many shortcuts I can have. That just sounds like windows. Surely the whole point of system variables is that you can find things without having to worry about exact structure. So far, they've ruined this in Boot by fixing and assuming the structure, I wouldn't like to see anyone try to do the same for the general layout of anywhere else.

And Happy New Year, too

john on 29/12/03 11:42AM [ Reply | Report ]

I think the problem is that everyone has different ideas about the "best" layout of apps and data. Certainly, the Iyonix updater got a lot of flak in the early days because it assumed that you hadn't changed your HD layout. This only subsided when Castle enhanced it to seek out anything that had been moved.

I certainly like a mix of data held with apps and data held under subject-based directories - I doubt anyone else would like my scheme.

So I have to agree with John and others. Let us drag and drop the apps where we want them but help out with duplicated/wrong version/system resources etc.

Tony

TonyStill on 29/12/03 3:13PM [ Reply | Report ]

TS: it simply isn't that simple. The issue is quite complex, and perhaps we ought to ask Graham to write us an article on it.

mrchocky on 29/12/03 3:27PM [ Reply | Report ]

A question for john and TonyStill:

Why does it matter where your applications are actually stored, provided they appear to be in the places you want to use them from?

(Please rest assured that MS Windows was not one of the operating systems that inspired RiscPkg - it is far closer in concept to Debian GNU/Linux.)

gdshaw on 29/12/03 6:05PM [ Reply | Report ]

I would prefer having to index the hard drive or/and dragging apps to the packager, than /have/ to use shortcuts and fixed locations.

It would be nice to be able to run a simple script file, (with the name of the app) in the correct folder and have the packager go and fetch the latest version and put it in place of the script.

jess on 29/12/03 6:10PM [ Reply | Report ]

gdshaw> cos a lot of the time I look inside the application for stuff (if you do that in Resources:$.Apps you see what I mean). Usually to hack the templates or maybe just a cursory look, but I do it often. Also you have the problem that maybe some link breaks and you have no idea where the app *really* is to narrow down the problem, or maybe just the !Run file has some silly RMEnsure in it or well, you ge the idea.

Perhaps I'm a hard core hacker control freak - I can certainly see the benefit for users who only "use" applications properly, but I just like to know what my computer is thinking, and I like to remove all (unnecessary) layers of complication.

MS windows analogys are handy cos everyone knows windows does everything badly

Anyway I'll leave it there for now, hope this helps you understand my opinion!

john on 29/12/03 6:46PM [ Reply | Report ]

In reply to john:

Your first point is a good one, and is a good reason to favour LinkFS-style links (which operate via an image filing system) rather than the Obey-file veneers you find in Resources:$.Apps.

As for directly patching !Boot and !Run files, you would have to copy the application to one of your working directories anyway (otherwise the package manager would overwrite any changes at the next upgrade).

One enhancement I had been considering, and in view of this discussion would probably be a very good idea, is to allow any path to be mapped - not just the top level of each hierarchy. In principle you could use this feature to achieve any disc layout.

(In practice, if you want that level of control then package management probably won't be a win for you. I'm not looking to convert everyone - just provide an option that doesn't currently exist.)

gdshaw on 29/12/03 8:14PM [ Reply | Report ]

I'd be interested to see an article if you're up for it!

Image files seem alright for copying on the local disc, but it'd be a pain if you, say wanted to copy it to someone else or share it over a network, or just shove it on the RAM disc. Also, AFAIK the filer doesn't allow image file applications (although there's a horrible hack called ImageFSFix, which is anything but) but that's a minor problem.

Your second paragraph is a good point not in favour of package management for me

I think I'm firmly in the "no package management" group!

I don't even like the way Boot is now, "do not touch, you don't need to understand it," so that says it.

Thanks for your time though, it's interesting.

john on 29/12/03 10:32PM [ Reply | Report ]

Adding to Johns 'hack' reason for it mattering where the applications are actually stored, the main benefit to the simple drag-and-drop installation method, in addition to being able to move the app where you please, is being able to delete the app in the normal filer way of selecting delete from the menu. The beauty of the system is the application is /just/ another directory in the filer. You don't need another application or system to get rid of something (part of RiscPkg, the uninstaller in Windows etc.), instead even the most inexperienced of users can easily and assuridly get rid of an application.

Anyway, in this sort of situation, rather than arguing the case for keeping things the way people like it, there should be arguments for changing things. 'The issue is quite complex' and 'benefits... from having control over part of the filesystem' hardly seem compelling reasons for change, in my view. It is just /easier/ to write RiscPkg if it does things with symbolic links, or is it actually /better/?

senduran on 30/12/03 12:10AM [ Reply | Report ]

To my mind, using symbolic links is overkill. Given that many apps will set one of App$Dir or App$Path, would it not be easier to enumerate these for the applications that RiscPkg knows about? Admittedly, that also has its limitations.

Another solution is to simply have the user install the program as usual (via drag and drop) and then have the user drag the application to RiscPkg, thus telling RiscPkg the program's location on disc. I like this approach more as it allows the user the same control over their system as before (including deleting a program by simply deleting its directory).

Whatever way is finally decided upon, there would still be a need for a standard location for some things (modules and shared libraries spring to mind). However, we already have a standard location for modules (the !System folder) and I should think that a similar system for shared libraries would not be out of place.

Another thing that springs to mind is how to handle the situation where the user has changed program related files (eg the boot or run file). Perhaps some form of checksumming would be a solution. For example: The upgrade package contains a file with checksum information for each file in the previous version. RiscPkg then generates a checksum for the existing files on the user's computer and, if the differ, ask if they wish to upgrade that file (with the option of backing up their old one).

I'll shut up now, sorry if this is a bit long, got a bit carried away

jmb on 30/12/03 2:09AM [ Reply | Report ]

To pick up a couple of the points above:

Simply deleting (managed) application directories using the filer is completely incompatible with the way RiscPkg works. What if something else depended on that application being present (because it provides a command-line tool, for example)?

There also seems to be an assumption that applications and packages are isomorphic. I can think of at least one major application (GCC) where this has been untrue for a long time - and if anything my preference would be to distribute in smaller chunks, not larger ones.

(A good way to think of RiscPkg is as a network administrator who provides a read-only share. You have limited control over what is in the share, but you can choose which bits of it you use, and you can make local copies of applications you want to hack around with.)

A system along the lines being described (an 'application updater' rather than a full-blown package manager) would no doubt be useful to many people, but the approach fundamentally different from RiscPkg. If anyone wants to write such a system then they have my full support and encouragement, but it really is a different project with different goals.

(The same applies to less radical variations. RiscPkg is GPLed and the back-end is LGPLed. You can take or leave as much or as little as you like.)

gdshaw on 30/12/03 9:28AM [ Reply | Report ]

Why I like to control where applications are stored: * Back-up. * For when things go wrong. * Selective booting (I know this is making a merit from a RO shortcoming but I'm used to it). * Scepticism (how long before the system is perfect and I can really lose interest in what is going on - maybe this is the 2nd point again). * Inertia (ie I understand what I've got and don't really appreciate the potential improvements).

The article sounds like a good way forward. Also, what proportion of the RO community has to adopt this before wecan forget the 'old' way? As an application author, I have to defer to the lowest common denominator.

And a random point to support John: I used Thump for the first time on the Iyonix yesterday; first thing I hit was that one of its libraries wasn't 32-bit so I had to dive in and update it. Trivial but these things happen. I'll now let Rick know and I'm sure he'll fix it quickly but fixing it myself got the Xmas photos off to the in-laws in a more timely fashion :-)

Tony

TonyStill on 30/12/03 6:09PM [ Reply | Report ]

OK, I suppose I'll have to write something. I MUCH prefer the simple RISC OS way of installing and removing applications to the package management approach used on Unix and elsewhere. Package management comes into its own when an application consists of lots of files scattered all over the disc, as on many Windowss and Unix applications. AFAIK this does not apply to most RISC OS apps. If there are problems with shared resources, would it not be better to try to build on the strengths of RISC OS. Package management is only necessary on other OS because of an inferior and more complex installation process compared to RISC OS. This leads to the sort of problems with DLLs that seem so prevalent on Windows. I much prefer the natural RISC OS approach of an app checking for the presence of the resources it needs, as with RMEnsure for modules for example. The RISC OS way is for the user to be in control of what happens. I for one would be unhappy if a library for instance was replaced with a later version without my knowledge. It is one of its major strengths of RISC OS that we do not suffer from the nightmare of Windows type DLLs. and all the complexity of managing them. I would hate it if this initiative encouraged application writers to move away from the simple RISC OS approach unnecessarily. After all, most RISC OS applications are very simple to install. A package approach will not make it any simpler in most cases. Most UNIX installations, even where packaging is used are not simple, you need to understand what you are doing.. Most Windows ones are not either, but the complexity is usually hidden from the user by the installer program, making installation simple provided nothing goes wrong. Of the three I prefer RISC OS.

Martin

mrtd on 30/12/03 7:23PM [ Reply | Report ]

In reply to TonyStill:

Backups are greatly simplified if you keep third-party code well away from your own data. (RiscPkg doesn't really support this yet, but with apt/dpkg all you really need to backup is your data plus a list of installed packages.)

I'll see what I can do about an article.

The 'old' way (not my choice of words) will never go away, and I don't expect upstream authors to start producing packages (unless they really want to). Even in the GNU/Linux world, upstream sources are usually distributed as tarballs - packaging is done by Debian or Red Hat or whoever.

Of course that does beg the question of exactly who will do the packaging. Please rest assured that the issue has not escaped me: writing the package manager is just the beginning.

gdshaw on 30/12/03 8:41PM [ Reply | Report ]

mrtd: I think you're getting overly excited, over what, for the most part, are non-issues. We've already discussed why shared libraries are so important in a previous article, and their faults under Windows really have nothing to with RISC OS. Equally, many of the things about packaging in Windows and Unix are largely irrelevant.

I would even assert that a package manager is the only way to ensure that use and deployment of shared libraries goes smoothly (c.f. Debian)

Furthermore, it's great to go on about the existing way of doing things is, but the decreasing number of RISC OS developers means that for RISC OS to have any kind of momentum in terms of development, developers must be able to do much more with less. Packaging like this solves are number of previously very manual problems.

graham: I'm sure the issue hasn't. In the case of the Unix Porting Project, the it is easily solved since for many of the programs, as the zipping up is already automated. Adding suitable control files and so forth is relatively simple.

mrchocky on 30/12/03 9:15PM [ Reply | Report ]

I don't see that a packager would have to interfere with the way RO works.

There is already the resources folder containing common resources.

If this also contained all apps that are used as resources by other apps, would this not fit the requirents?

These apps themselves could be launched from a simple appdir that checks for the presence of the app in resources, uses the packager to fetch it if it isn't, then runs it.

jess on 31/12/03 3:40PM [ Reply | Report ]

Please login before posting a comment. Use the form on the right to do so or create a free account.

Create a new account
Forgot your password?

Search this website