RISC OS news and information from Drobe


Beta! \| About us \| Contact \| RSS	Webspace \| Tech docs \| Downloads \| BBC Micro \| Gallery \| Wallpaper

The Case for a RISC OS Package Manager

By Graham Shaw. Published: 18th Jan 2004, 19:13:41.

Graham Shaw talks about RiscPkg

A package manager is a program for installing, upgrading and removing other programs (and sometimes data too). Other possible features include the ability to:

download packages over the internet;
resolve dependencies between packages.

The concept of package management is new to RISC OS, but on GNU/Linux systems it is well established. Examples include apt and dpkg (used by the Debian Project), rpm (the Redhat Package Manager) and Portage (part of Gentoo Linux). Packaging is typically done by the distribution, so (for example) the upstream sources for Mozilla would be written by the Mozilla Project then packaged by Debian, Redhat and Gentoo in slightly different ways.

Traditionally most RISC OS software has been installed by manually copying files. This works well most of the time because files tend to be bundled into application directories, and these can be moved quite easily by drag-and-drop. It falls down as an installation method when:

there are large numbers of applications to be kept up-to-date;
there are many files which do not belong to any single application.

RISC OS provides some support for merging files into the !Boot directory, but this is strictly a one-way operation with no way to remove files that are no longer needed. It also detracts from the simplicity of drag-and-drop installation.

A package manager can automate the process of installing shared files, saving effort and reducing the risk of error. For example, if an application needs a particular module in order to run, this requirement can be declared as a dependency when the application is packaged. The package manager then knows to fetch the module if the application is requested and the module is not already installed.

Unfortunately it is neither feasible nor appropriate to simply port a GNU/Linux package manager to RISC OS. The most important difference is that RISC OS does not have a standard filesystem layout in the same way that GNU/Linux has. This means that packages cannot simply specify where each file should be copied to because the destination will depend on the layout of the target filesystem.

RiscPkg handles this issue by structuring packages in terms of 'logical' pathnames. These are converted into 'physical' pathnames by the package manager using a translation table. The table can be customised to suit the target filesystem. It can also make use of system variables to track movable destinations such as the !System directory.

This works well for modules, fonts, and most other types of shared resource. By itself, it is not a good solution for applications because their placement directly affects the end user. Some users put applications in relevant working directories, others keep them away from user data. Any attempt to impose a standard layout would be unacceptable to many ... or at least, it would be unless the effects can be hidden in some way.

The Apps directory on the icon bar provides the start of a solution. When an application is added to the Apps directory its files are not physically copied. Instead, RISC OS creates a small stub application which points to the real copy. This makes it much less important where the real copy is stored, because the user (for most purposes) never has to go there.

Putting every application into the Apps folder is not an option, and in any case this would not achieve the objective of simulating the user's preferred layout. However, the same concept can be used elsewhere in the filing system so that applications appear to be in the places where users want them.

An alternative (but superficially very similar) solution is to use symbolic links of the type produced by LinkFS. A planned upgrade to RiscPkg will provide users with an easy method for creating either links or stubs at their preference.

Some users will still want to control exactly where each real applications is placed. It is possible to do this by adjusting the paths file but I strongly recommend that you don't. It makes little or no difference to day-to-day use of the system, and package management works best when there is a clear distinction between files owned by the package manager and files owned by the user. For users who are adamant that they need this level of control I am looking into ways to partially automate the path changes.

The package files used by RiscPkg are zip files with a specific internal file layout. There are two reasons for this:

zip files are easy to create using existing tools
they are easy to unpack without using the package manager

This second point is important because it ensures that packaging is not a one-way process. Not everyone will want to use RiscPkg and I didn't want to lock software away in files they could not access.

There are just two essential parts to a package: the control file and the payload. The purpose of the control file is to provide basic information about the package and its relationship to other packages. This includes:

the name of the package

the version number

labels to classify it by purpose and importance

the type of licence (free or non-free)

the identity of the package maintainer

other packages which must be present for this one to work

The payload must be moved to a subdirectory which corresponds to the logical pathname where it will be placed. There are other files which can be added to a package but none of them are essential. This makes packaging (for your own use at least) a very quick and simple operation.

(Those familiar with the Debian package management system will find many of the internals of RiscPkg very familiar. That does not mean that RiscPkg can do everything that apt and dpkg can - far from it. Some, such as alternatives and pre/post installation scripts, are planned but not yet implemented. Others will be added if and when a need is encountered.)

The design of the package manager goes some way towards defining the differences between GNU/Linux distributions such as Debian, RedHat and Gentoo. More important is the package collection which each distribution provides. Packaging is usually performed by the distribution rather than the upstream developers.

The RISC OS Packaging Project aims to do much the same for our platform. That means more than just making software compatible with the package manager: it must also be compatible with the system as a whole, legally distributable by the project, and of a standard fit for distribution. This is not a licence to make changes at the whim of the package maintainer. It is an opportunity to create a more robust and coherent system than would otherwise be possible.

I am currently preparing a policy manual which will set out the rules which packages are expected to follow (if they are part of the packaging project). This will initially concentrate on the technical requirements of the package manager but need not stop there. Provided a consensus can be reached it can play an important role in setting standards for RISC OS software (or even just enforcing the standards already set by Acorn and its successors).

The current state of play is that the package manager RiscPkg exists with a very basic set of features. There are a handful of packages with which the system can be tested, but initially these are just the package manager and its supporting libraries. The policy manual will be published shortly and once this happens the package collection can begin to expand. There are a number of enhancements planned for RiscPkg (and in particular to its user interface).

The case for using RiscPkg is dependent on both the number of packages available and the extent to which it is able to co-exist with existing methods of working. Both of these will gradually improve with time. The key milestone will be the launch of a 'stable' version of the package collection: that is the point at which the packaging project will be suitable for use by end users rather than those with an interest in development.

Links
RiscPkg

Discussion

Viewing threaded comments | View comments unthreaded, listed by date

Sounds good

Phlamethrower on 18/1/04 11:23PM [ Reply | Report ]

Your computer works best when the number of levels of abstractions is low, so you understand what's happenning and can quickly fix things. If the app you double click on isn't where the files are that's an extra thing to remember, just like a Shortcut file in your Start menu on Windows.

Comparing searching for a module in the old !System to doing it in the new one with 310,350 etc dirs.

mavhc on 19/1/04 12:57AM [ Reply | Report ]

Not quite looked at it yet, but: have you used RISC OS's inbuilt system - the Installer module? This is a very useful tool for performing SysMerge, BootMerge and FontsMerge. It also rather nicely does a nice upgrade procedure for /any/ generalised application. It /does/ warrant a decent application to make generalised use of it though - eg. it can't handle uninstalls, needs a decent script with module versions etc.

If you haven't already, you should also seriously consider SysLog for any upgrades. Both can be used on RISC OS 3.1 upwards, and are established platform 'standards'.

Nice work though - definitely needed doing. Bonus marks if you use a further standard, eg. XML, for the package script ;o)

. Would mean that a suitable database could query upgrades easily :o)

md0u80c9 on 19/1/04 1:07AM [ Reply | Report ]

Good, clear, article which I enjoyed reading and which explains the isues in an understandable way to someone (like !) who's new to them. Sounds like an ambitious, but interesting, project - good luck with it !

martin on 19/1/04 7:21AM [ Reply | Report ]

Some points:

1. The control file with the name, version and license ist something I really think is a good feature since unfortunately for quite a bit of software it is pretty difficult to find the license condition. Please make sure that the licence contains the base info like "freeware", "shareware", ... as well as the info about distribution "may be copied freely", "charge for copy may cover cost for postage and disc only", "need consent from author before passing on", "may be gotten from authors website only", "may pe put on cover discs" and the like.

2. I can understand your idea of a base installation path since it makes things easier for you. If you indeed isist on definig a path where to install the app and then setting up links or stubs: Please do let the user define that base path for the application so that e.g. I can decide where the software may sit on my disc. E.g. my root directory is pretty empty containing "!Boot", "Programs", "Data" and "Work" only. The original folders like "Apps", "Utilities", ... have been moved inside "Programs".

3. As for that installation path perhaps this might be a possible idea for a bit more flexibility: I assume that the package manager keeps a list of what it installed on the local system. If you let the user specify where an app goes it could put the location down in that list.

4. Furthermore some scanning tool to search a directory or a list of them for the installed apps for the package manager is sensible so that the user can even move an app some place else and the aforementioned database is rebuilt. (Don't scann the full harddisc since that might take a bit too long and I can't hide e.g. previous versions of some app I backed up just in case).

5. As for the links or stubs: There is the odd case where I need to go into an app directory. That is no fun with those stubs and I'm nore sure about the links since I never saw any sensible link type solution for RISC OS yet.

6. One of the things nice in RISC OS is that I as the user can decide where I want to put my apps and thus have the odd subdirectory in my folder "Programs" to get some structure to what is where - this is partially vital since we don't have that start menu where I find all the apps (o.k. there is the odd tool to address that but they need to be manually filled). Thus my applications are grouped and with Filer- or Director I have a sensible menu structure to run them.

HzN

hzn on 19/1/04 8:23AM [ Reply | Report ]

Reasons why drag&drop fall down:

"there are large numbers of applications to be kept up-to-date;"

Does anyone find this? Personally I find I only need to update programs now and again. Perhaps 2 a month if I'm lucky.

"there are many files which do not belong to any single application."

Yeah, this is more of an issue. Keeping !System and Resources up to date and well stocked.

"RISC OS provides some support for merging files into the !Boot directory, but this is strictly a one-way operation with no way to remove files that are no longer needed. It also detracts from the simplicity of drag-and-drop installation. "

I can't remember the last time I wanted to remove something from !System or Resources. Does anyone find this a common problem? SysMerge, et al may detract from the drag&drop system but I'm assuming that RiscPkg isn't d&d either.

As I see it RiscPkg addresses the problem of keeping the files in !Boot updated but brings up a new problem of storing apps in a standard place.

My suggestion would be that an installer for !Boot components alone would be more approprite. Something that syncs with an online database of where to find the latest version of each component. As long as Programs' !Run files inform the user of which module, etc is missing the user can type the name into the !Boot installer and, provided the item is packaged properly, it will be downloaded and put in the correct place. The upside of this being that this system works with all existing programs and future apps need not follow any restrictive format.

This is even a system that could coexist with RiscPkg as long as it could make use of RiscPkg's package format for modules and resources.

Spriteman

Spriteman on 19/1/04 10:06AM [ Reply | Report ]

Personally I think people would also like a package manager to be able to find software to allow them to get a job done. Far too often we see in the Newsgroups and mailing lists people not being able to find software either by name or via a description by using normal web searchs.

Wouldn't it would be much nicer if they had an app that did this and just presented them with a list of suitable apps and a simple point and click download facility?

I know I would. :-)

quatermass on 19/1/04 10:19AM [ Reply | Report ]

Personally I prefer to know where everything is, which is why I prefer to know what's in my Boot and it not be a huge mess that we're not supposed to look in. A package manager is a nice idea, I can see it opening the way up to use of shared libraries more, which are in beta AFAIK. However, as a power user, I think I'll always try to avoid a package manager just because of how I like to work. Flexible paths seems a nice idea - since I keep my apps and data separate anyway I won't ave problems there if I use it.

john on 19/1/04 10:52AM [ Reply | Report ]

I'm with Spriteman on this. Really, applications should have as much of their stuff in the directory as possible, the only exceptions being shared modules and the like. !Boot and !System can accept most things (although my understanding of the innards of boot is limited, and I personally think it is too complicated.)

As I see it the only problem a package manager would really solve would be uninstalling shared files. And I don't think this is a huge problem. On the other hand, the amount of complexity introduced would be significant.

So what I'd really prefer is a more understandable boot. Maybe I'm just misunderstanding. If this is just a proposal for a !System (And predesk and the like) which move old versions of modules out to an archive directory[1] then it is OK. But if it is suggesting installing pseudo-applications which then break if they are moved, I think it is a crappy idea.

[1] Because some programs break if they hit a newer version module, they still can't be deleted; instead there would have to be a new 'maximum version number' option which could be retroactively applied to applications - thus the !Run files would have to be in some parseable format, and it *still* all gets horrid and icky.

Loris on 19/1/04 6:03PM [ Reply | Report ]

The issue of all files for an app in one places isn't really here nor there with regards to this.

The real issues, which RiscPkg attempts to adddress, are dependencies (which are genuine on RISC OS, despite some attempts to gloss over them), version checking (RISC OS apps are widespread, and it's not always obvious where to get the latest version), and automatic installation (easy on RISC OS, but that doesn't mean it can't be easier).

As I said on OSNews, once this system is in place, I expect people will begin to see the benefits. Just because you think a certain feature is of no use to you, doesn't mean it's not extremely important to other, less experienced RISC OS users.

mrchocky on 19/1/04 6:44PM [ Reply | Report ]

I agree with the issues Chocky lists right up to the automagic installation bit.

What might be interesting is fining out from a Mac OS user what they do since, as I understand it, Mac OS still uses drag and drop installation. (NB I refer to true Mac OS X apps and not unix-style, ported programs). What happens with dependencies and version checking?

Spriteman

Spriteman on 19/1/04 8:39PM [ Reply | Report ]

Well, disagree what you want, but I didn't use that word ;-)

"automatic" could include any number up things, and doesn't rule out drag and drop, nor does it have much to do with the fixed paths of Unix, or the relative opacity of installation of Windows apps.

The precise details of how a given object is installed, I think, are something that will have to be worked out by trial and error, and it's probably not worth debating them at this point until we've seen them in action.

mrchocky on 19/1/04 9:56PM [ Reply | Report ]

Personally I love the simplicity of drag and drop installation. It beats the pants off the methods used on Windows and Unix. I don't think we need to cater for people who don't understand the concepts of files and directories (as Windows seems to), and like many other RISC OS users I like to know what is going on when I install something. For example, when I get an upgrade to the Iyonix from Castle via the update watcher, I always open the directory first to see what it does, then make backups of the appropriate directories before launching the installation (yes, I know the installer makes backups automatically, but I like to play doubly safe). So I would hope at the very least that a packager will enable the user to find out what it is going to do before installation, and perhaps where necessary it should make automatic backups too. Oh, and leave a log of what it has done. Just in case we can't trust the backout process. Or in case it fails part way through. Perhaps I've learned from hard experience not to trust new software until I've seen it work!

Martin

mrtd on 20/1/04 12:53PM [ Reply | Report ]

We need to cater for everyone as much as possible. That kind of thinking will leave RISC OS with a small user base. Regardless of the reservations some users think they have (and I expect will be shown to be unfounded in practice), it still needs to be easy as possible.

mrchocky on 20/1/04 1:04PM [ Reply | Report ]

Absolutely. But drag and drop installation is already pretty easy. It also has the merit of simplicity, transparency and reliability. I'm not convinced I want to sacrifice that simplicity and transparancy for a marginal improvement in ease of use. I would think that learning to use drag and drop is something that very few people would be incapable of. Your point about dependencies is IMO the only justification, but this is less of a problem on RISC OS than on other platforms. Doing something just because other platforms do it that way (and it might be what some people expect) is a poor justification IMO. That way, we would just make RISC OS a clone of Windows and forget it.

mrtd on 20/1/04 1:54PM [ Reply | Report ]

Could someone give an example of such a dependency, please? Is there anything more involved than updating !system?

...

When I got a USB podule from STD it had an installer which would put stuff in various places in !Boot. Except it didn't work, because I didn't have a new enough boot structure. Actually getting a newer boot would actually be quite tricky, since the machine isn't internet connected. But fortunately I found I could just copy stuff to the appropriate places. This is an example of apparent simplicity actually making it more difficult for some users (which is what most of us hate about PCs I guess). So... won't a package manager just end up being another source of frustration?

Loris on 20/1/04 3:36PM [ Reply | Report ]

mrtd: no, no, and er, no. Your argument is full of assumptions.

Firstly, I haven't claimed anyone is "sacrificing" anything. The simplicity remains, and it will make other operatations which are not currently simple easier.

Secondly, no one said "because other platforms do it". We said "because RISC OS needs it". Which we do. There are many cases where there dependencies, and the mythical drag and drop installation doesn't apply. As Graham has said in the past, there are various programs which aren't isomorphic, !GCC being an obvious example.

If you think dependencies is the only justification, then you haven't read the article fully.

"Drag and drop installation is pretty easy". Is it? Yes, in general it is. But how do you know you have the latest version, or where to get the latest from? How you can because you have all the parts of the application? How can you be sure you have all the bits it relies upon? How do you know to change the config file if its format has changed? None of these questions can be answered reliably if done manually, and often are very frustrating even for seasoned users.

Please stop making assumptions about a system you haven't yet seen in action.

Loris: These types of installers are another reason why a system wide packager manager is good. It can be done once, and done properly, instead of ad-hoc solutions such as your example.

mrchocky on 20/1/04 4:55PM [ Reply | Report ]

oops: s/because/be sure/

mrchocky on 20/1/04 4:57PM [ Reply | Report ]

So basically, this is a system which would replace all of the random places that things can go in Boot? Or rather, be an interface to them all? Like I said before, I really don't mind the idea. But I do worry about applications becoming less 'atomic'. Having had to install programs on a PC, and know that it might add or modify things over the whole directory tree - after which things would never work properly again. Thus far, it hasn't been a problem on the RPC.

What does GCC do which is so special? I mean, I know its a compiler, and believe it to require a lot of configuration...

"But how do you know you have the latest version, or where to get the latest from?" How could the package manager tell you this? Check the website for you then perpetually bug you to update, or update without you knowing? What if I want to keep the version I've got?

"How [can you be sure] you have all the parts of the application?" If it doesn't work, thats a clue

(I'm sure you know what happens at the moment; I don't see how a package manager could improve matters beyond this. If it can't find module X then having an installer inform you is not really any different to having the Run file tell you.)

"How can you be sure you have all the bits it relies upon?" Because if it doesn't, it should tell you and tell you how to get them. (Isn't that the same question as above, basically?)

"How do you know to change the config file if its format has changed?" Um, dunno. How would the package manager do that? A custom updater could of course supply a tool to do it if necessary.

"None of these questions can be answered reliably if done manually, and often are very frustrating even for seasoned users." As you can see, I really am in the dark as to how a package manager would help matters. Please don't think I'm being sarcastic, I just don't 'get' it.

If the answer to this series of your questions is really just that it saves programming effort, and standardises the amount of information given for errors that is fine, that is a noble effort, but you seem to be promising that it will fix problems and I can't see how it will really make any difference for those to the end-user.

Loris on 20/1/04 6:27PM [ Reply | Report ]

"So, basically, this is a system which would replace all the random places that things can go in boot?"

No, no one said that at all. Anywhere. !Boot is just one part of a RISC OS system.

....GCC doesn't require any configuration. But that's not the point. It's not atomic, which is part of the problem. It contains a lot of sub utilities too, and also comes in parts.

The questions of mine which you've quoted are largely rhethorical, so I won't answer them directly, but I think you can answer many of your own questions youself.

However, the traditional way a package manager might work (and this slightly reflects how RiscPkg _currently_ works) is that it fetches a list of packages from a given URL source(s) which contain information about available packages. This is compared with what it knows you have installed, and you can choose to upgrade if you have an old version, or install something new.

If the new thing requires somethig else, then that is installed too - if there's genuinely anything missing (rare), then that's the fault of the packaging, and not something you've failed to download as in the manual case. The verbosity/user control is a matter of implementation detail/configuration and there is lots of scope.

The packaging will also know which files are configuration files, and will be able to deal with them specially if need be.

But referring to one question you ask "Because it if does, it should tell you". Which "it" would this be? Again, we have to resort to an adhoc system. And a program can't always know that it's missing part of itself. It might just crash. Better to have one system that avoids this.

mrchocky on 20/1/04 6:56PM [ Reply | Report ]

I think what Peter is saying is that a package manager would be very useful for installing non-"atomic" applications, such as some of the ones he ports from Unix. As a Unix user I can see where he is coming from. With most native RISC OS apps though the situation is less clear, since many of these are self-contained, and RMEnsure already deals with modules. The main problem that this solves is with shared dynamically linked libraries, something that we don't really have on RISC OS, although the Shared C Library achieves similar functionlity by using a module, with functions invoked by the stubs that are statically linked to the program.. If proper DLLs ever do get implemented on RISC OS, some sort of package management system will become eessential. So perhaps a package manager (and DLLs) are worth having if only to aid porting from other platforms. I would hate to think that it would encourage programmers to move away from the application directory concept, with everything except modules kept together. It is one of the strengths of RISC OS. But on a minority platform like ours, it makes sense to make porting from elsewhere as simple as we can.

mrtd on 20/1/04 8:40PM [ Reply | Report ]

No, I haven't said that. It is, but that's merely one detail. Shared libraries are also one small thing, and certainly not "the main problem this solves". The problems it solves I have already highlight in much earlier posts.

The situation isn't unclear with "normal" RISC OS apps - it's precisely what Graham has already stated. RMEnsure doesn't "deal" with modules - that's simply a mechanism for loading things at runtime, and the functionality of a package manager is completely different to this.

I'm afraid I can't help feeling you aren't really following the discussion, but are just picking up points slightly at random to comment on.

mrchocky on 20/1/04 9:10PM [ Reply | Report ]

For the record:

RiscPkg does not force you to put applications in any particular location. By modifying the path translation table it is possible to achieve any desired filesystem layout. At present this process isn't exactly user friendly, but that can be fixed.

However the recommended method of use is not to worry where the applications are really stored, and simply fake your preferred layout using softlinks of some description.

The next version of RiscPkg will provide a facility for creating softlinks. I plan to support both application stubs (of the type used to implement the Apps folder on the icon bar) and LinkFS image filing system links.

On a more general point, RiscPkg is written (and licenced) to be easily modified, so there is plenty of scope for taking an alternative view of how the filesystem should be managed.

What does need to be fixed in the very near future is the package format, so if anyone has views about this then now is the time to say.

(Draft copies of the policy manual, which contains a description of the package format, are available on request.)

gdshaw on 21/1/04 8:48AM [ Reply | Report ]

OK thanks folks, the concept is starting to get clearer to me.

Chocky, for most of your questions I was considering the extra things to be modules, so perhaps this explains my confusion.

"The questions of mine which you've quoted are largely rhethorical, so I won't answer them directly, but I think you can answer many of your own questions youself." Heh, most of my questions were also rhetorical, the point being that having a special manager wouldn't magically solve these problems. If I've not got a new enough shared C library (which is usually my problem) then on my net-less riscPC there is nothing it can do except ask for me to get it (or as you just mentioned, include it in the package, which is not done in many cases for legal reasons).

"But referring to one question you ask "Because it if does, it should tell you". Which "it" would this be? Again, we have to resort to an adhoc system. And a program can't always know that it's missing part of itself. It might just crash. Better to have one system that avoids this." You see I was thinking of the case where you run the program and it says "cannot find component X" (or update your C library or summut). If the additional file is part of the package as well then I can see how the package manager could check to see if it was present and install as necessary. And if the missing components are apps rather than modules then it makes even more sense. Although on the other hand this might mean that the zip-files will get much more bloated. Since I have to transfer everything from work, (and this used to be via 1.44M disk) you can see the problem.

"I'm afraid I can't help feeling you aren't really following the discussion, but are just picking up points slightly at random to comment on." Perhaps he is, and just didn't understand (like me). If you are not deeply aware of the concepts of Unix or PC systems then describing it with reference to them doesn't help, so I(/we) have been struggling to understand on the basis of how RiscOS works.

Loris on 21/1/04 2:39PM [ Reply | Report ]

An example of how using a package manager makes life easier (on Debian):

apt-get update; apt-get upgrade Updates the local list of which packages are available, finds out which I have that have newer versions, downloads and installs them. I can keep up with security updates by just doing this at regular intervals.

apt-get install fooapp Finds the latest version of fooapp, and all the other packages it depends upon, and downloads (or gets them off CD), and installs them.

apt-get remove fooapp Removes all files relating to fooapp, plus finds if there are any packages that it depends on that are no longer used by anything, and removes them.

OK, so this is Linux so it's done by typing commands, but all three actions could be done by a few menu clicks. I think this is much easier than going to the author's website, downloading an archive, unzipping it, moving the old copy out of the way, installing the new one, checking if the config file format has changed and so on. It also handles setting up the application better - generating default keys, config file wizards ('do you want black or white backgrounds in Zap?') and so on.

Perhaps the RO method might be augmented by a module that logged when applications were moved around by sitting on a vector, so that the user can move apps around and the package manager can still keep track of them. There's the risk of getting out of sync of course, but a rescan of the disc gets around this in the worst case (scheduled for 2am maybe?). So applications initially get installed somewhere central, and the user can move them around if they really want to.

caliston2 on 21/1/04 5:56PM [ Reply | Report ]

No Unix or PC concepts were introduced by me, everything I explained was based on RISC OS - in fact, they were introduced by others who tried to object to the concept on the basis of their understand of the idea on these systems.

I don't see how your Zip files will get more bloated. For one, Shared Libraries will make things smaller, and files related to packaging will generally only be a few hundreds of bytes, and rarely more than a few K.

mrchocky on 21/1/04 5:58PM [ Reply | Report ]

Why does the packager need to keep track of an App? (Assuming it's NOT used as a resource by another app).

Could a call be made on filer boot to check for new versions (defered if not online) and download ready (if so configured)? Then on filer run the app could be updated.

jess on 21/1/04 6:54PM [ Reply | Report ]

No.

piemmm on 21/1/04 7:26PM [ Reply | Report ]

One useful feature of package managment is the ability to issue a simple command and get a list of all the packages that are installed on your computer, along with version information (a bit like the *ROMModules command does for modules). This information can also be used by the package manager to check out the dependencies when a new package is installed.

Peter. The RMEnsure command can do much more than simply load missing modules at runtime. It executes a COMMAND if a module is missing or its version number is too low. A command can do anything the programmer chooses. Also, it isn't restricted to runtime. It can be included in any obey script, for example on filer-boot of an app or machine boot, or an obey file explicitly run by a user or by a timer. I'm sure you are aware of that. Presumably an app that is installed by drag and drop will automatically be "seen". So this could be used to check for missing modules at install time and perform some corrective action. But this is getting off the subject. Automatic checking for new versions of software is enminently possible, Castle's Iyonix Update Watcher already does exactly that. Doing this by running an app from the boot seqence that detects going on line as the update watcher does thouh is a better way of doing it IMO than using filer boot. That way it can be done centrally for all packages that the packager knows about. It also means that only a single watcher app is running, rather than a separate one for each package. The information held by the packager would either need to include a URL for each package, or alternatively the watcher would access a website that has a central version check and download facility for asll packages.

mrtd on 21/1/04 7:35PM [ Reply | Report ]

In reply to jess:

You can never know (for certain) that a package is not used as a resource by another application.

Other complications are that an application may be composed of several packages, and there may be unmanaged copies of an application present in addition to the managed copy.

(In the interests of simplicity, the lowest levels of the package manager know nothing about applications and modules: everything is a collection of files. I would quite like to keep it that way.)

gdshaw on 21/1/04 8:41PM [ Reply | Report ]

mrtd: No, the whole RMEnsure thing is a red herring. The correct modules should be installed when you install the program, if it's not, then it's a packaging failure.

Never mind that modules are just one small part of things on your system that can be upgraded, but the whole idea of "dynamically" checking versions will just cause frustration in the break up of the user/GUI experience, when the correct thing should have been installed in the first place. Not to mention the issue that Graham just hightlighted, that you can't be sure that an old version isn't in use.

No, please don't presume. An app that is "seen", can't reliably be checked for module dependencies without running it, or checking the dependency information when it is installed. It is at this latter point this should be fixed up.

Yes, we _know_ checking for new versions of software is possible. This is the entire point of this article and of RiscPkg. The rest of what you've said in your comment is no different to what has been said all along, so I'm not sure of your point.

mrchocky on 21/01/04 9:16PM [ Reply | Report ]

Could the package info include details of what file formats an application can handle?

Then if you had a file of some type you were unsure what to load with, you could, say, drag it to RiscPkg, and it could tell you what apps (if any) you have that would import it, or offer to install you one that could...

anon/131.111.137.75 via 131.111.8.103 on 21/01/04 10:06PM [ Reply | Report ]

Chris: I think a mechanism of this sort is possible, although in what form I think Graham might have to think about and comment on. This is the sort of typical and powerful query that becomes possible with a package manager.

mrchocky on 21/01/04 11:14PM [ Reply | Report ]

In reply to Chris:

There are already plans for an 'alternatives' mechanism which will have the necessary information to do this for already installed packages.

To do the same for uninstalled packages it would be necessary to add the information to the package control record. My only question is how useful the facility would be in practice (bearing in mind that the files which cause most difficulty tend to be those produced by (a) discontinued commercial software or (b) other operating systems.

I'll open up the question on the mailing list as to what if any metadata should be provided for searching (beyond the section, priority and free-text description). There are already quite a few fields to complete, and I don't want to make packaging more difficult or time-consuming than it needs to be.

gdshaw on 22/01/04 06:12AM [ Reply | Report ]

There might be some merit in a method of retropectively "registering" applications on that are installed on a machine but unknown to the package manager (ie ones that were installed by other means). For the sort of query Chris mentions to be useful, the packe manager would ned to know about ALL installed applications, not just the ones that came as packages.

mrtd on 22/01/04 08:55AM [ Reply | Report ]

(In reply to Chocky - but don't think I haven't read any later posts) "No Unix or PC concepts were introduced by me, everything I explained was based on RISC OS - in fact, they were introduced by others who tried to object to the concept on the basis of their understand of the idea on these systems." Actually they were introduced in the article itself. No insult was intended, merely explanation.

"I don't see how your Zip files will get more bloated. For one, Shared Libraries will make things smaller, and files related to packaging will generally only be a few hundreds of bytes, and rarely more than a few K." If for example every graphics program which relied on ChangeFSI included a copy of ChangeFSI, the packages would get larger. Please note I realise this is a spurious example because everyone should have changeFSI, but suppose a new, modified version was released... Similarly, every archive would need to include a copy of any Shared Libaries.

If the required programs are *not* included in the package then you are back to square one, in that you can't assume that you can actually obtain the file automagically at all. Its not going to help me, because I don't have an internet connection from that computer. People on dial-up may not thank you if the machine decides to phone up the internet automatically if you try to install a program during the day - at peak-rate. The website at which the required files may have moved, or be temporarily unobtainable, or the file may have been moved, and so on...

Loris on 22/01/04 12:15AM [ Reply | Report ]

Tony: you seem to have missed the point of packaging and shared libraries.

If something relies upon ChangeFSI (i.e. has a dependency on it), then the ChangeFSI package (which is separate) is also install - once, and once only (unless of course there's a later update) at the same time as your graphics app was installed. Likewise shared libraries - with the criticial word here being _shared_ - they are one resource used by everyone, and therefore only need to be installed once.

There's no requirement to dial up to the internet automatically (and I would discourage this feature in any case).

mrchocky on 22/01/04 12:34AM [ Reply | Report ]

mrtd: Again, there's no method of reliably determing what versions you have installed. Yes, you could use a guessy version/timestamp method, but it's boggy ground. At best, you could simply say it was installed, with an unknown version, and a properly packaged version that came along later was an upgrade.

mrchocky on 22/01/04 12:37AM [ Reply | Report ]

In reply to Peter:

I think I would go for prompting the user to enter the required data at registration time (with the option to leave items blank where not known) rather than trying to automate its derivation. This would be much the same information that is supplied during the packaging process.

It should also be possible to re-register of course, in case an application has been updated without installing a package.

In fact this sort of thing might be essential, to avoid complaints on installation about missing software that you actually have, but wasn't installed as a package.

mrtd on 22/01/04 1:30PM [ Reply | Report ]

Replying to Chocky: "you seem to have missed the point of packaging and shared libraries." I probably have!

"If something relies upon ChangeFSI (i.e. has a dependency on it), then the ChangeFSI package (which is separate) is also install - once, and once only (unless of course there's a later update) at the same time as your graphics app was installed."

So. A shared library/ChangeFSI etc are provided as separate packages. If a required component like that is not found the package/installer merely tells you that you need it (& potentially where it might be available).

May I attempt to sum up?

Basically it moves error reporting which currently happens at run-time to install time. This is an advantage for complex installations because you get a list of further requirements instead of finding them out one by one. This also applies to upgrading a piece of software. It also means that you can uninstall things and have the option of disposing of unused components. Finally, you can get a list of software with its versions, when you last upgraded etc.

I hope that is right.

Loris on 22/01/04 4:04PM [ Reply | Report ]

In reply to Loris:

The normal mode of operation is for RiscPkg to install the package(s) requested along with any other packages needed to make them work, all in one operation.

I dare say there will be an option to turn off dependency resolution. It is also possible for a package to 'recommend' or 'suggest' other packages, and I hope to include support for these relationships at some point in the future.

There will be a status bar to tell you how many packages and how many megabytes you have selected before you commit to downloading them.

gdshaw on 22/01/04 6:48PM [ Reply | Report ]

How then will this work on machines which don't have net access? What will happen if one or more of the required components are not able to be downloaded at request-time?

Would it work like this: I download all the packages I think I want on this PC and copy them to a special install directory on my RiscPC. I then run the install program. It scans this directory for packages which are actually executables (as opposed to shared libraries) then asks me which of these are the ones I want to install. It then attempts to install my selections, but finds it cannot do some of them because it is missing some required packages; therefore it prints out a list of required packages for each program-package and places where I could get them from. (If I'd had a network connection it would have given me the option of attempting this itself.) The packages which couldn't be installed, or ones which didn't need to be because they were not required components remain in the install directory, the other (zipfiles) are moved away.

?

Thankyou for your continued patience.

Loris on 23/01/04 10:28AM [ Reply | Report ]

Once again, I ask you to wait and see. There are various solutions here, but I won't outline them, because it'll just cause needless speculation over unimplemented details.

mrchocky on 23/01/04 10:50AM [ Reply | Report ]

In reply to Loris:

Actually, RiscPkg has already reached a state where it could install from a CDROM set. (There are a couple of issues which make this a bit more complicated than it needs to be, but these will disappear once RiscPkg is able to resolve relative as well as absolute URLs.) The main reason no CDROM set has been produced is the small number of packages available to put on it.

Other package transport schemes could be implemented as necessary, but if this left you having to manually download packages one at a time then you could reasonably question whether it was all worth the effort.

gdshaw on 23/01/04 5:47PM [ Reply | Report ]

Please login before posting a comment. Use the form on the right to do so or create a free account.

Create a new account
Forgot your password?

Search this website