Drobe :: The archives
About Drobe | Contact | RSS | Twitter | Tech docs | Downloads | BBC Micro

RISC OS Memory Protection

By Peter Naulls. Published: 25th Apr 2005, 12:20:20 | Permalink | Printable

Corruption at the lowest levels

Memory ProtectionOne common complaint or feature request for RISC OS improvement is to add "memory protection". This is largely a result of the relative ease of which single programs can take out the entire operating system, combined with a misunderstanding of what precisely memory protection is.

In this article, I'll try and cover some of the issues around memory protection, and why RISC OS is often so susceptible to breakage and some of the measures which can be taken to improve the situation.

RISC OS and Memory Protection
Of course, even a basic grounding in computer architecture will teach you that RISC OS does indeed have memory protection, and that it's essentially the same mechanism that's used in most other operating systems with any degree of complexity. I won't get into the nitty gritty of the memory protection system and the difference between physical and logical memory (sometimes called virtual memory, which is not to be confused with swap space), but it should be sufficient to say that a running application cannot see the memory used in wimpslots of other applications. It also cannot see some parts of memory used by RISC OS itself. Other parts of memory will be readable only, and some parts will be freely readable and writable by all programs.

Some simple operating systems found in embedded devices do not have any mechanism of this type, either because the CPU does not support the required functionality or the lack complexity of the software does not warrant it: they use what is called a flat memory model, where all the memory is visible to the entire system. It's only these systems that you could really say lack memory protection.

The Problem With RISC OS
There are several issues that can leave to instability in RISC OS. The root cause of all of many of these is the ability to read and write certain areas of memory with impunity, and they can be categorised into three main problems:

Low Memory Access. In the C programming language, a null memory reference is represented by a value of zero, meaning a value which does not refer to anywhere. There are instances where zero value memory references are used in BASIC, but that tends to occur less often because of things like BASIC's comprehensive string handling.

Problems arise when programs erroneously try to access memory at location zero, or small increments after it. This memory contains nothing of interest to normal applications, and it is a program bug to try and access it. On most systems, this would correctly cause an immediate crash. But not on RISC OS - instead, the values (often, ARM instructions) are read, and the program may continue on for a time until it later tries to use the values resulting in a crash well removed from the point of the program fault. Or, it might try and display the values because it thought it was looking at a message - this is precisely the cause of the "ofla" bug seen in some RISC OS programs.

Note that it's rarely useful for a program to try and stagger on after a memory access failure - not only does the programmer want to know about any failures as soon as possible, but further indiscretions the with bogus data could lead to other crashes elsewhere in the system or data corruption in the document the program is currently handling.

The situation on RISC OS prior to RISC OS 4 is even worse: low memory areas were writable as well as readable. But the bottom line is that there shouldn't be any reason at all for applications to be reading or writing memory below its start address (the 32k between zero and the 0x8000 wimpslot address), and attempt to do so should result in an immediate signal being sent to the application (which normally means the application immediately quits - some programs may attempt to die more gracefully).

At present, as mentioned earlier, programs attempting to read low memory will succeed, with a value that will often cause grief much later in the program - it's these bugs can have caused a huge waste of developer time. Being able to immediately pinpoint the point of failure can be very important.

So why isn't this the case? Well, it can be, but there are a number of minor obstacles. Some SWIs such as OS_GetEnv return workspace in this memory, and the ShareFS filer also has memory it validly accesses here. Of course, these could be moved, and it's very likely that there are a number of other issues that would have to be addressed before this could become the default in a version of RISC OS. In the meantime, Adrian Lees has written a program "Prot1K", which can be fetched from his pages. This prevents RISC OS applications reading the first 1KB of memory - this will still allow OS_GetEnv and friends to work, but it will crash the ShareFS filer. Using Prot1K allowed me to immediately spot some long-standing low memory accesses in Unixlib and some of my other code. I encourage you to try it on your own programs.

The RISC OS module model. RISC OS modules are very powerful; they run with the same status as the rest of the operating system (which is mostly built from modules itself), which means that any badly written module has free reign to trash whatever it likes. On top of that, the module area, where modules themselves live and their workspace is allocated from, is readily accessible from RISC OS applications. This has advantages - it means that modules can directly pass back references to buffers they have, avoiding a copying mechanism. This design accounts for much of the responsiveness of RISC OS on relatively low specification hardware, but is clearly also wide open to abuse by misbehaving applications or other modules.

There's not a great deal that can be done about this - changing the system would require huge changes in RISC OS and many applications. One mitigating suggestion that has been mentioned is to add a module flag that could be used by new modules (or set in old modules in which it is possible to release new versions of). This flag would indicate that the module code ought to be loaded into memory that is read only to the module and the rest of the OS, and completely inaccessible in RISC OS applications. Buffers can still be passed back to programs via module workspace, which many modules claim anyway, and is readable by everyone.

One of the reasons that modules are so pervasive in RISC OS is that there are a number of OS facilities that simply can't be accessed by regular programs. To get access to certain data or be informed of certain types of events, you need to have a module. Clearly there is room here for improvement in RISC OS to provide new APIs to access such services, and reduce the need for modules.

Garbage in, Garbage out. Passing nonsense to a SWI has traditionally been an excellent way to take out the system. Often the reason for this is really the same as the above - the ability of modules to access anywhere in memory. For speed, or perhaps just laziness, parts of RISC OS don't do as much checking as they should, and likewise many third party modules and applications.

This is a situation which can, and has been improved. Many of the improvements in Select, although often seemingly minor, have been to considerably improve the robustness of RISC OS, part of which is to include such checking. This is why Adjust/Select is generally considered much more stable than previous versions.

As for avoiding passing rubbish around - either within an application itself, or to parts of the OS, there are freely available tools to help C/C++ programmers.

One of these is Fortify, which provides wrappers for the memory allocation facilities used in C - the calls to malloc, free, et al. Fortify is not written for RISC OS, but works perfectly well with it, and has been invaluable in spotting buffer overruns, naughty use of memory, and other bad behaviour. Providing very similar facilities, but arguably better tuned for RISC OS, is MalCheck.

These tools can prove to be very important, but one of the down sides in using them is that you need to recompile your entire program, and make sure that the memory check header is used in all files where allocation functions are used. If libraries the program uses expect your program to free memory they've allocated or similar, then these libraries will need to be compiled with the headers too.

Recognising this issue, I've made some changes recently to GCCSDK. This makes use of an obscure linker feature whereby the names of symbols are changed when the program executable is generated, so that the memory allocation functions used in all parts of the program can be redirected via the memory checking functions if a single option is specified. This means that the single step of linking is only required to generate a binary either with or without memory checking (which naturally degrades performance), and also means that there's no penalty when it's turned off. You can read more about this in my post to the GCCSDK mailing list. This has already proved to be valuable in finding issues that would otherwise be very hard to spot.

Wrapping Up
There's perhaps quite a bit more that could be said, but I think I've covered the main issues, and you'll understand my glossing of some points in the name of brevity. But now you know; the next time someone asks for memory protection in RISC OS, point them at this article.

Links


Fortify
MalCheck
Prot1K
GCCSDK

Previous: "Vast majority" of ROS 4 now 32bit
Next: Wakefield 2005 theatre details

Discussion

Viewing threaded comments | View comments unthreaded, listed by date | Skip to the end

I've been trying to explain to people for many years why wide-covering memory protection for RISC OS either won't work all too well, or will require massive rewrites to huge chunks of software. One of the reasons of course is that so many applications expect to be able to write to the module area, which of course means they can write (indirectly) to the kernel. Which is a bit of a mess.

I've always been an advocate of the hope that when RISC OS became 32bit, they'd change the APIs sufficiently to fix most of the obvious issues, rather than just doing the bare minimum. After all, most programs needed a lot of work doing to them. It was a missed opputunity to fix many of RISC OS's braindeadedness. :(

 is a RISC OS Usernunfetishist on 25/4/05 9:08PM
[ Reply | Permalink | Report ]

In my opinion, it's too late to try and change RISC OS by adding memory protection.

Reason 1 is that too many programs would need modification.

Reason 2 is that, once a program is developed and debugged, it probably won't need it. Yes, it may be a good idea on a development system for debugging software, but how much software is released with a memory corruption bug still in ?

 is a RISC OS Userdemondb on 25/4/05 9:15PM
[ Reply | Permalink | Report ]

demonb: In light of what I've written above in the article, your point doesn't make much sense. Memory protection to the degree I advocate is clearly both possible and crucially important in saving considerable amounts of developer time. You don't explain why you think programs would need modification - if they're buggy, then I certainly hope they would be fixed. This is perhaps more practical on a 32-bit RISC OS, where running programs have some degree of source available.

As for your reason 2, loads of software is released with memory corruption, or dodgy accesses in it, precisely because of the lack of memory protection in the past. The truth is that such programming errors are surprisingly common, although in many cases they go unnoticed because they don't do anything obviously bad (most of the time).

 is a RISC OS Usermrchocky on 25/4/05 9:42PM
[ Reply | Permalink | Report ]

overdue isn't too late some programs will crash earlier but exactly where the mistake is so it easyer to fix

 is a RISC OS UserJaco on 25/4/05 10:31PM
[ Reply | Permalink | Report ]

To mrchocky

It would depend on how the memory protection is implemented. If the module area is suddenly going to be a protected area, then any software which reads or writes to it probably would not work. I would imagine that this is a lot of software.

Memory protection is something that is usually implemented in an OS from the outset, I would imagine it would be difficult to add it later without compromising a lot of existing software. Bear in mind that a lot of software in use is now no longer developed. Also, users may not be able to stump up money to upgrade a lot of applications.

In terms of your last statement, it's a bit of an oxymoron to say lots of software is released with memory corruption, but they don't do anything bad. Surely if it does not crash, then it works !

I very rarely find software, either free, shareware or commercial which has bugs in which cause it to crash. Maybe I'm just not running the right (or wrong) software.

I'm not knocking the idea of memory protection, I could certainly have used it a few weeks ago, but would argue more effort could be put into fixing existing problems with modules such as the WIMP, where very nasty things can happen if you give it a wrong window handle !

 is a RISC OS Userdemondb on 26/4/05 7:01AM
[ Reply | Permalink | Report ]

demondb: You very rarely find software with bugs that cause crashes? Do you ever turn your computer on? Sometimes just the program itself crashes, which is a bit annoying but fine, but it's still possible to lock up the whole machine. Even if the machine still appears to be running there might be some "below the surface" problem. Reducing the possibilities of this happening sounds like a very sensible move, even if it isn't complete memory protection.

"it's a bit of an oxymoron to say lots of software is released with memory corruption, but they don't do anything bad. Surely if it does not crash, then it works!"

Have you ever written a program? Do you know anything about software? It's very easy for a program to write to where it shouldn't do, but for nothing bad to appear to happen. Why nothing appears to happen might very well be because you've got lucky (or unlucky, depending on your point of view), and as soon as someone else uses the software on a different machine running different things it may crash. There is no way of predicting what could happen, so leaving a known bug in because it still appears to work anyway is a recipe for disaster.

If the brakes on your car stop working you'd like to know about it ASAP, when you might still be able to bring it to a stop under control, even though you might be sitting on the motorway and not even need to touch the brakes for another two hours.

 is a RISC OS UserSimonC on 26/4/05 10:19AM
[ Reply | Permalink | Report ]

Err yes, I have written programs before...let me see, Boots the Chemist, Cooper lighting and security, oh and RiscCAD of course...

and I didn't say known bugs should be left in.

and yes, I run my computer at least 8 hours a day, 7 days a week, and I still maintain I find very few programs which crash.

 is a RISC OS Userdemondb on 26/4/05 10:42AM
[ Reply | Permalink | Report ]

Actually leaving a known bug in because it still appears to work happens all the time in big software packages - for the simple fact that trying to fix it would cost too much.

There are plenty of bugs in Microsoft products, but because they will only affect a minority of users they are not fixed. Testing and fixing bugs costs money, and at some point a company has to draw a line under their testing and release the product. The notion of 100% reliable software is a myth, I'm afraid.

 is a RISC OS UserWalks on 26/4/05 12:26PM
[ Reply | Permalink | Report ]

As a non-programming user I may have mis-understood the article, but to me it said. 1. RISC OS has some memory protection, which has been improved, but its design rules out more robust methods for every day use. 2. There are tools to increse memory protection which developers can use which will catch a majority of the major culprits during development work and may speed up such work by identifying mal behaviour in a timely manner. 3. Applications developed in this manner/enviroment should have fewer cases of inappropriate memory access and therefore require less support. I could be mistaken however.

 is a RISC OS UserHeloWorld on 26/4/05 1:00PM
[ Reply | Permalink | Report ]

Oh, and I am very appreciative of all the work that the developers do.

 is a RISC OS UserHeloWorld on 26/4/05 1:01PM
[ Reply | Permalink | Report ]

In reply to SimonC

"If the brakes on your car stop working you'd like to know about it ASAP, when you might still be able to bring it to a stop under control, even though you might be sitting on the motorway and not even need to touch the brakes for another two hours."

Maybe, but you don't actually buy the car with duff brakes.

 is a RISC OS Userdemondb on 26/4/05 1:45PM
[ Reply | Permalink | Report ]

You won't knowingly buy a car with duff brakes. Most of the time, the manufacturer won't sell you a car they know has duff brakes. That doesn't mean that it's brakes aren't duff, and they won't recall a batch of them to replace the brakes.

 is a RISC OS Usernunfetishist on 26/4/05 1:51PM
[ Reply | Permalink | Report ]

They will recall them, trust me. Ford were once very badly stung in the US and sued because they did not recall a car with a fault. An internal memo revealed the top brass decided the cost of recall would be less than the lawsuits from dead peoples relatives.

My point is this, if memory protection was SUCH an issue, based on the fact that lots of software crashes regularly due to memory corruption, then it would have been implemented a lot earlier. The fact that it isn't, must mean that the vast majority of software is OK. I would think you simply avoid that which does not work.

 is a RISC OS Userdemondb on 26/4/05 4:30PM
[ Reply | Permalink | Report ]

Sorry, that should read "the cost of recall would be more that the lawsuits from dead peoples relatives."

 is a RISC OS Userdemondb on 26/4/05 4:31PM
[ Reply | Permalink | Report ]

The big problem with memory corruption on RISC OS is that you often can't easily find out which application is at fault, and even more often the results are so obscure that the developer can't track the problem down without the help of the measurements Peter has described.

So which software should I avoid if I don't know which is at fault?

 is a RISC OS Userhubersn on 26/4/05 4:45PM
[ Reply | Permalink | Report ]

demondb: Perhaps my statement was very slightly too ambiguous - I'm saying that they *would* recall them, once it was discovered. Not that it actually has any relevance to the meaning of my reply to you.

hubersn: The cynical troll inside me says you should avoid RISC OS if you don't know which piece of software is at fault. :)

 is a RISC OS Usernunfetishist on 26/4/05 4:58PM
[ Reply | Permalink | Report ]

To nunfetishist

I know what you meant, just could not resist :-)

I'm not saying memory protection is a bad idea, just with very recent problems I have had with the WIMP reporting Illegal Window Handle, I would much rather work go into more checking of data on these modules, rather than memory corruption, which I don't have much experience of.

 is a RISC OS Userdemondb on 26/4/05 5:34PM
[ Reply | Permalink | Report ]

demodb: Perhaps it's because there hasn't been much in the way of protection that RISC OS apps haven't been prone to crash too often - forces the programmer to take care, which is why I raised my eyebrows at the comment about if it doesn't seem to crash then it's OK. I took it to mean that you thought it doesn't matter even if you know it's doing something naughty, as long as it seems to work.

 is a RISC OS UserSimonC on 26/4/05 6:11PM
[ Reply | Permalink | Report ]

demondb: "My point is this, if memory protection was SUCH an issue, based on the fact that lots of software crashes regularly due to memory corruption, then it would have been implemented a lot earlier. The fact that it isn't, must mean that the vast majority of software is OK. I would think you simply avoid that which does not work."

Who says it's such an isue? It's nice to have and it saves time, especially for begining programmers and when porting or changing other peoples programs.

 is a RISC OS UserJaco on 26/4/05 6:48PM
[ Reply | Permalink | Report ]

nutty:

"you should avoid RISC OS if you don't know which piece of software is at fault"

Baby and bathwater ;)

 is a RISC OS UserSpriteman on 28/4/05 12:54PM
[ Reply | Permalink | Report ]

More like frying pan and fire.

 is a RISC OS UserLoris on 28/4/05 6:56PM
[ Reply | Permalink | Report ]

Perhaps the way to attack this issue is to have memory protection that can easily be switched on and off by the user. This way developers could test their programs with the module area write protected to see what happened. They could then fix any prblems before releasing the software. Users would continue to run with write enabled, so that the memory protection does not break existing appls, until their developers have had chance to fix the faults. As someone who writes safety related software for a living, the thought of not doing all that you can to eliminate such bugs horrifies me. We use an array of tools to perform both static and dynamic analysis on our code, we compile it with different compilers on different platforms, which can sometimes result in the compiler throwing up new errors, we are starting to use a new technique called abstract interpretation, which models the code and then interprets it to find things like unitialised pointers and other variables, dead code, writing ouside array bounds etc. We also tend to avoid certain types of construct like the use of dynamic memory allocation, a rich source of memory leaks, and explicit use of pointers. In concentrates the mind to know that peoples lives may depend on that code working correctly. The lack of full memory protection is of course one reason why we could never use RISC OS (in its present form) for such applications. We don't use Windows either.

 is a RISC OS Usermrtd on 29/4/05 9:29AM
[ Reply | Permalink | Report ]

The issue is that a badly behaved app shouldn't take out the rest of the system - that's just unacceptable.

 is a RISC OS Userjonix on 29/4/05 12:24PM
[ Reply | Permalink | Report ]

In reply to jonix: In that case you'd also have to prevent programs from claiming vectors and other entities that can take down the system if not properly released upon program termination and before task switching. Under RISC OS the onus is entirely on a program's exit handler to do such tidying up. Given that the exit handler may be badly written, bugged or simply non-existent, the potential for disaster is large. Also, the shared C library's exit handler relies upon having a valid stack frame, which will often not be the case if the program contains non-APCS code or an abort occurs elsewhere. (You get a 'No stack for trap handler' error, possibly followed by the system crashing due to unreleased vectors etc.) In short, what you are asking would require more changes to RISC OS than simply memory protection.

 is a RISC OS Userthesnark on 29/4/05 11:01PM
[ Reply | Permalink | Report ]

thesnark:

One step at a time at least gets you closer to what you want.

 is a RISC OS UserJaco on 30/4/05 9:54AM
[ Reply | Permalink | Report ]

Please login before posting a comment. Use the form on the right to do so or create a free account.

Search the archives

Today's featured article

  • Running a fashion label with RISC OS
    Independent t-shirt design in Germany
     3 comments, latest by Sawadee on 22/10/05 9:36AM. Published: 16 Oct 2005

  • Random article

  • Castle announces ARM9 MiMagic 5 support
    Only 220MHz but still exciting, if you want ROS in a PDA [Updated]
     25 comments, latest by jess on 1/12/04 9:49PM. Published: 29 Nov 2004

  • Useful links

    News and media:
    IconbarMyRISCOSArcSiteRISCOScodeANSC.S.A.AnnounceArchiveQercusRiscWorldDrag'n'DropGAG-News

    Top developers:
    RISCOS LtdRISC OS OpenMW SoftwareR-CompAdvantage SixVirtualAcorn

    Dealers:
    CJE MicrosAPDLCastlea4X-AmpleLiquid SiliconWebmonster

    Usergroups:
    WROCCRONENKACCIRUGSASAUGROUGOLRONWUGMUGWAUGGAGRISCOS.be

    Useful:
    RISCOS.org.ukRISCOS.orgRISCOS.infoFilebaseChris Why's Acorn/RISC OS collectionNetSurf

    Non-RISC OS:
    The RegisterThe InquirerApple InsiderBBC NewsSky NewsGoogle Newsxkcddiodesign


    © 1999-2009 The Drobe Team. Some rights reserved, click here for more information
    Powered by MiniDrobeCMS, based on J4U | Statistics
    "An increasing number of Drobe articles don't relate to RISC OS. Who cares if the Finn brothers sell Sibelius to a bunch of yanks"
    Page generated in 0.0778 seconds.