rmac (+2.0)
19/10/07 4:44AM |
With the possible release of ADFS what are the implications for access to the hard drives commonly found on sale today? Will it be possible for improvements to allow us access to 500Gb or even 1Tb drives? Also to be access at an acceptable speed.
Currently it appears that a working limit is around 128Gb (at an acceptable speed) or 200 something with speed penalties.
As a RISC OS user who takes many digital images the lack of large storage is now a serious problem. To my mind one of the important bottlenecks that needs addressing.
How difficult is the task of changing this?
Do we have someone capable (with the time) to do it?
Would it be possible to suggest a contribution scheme from all RISC OS users to ensure it happens? |
bluenose (+3.0)
19/10/07 7:25AM |
The 128GB limit is on the Iyonix a hardware limitation with UDMA I believe. On the RiscPC then again their are hardware limitaions of a different sort. I suppose all these could be worked around with an addition interface card. There are limitations with the addressing that ADFS can do which is I suppose more important and again if ADFS could see more than one partition then I believe that could be one answer though thee best would be to address the underlying addressing limit.
Great to see the ROOL initiative oving forward. |
druck (+2.0)
19/10/07 8:59AM |
The RISC OS 5 version of ADFS has a defined 64bit interface to FileCore for use when the hardware sensibly supports larger drives. But remember ADFS is a block level driver, the native disc format is implemented by FileCore, and that needs changes to support more than 256GB. An extension to 1TB is easy, but decreasingly space efficient. More than that will need a different format entirely. |
rmac (+2.0)
19/10/07 9:06AM |
In reply to druck:
Is the change to a different format possible? How much would it cost to do a) in time & b) in money?
Would it be compatible with the rest of RISC OS? |
druck (+2.0)
19/10/07 9:10AM |
There needs to be big changes to the entire filing system stack to support greater than 2GB files, but of course existing APIs and compatibility with FileCore would retained. As for cost, who knows, thats the point of open sourcing it. |
hubersn (+2.0)
19/10/07 11:37AM |
In reply to rmac:
I am not so sure if tackling the disc size problem at the native access end is our best bet. As Druck said, we would need to replace Filecore for 1 TB HDs anyway, and then there are the hardware limitations (UDMA limit on IYONIX, no UDMA on A9 and Omega, PIO0 on RiscPC). For IYONIX and Omega, this would be theoretically solvable by writing a driver for one of the S-ATA PCI controllers, but it would be difficult to be able to boot from such a disc.
Maybe the best idea would be to leave the native HD access as it is and focus on NAS technology. We already have a very competent and fast NFS client, what we need is an optimized IP stack and an updated SMB client. With the IYONIX' gigabit networking, transfer rates of 40 MB/s should be possible, so it's in the same ballpark than native UDMA.
Of course, extending Fileswitch to handle files > 2GB would be necessary in any case. |
davehigton (+1.0)
19/10/07 1:07PM |
The simplest way I can see to deal with big discs is to partition them. Other common OSes partition discs; RISC OS could if it were so designed. I have a feeling that it could be added with only medium (at most) difficulty. It would certainly need thorough testing! |
nijinsky (+1.0)
19/10/07 1:09PM |
All the talk about huge hard drives makes me wonder how much data people actually have. Or what they exect their system to do. If they really think that RiscOS is the best system to be manipulating Tb of images they should think again.
Take me for example, I'm a biologist who have a 160Gb external drive I use for files. I've just spent 5 months doing video microscopy. Small image files 40Mb multipage 8 bit TIFF's. I have my desktop, my laptop and my home computer backed up on the drive and I've still only used 100Gb. All my old 500Mb image files from my last project are backed up on DVD.
If people really need 1Tb for their images must be taking 12Mb images by the ton and are a pro photographer. If that is the case then they should NOT be using RISC OS. They should be using a machine that can manipulate these large images quickly. eg a Dual Processor Mac and photoshop. Or like me use my 3Ghz Linux machine and Pixel32.
As for open source of the OS. Well I welcome that wholeheartedly.
All the best
Bob |
fwibbler (+0.2)
19/10/07 2:25PM |
Ahh! The old 'I don't need it and I can't understand why anyone else should need it, so no one else should' attitude.
Personally I think that kind of attitude has done a lot of damage to the RISC OS market over the years.
It has been far too prevelant in the RISC OS market for too long and has helped to hold back development and thus damaged the market.
It's like saying 'I don't need more than 640K of ram so I won't provide for it in the future' (or something like that) |
rmac (+1.0)
19/10/07 3:51PM |
In reply to nijinsky:
My Iyonix had two 80Gb HDD and my wife and I have quantities of images and documents/manuscripts. When appropriate I do go through and burn DVD's to enable clearing of some material. However when researching and collecting material for a book over a period of years you tend to accumulate.
To have a larger HDD that is fast enough would be a plus. Even a partitioned one would be fine. Just give us the space. |
riscosopen (+1.0)
19/10/07 6:33PM |
@nijinski: What about burning DVDs? The ISO image for a single DVD is several GB in size, a compressed archive of all the RISC OS sources is over 2GB. Developers tend to have multiple build environments/source trees for different things and these aren't compressed so that's tens of GB. Then lots of us have tens of GB of digital photos from over the years plus tens of GB of audio files.
I do testing of digital set-top boxes, too. The average video test file is several of GB in size. It's very easy to fill a >100GB hard disc nowadays for most of us. |
rjek (+1.0)
19/10/07 6:47PM |
Another good step, even if some of us still have reservations about the licence. I wonder when enough will be released that we'll be able to build RISC OS images ourselves?
In reply to nijinsky:
I'm not a pro photographer, but a hobbyist. My camera spits out 10MP images in RAW format. They're huge. Of course, you'd be mad to process these on RISC OS. But there are still important things that you'd want to do on RISC OS that requires large files. DVD burning has already been mentioned. Others include large image file systems (large tarballs, zips, ISO images, DOSFS images etc). |
rjek (+1.0)
19/10/07 6:49PM |
In reply to davehigton:
You could do a similar hack to Clares' BDFS reasonably easily. It's basically a new FileCore file system that uses ADFS simply as a block driver using its sector read/write calls.
I'm still in favour of replacing all of ADFS, FileCore and FileSwitch completely, though. You still hit the strange limitations of 8 drives per file system, and such, even ignoring drive and file size limitations. |
hzn
20/10/07 7:59AM |
In reply to several:
I agree that in most cases a harddisc beyond 100GB (probably even beyond 40GB) tends to be mostly empty since RISC OS software has such a nice small footprint and video processing is probably not done in RISC OS.
But perhaps this is looking from the wrong side: Some things are not done on RISC OS due to the 2GB limitation.
Currently there are two issues with the dammn 2GB limit:
- DVD burning (but, what the heck, Steffen Hubers CDVDBurn can cope by splitting the image) so not really a pain in the a.
- But the fact that the 2GB limit applies to DOSFS since that is an image filing system is really a drag and should be fixed ASAP - be it by removing the 2GB limit (with all the good benefits that gives you in other areas), or by rewriting a real DOS Filing System, that is not making it an image filing system.
- And with bigger file sizes, perhaps video processing and playing might come to RISC OS since we'd then have a chance to store longer films to start with. |
hzn
20/10/07 8:06AM |
In reply to riscosopen:
BTW, will Printers be in the next open source bundle - I was thinking about Martin Würthner's and John Tygat's Postscript work... |
AMS
20/10/07 4:42PM |
In reply to hzn:
Agreed the 2GB limit is a problem alright. The issue is how do you sensibly get around it? Many APIs assume a 32bit value (in BASIC EXT#, PTR#) - you'd need to modify the way BASIC works and also add to or modify OS calls like OS_GBPB and so on - but in such a way as to preserve backwards compatibility and without introducing "silly" solutions (think of the page+offset scheme that was on the old x86 as an example of how not to do things).
I note ADFS/Filecore etc., are included in the next ROOL release - perhaps with these in the open it is time to start discussing how to take Filesystems on the RISC OS platform forward and come up with sensible solutions for this. |
wibli (+1.0)
20/10/07 10:48PM |
Take a peek at the following for source code and (hurrah!) binaries:
[Link: www.riscosopen.co.uk]
Fabulous  |
thesnark (+1.0)
21/10/07 12:27AM |
I don't want to rain on anyone's parade, but as someone who has contributed to the RISCOS Ltd branch of RISC OS in the past, I have extremely mixed feelings about this release. I doubt that I am the only one.
With some of this software such as Browse, there is no risk of conflict with a parallel RISCOS Ltd version, and I welcome its release wholeheartedly. However, in other cases - such as the Toolbox - there seems a high risk of duplicated effort and version number confusion (lucky users). The worst case scenario is that features will diverge, leading to incompatibilities (just what we need to retain developers). Colin Granville posted to usenet about this several months ago:
[Link: groups.google.com]
I am not interested in reinventing wheels, filing duplicate bug reports, or repeating past discussions. Frankly, I am more likely to become disillusioned with both using RISC OS and developing software for it. Hey, I might even get a life instead! Sadly, there are few people left who are both capable and willing to work on RISC OS, and the last thing that we need is the worsening of the current schism. I fear that is what this latest source code release may precipitate, no matter how well-intentioned.
That is merely my personal point of view. I have no influence over the directors of either RISCOS Ltd or Castle Technology Ltd, nor much interest in their alleged past disputes. |
hubersn (+1.0)
21/10/07 1:08AM |
In reply to thesnark:
I think the past has clearly shown that neither RO Ltd. nor Castle really have the development resources to take RISC OS forward. And both have no intention to make many of their developments available to the other party (and therefore to the whole RISC OS userbase).
I think it is highly unlikely that both branches of RISC OS will merge. Therefore, I see the RISC OS Open source soon becoming the de facto standard for all RISC OS users, since this will be the RISC OS branch which allows OS development for all users. In the long run, it will help to prevent future incompatibilities and future wheel reinvention.
I believe this is the only way forward for RISC OS. The only way to bridge the gap between the currently existing branches. The only way to attract new users. |
thesnark
21/10/07 1:47AM |
In reply to hubersn:
"I think it is highly unlikely that both branches of RISC OS will merge. Therefore, I see the RISC OS Open source soon becoming the de facto standard for all RISC OS users"
If you are right then a stupendous amount of time will have been wasted by testers and developers who worked on the other branch. |
markee174
21/10/07 8:48AM |
In reply to thesnark:
The IT world is full of such waste, sadly (just look at the number of 'dead' projects on sourceforge). Regretable it is, but as Hubersn says, at least there is a possible way forward (although not the one we would really like). |
jess
21/10/07 9:29AM |
RISC OS is modular. As long as both branches converge on the way they are modularised and keep the interfaces consistent, then there should be a sensible way forward.
If the ROL toolbox is better (and remains free) then that will be used by most systems. I
f a component of RO 5 is better, then 4/6 users would use that in preference.
Convergence could happen naturally over time. |
sa110
21/10/07 10:49AM |
In reply to hubersn:
You wrote: "Therefore, I see the RISC OS Open source soon becoming the de facto standard for all RISC OS users, since this will be the RISC OS branch which allows OS development for all users. In the long run, it will help to prevent future incompatibilities and future wheel reinvention. "
How can the ROOL RISC OS become the defacto standard when it only runs on the Iyonix?. ROL do at least have all the old RPC's and A7000, all the new Virtual Acorns and A9homes. For ROOL's version to become the standard, then this first needs to run on at least Virtual Acorn and the A9home. |
AMS
21/10/07 12:28PM |
In reply to sa110:
How can Select or RO6 be considered the defacto standard when *it doesn't* run on Iyonix ?
We have a split, there is no defacto standard only two related OS'es with a considerable degree of API commonality.
RO5.xx is at least open to all - that will give it traction. You may find some of the modules that are not hardware specific may run as is on A9 - some others would require work - but with the source that becomes feasible.
The most likely area to see rapid improvement is RO5.xx - so be it defacto standard or not it's features will become more desireable. That will either cause (i). ROL to adopt some of these implementing their own "workalikes" (ii). ROL to license from Castle the fed back improvements from RO5.xx and not have to use their own development resources to implement these or (iii). A9 owners with some programming skills to contribute to and take advantages of the improvements on their machines. (i) and (iii) being the most likely.
In any event you'll see improvements. To benefit first you'd be better off on an Iyonix (the source is for *it's* OS and can be used directly), overtime though others can take advantage too. In the absence of one defacto standard that would make RO5.xx the version that sets the standard and sets the pace - not a bad place to be methinks. |
hubersn
21/10/07 1:41PM |
In reply to sa110:
I expect much of the development on the RISC OS Open source to be runnable on other RISC OS versions than RO 5.
Until there is a RO 5 version for the old RPC architecture, the only way forward for the whole platform is to go the same way as Acorn did when they released the Nested Wimp, Castle when they released the 32bit SharedCLib and RO Ltd. when they released the Toolbox modules. |
hubersn
21/10/07 2:05PM |
In reply to thesnark:
yes, unfortunately RO Ltd. decided to not support the IYONIX, so for a significant share of RISC OS users, much of the time invested in the RO Ltd. branch could be described as "wasted". The same can be said of the Castle branch wrt RPC/A7000/VRPC/A9 users.
This is sad, but we are living with this kind of divergence for many years now. With the RISC OS Open initiative, at least the community itself has a chance to bridge the gap. The involved companies have already shown that they won't. |
hzn
21/10/07 2:14PM |
In reply to thesnark & hubersn:
I agree with hubersn that it does seem that neither ROL nor CTL really have the development resources to take RISC OS forward. The main difference between these two is that CTL decided to open source their RO so that there is a chance for enhancements by others whereas ROL simply delays releases...
But there are fair chances that enhancements done to RO5 will be usable on RO4/6 too (unless RO changed the interfaces too much).
And as for VirtualAcorn: AFAIR RO5 licenses are to be comparatively low so that it is perhaps worth for VA to look into getting their virtual machine running with RO5 and thus being able to sell it at a cheaper rate.
|
stevef
21/10/07 11:25PM |
In reply to thesnark:
"If you are right then a stupendous amount of time will have been wasted by testers and developers who worked on the other branch."
If Paul Middeton's reported comments at ROUGOL are true (that Select will never appear for the Iyonix), then a stupendous amount of time has already been wasted by testers and developers as far as Iyonix owners are concerned. |
caliston2 (+2.0)
21/10/07 11:42PM |
I've been having a brief poke around the source of ADFS, FileCore and FileSwitch.
Firstly there's a serious documentation need. All the documentation is in little text files scattered around - ideally it needs someone writing a basic overview of what happens where. Secondly FileCore and FileSwitch are inherently harder to understand (for me, anyway) because they do lots of structure twiddling which is harder to understand in assembler.
I'd suggest:
Retaining ADFS as-is for its floppy support (on any machines that still have drives). All the floppy foibles take up a fair amount of code and I think it's best left alone.
Re-implementing the hard drive support as a new IDEFS (davehigton and I had a go in 1997 - it wasn't too hard to get something working, the messy bit is coping with all the variations in drives out there)
Bin RAMFS. Fix bugs in Memphis and use that instead.
A new interface between block drivers and filing systems, with a FileCore 'personality' for compatibility. This might do disc partitions too.
FileCore retained as-is, for supporting existing drives, but accessed through the new interface
A module for a new disc format (FAT? NTFS? ext3?). If it can work on an extended image filing system API, so much the better (but would have to be reentrant)
Lower priority: Rewrite HForm into a cross-platform FileCore filesystem generator - maybe plug it into acorn-fdisk that already exists.
Then it gets messy. FileSwitch is more closely tied in with the kernel, and there's no nice list of tie-in assumptions in the source as there is for ADFS. I'm not convinced rewriting FileSwitch would help - you'd have to rewrite bits of the kernel too. Extending to 64 bit addressing would be a must, perhaps at the same time simplifying the API and putting the old one in a 'personality' module for compatibility. Are there any other problems with FileSwitch other than the 32 bit file size issue? Could they be solved by helper modules intercepting calls between FileSwitch and the lower layers in the stack? |
CKH2
27/10/07 9:51AM |
The most important and urgent task now that the source code has been released is for the developers to ensure that the code is tweaked so that it runs seamlessly on both RiscPC/Virtual Acorn type machines as well as the newer 32bit A9home/Iyonix type machines. The use does not want or need to worry about the history. Other development can wait until that is done. The sources also need to be distributed with a !SysMerge utility that will update any of these machines to have the latest modules (and module numbers) so that they are 32bit neutral (and RISC OS fork independent). Raising money against this aim would be much more worthwhile than the current 'Select' scheme. |
| Use the forum for more comments on this article |
RISC OS Open, the community driven organisation tasked with opening up RISC OS 5, is set to release more operating system components this weekend. The ROOL team plan to publish the next batch of RISC OS source code on the day of the South East show in Guildford, Surrey.
