*** Updated March 1, 2000 ***
This page contains a letter from IBM, follow-up clarification, a rebuttal
from a Project supporter, and another IBM letter.
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Glenn Booker responds on March 1, 2000 to the latest letter below:
Well, I was still hoping for some positive support from IBM, but I guess I'd have to be insane to hope for that now.
This project has collected over two dozen people from at least ten countries who are interested in seeing it happen. We realize that the technical challenges are daunting. This project is an opportunity to learn about a different computer universe, and see what we can do with it. Yes, most people on the project are ill equipped to take on such an ambitious effort. So what? Even if we never "succeed," we will have learned a lot and, perhaps, set the stage for someone else's project.
Performance is not the issue. Heck, if we wanted a high performance machine, we could buy an old Pentium for $100 (US) and it would probably run circles around a CISC AS/400.
It's a bit confusing, however. Since we obviously aren't all Linuses here, and the hardware is so challenging to unravel, why is IBM wasting their time on us at all? After all, this project could lead to continued use and upgrading of otherwise useless machines. So isn't it in their best interest to support our efforts?
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Letter on March 1, 2000 from Dennis Towne of IBM:
[this letter was intended as a response to Clay Atkins' remarks on the page
http://users.snip.net/~gbooker/AS400/ibm.htm.
I am sending this to you as
it seems to be the most appropriate thing to do at this time. Please
forward this to him if possible; you may also make it available on
the web page
or mailing list if you wish. I hope it clears up some questions.
-dT]
Hello, my name is Dennis Towne; I also work for IBM, just down the hall
from Mr. Corrigan. While I was not originally involved in this discussion,
I
have an interest in this issue and felt I could be of some assistance here.
First of all - and note that I do not intend this to be rude, merely a
statement of fact - you have a major misconception about what 'Linux on the
AS/400' means. In fact there are two reasonable meanings, of which
Corrigan
is using one and you are using both. The two separate meanings are quite
different, and it is not sane to combine them.
--------------------------------------------
The first meaning of the phrase is the most common, and involves a port of
the
actual linux kernel to run on AS/400 hardware. This the issue that
Corrigan
discussed in his reply, and his response is quite accurate. Think of it
this
way: The CISC hardware of old AS/400 machines was custom built for OS/400
-
and it decidely lacks features needed to make linux run. It is almost
exactly
like trying to put linux on an 8086: the 8086 lacks process separation,
hardware protection modes and many other hardware features that are simply
required for linux to behave properly.
(Note that there is indeed a linux version working on the 8086 - however it
does not have true pre-emptive multitasking or proper memory management
for
the reasons listed above. Also note that without these things, any
misbehaved application can take the machine down. Needless to say
there
is
very little incentive to continue with the linux-86 project, and as far as
I know it remains nothing more than an intellectual curiosity as a
result.)
As I mentioned above, this is what Corrigan is discussing. Specifically,
from his response:
CISC AS/400 hardware supports only a single address space.
This is a showstopper for porting linux or any unix
derivative directly
to
the hardware. All modern unix derivatives, including
windows NT,
require
separate page tables and separate process address spaces.
This could
possibly be emulated, but without hardware support it would
be
impossible
to have proper memory or address space protection.
CISC AS/400 hardware has no concept of user/kernel mode.
This is also a showstopper for porting linux or any unix
derivative
directly to the hardware. All modern unix derivatives
require that
only
the kernel have access to certain devices, commands,
instructions, etc.
On CISC hardware, any process can execute any instruction
from any
state
without kernel support. Yes, any user process can
memcpy() right over
the top of the entire kernel, and there is absolutely nothing
you can
do
about it. There is no hardware support for restricting
this action.
I'm sure the question is now rising in your mind about how hardware like
this
can possibly do the job you see it doing in OS/400 - the answer is that the
structure and design of OS/400 is catastrophically different from any
modern
unix. The memory protection mechanisms are so different as to be beyond
incompatible.
Not that this kind of port is impossible - however:
It will take hundreds of man years, by extremely competent kernel
programmers.
It will be slow as hell, due to the lack of hardware support.
It will be insecure as hell, due to lack of hardware support.
It will be unstable as hell, due to lack of hardware support.
And finally, since the cisc hardware is the equivalent of a 386 or so, it
will
be so slow as to be almost useless anyway.
--------------------------------------------
The second meaning of the phrase is quite different, and is more accurately
expressed as 'porting the linux user environment to the AS/400 MI
interface'.
For the most part, this seems to be what you mean in your response.
This is an interesting idea, but on CISC hardware, it is probably even more
doomed than the idea Corrigan was addressing. The principal reason is that
many of the kernel interfaces in unix-like operating systems simply do not
have an equivalent MI instruction. For some interfaces, it isn't even
possible for VLIC to do the function requested. An excellent example is
the fork() system call, which has no equivalent on the AS/400 and cannot be
implemented due to the lack of separate address spaces.
You also brought up another issue with this quote:
"It will take some work. Especially when you start dealing with
interface
cards, like comm cards and device cards. But that just takes
manpower."
This begins to delve into porting the actual kernel to control and run
these
devices, which is quite different from simply porting the unix environment
to the platform. Also, be aware that the way OS/400 handles these devices
is very different from unix. The MI -will- lack some critical interfaces
that
you will need for unix applications - and there is absolutely nothing you
can
do about it.
Additionally, since the compiler on OS/400 is trusted, it will be
impossible
to port GCC to the platform. Without gcc, it will be very difficult to
port
unix apps to the MI. If you manage to hack around VLIC and make gcc the
compiler, you will forfeit all system stability due to the lack of
protected
address spaces.
--------------------------------------------
In short, I'm not really sure which of these options is the worst. Both
are
pretty terrible. Neither is likely to be of any value - the kernel due to
lack of hardware support, the environment due to lack of MI support. You
are welcome to try, but I think you would be much better off trying to port
the environment into a PASE-like address space on the newer RISC hardware.
Even porting the kernel onto newer machines would be better, because the
RISC
hardware supports the things linux and unix need to run. As an added
bonus,
a lot of the port has already been done for you by the Linux-PPC project.
Side notes:
--------------------------------------------
The GNU kernel is the HURD, not herd. The HURD also will rely on hardware
support that does not exist in the CISC processors.
As I'm sure you know, a 'recompile' on an as/400 need not be user
initiated,
and is transparent unlike a recompile in the Linux world. An MI recompile
is more like having a JIT compiler than rebuilding from source.
We understand open source. Our lawyers do not (yet).
As a final note - your impression of Corrigan is, in my eyes, incorrect. He
is not some suit-and-tie engineer that sits in front of a green screen all
day. He knows very well what linux is, where it came from, and even how
some
of it works. He is far more familiar with this porting issue than you seem
to be, and has not tried to mislead you in any way. The concerns in his
rebuttal are valid, and you would do well to investigate them if you do not
understand them.
-dennis towne
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
The following letter was received from IBM's Mike Corrigan, who works on the AS/400. We need to identify how we can overcome the physical limitations of the AS/400 architecture to produce a usable Linux port. Please respond to the mailing list with your ideas.
A request was sent to Mike to ask for further information, in accordance with his generous offer.
------------------------------------------------------------------------------------------------------------------------------------------
Glenn,
Daniel Frye asked me to contact you about your project to port Linux to the CISC
AS/400 hardware. I've read what you have on your website, but it is clear
that
you would need much more information to even start. In your note you asked
for
information regarding the low level machine instructions. I believe that
is
still considered proprietary, although I would have to check. I believe
you
would need to know much more than just the machine instruction set (known as
IMPI). The I/O attachments are all proprietary as is the code in the
service
processor (which initializes the main processor and loads its microcode).
There are some aspects of the CISC AS/400's that you should know about that
might affect your plans.
First, the CISC AS/400 hardware supports only a single address space. By
this I
mean that there is a single, system wide, page table. All pages that are
addressable by any process are always addressable by ALL processes. The
hardware doesn't support switching page tables after the system is booted.
This
doesn't fit very well with the unix (and Linux) model of a separate address
space per process. A function like 'fork' is basically impossible as there
is
no way to make a given virtual address access different pages for different
processes.
Second, there is essentially no storage protection in the CISC AS/400 hardware.
There is no mode which controls whether addresses are translated or not.
High
order bits within an address determine whether the hardware interprets an
address as virtual or real. Thus, all real storage is accessible to all
programs at all times. There is no notion of privileged vs. non-privileged
states. All instructions (including those which could easily crash or hang
the
system) are executable by all programs.
The AS/400 VLIC (which is written in the low level (S/370 like) instruction set
implements a higher level "virtual" machine. This virtual
machine is what the
rest of the operating system (OS/400) and applications are written to.
Programs
written to this high level virtual machine (known as MI programs) are processed
via a combination of direct translation and interpretation. It is through
the
trusted translation and interpretation that the system's security is maintained.
This high level virtual machine is even worse suited to Linux than the
underlying hardware.
I hate to discourage efforts like this, but I think you would find this a
monumental task and that the hardware is quite ill suited to Linux.
Let me know if you want to pursue trying to get documentation on the instruction
set and I'll see what I can do.
Mike Corrigan
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Mike clarified his letter a couple of days later:
"I will check further into getting more information released, but I
suspect that
we won't be able to do that.
The issues I brought up with the CISC AS/400 architecture (we call that IMPI)
are more than just security issues. The single address space precludes
functional things like fork as well. I don't believe that you could
ultimately
work around this.
A lot of the I/O attachment architecture (and hardware) is the same between the
CISC and RISC AS/400's. I think it extremely unlikely that we could
release
information on that.
Mike Corrigan
Distinguished Engineer
Server Group; Rochester, MN"
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
A reply from Clay Atkins:
I read the letter from IBM, but the guy is being a little intellectually
dishonest.
I'm not an IBMer, but I have written a considerable amount of MI. I even
have, in some box, a copy of the MI programmer reference manual! Try to
get one of those. My biggest MI project was to port LZW to the AS/400.
It ran quite well.
I think the guy from IBM doesn't understand what Linux really is. As you
know, Linux is the kernel, while the operating system, the Posix stuff, is
GNU. GNU is developing a new kernel, the Herd, that will be available as
a replacement for the Linux kernel.
The VLIC, Vertical License Code, is the "kernel" for the AS/400.
OS/400,
the "current" operating system is like GNU unix. Not that OS/400
is unix
by a long shot, but can you see the correlation.
Where the Linux kernel provides say, ioctl(), the VLIC provides an MI
instruction to fetch an object descriptor (don't get too excited, it's not
really an OO object).
What makes the AS/400 REALLY cool, is that unix on the AS/400, written to
the MI layer, will be binary compatible with all AS/400s! Well, you'll
have to recompile for RISC boxes, but still, hey, who in the Linux world
isn't use to a little compiling?
Another great feature about the AS/400 is the hardware stability. The
stuff just runs. Period.
He is right about one thing. It will take some work. Especially when
you
start dealing with interface cards, like comm cards and tape device cards.
But that just takes manpower. Perhaps the one hurdle that you might find
near impossible to overcome is the lack of information. IBM will be very
very very very hesitant to give you any information about the AS/400.
It's a major part of the midrange income, and I don't think their going to
want you to monkey around. They kinda' have that Apple computer mindset
when it comes to open systems and open source.
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::