Subject: Re: [netatalk-admins] AS IP speeds
From: Mark Donnelly (mark@coe.missouri.edu)
Date: Thu Jan 07 1999 - 13:20:56 EST
On Wed, 6 Jan 1999, Paul Krohn is reputed to have written:
> I can think of a few things that might sway the results in favor of the
> AppleShare IP 6 server.
>
> 1) identical file system - needn't do any fancy cr/lf stuff and so forth
Most likely the case - more on this in a moment.
> 2) powerpc/g3 at 300 MHz is most likely substantially faster that a
> PII-333 - are both fast enough to saturate a 100 Mbit wire? i dunno - i
> have no PII's and no 100 Mbit wire.
Neither do I, but I know that a FreeBSD box (wcarchive.cdrom.com)
completely saturates a 100MB wire, and it's a PentiumPro (probably a
slower MHz than the G3). I don't think it's the processor speed,
especially if that's a 333 processor on a 100 MHz bus.
I'd say that more likely than the processor speed is the drive speed.
What type of drive is it? What's its average access time? Do you run
disk utilities (defrag especially) on it? My guess is that your Lintel
box has an IDE disk, whereas the ASIP6 box is probably more of a server
and has faster SCSI disks.
Adrian: Have you looked into the mechanisms used by FreeBSD to achieve
this? I hear that the guy at wcarchive implemented a SendFile() call to
grab files directly out of VM and send them on the wire. I also hear that
this either already exists or is in the works for Linux as well. Would
these be of any use to ASIP? I don't know the file sharing protocol at
all really, but if you ever need to send entire files at a time, this
could be really useful.
> all the parts separately. we have adrian - bless his heart. thinking of
> adopting the pizzaware philosophy, adrian?
Huzzah! The Windoze connectivity people (SAMBA) do it - why not the Mac
connectivity people? Adrian & Wes: name a place! I'll be happy to send a
bit of a donation.
> 4) i've always noticed a huge drop in file copy performance on mac when
> copying a large number of small files (documented below). it makes a big
> difference what you copy. ever accidentally copied someone's netscape
> cache directory over the network? yikes. and fonts seem to especially
> take forever.
Ah, here's where the real culprit shows its face, I suspect. The
difference is that UNIX filesystems don't really support Macintosh files.
They have no concept of 'forks' or 'resources.' So, an artificial
structure had to be put in place to simulate the resource fork. So,
there's more done automagically by MacOS than UNIX. From what I
understand, this might be what the corresponding processing would look
like for MacOS and netatalk servers (and please forgive me, I've only
lightly touched Mac programming, and that was 3 years ago):
MacOS netatalk
=-=-= =-=-=-=-=
1) Get a directory listing. 1) Get a directory listing
2) Check for .AppleDouble directory
3) Check for the corresponding
.AppleDouble file
4) Open it
5) Parse it out for file type/creator
5.5) Close .AppleDouble file (*)
2) Open Data Fork for file 1 6) Open file 1
3) Read Data Fork for file 1 7) Read file 1
4) Send the file 8) Send the file
5) Open Res. Fork for file 1 8.5.1) Check for .AppleDouble dir (*)
8.5.2) Check for .AppleDouble file (*)
8.5.3) Open .AppleDouble file (*)
8.5.4) Parse out all the resources (*)
6) Send each resource 9) Send each resource
7) Close Data/Resource fork 10) Close file/.AppleDouble file
[ Note: Items marked with an (*) might or might not happen, depending on
how intelligent Wes/Adrian made their code. If they generalized
it, then a directory read will always close the .AppleDouble file
when it's through; if they made cases for copying files, then it
won't.]
[ Note2: This also assumes that the network has already been taken care
of, and that the IP stacks are of about the same caliber, which
they're not - things like FreeBSD are almost certainly better.]
So, each file copy out of a netatalk volume likely results in twice as
many disk accesses as on a MacOS volume, and that doesn't even take into
account any optimizations Apple might have thrown into HFS (like having
type/creator inherent in the directory structure, or perhaps having
data/resource forks contiguous so that one disk read gets both of them,
etc, etc). Yes, things like parsing the resources do have to happen in
both cases, but since Apple controls the OS, that's likely highly
optimized.
That doesn't even take into account things like directory ID's and unique
file ID's.
The end result is that the UNIX filesystems can't handle Macintosh files
without some juggling, which is always going to result in loss of speed.
That said, netatalk seems to handle things a lot better than WinNT server
does! Let's hope that Adrian's DID improvements help in speed some as
well.
Thanks Wes & Adrian!
--Mark Donnelly
This archive was generated by hypermail 2b28 : Sat Dec 18 1999 - 16:16:08 EST