dschmenk
PLASMA 2 CAT is slow - excessive disk activity
This shows how slow the program +CAT is under PLASMA 2. Video playback control (pause and frame tracking) can be used to see which tracks and sectors are involved. There is a LOT of disk activity. It can be seen in AppleWin sector/track indicator and mapped to ProDOS file locations.
Trying to understand which file are being accessed in this disk activity, an issue in the AppleWin project was created -- to make track/sector indicator more consistent, and also to provide logging feature for accessed tracks/sectors over time in the AppleWin log:
Well, you've discovered the downside of using loadable modules. CAT, COPY, DEL, and NEWDIR uses a host of library modules to keep the SYS/ directory small and consistent. The module search process starts with the current directory then looks in the SYS/ directory. On a floppy system, that is particularly slow. With a hard disk, RAMdisk, or SSD, the searching is quite a bit faster and doesn't impact the perceived load time as much. As noted, PLASMA is much better suited to this kind of installed environment over floppies. Have you tried doing the same operations on the old Apple Pascal system? Ugh.
I was thinking about module loading. And it is still looking like too much activity. That's why I wanted to see exact logging of sector and file access.
The analysis below applies to 140k floppies, which could be the media of choice for the more classic Apple II's up to IIc.
It doesn't seem to be an issue on 800k drives. (* - below). Maybe it has do with better ProDOS directory caching on SmartPort or HD?
💡Here's a thought: Why start looking for modules in current directory instead of app directory? Starting in app directory would skip the unnecessary scan in case of the tools already residing in SYS. Also if you run an app from outside the current directory (other than SYS) and it has private modules: why would they reside in current directory and not in the app directory.
In AppleWin this use case is nicely illustrated, when inserting the two 140k disks PLASMA2-SYS.PO and PLASMA2-BLD.PO in the two drives. Then
P /BLDand+CAT. Observe the drive lights flipping between drives 1 and 2 about 7 times - this is for loading modules, when it first scans the current directory. WhereasCATis inSYSand all its dependencies are inSYS, which is also the app directory. So with this alternative module loading rule it would've saved around 1/3 to 2/5 loading time (as shown in timings below).
Other potential workarounds for 140k disks:
- All modules in a single folder, thus avoiding deferring to SYS.
- Static linking (of selective modules), which is contrary to purely modular systems like Java, but might be a viable alternative.
- Rewriting certain modules are source-level includes. Or consolidating smaller modules. This is illustrated by
EDandPLASMbeing larger but loading faster than combinedCATmodules.
Further analyzing 140k: if you slow down the video,
- it seems like it's going over the same sectors more than once (these are probably directory scans)
- in any case, there's too many sectors being accessed
To estimate how many sector reads are needed for CAT and its 6 imports,
- 6 CAT (main app)
- 0 CMD (should be in memory)
- 1 LONGJMP
- 1 ARGS
- 3 MATCHFILES
- 10 FILEIO (but maybe should also be in memory)
- 6 INT32
Total: 27 sectors.
Plus scanning directories:
- 6 SYS
- 2 current
Considering how much time it takes to run on these 140k disks:
- 17 sec PLASMA2-SYS.PO
- 10 sec PLASMA2-MIN.PO (only root files, CAT and its dependencies -- all in root)
(* Interestingly on the 800k
PLASMA2.2mg, the CAT time is 2-3 sec from BLD folder.)
It looks like the directory scans contribute on average 1/2 of their sector scan time per each import:
- performed for each file
- but partial -- until the file is found
Further developing the idea of app directory for relative modules loading, with nested modules imports becoming a new relative directory, as opposed to current directory. Here's some code changes that make it possible. It may need some optimization, but it runs and provides a proof of concept.
-
cmdjit.pla --
CMD128
The cost is 120 bytes in 4,128 bytes CMD128 module.
There are some considerable benefits for 140k disks: about 27% reduced time for +CAT from root in PLASMA2-SYS.PO (from 17 down to 12.5 seconds).
For the mini system all in root PLASMA.MIN in the original post, it's only about 0.5 sec difference. But it's not worse.
For 800k disks the difference is hard to measure, but there is no degradation either.
The functional benefit is the ability to run apps outside current directory, which have local dependencies.
For example, when building and running TEST.PLA and it dependency TESTLIB.PLA keeping them in the SAMPLES directory, on the default distribution PLASMA2.po 800k disk, it just gives "File not found" error:
whereas with the proposed changes in place, it runs as expected:
Thanks for all the work looking into this. I'm not opposed to changing the order of library loads, but my initial thinking was that apps would have many local modules and only a few system modules. You're the first to point out the slow loads. I want to also look at all the prefix setting. That may also be adding to the overhead. Life has gotten busy lately, but I hope to get to this in the next week
my initial thinking was that apps would have many local modules and only a few system modules
It's not so much about local vs system. It's about what local means. It seems currently local means current. Instead we propose to make local mean app directory -- only for module loading, not accessing data files. Such as in case of a compiler, it loads it's modules from the app (compiler's) directory, but (after it loaded its modules) it still maintains the current directory to load the source file from the arguments.
System (path) will also be checked last -- same as now. It's just happens that for many utilities app folder=sys. So loading will be faster, since it doesn't need to scan current folder for modules whatsoever.