Directly programming the Amiga hardware was not a bad practice!

Rivers of words have been pouring out on social media because of the latest article on the bad practices of several developers, to which I will not return (having already been sufficiently covered), but some comments have emerged that are not exactly in favour of direct hardware programming, which is seen by some as something “dirty”, which should not have been done, always having to go through the OS.

While we should all agree that using the OS should always be the first choice when making something (because it is the one that guarantees the greatest compatibility. In addition to also offering various tools that make life much easier for programmers), I cannot, however, share the general diktat of abjuration (and, even worse, contempt) of so-called “bare metal” programming, if only because it had not been publicly deprecated (with indications such as: “it will no longer work in the future, so it would be better not to do this any more”) or even banned.

So while one could rightly say that: “if something is not forbidden, I can do it”, on the other hand, the freedom granted should always be accompanied by common sense in the choices one makes, not necessarily having to go looking for whatever solution the mind can come up with.

On the other hand, there was also no absolute freedom regarding the direct programming of the Amiga’s hardware, as already anticipated and partially discussed in the previous article, even if the stakes were not all set by Commodore from the very beginning (a good part of them were), but some of them arrived “in the works”, as time passed and its publications received corrections, updates and new editions.

This could be considered the main flaw in the parent company’s behaviour, but this does not exempt those who had to develop software for the platform from knowing and respecting these initial rules, and those that came later.

The evolution of the guidelines

Of paramount importance in all this are, precisely, the guidelines, which were supposed to direct programmers in writing correct code that would work anywhere, whatever the configuration of the user’s machine. Unfortunately, and as already mentioned, they did not arrive all at once, and were also scattered in different places.

Starting with the first edition of the Amiga Hardware Reference Manual (from now on called Hardware Manual for simplicity’s sake), in fact, it can be noticed that the first chapter, introducing the machine, lacks any indication about it (which, instead, will come from the second one) and is presented as a guide to know the inner workings of the custom chips and a tutorial on how to program them, for those who needed to get higher performance (compared to what the OS offered), or for those who needed to make new peripherals.

Actually, and as you go on reading, you discover how various rules are scattered throughout the book, from the various chapters to the appendices, and that they are mentioned more or less implicitly where they occur, depending on the particular context. A good number of them I have collected and listed below:

• peripherals and other memory can be added via the external connector (p. 6. Reference is made to the pages of the PDF for ease of access and control, with page 1 of the book in paper format corresponding to page 15 in the PDF).
• Copper‘s MOVE instruction is used to set a value in one of the custom chip registers (p. 23), which are listed in Appendix B (p. 24).
• In the Copper’s MOVE instruction there are unused bits which must be set to zero (p. 23).
• Copper cannot write to the first 8 registers (up to address $10), or to the first 16 (up to $20) depending on whether the Copper Danger bit is set or not (p. 28).
• HAM mode uses 6 bitplanes (p. 92).
• the minimum period for audio channels is 124 (p. 152).
• the data fetch start of the video controller must be a multiple value of 8 (p. 200).
• The table showing the Blitter‘s pipeline for various cases is purely illustrative and Commodore reserves the right to change its operation (p. 204).
• The example on pages 205 and 206 shows how to programme the Blitter and the waitblit procedure to wait for the completion of the previous operation.
• Bits 3 and 13 of INTENA are reserved for external interrupt sources (p. 221).
• hardware requires a special sequence to start a disk operation (p. 242).
• After receiving a byte from the keyboard, the processor must set the SP line to zero for at least 75us (p. 247).
• Appendix B reports that there are registers used only by the DMA (p. 267).
• Registers are read-only or write-only (p. 268).
• the memory map (p. 276) shows blocks of addresses not to be used or reserved for future use (including $C00000 for the famous Slow memory).
• still the memory map shows that the beginning of the ROM has not yet been defined: $FC0000 is only indicated as a possibility.
• when accessing the two CIA chips as bytes (and therefore not as words = 16 bits = 2 bytes), bit 0 of the address must always be at 0 for the CIAA and at 1 for the CIAB (p. 294).

I omitted all the pages where unused bits in the registers were reported, or which had to be zero, as well as all the DMA channel pointers which must have bit 0 at zero (because word = 2 bytes is always accessed), because otherwise the list would never end.

The only exception here is the registers of the two I/O chips, the aforementioned CIAs: unused bits are ignored in writing and returned as zero in reading (p. 299), so you can basically do whatever you want (although it would always be wise not to go touching them).

However, the implications arising from the various points in the list should be obvious, and so I have omitted them so as not to make the discussion too long, but I give just one example with the first. Which states that the system configuration can change by taking advantage of the expansion connector, which means more memory (Fast, in this case) and/or new peripherals (which also include a new processor). The consequence of this should be that programmers should obviously (!) have taken this into account.

Just to be clear and dispel any doubts, this is a list of constraints that, as such, should be respected when writing code that touches these precise areas of hardware. But they are not the only ones, as we shall see.

The three programming models

Also because this manual is lavish with details on the hardware and indications on how its components should be programmed, but it remains a mystery as to… how to get started! How, in short, to put all this knowledge to use.

In general, with the Amiga there are basically three models for making software for this platform:

• exploiting the API of the OS –> only using the OS;
• directly programming the hardware –> taking out the OS;
• exploiting the API of the OS and directly programming the hardware when necessary –> a model that we could call “hybrid”, because it combines the two “worlds” (we could say the best of both).

The first and third cases are the domain of the OS, so the guidelines for proper development found in the other manuals of the Amiga Technical Reference Series (Amiga ROM Kernel Reference Manual: Exec, Amiga ROM Kernel Reference Manual: Libraries and Devices, Amiga Intuition Reference Manual) must be followed. The third case, however, is even more restricted, because it also requires the Hardware Manual to be followed, for obvious reasons.

The second case should, theoretically, rely exclusively on the latter, but this is not exactly the case, because it also depends on the others, and not only for a question of belonging to the series that collects them all, as we shall see later.

In fact, the main reason is that, while all the information found in the three aforementioned manuals is sufficient to create an application that runs on the OS, there is nothing in the Hardware Manual that tells us how to develop one that cuts out the OS, allowing us to take full and exclusive control of the machine from the floppy disk inserted before booting (or following a reset).

So this manual alone would be of absolutely no use if we wanted to implement something in the second of the three cases. In the third case, on the other hand, it would obviously make a contribution, because the direct programming of the hardware would take place within the ecosystem represented by the OS, and primarily following its rules.

In fact, in the third case, one could prepare a standard Amiga floppy disk, which would immediately load an executable to take full control of the system, and then use its own track loader (code that takes care of reading the tracks from the disks. In the OS it is represented by the trackdisk.device component, but games that take full control of the system have to rewrite it) to read the data.

This stratagem makes it possible to achieve a hybrid approach, in fact, partially exploiting the OS for the start-up and subsequent launch of the actual code. But it involves some complications (e.g. the first disc would have to serve only that purpose, perhaps loading only the game’s introduction and leaving the other discs in their proprietary format) and does not represent the method popular at the time, i.e. floppies of games or demos that started immediately once inserted in the drive.

These, in fact, were called NDOS (Non-DOS. AmigaDOS was the more high-level management part of disks et similia. Just to simplify), and they worked very differently, because their format generally had nothing to do with that of the Amiga’s standard floppy disks. For instance, sometimes only the first track was readable by the OS, just to allow the game code to boot, while all the others could be totally unreadable (perhaps because they used a particular MFM format for protection). It remains, however, a mystery as to how it all worked.

The bootstrap

The doubt is dispelled by reading what is the first manual in importance when it comes to the Amiga: Amiga ROM Kernel Reference Manual: Exec. In fact, scrolling through its pages one finally arrives at the penultimate appendix (the C), which, starting from page 265 (here too, reference is made to the pages of the PDF, with page 1 of the paper version corresponding to page 21 of the PDF) begins to speak explicitly of direct access to the hardware.

In the next one come the first indications for those who want to take full control of the machine at start-up, immediately after the Kickstart, followed by a few pages about the system’s memory map and the list of registers.

The coveted information, on the other hand, is found on pages 273 and 274, which give details of the system’s boot phase, with descriptions of the data structures that must be present in the first two sectors of the first disk track, where the code to be executed must be located, what is available in certain registers, and what to return to Kickstart to complete the boot procedure.

Information is also given on the physical format of the floppy, i.e. how a track is structured and the precise format of the individual bytes that make up its sectors. The last part is dedicated to the C sources of a couple of routines for encoding and decoding data in MFM format.

At this point, everything is really there, but it was necessary to get to the bottom of a manual that, on paper, should have served for something else entirely, and which instead became not only important, but even indispensable and compulsory in order to be able to develop software with the second type, thus programming the hardware directly and cutting out the entire system.

Interestingly, the scrolling through the pages of this manual leads to further guidelines (scattered between pages 10 and 11), which not only serve to write correct code when the software runs with/on the OS, but are also general indications, including, for example, those for processors other than the 68000 (the 68010 and 68020 are mentioned in particular, and the differences that exist with the 68000 regarding the infamous MOVE SR instruction. Which is freely usable in the latter model, but privileged in the other two).

The summary of all this is that the guidelines are there, and they serve to put various stakes in the programming of the machine, whichever way you do it. What emerges, above all, is how the Amiga is a far cry from the concept of unique, practically unchangeable hardware that had been handed down to us by the consoles and, in particular, by the home computers of the previous “8-bit era” that our wonderful platform helped to close (to inaugurate the “16-bit” one), as already mentioned.

With such rules it would, therefore, have been possible to write code with direct and exclusive access to the hardware, and able to run on all Amiga machines, even future ones and/or with different configurations, being able to count on a minimum common denominator well delineated (in the hardware manual), but without compromising the exploitation of the greater resources available (in particular amount of memory, frequencies, and new processor models) if developers had wanted to (and in some cases they did: there are games that benefited from it).

“The origin of all evil”: the self-modifying code

Particular mention must be made, however, of the infamous self-modifying code, which has generated a lot of diatribes, since many consider it “dirty” (for a change!) and even illegal: to be counted among the bad programming practices that should never have been followed, because it could have compromised execution on machines equipped with cache memory.

Evidently, these are people who have never delved into the workings of processors, their architectures and micro-architectures, unaware of the concept of functionality (that of having caches, for instance) and its use or non-use for the purposes of the software’s objectives. It was probably enough for them to learn the use of a few instructions, just enough to throw down a few lines of code, ignoring everything else. But ignorance is never a justification: if anything, it is an aggravation for those who engage in certain arbitrary as well as baseless assertions.

In any case, the situation in this particular case was never clear even at Commodore. Whereas it was, on the other hand, for Motorola, which with the arrival of new models for its glorious 68000 family updated the guidelines for their correct use. Besides the fact that processors equipped with a cache had it disabled at start-up (for compatibility issues). On the other hand, and as already mentioned, it is still a feature, which can be enabled as and when one wishes according to the particular needs of the moment.

In this case, the problem is that the first edition of the Hardware Manual and that of Exec (remember that there are no APIs to manage caches, which will only be introduced in 1990, with version 2.04 of the OS) are completely silent on this precise subject, effectively giving the green light to the use of this practice without any restrictions.

Since it is well known that elements other than the 68000 may be present, the processor on which the code runs would in any case have had to be handled correctly. Here, of course, Motorola’s manuals on how to correctly program its processors are The Source (Commodore can certainly not entirely take over this task!).

The second edition finally touches on this subject and clarifies it (on p. 28 in the PDF and p. 10 in the paper format) in a way that is, to say the least, lapidary: no self-modifying code is to be used. There are no special exceptions or use cases: you just must not use it!

A frankly absurd position, for two reasons. Firstly, because it is the exact opposite of the previous one, which contributes to generating chaos for developers who have decided to use this technique (since it was not previously prohibited).

The second, and equally important, is that it shows how they too were unclear about how processors work (as illustrated at the beginning of this section), claiming that self-modifying code can “vanish” (I guess this is meant as not having an effect) by the caching capacities of processors. This can happen, it is true, but it is by no means certain to occur, as there are ways (note the plural) to handle it properly.

Common sense arrived only a couple of years later (in 1991), with the third and final edition of the Hardware Manual, which discourages the use of this practice (p. 28 of the PFD and p. 13 of the hard copy), but does not forbid it, while providing valuable pointers on how to behave in such cases.

In particular, it dwells on how to flush the caches by exploiting the appropriate API of the OS, and also provides a piece of code in the case of having taken full control of the hardware (thus making it impossible to call the new API), rightly warning against the possibility of it not working on future models of the family (68020, 68030 and 68040 are mentioned. Missing from the list is the 68060, which had not yet been marketed).

A nice step forward that finally puts things in the right place, again leaving freedom of choice (but not absolute: there are still guidelines to be followed, and attention must always be paid to the prefetch logic of processors) to developers on whether or not to use self-modifying code.

In reality, the picture is by no means complete, since there is no mention of the scenario that guarantees absolute compatibility with any processor, even future ones: completely disabling caches!

The manual, in fact, focused exclusively on the one in which the caches are always enabled, and rightly need to be flushed following the generation of the self-modifying code. This is certainly the most important one, and one that one should always aim for if possible, as the caches make a considerable contribution to improving processor performance, but… it must be managed correctly, and this may not be possible.

The biggest problem here is solely due to Motorola, which has been dedicated to changing the specifications of the architectures of its processors, making them incompatible (the successor has always been incompatible with the predecessor due to some modification). Therefore, it is by no means guaranteed that future processors would provide that precise instruction for flushing the caches, so Commodore had to get its hands on that portion of the code it provided.

But the ultimate solution to the problem I have provided above: disabling the caches altogether makes the self-modifying code future-proof (always taking the prefetch into account! But it is little stuff = few bytes to deal with). This is certainly a more computationally onerous solution than flushing them, but at least it is safe.

At this point the ball is once again in the hands of the programmers, who must decide how to exploit this scenario (permanently disabling the caches, or disabling and re-enabling them programmatically), according to their specific project requirements.

In the case of programmatic management, the best solution is to keep the value of the CACR register at start-up, create a version of it with disabled caches (only the one for code, for example, leaving the one for data intact), and load one of these two values into it to enable or disable the caches when necessary, so as to minimise the performance impact (more information on the WHDLoad site).

Conclusions

With the most controversial and contested practice also unravelled, it is time to draw conclusions. In my opinion, the contrast between developers who programmed the hardware directly and those who, instead, relied exclusively on the OS, has no reason to exist: both are choices dictated by the vision of the programmers and, ultimately, by the purpose of the projects they worked on. With the addition of the hybrid scenario represented by the software written for the OS, but which sometimes requires and executes direct access to the hardware, which represents the (delicate) junction bridge between the two completely different worlds.

There is, therefore, no blame to be laid at anyone’s door who chose one of the three models, because it would come down to purely ideological issues, dictated by a vision, if not short-sighted, certainly very limited of what the Amiga was and the needs of the time. In fact, I’d be really curious to know, otherwise, how Fighin’ Spirit and USA Racing (the games I worked on. The latter not released) could have been made using only the OS, just to give a couple of examples dear to me.

The only discriminating factor, partly already mentioned in the previous article, should only be whether or not Commodore’s guidelines were followed so as to make the software robust and resistant to future innovations, both from the system software and the machine & peripheral hardware, and thus able to run on any model in any configuration.

The non-homogeneous as well as scattered documentation did not help, and for this the main responsibility lies with the parent company, which in any case was faced with a new and revolutionary product that changed the way home computing was conceived until then, laying the foundations of the modern systems that we still use profitably today.

But it was there! The documentation was there, and even long before the Amiga Technical Reference series was available to the general public. There are traces of this in the Hardware Manual, where it is clearly stated on page 301 that Appendix G (on self-configuration), which had been published in the initial versions (plural!), had instead been removed in the first edition. It is even better to read this in the Exec manual (also first edition), where it is stated on p. 267 that the autoconfig guidelines were to be published in December 1985.

And it could not have been otherwise, if we take into account the fact that applications and games were already available when the Amiga 1000 was introduced: software houses had to draw on documentation in order to write and publish them!

A final note on the “loyalists” of developing using only the OS: it is understandable and I have already reported that it would be the best solution to follow, if possible, as it is the one that guarantees a greater capacity to cover the entire machine fleet, even in the future. But this does not mean that it should be considered the only acceptable one and that we should crucify and blame those who, by choice or necessity, have decided to follow the more complicated path (from this point of view).

For some, this stance is due to the fact that it is generally not possible, or is very difficult (even if one has the sources), to port software that directly accesses the hardware to “evolutions/rewrites” (AmigaOS4, MorphOS, AROS). Reasonable objection, but irrelevant: those machines are not Amiga, and the software we enjoyed ran on the Commodore machines that could bear that name. The solution, in these cases, remains to rely on the good, old, UAE, and the problem is solved.

For those who, on the other hand, still continue to have a religious view on the matter and are firmly convinced that the only way forward should have been to use the OS and that’s it, they have but… to prove it!