Memory Layout

[sundance2]

Reproducing below a post by @swat
which is a treasure trove of information when it comes to memory addressing on Dreamcast.

I have a few follow up questions, but I presume many users will be uninterested by these technical details,
so it felt more appropriate in the "programmer" section.

(27.11.2020 07:42)SWAT писал(а):  Loader memory usage:
params (1KB) + loader code (nKB) + buffer (32KB - params - loader). Max 32KB.

In best case, for buffer need 2KB (one sector). This buffer uses only for non-optimized images and little for CDDA (track name). So the loader should be less than 29KB.
In future this buffer I will made as *dynamic* too, but it's working not in all games yet.

Also loader uses additional memory (up to 32KB) for CDDA and/or CISO buffers.
For CDDA now supported dynamic, for CISO still not, it uses only hardcoded addresses.
And CDDA can use hardcoded addresses if dynamic disabled (not supported).
You can see here:
https://github.com/DC-SWAT/DreamShell/bl...?ts=4#L220

Application (KATANA game) memory usage:
Params: 0x8c000000 - 0x8c000100 (also here vectors for syscalls)
Syscalls: 0x8c000100 - 0x8c004000
Stack: 0x8c004000 - 0x8c00f400 (in some cases at 0x8d000000)
VBR: 0x8c00f400 - 0x8c010000 (in some cases in heap)
App binary: 0x8c010000 - ???
Heap: ??? - 0x8d000000

So we can place loader (and buffers) in the heap (carefully with stack if it's at the end) or at 0x8c000100 - 0x8c004000 because loader can emulate all syscalls.
Best choice it's a 0x8c004000, we don't touch original syscalls (just change some vectors) and don't conflict with game, because stack here is usually does not grow much and VBR code block is 3KB, so we should have ~ 32KB for free.
But some games can write in this memory area some "loaders" or stack grow so much and do damage loader code. So we should move loader at high memory (heap) and hopes we are lucky

First, thanks @swat for these details, it clarified so much.

Now, "of course", as a programmer understanding more generally means having new questions.
Many are probably well known by dreamcast programmers, and may seem basic to you, sorry in advance, they are just not obvious from a general programmer perspective with no specific knowledge of Dreamcast platform.

Цитата:b]Application (KATANA game) memory usage:[/b]
Params: 0x8c000000 - 0x8c000100 (also here vectors for syscalls)
(...)
Heap: ??? - 0x8d000000

These few lines are packed with information, much of which seems implied. So here are some questions:

- KATANA game : I presume it means games that use the KATANA game development kit from Sega. It's probably a majority of games, but it also means that games using a different toolkit and set of conventions can work quite differently when it comes to memory layout. For example, maybe Window CE games. And maybe also more modern production using KallistiOS ? Anyway, this merely helps to understand that this is not a "universal" design.
- Params: 0x8c000000 - 0x8c000100 : nit-picking here : this seems to represent 256 bytes ( < 1 KB). Still, likely large enough for parameters (I have no idea how much space "vectors for syscalls" is using, but that's probably irrelevant anyway)
- The entire memory design seems to imply an MMU-less scheme, where memory addresses are directly mapped to the physical RAM
- It seems to imply that 0x8c000000 is "address zero" ?
- The Dreamcast (normal) features 16 MB of RAM, or 0x1000000 in hexadecimal. In a direct addressing scheme, it seems it would go from 0x8c000000 to 0x8d000000. Yet, later in the list, 0x8d000000 is mentioned as a start (?) address for heap (and maybe sometimes for stack). This seems contradictory, unless heap allocation goes _down_ from 0x8d000000.

Цитата:Stack: 0x8c004000 - 0x8c00f400

This seems to represent a memory space of 64KB - 16 KB (syscalls) - 3 KB (VBR) = 45 KB .

What initially was unclear to me was "a stack space for _who_?", and I guess it means stack space for the KATANA program.

Цитата:So we can place loader (and buffers) (...)
Best choice it's a 0x8c004000, (...) because stack here is usually does not grow much and VBR code block is 3KB, so we should have ~ 32KB for free.

So the idea here is to "steal away" memory space from the stack, on the ground that it's not entirely used in most cases.
It implies that stack space growth _downward_, from 0x8c00f400 towards 0x8c004000.
I presume this is generally true : in such an environment, programmers should not "toy" with stack space limits, and keep the stack for very basic structures and variables, thus pushing any table or large structure into heap space instead. In the end, they may only need a handful for KB.
But by the very nature of this bet, it's a bit "random", meaning that some games use more stack than others, so some games will overwrite the loader during gameplay. So now it becomes a "by game" setting.

I guess it explains why 0x8c004000 is "generally fine", except in some cases.

It also explains why the loader should remain small, although I don't see any hard 32 KB limit here. It seems more like a rule of thumb : using 32 KB for the loader only leaves 13 KB for the game itself, so now it becomes more and more probable that the game will overwrite the loader during gameplay.

Is that a correct assessment ?

[kof888]

Let me answer something I know, please point it out if you’re wrong

1 Regarding the stack, for 99% of the KATANA game, after loading the game main program, r15 will be assigned first. This r15 is the stack register, which is usually assigned to 8c00f000 or 8c00f400, and then the memory in the middle of 8c00c000-8c00f400 will be written into sega characters. initialization

2 In the case of 16M memory bios (32Mbios excluded), 8d000000==8c000000,
example
jmp 8d100000 == jmp 8c100000

So 8d000000 is also possible as the stack register address of r15, because the stack is all subtracted
and so
in case
r15 = 8d000000
mov.l r14, @-r15

r15 = 8d000000-4 = 8c000000-4 = 8cfffffc

Finally, 8c004000-8c008000 is not used in general games.
(By the way, 8c008000-8c010000 is the loading location of the ip.bin file) So the location of 8c004000 is generally safe, but only 16kb in size

[sundance2]

> So the location of 8c004000 is generally safe, but only 16kb in size

That's another mystery for me.
The loaders are more in the ~30 KB range, which is way above 16 KB.
That being said, this is their binary size. I would expect that correlate somehow with RAM occupation size, but I don't know how representative that is.

[SWAT]

"KATANA game" is application written on official KATANA SDK. Also exist a part of games on official WinCE SDK and homebrew on KallistiOS.

"0x8d000000" - is HEAP end or STACK start (heap end lower in this case). This is the end of RAM in this area.
For more information about memory map you can see here: http://mc.pp.se/dc/memory.html or look for SH4 Hardware Manual PDF.

"Params: 0x8c000000 - 0x8c000100" is some struct with states and data for bios syscalls. Applications get from this area vectors for syscalls functions.

"stack at 0x8c00f400" is used only for syscalls, startup initializations and kernel thread I guess. For other threads used another stack address.

Also about MMU:
KATANA doesn't use MMU as it should be, but some games used it only for memory protection, no mapping.
WinCE uses MMU fully.
Homebrew doesn't use at all.

(29.11.2020 06:16)sundance2 писал(а):  > So the location of 8c004000 is generally safe, but only 16kb in size

That's another mystery for me.
The loaders are more in the ~30 KB range, which is way above 16 KB.
That being said, this is their binary size. I would expect that correlate somehow with RAM occupation size, but I don't know how representative that is.

We have 16KB only if use IP.BIN at 8c008000 (applications from GD drive starts from IP.BIN by BIOS), but the loader executes 1ST_READ.BIN (main app binary at 0x8c010000) directly by default, so we have more memory

[MilkeyWilkey]

Someone should organize all this in a table with notes

[SWAT]

The KATANA SDK application:

Код:

Syscalls state: 0x8c000000 - 0x8c000100
Syscalls code: 0x8c000100 - 0x8c004000
Reserved: 0x8c004000 - 0x8c008000 (16Kb)
IP.BIN: 0x8c008000 - 0x8c010000 (32Kb)
- Bootstrap 1: 0x8c008300
- Bootstrap 2: 0x8c00b800
- Stack: 0x8c00e400 - 0x8c00f400 (4Kb, but can be more and at 0x8d000000)
- VBR: 0x8c00f400 - 0x8c010000 (3Kb)
Application: 0x8c010000 - 0x8cxxxxxx

The WinCE SDK application:

Код:

Syscalls state: 0x8c000000 - 0x8c000100
Syscalls code: 0x8c000100 - 0x8c004000
Reserved: 0x8c004000 - 0x8c008000 (16Kb)
IP.BIN: 0x8c008000 - 0x8c010000 (32Kb)
- Bootstrap 1: 0x8c008300
- Bootstrap 2: 0x8c00b800
- Unused: 0x8c00e400 - 0x8c010000
Application: 0x8c010000 - 0x8cxxxxxx
Stack: 0x00xxxxxx (MMU mapped)
VBR: 0x8c0120f0 or 0x8c012110 (depends on the version)

The KallistiOS application:

Код:

Syscalls state: 0x8c000000 - 0x8c000100
Syscalls code: 0x8c000100 - 0x8c004000
Reserved: 0x8c004000 - 0x8c008000 (16Kb)
IP.BIN: 0x8c008000 - 0x8c010000 (32Kb)
- Bootstrap 1: 0x8c008300
- Bootstrap 2: 0x8c00b800
- Unused: 0x8c00e400 - 0x8c010000
Application: 0x8c010000 - 0x8cxxxxxx
Stack: 0x8cxxxxxx
VBR: 0x8cxxxxxx

In IP.BIN, stack and VBR sections cleared to zeros, so the applications setup VBR code by self. But maybe this is not the case everywhere.
All applications use multiple stacks, because it's multithreaded applications. In this description I mean *kernel stack*.

BTW, Visual Concept games uses another VBR address with different entry code, so loader with IRQ handling injection (full loader, not "se") doesn't support it yet.
Also some games that uses MMU for protection, can use different VBR. I think it should be the same, just need more reverse-engineering to add support for IRQ handling injection.

Some games like Shenmue 1/2 or Atomiswave ports write some data to 0x8c004000 memory, so we should place the loaders at high memory.