Question about reading and writing data with sd card

[kof888]

HI, megavolt85 swat, some time ago I passed this post

https://segaxtreme.net/threads/sdloader-...ves.25275/


Learn about using SD cards on the sega Saturn（Thanks to Murzik for writing）

Now I feel that I have almost mastered it,

so I started to play with the SD card of DC again.

After learning the source code of DreamShell, I finally finished writing a small program, which can be executed directly from the application file manager, or can also be made into a cdi format run

The function of this program is very simple, it will detect the SD card, if the detection is passed, it will generate the bios (DC_BIOS.BIN) and flash (DC_FLASH.BIN) files of your dreamcast host on the root directory of the SD card.

Because I wrote the assembly directly, the file is not big, 5kb after decompression
In order to reduce the complexity of the program, I do not consider cards smaller than 4G, so this program only supports SD TF cards greater than or equal to 4G

（Thanks to megavolt85 swat Murzik, and everyone who contributed to DreamShell）




I encountered some problems in the process of writing the program, so I would like to ask for advice

1 If you use the SD card on the DC, you will use the spi mode like Saturn, right?
If yes, by looking at the data, the CRC check digit of the SPI mode command (command) is not needed
(Only need 0x95 of CMD0 and 0x87 of CMD8, others can be directly replaced by 1)


So the first question is, if the sd card is used on the dc, does the CRC check digit of the command (command) have to be calculated and filled in?
(I found that the source code of DreamShell has been calculated and used, but in the program I wrote myself, I have not used it, but it is normal after testing)



2 In my program, I don't use the cache, but the source code of DreamShell uses the pref instruction to transfer data to the cache when writing data to the SD card. Does using the cache improve the writing speed? (Because I tested it myself, using the pref command did not increase the speed, but the speed decreased. Is the way I used it wrong?

Код:

sub_cache:
        add        r4, r5             r4 = memory address (32-byte alignment)
        mov        #0xe0, r0          r5 = 0x200
        and        r0, r4
loc_cache_1:
        pref       @r4
        cmp/hs     r4, r5
        bt/s       loc_cache_1
        add        #0x20, r4
        rts
        nop

3 In the source code of DreamShell, 5 nop idle cycles are added to most instructions of the sd card. Is it to consider the compatibility of some SD cards?
(In my sd cards and tf cards, I have tested that if the 5 nop idle cycles are removed, there will be no error. Maybe I missed something?



4 The last question, why can't I go back to the application file manager after executing my program from the application file manager


There are a lot of questions, thank you for your reply~


please forget my bad language, i don't know english, i use google translate


This program has passed the test in the following tf sd card









This program failed the test in the following tf sd cards
The reason why the 2 TF cards failed the test was that they were smaller than 4G

But this 16G Toshiba sd card failed the test, which is more interesting, because even if I copy the ds folder in, start the DreamShell image with gdemu after booting, it still prompts that the SD card is not found

but this card directly uses GDEMU to play games. Normal, I guess this kind of card does not support spi mode?

[SWAT]

1. SCIF in SH4 doesn't support hardware SPI mode, this support only SCI (second serial), but this port of SH4 is muted on the motherboard and is not output outside as SCIF. So on SCIF it's works as bit-bang SPI (software emulated). Using CRC is not necessary. In DS core it is used, in iso loader it's disabled to save memory and speed up.

2. Depends on the data used, if it is already in the cache, then this makes no sense and will take extra time. It is not necessary to use.

3. Depends on the SD card class I think, this is a delay for bit-bang, because it is emulation of SPI not a hardware. But usually this delay is generated by the bit-bang itself and it's enough for most cases.

4. Because DS is shutting down if you execute another binary. And you rewrite some memory with DS core. So it's not possible to return. You need write a command or module for DS if you want save DS core and execute own programs.


And yes, not all SD cards support SPI mode. Also small cards are very slow, maybe need more delays for SPI emulation.

[kof888]

Thanks for the quick response.
I also prefer sd card over tf card.
Because I have tested using the spi mode of the SD card to run commercial games on Saturn, for some games that do not require high reading speed, the result is still ok

So I want to try to modify and optimize the assembly instructions to read how fast it can be read on DC's commercial games (most of the functions have been slightly modified)

I would like to ask, for the iso loader using sd card, which system functions are replaced by the iso loader? Can you briefly write the process of the iso loader?
I want to test the reading speed of a certain dc commercial game, do you know of a more suitable game? (It is very intuitive to see whether it is stuck in the game, not CG)
I remember watching the speed comparison video of using sd card hdd and gdrom to read the game doa2, but I can’t seem to find it.

[SWAT]

(30.12.2022 10:23)kof888 писал(а):  So I want to try to modify and optimize the assembly instructions to read how fast it can be read on DC's commercial games (most of the functions have been slightly modified)

Manual assembler doesn't help you increase speed a lot.
The main problem is the long reaction of the port pins in the software emulation of SPI itself, it takes CPU time and can't be better than hardware interfaces in general. Don't waste too much time on this. Just a little unroll loops, do not move pointers every byte, write to RAM at least 32 bit and that's all.
Modern versions of GCC are good at doing optimizations on their own, as they manage superscalar CPU well (arranging instructions in a certain way to execute them in parallel). Manually it's tricky and confusing, I've tried this here. You can't do it much better, because you need to keep a table with instructions in your mind that can be executed by SH4 in parallel and arrange them in the right order. It is not enough just to write in assembly language to be effective.

(30.12.2022 10:23)kof888 писал(а):  I would like to ask, for the iso loader using sd card, which system functions are replaced by the iso loader? Can you briefly write the process of the iso loader?

It's not easy and it has been developed over the years to get good results.
All functions here that name start from "gdc" is a BIOS system calls replacements for GD-ROM:

https://github.com/DC-SWAT/DreamShell/bl...lls.c?ts=4

(30.12.2022 10:23)kof888 писал(а):  I want to test the reading speed of a certain dc commercial game, do you know of a more suitable game? (It is very intuitive to see whether it is stuck in the game, not CG)
I remember watching the speed comparison video of using sd card hdd and gdrom to read the game doa2, but I can’t seem to find it.

In my videos you can find SD and HDD tests for the same game:
https://www.youtube.com/@SWAT-DC/videos

But you will never reach the speed of even the drive, and even more so HDD.

[kof888]

(30.12.2022 13:56)SWAT писал(а):  Manual assembler doesn't help you increase speed a lot.
The main problem is the long reaction of the port pins in the software emulation of SPI itself, it takes CPU time and can't be better than hardware interfaces in general. Don't waste too much time on this. Just a little unroll loops, do not move pointers every byte, write to RAM at least 32 bit and that's all.
Modern versions of GCC are good at doing optimizations on their own, as they manage superscalar CPU well (arranging instructions in a certain way to execute them in parallel). Manually it's tricky and confusing, I've tried this here. You can't do it much better, because you need to keep a table with instructions in your mind that can be executed by SH4 in parallel and arrange them in the right order. It is not enough just to write in assembly language to be effective.

Yes, I have checked the software manual of Renesas sh4-A, and Chapter 4 is about pipelining (page 36-55)
It details which instructions can be executed in parallel.
So, writing assembly by hand paying careful attention to every instruction can be confusing and frustrating indeed.
Hahaha, these are not the key points, the point is that the single-line transmission is indeed too slow

(30.12.2022 13:56)SWAT писал(а):  It's not easy and it has been developed over the years to get good results.
All functions here that name start from "gdc" is a BIOS system calls replacements for GD-ROM:

https://github.com/DC-SWAT/DreamShell/bl...lls.c?ts=4

It seems very complicated, but if I only write to the address 8c004000, I should be able to apply this template directly, and it should be enough to write the memory here, thank you very much

(30.12.2022 13:56)SWAT писал(а):  In my videos you can find SD and HDD tests for the same game:
https://www.youtube.com/@SWAT-DC/videos

But you will never reach the speed of even the drive, and even more so HDD.

Of course, I know that the spi mode is the slowest, but I just want to compare it, how much difference can it make with CD-ROM reading cdr? By the way, is it 1-6x faster to read cdr?

Thank you very much for your reply

[SWAT]

I also forget to say about some asm for gdc syscalls: https://github.com/DC-SWAT/DreamShell/bl...all.s?ts=4

Soft SPI on DC approximately equal to 4x drive speed: 600-700 KB/sec.

[kof888]

(31.12.2022 18:19)SWAT писал(а):  I also forget to say about some asm for gdc syscalls: https://github.com/DC-SWAT/DreamShell/bl...all.s?ts=4

Soft SPI on DC approximately equal to 4x drive speed: 600-700 KB/sec.

thank you~

600-700 KB/sec. is not too slow

I forgot to ask, if the program is directly executed from the file manager, where is the program loaded into the memory? How is it implemented?
I think, if I am careful, the file manager should not be damaged. After executing my program, can rts return directly?

Also, happy new year~

[SWAT]

(01.01.2023 04:38)kof888 писал(а):  600-700 KB/sec. is not too slow

Yes but it's PIO and spends a lot of CPU time. Most games uses DMA transfer that not used CPU in general. And this is main problem.

(01.01.2023 04:38)kof888 писал(а):  I forgot to ask, if the program is directly executed from the file manager, where is the program loaded into the memory? How is it implemented?
I think, if I am careful, the file manager should not be damaged. After executing my program, can rts return directly?

Also, happy new year~

This executed at default address - 8c010000.
In any case DS shutting down before executing binary because binary loaded to default address where DS core executed too. Compile your program as command or module for DS and you can use it without shutting down DS.
Happy new year!

[kof888]

Thank you for your reply~

At present, I am trying to re-inject the new code into the main program of SEGA TETRIS by learning the sd.bin file

When the game is ready to try to read the file data, let it receive data directly from the SD card

After the game reads N files, the result is successful at first, but when the screen displays 3 logo pictures

When the game program is preparing to read the Nth file, there will be a problem

The reason for the problem is that when the file is ready to be read, the cmd read command 0x51 or 0x52 is sent to the sd card, but the return value of the command will become 0xff, which is very strange, because it is normal to read N files at the beginning, but An error occurs when preparing to read N+1 files. Is this because some registers are modified by the game program? ?

BTW, I have the same problem when running this game on DS

where is the latest SD card compatibility list?

I want to know if this game can run normally on sd?


edit:
I found the specific address. In the following function, the following registers will be initialized.

FFE80000

FFE80004

FFE80008

FFE80018

Код:

loc_8C087876:             ! r4 =  0x1C200
        add    #-0x4, r15
        mov.l .value500, r1!  FFE80000
        mov    #0x0, r6
        mov.l .value501, r2!  FFE80018
        mov    r4, r0        
        mov.l .value502, r7!  FFE80008
        mov    #0x6, r3
        mov.l .value503, r5!  FFE80004
        mov.w    r6, @r7
        mov.w    r3, @r2
        mov.w    r6, @r1
        mov.l .value504, r1!  9600
        cmp/eq    r1, r0
        bt    loc_8C0878A8
        mov.l .value505, r1!  1C200
        cmp/eq    r1, r0
        bt    loc_8C0878AE
        mov.l .value506, r1!  38400
        cmp/eq    r1, r0
        bt    loc_8C0878B2
        mov.l .value507, r1!  70800
        cmp/eq    r1, r0
        bt    loc_8C0878B8
        bra    loc_8C0878BE
        nop
! ---------------------------------------------------------------------------

loc_8C0878A8:
        mov    #0x28, r3
        bra    loc_8C0878C0
        mov.b    r3, @r5
! ---------------------------------------------------------------------------

loc_8C0878AE:
        bra    loc_8C0878BA
        mov    #0xD, r0
! ---------------------------------------------------------------------------

loc_8C0878B2:
        mov    #0x6, r1
        bra    loc_8C0878C0
        mov.b    r1, @r5
! ---------------------------------------------------------------------------

loc_8C0878B8:
        mov    #0x2, r0

loc_8C0878BA:
        bra    loc_8C0878C0
        mov.b    r0, @r5
! ---------------------------------------------------------------------------

loc_8C0878BE:
        mov.b    r4, @r5

loc_8C0878C0:
        mov.l .value508, r4!  186A0
        bra    loc_8C0878CC
        mov.l    r6, @r15    ! 00
! ---------------------------------------------------------------------------

loc_8C0878C6:
        mov.l    @r15, r2
        add    #0x1, r2
        mov.l    r2, @r15

loc_8C0878CC:                !
        mov.l    @r15, r3
        cmp/ge    r4, r3
        bf    loc_8C0878C6
        mov.l .value501, r1!  FFE80018
        mov    #0x30, r3
        mov    #0x0, r0
        mov.w    r6, @r1
        mov.w    r3, @r7        ! FFE80008
        rts
        add    #0x4, r15

I will upload the program and image file I made later

Of course, we can also modify the image file ourselves

Search for the hexadecimal value of the image file:
FC7F47D1 00E645D24360

You can use the sd card to play the game on the ds

0b000900 00E645D24360

The link below is the iso file I made, download the compressed package in the link, and then decompress it to the root directory of the SD card (a bin file and an iso file).

The image is not big, only 21M after decompression

You must use an SD card greater than or equal to 4G




But you can't start the iso directly in isoldr, because I have made a lot of changes to all the files of this game, and made some optimizations for the sd card mode, so I can only run the TEST.bin file in the file manager of dreamshell

Of course, you can also use the above method to modify your own image file, so that you can directly start your own image file in isoldr



https://mega.nz/file/UGpD3YrC#iXaO4qiGHB...kcV3_107ws

test video：
https://www.bilibili.com/video/BV1xD4y1V7aP/


Thanks swat for guiding me~~~~

[kof888]

After testing, direct nop instruction has the same effect

mov.w r3, @r7 ! FFE80008 = 30

Then, we only need to re-initialize the register FFE80008 to 0 before issuing the cmd command to read the SD card data.

In this way, there is no need to modify the image. This method is effective in testing and should improve the tolerance of the SD card to other games.

[SWAT]

All games re-init SCIF and broke SPI setup. You should re-init it for SPI after game prepare hardware.

[kof888]

(06.01.2023 13:02)SWAT писал(а):  All games re-init SCIF and broke SPI setup. You should re-init it for SPI after game prepare hardware.

Yes, it looks like this, but this game is special, this game reinitializes SCIF after reading N files

Maybe I should test a few more games

[kof888]

I recorded a video of KOF98 using gdemu+SERIAL to transmit data with 32M large-capacity memory

Music doesn't stop playing after each round

GDEMU+SERIAL_SD+32M_RAM

video：https://www.bilibili.com/video/BV1rG4y1S7iY/

[SWAT]

Cool! I also want 32MB for ISO Loader debugging )

[kof888]

32M RAM can do many things. I just use it for caching