cso support

[SWAT]

(18.03.2020 10:47)sundance писал(а):  I've been looking for *.cso (compressed iso) support on DreamShell.
This capability is mentioned here and there, in a few places, and more importantly, there are traces of this support in many places throughout the code base.

However, every time I tried to load a .cso file, it resulted in a crash.

I was lucky enough to receive a hint from @ChristinaDreamcustDC, stating that .cso support might have been dropped in recent versions of DreamShell.
There is, indeed, one short line on the topic in the middle of RC4 release note, firmware section :

> Removed support for CSO in IDE loader to save memory =(

So that's interesting for a few reasons :

1) I'm not using IDE, but a SD Card reader on serial port. From the spelling, one could believe that it should not be impacted by the change on IDE, since after all, within `DS/firmware/isoldr`, there are distinct `ide.bin` and `sd.bin` files.

Well, the Serial Port config is clearly impacted. To confirm, I downloaded RC3, and attempted again to load the *.cso file. This time it worked.

So something broke support of *.cso files for all storage types in RC4.

CISO will be supported only in 0.6.0, for other 0.6.x it's disabled for all loaders. I can made build with CISO support if you want.

(18.03.2020 10:47)sundance писал(а):  2) "to save memory"

OK, this one is even more interesting. I've been looking at the code in `modules/isofs` pretty closely, and, provided that the *.cso file is using LZO compression, there is nothing that makes it consume more memory than a direct read on *.iso file. LZO decompression only costs a handful of stack variables, nothing significant.

Note that this is fairly different from zlib compression, where there is a state, which costs memory. I want to underline that I'm targeting LZO-compressed *.cso file as my primary application, precisely because it costs less memory and cpu at decompression time.

You are absolutely right, just did not take into account one more nuance. The LZO code itself takes up memory ~2 Kb and it's essential.
Also the current implementation of reading lzo-compressed images in the loader requires additional *dynamic* memory, unfortunately. The implementation that does not use additional memory is disabled there were some problems with it.

(18.03.2020 10:47)sundance писал(а):  3) The code is still there.

As mentioned, I've been looking pretty closely at the code in `isofs`, and a little bit at `isoldr` and `firmware/sd`.
It's pretty clear that the ciso code is still here.
More than that, it's still active, it's not gated, and it ends in the produced binary file, meaning it consumes space, and exposes all expected symbols.

Yet, I've not found where exactly the *.cso support is disabled, if it is.

Presuming I would like to re-enable this support, is there a place to look at ?
Can this support be enabled with a simple build flag at compilation time ?

I'm interested in this topic, and may even provide some patches to improve performance of this code path. That is, if there is a way to test the code changes ....

Of course you can! Just uncomment ENABLE_CISO=1 in Makefile.cfg of isoldr firmware. The code is still there, but it doesn't compile when disabled.
It's should be works for all loaders.

(24.11.2020 20:58)sundance2 писал(а):  

(30.03.2020 12:41)SWAT писал(а):  CSO doesn't support in IDE loader since RC4, only SD loader support it.

In this post, @SWAT seems to confirm that
CSO support is a thing, although limited to SD loader (presumably SDcard on SPI).

According to my tests, including on recent 0.6.11 loader, CSO doesn't work at all, not even on SPI. It used to work in RC3, but seems broken since RC4.

Considering that the loader code seems to receive some attention and updates these days,
could it be possible to check this part?

It would be a pity that CSO support occupies some precious memory space if, ultimately, it doesn't work. On the other hand, if it's supposed to work, it would be great to confirm it by actually testing it. If there are a few tricks to know to get it working with RC4, we would be glad to know them.

http://www.dc-swat.ru/forum/thread-2646-...l#pid41022