SID dump file format

[simon]

Hello,

available SID files come with an own player routine written for 65xx CPUs. Hence all PC based SID players emulate this CPU or even more parts of the C64. This approach is probably not the way to go as it needs too much computing power.

To circumvent the emulation layer I used siddump from coverbitops to dump all SID registers 50 times per second. The output looks like this:

Code: Select all

| Frame | Freq Note/Abs WF ADSR Pul | Freq Note/Abs WF ADSR Pul | Freq Note/Abs WF ADSR Pul | FCut RC Typ V |
+-------+---------------------------+---------------------------+---------------------------+---------------+
|     0 | 0000  ... ..  00 0000 000 | 0000  ... ..  00 0000 000 | 0000  ... ..  00 0000 000 | 0000 00 Off F |
|     1 | 15ED  ... ..  01 00AA 700 | 1167  C-4 B0  51 0059 ... | 0459  ... ..  01 00C8 E00 | .... F4 Low . |
|     2 | 15ED  E-4 B4  21 .... 730 | ....  ... ..  .. .... ... | 08B3  C-3 A4  41 .... DC0 | C000 .. ... . |
|     3 | ....  ... ..  .. .... 760 | ....  ... ..  50 .... ... | 0459 (C-2 98) .. .... D80 | A000 .. ... . |
|     4 | ....  ... ..  20 .... 790 | ....  ... ..  .. .... ... | ....  ... ..  .. .... D40 | 8000 .. ... . |
|     5 | ....  ... ..  .. .... 7C0 | ....  ... ..  .. .... ... | ....  ... ..  .. .... D00 | 6000 .. ... . |
|     6 | 159B (- 0052) .. .... 7F0 | ....  ... ..  .. .... ... | ....  ... ..  .. .... CC0 | 4000 .. ... . |
|     7 | 1549 (D#4 B3) .. .... 820 | 15ED  E-4 B4  51 .... ... | ....  ... ..  .. .... C80 | 2000 .. ... . |
|     8 | 159B (E-4 B4) .. .... 850 | ....  ... ..  .. .... ... | ....  ... ..  .. .... C40 | .... .. ... . |
|     9 | 15ED (+ 0052) .. .... 880 | ....  ... ..  50 .... ... | 0442 (- 0017) .. .... C00 | .... .. ... . |
|    10 | 163F (+ 0052) .. .... 8B0 | ....  ... ..  .. .... ... | 042B (B-1 97) .. .... BC0 | .... .. ... . |
|    11 | 1691 (+ 0052) .. .... 8E0 | ....  ... ..  .. .... ... | 0442 (C-2 98) 40 .... B80 | .... .. ... . |
|    12 | ....  ... ..  .. .... ... | ....  ... ..  .. .... ... | 0459 (+ 0017) .. .... B40 | .... .. ... . |
|    13 | 1A13  ... ..  01 .... 700 | 1A13  G-4 B7  51 .... ... | 0470 (+ 0017) .. .... B00 | .... .. ... . |
|    14 | 1A13  G-4 B7  21 .... 730 | ....  ... ..  .. .... ... | 0487 (C#2 99) .. .... AC0 | .... .. ... . |
|    15 | ....  ... ..  .. .... 760 | ....  ... ..  50 .... ... | 0470 (C-2 98) .. .... A80 | .... .. ... . |


To play such a dump the player routine is called 50 times per second (for example via a vertical blank interrupt). Every time it is called the player routine writes the corresponding values into the SID registers. As you can see, there are many registers which doesn't change with every frame ("...." means don't touch this register in this frame).
The dump above is shown in ASCII format. Wenn using binary format all ".." have to be replaced with a byte value which is not used elsewhere in the SID dump. Call it magic byte for example. If the player routine reads this magic byte it just skips it.

All this works, but has some disadvantages. First I have to find a byte value which can be used as magic byte. Second the resulting dump file can be very big - due to repeating values every frame even if there are no changes.

What I would like to have is a better file format. Something which comes compressed and where I don't have to fiddle with magic bytes.
Any ideas?

Simon

[SteveMoody]

I'm writing a bit of code to convert the output of siddump format into an easy to use format. It should minimise the amount of space that will be required to store music.

Hopefully will have something to show this weekend.

[SteveMoody]

I was asked if i could share my progress on this so i''ll post it here.

All i've done is modified the SIDDUMP program to generate a file that contains only the registers that have changed between frames. It can generate either a binary file or a c header file.

For each frame it generates a count of the number of registers changed with the register that has changed and the value for the register. All values are bytes and an this is an example of the data that is produced (header file).

Code: Select all

   0x01, 0x10, 0xC0,
   0x01, 0x10, 0xD6,
   0x04, 0x0B, 0x43, 0x0C, 0xC4, 0x0D, 0x0F, 0x10, 0xEC,
   0x02, 0x0B, 0x80, 0x10, 0x02,
   0x01, 0x10, 0x18,
   0x06, 0x05, 0x00, 0x06, 0x00, 0x0B, 0x42, 0x0C, 0x00, 0x0D, 0x00, 0x10, 0x2E,
   0x01, 0x10, 0x44,
   0x01, 0x10, 0x5A,
   0x07, 0x04, 0x15, 0x05, 0xFB, 0x06, 0x0D, 0x0B, 0x43, 0x0C, 0xC4, 0x0D, 0x0F, 0x10, 0x70,
   0x03, 0x04, 0x80, 0x0B, 0x80, 0x10, 0x86,


The header file contains a define that shows the number of frames to play.

For the binary file it starts with a 4 byte header "SSND" followed by a 16 bit value which contains the number of frames. After that is contains a series of bytes in the same format as above.

I've not got as far as a player yet, so there still could be bugs in the data.

[mstram]

Hi Steve,

Thanks for posting your code.

I compiled it and ran it against my favorite sid ("Commodore 64" by Jeff / Soren Lund).

The output was exactly the same as the standard siddump program.

Re: a player:

I'm more interested in getting the siddump format back into an editor format, Sid Wizard, Goattracker would be my preferences.

And probably the most achievable for those two trackers, as AFAIK they are the only ones whose source has been released.

A siddump -> Renoise converter would also be very cool.

Mike

[simon]

Thank you Steve,

I haven't the time to test your code yet. I'll do this in the next days.

Indeed, it would be nice to get an editable format out from the siddump. But this is not that trivial as the program would have to analyse the player routine to get more information about the SID data. The goal of this siddump is to get an easy way to play any SID tune on different - not compatible - hardware than the C64. If we have achieved this we can think about the way back. If you like to contribute you are welcome.

Simon

[SteveMoody]

Hi Steve,
The output was exactly the same as the standard siddump program.

Sorry i wasn't very clear there. I've added extra options to the program to generate the extra files. If you run siddump without any parameters you will see them. The -b and -h options will generate the extra files, the output on screen will still be the same as the siddump format.

I've not looked at any of the editors so i've no idea what format they use.

[simon]

I pretty much like the format, Steve. But I have problems to play a generated tune. Could you look at the attachment and check my code for errors? I tried two different tunes I used somewhere else.
The actual player is in ISR1. I wonder if I made a mistake or if there is a disorder in setting the registers.

[Edit]It seems that the siddump output misses the frequency register values. I try to check the code.[/Edit]

[simon]

Steve,

I stumbled upon this code:

Code: Select all

                        if ((chn[c].freq & 0xFF) != (chn[c].freq & 0xFF))
                        {
                            registers[regcount * 2] = regbase + SID_FREQ_LO;
                            registers[(regcount * 2) + 1] = (unsigned char)(chn[c].freq & 0xFF);
                            regcount++;
                        }
                        if ((chn[c].freq & 0xFF00) != (chn[c].freq & 0xFF00))
                        {
                            registers[regcount * 2] = regbase + SID_FREQ_HI;
                            registers[(regcount * 2) + 1] = (unsigned char)((chn[c].freq >> 8) & 0xFF);
                            regcount++;
                        }


What are you doing here? If I comment out the if-conditions I get a working dump. Well sort of. It still doesn't sound as it should.

[simon]

I found the error. The if statements should be

Code: Select all

                       if ((chn[c].freq & 0xFF) != (prevchn[c].freq & 0xFF))


Anyway, it doesn't sound correct. I am not sure why. Maybe we have to set both bytes of 16-bit registers (frequency, pulse) even if only one changes. In addition we might want to move registers 0x04, 0x0b, 0x12 to the end of each frame as setting the gate bit starts playing the note. Compare here: http://www.oxyron.de/html/registers_sid.html

[SteveMoody]

I should have said expect bugs I've not done much testing with this code yet.

That's a useful reference. It may be that 16 bit values have to be written. That should be easy enough to sort out. I need to get back to reverse engineering the rick dangerous sound code. It will help a lot to understand how the SID works a little better