GoldenEye 007 Nintendo 64 Community, GoldenEye X, Nintendo 64 Games Discussion

[K1lo]

Hey all, I've done a bit of searching but I can't find any documents explaining how the compressed block between 0x21990 and 0x331E0 is formatted. I've started picking it apart in a hex editor.. but if there are any documents / prior work I can look at anywhere I'd really appreciate it.


EDIT:

I've discovered what I believe is the file table starting at 0x5A84 and repeating with the following structure:

Code:

uint8_t m_magicByte1;                 // always seems to be 0x80     uint8_t m_magicByte2;                 // always seems to be 0x05     uint16_t m_entryIdentifier;           // ?     uint32_t m_addressInROM;     uint16_t m_magicWORD;                 // always seems to be 0x0000     uint8_t m_unknownFlag;                // Observed values 0x00 - 0x02

I've found reference to the setup locations in memory in this block of data so I'm fair sure I'm on the right track, however there is a huge amount here that I still don't understand..

Last edited by K1lo on Thu Jul 20, 2017 12:08 am; edited 6 times in total

[SubDrag]

Zoinkity's documentation has a gigantic file on everything he found out.

[K1lo]

Could you point me to where to find it? I did look but only found a download link to a site that no longer exists

[SubDrag]

Looks like Zoinkity's media fire link to it is down. Hopefully he can post when he sees this latest link.

[Wreck]

Or try sending him a PM in case he misses this thread.

[Trevor]

This is everything I have from zoink from a few years back.

In the parent directory I have modifications of some of his docs to correct display list stuff - mainly an updated ucode05.txt and because some branched into in-depth I split them off into their own files.

There is also Kholdfusions disassembly project that holds a copy of zoinks docs

Hope these help

Trev
_________________

[K1lo]

I did PM Wreck but I guess he's not seen it yet.

Thanks for the link Trevor, was planning on spending some time today working more on Goldeneye so I'll start by reading through what you linked.

[K1lo]

Hey guys, I'm still confused after doing quite a bit of digging.

I am trying to identify where the setup offsets are within the ROM - specifically these values from rom-address-table.txt

Code:

solo setup 8004*number *word *ROM addy *word value @ pointer 7d40:00000269|8005b26c|008ab210   UsetuparchZ 7d4c:0000026a|8005b278|008af820   UsetuparkZ 7d58:0000026b|8005b284|008b33b0   UsetupaztZ 7d64:0000026c|8005b290|008b5cb0   UsetupcaveZ 7d70:0000026d|8005b29c|008b9b10   UsetupcontrolZ 7d7c:0000026e|8005b2ac|008bd610   UsetupcradZ 7d88:0000026f|8005b2b8|008bf240   UsetupcrypZ 7d94:00000270|8005b2c4|008c10d0   UsetupdamZ 7da0:00000271|8005b2d0|008c53a0   UsetupdepoZ 7dac:00000272|8005b2dc|008c8330   UsetupdestZ 7db8:00000273|8005b2e8|008ca680   UsetupjunZ 7dc4:00000274|8005b2f4|008cdd80   UsetuplenZ 7dd0:00000275|8005b300|008ce350   UsetuppeteZ 7ddc:00000276|8005b30c|008d12d0   UsetuprunZ 7de8:00000277|8005b318|008d2b30   UsetupsevbZ 7df4:00000278|8005b324|008d5190   UsetupsevbunkerZ 7e00:00000279|8005b338|008d6bc0   UsetupsevxZ 7e0c:0000027a|8005b344|008daed0   UsetupsevxbZ 7e18:0000027b|8005b354|008defc0   UsetupsiloZ 7e24:0000027c|8005b360|008e1a10   UsetupstatueZ 7e30:0000027d|8005b370|008e41e0   UsetuptraZ

To take the first entry - 0x008ab210 is the offset within the rom that the compressed setup for archives can be found. But where does this value 0x008ab210 come from? It must be hardcoded somewhere?

EDIT:

After much more digging I found that this table appears to occur within the decompressed 21990 block at offset 0x26FB0.

The structure appears to be exactly as described in ZoinkDocs:

Code:

uint32_t id; uint32_t word; // not sure yet what this is uint32_t offsetInRomToCompressedBlock; // Bingo

[zoinkity]

Middle value is a pointer to the filename string.
GE loads files based on filenames. This is why there's a minimum of two instances of every string in the game; one is from a table of all the files of a given type, corresponding to their IDs; the second is the one in this table, used for comparison when loading the file.

When a request is made, first it walks through a fixed number of entries in this table and compares the requested filename with the string assigned for each entry. If found, the file is loaded an an instance is filled into the corresponding slot in a table somewhere in the 8007-8008ish range. For the most part these entries are boring, unless...

If it isn't found, then it looks at a different table which doesn't exist, found at a memory address on the dev board. It continually looks at the list until it is found (since it presumes you will eventually load it), and once found fills in that 8007-8008ish fileentry with a pointer to the uploaded data and some of the features of it like filesize.

This might seem like a footnote, but it's also how saving and loading ramrom replay files works, how the Spectrum emulator really got its data, and how debug features like screenshots were taken.

+_+

Might as well mention that the setup editor makes some changes behind your back if you save a ROM in there--even if you make no changes to it. Most important is that it completely rewrites the image lookup table in a different, compacted format.
_________________
(\_/) Beware
(O.o) ze
(> <) Hoppentruppen!

[K1lo]

Hmm so I ran in to a bit of a brick wall and took a break for a day or two.

This is what I've done so far:

- My code iterates over all the E and J data treating each E and J block as one 'entry' recording where the pointer to the data entry exists for both E and J in the 21990 block and taking a copy of the raw data referenced for E

- I then set all the bytes in the text block which contains both E and J data to 0x00

- Lastly I start at the end and write the copied data from step 1 from the end of the zero'd out space backwards, updating the pointers within the 21990 table setting both E and J to point to the same E data

This results in 33716 bytes saved, Japanese data gone and English data packed nicely at the end.

This all works perfectly (at least looking at how I change the ROM, layout the data and update the pointers), however when I start the ROM in PJ64 I get this:



.. so something's not right.

I suspect it relates to the next section described in http://fgfc.ddns.net/PerfectGold/ZoinkDocs/GameShark%20-%20GE%20specific/multi%20load%20information.txt :

Code:

----------------------------------------------- pointers to replicate L~J/L~E table - RAM 800484dc-8004863c [target list] 8005 b880|LameE   |b887   b888|LameJ   |b88f b890|LarchE  |b897   b898|LarchJ  |b89f b8a0|LarkE   |b8a7   b8a8|LarkJ   |b8af b8b0|LashE   |b8b7   b8b8|LashJ   |b8bf b8c0|LaztE   |b8c7   b8c8|LaztJ   |b8cf b8d0|LcatE   |b8d7   b8d8|LcatJ   |b8df b8e0|LcaveE  |b8e7   b8e8|LcaveJ  |b8ef b8f0|LarecE  |b8f7   b8f8|LarecJ  |b8ff b900|LcradE  |b907   b908|LcradJ  |b90f b910|LcrypE  |b917   b918|LcrypJ  |b91f b920|LdamE   |b927   b928|LdamJ   |b92f ...

I don't understand what is meant by these address ranges 800484dc-8004863c and this section at all

Quote:

Middle value is a pointer to the filename string. GE loads files based on filenames. This is why there's a minimum of two instances of every string in the game; one is from a table of all the files of a given type, corresponding to their IDs; the second is the one in this table, used for comparison when loading the file.

Can you explain how I convert a 80xxxxxx address into an offset from the start of the ROM file?

If I understand you correctly (and I'm not sure if I do to be honest) I suspect the issue is this second table still has the old offsets which no longer match the ones I've updated in the 21990 table.

[K1lo]

Actually the problem appears to be in the way I write the 21990 block back in to the ROM. Just reading in the 21990 block, decompressing, recompressing and writing it back out seems to causing the infinite loop.

*sigh* I was sure this was working before but apparently not.. still investigating.. at least it gives me something to go on

[pavarini]

Not sure if you consider this, but try updating the ROM's CRC.

[K1lo]

pavarini wrote:

Not sure if you consider this, but try updating the ROM's CRC.

I don't think this is it as I haven't recomputed the CRC at all so far and not run in to problems. If the checksum needed to be correct I would have come across this by now.

I'm rather stumped currently as to why my completely unaltered 21990 block when..

- decompressed (verified correct in content and length against http://fgfc.ddns.net/PerfectGold/ZoinkDocs/21990%20index%20extension.txt)
- recompressed
- the previous data from 0x21990 - 0x3358f set to 0x00
- newly compressed data written starting at offset 0x21990

... causes the ROM to go in to this infinite loop.

The newly compressed data is smaller than the original, so it's not exceeding the space available or overwriting any data later.

It suggests somewhere the emulated CPU encounters something it wasn't expecting or fails to decompress the new 21990. However if there was a problem with my decompression / compression I would have seen it elsewhere with the compressing of setup files that I change and re-inject into the ROM.

Really puzzling.. I don't think there's a length specified anywhere for the compressed 21990 block as the decompression handles null padded data.

SubDrag - did you have to do anything special when writing out updated 21990 blocks for the Goldeneye Setup Editor?

[SubDrag]

If you edit anything < 0x100000, you need to redo the CRC. That includes 21990 file. Run rn64crc to redo the CRC.

[K1lo]

Gargh.. I used the tool in the SetupEditor and it works now.. can't believe I lost so much time trying to figure out what was happening. Thanks SubDrag and pavarini! I must have gotten lucky with the placement of the setup blocks in the ROM.

Now I'll put together a quick CRC calculator into Janus to do this automatically.

UPDATE:

Actually it was even easier than I thought since there was already a GPL'd implementation available here http://n64dev.org/n64crc.html . From studying that I was able to piece together what I needed quickly.

Right.. I think that's enough for one night. Hopefully tomorrow I can start making good progress again.