ShootersForever.com Forum Index

GoldenEye 007 Nintendo 64 Community, GoldenEye X, Nintendo 64 Games Discussion
GoldenEye Cheats, GoldenEye X Codes, Tips, Help, Nintendo 64 Gaming Community


File table 0x21990 schema
Goto page 1, 2, 3  Next
 
Post new topic   Reply to topic    ShootersForever.com Forum Index -> Q-Lab Hacking Department
View previous topic :: View next topic  
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Wed Jul 19, 2017 5:49 am    Post subject: File table 0x21990 schema Reply with quote Back to top

Hey all, I've done a bit of searching but I can't find any documents explaining how the compressed block between 0x21990 and 0x331E0 is formatted. I've started picking it apart in a hex editor.. but if there are any documents / prior work I can look at anywhere I'd really appreciate it.


EDIT:

I've discovered what I believe is the file table starting at 0x5A84 and repeating with the following structure:

Code:
 
    uint8_t m_magicByte1;                 // always seems to be 0x80
    uint8_t m_magicByte2;                 // always seems to be 0x05
    uint16_t m_entryIdentifier;           // ?
    uint32_t m_addressInROM;
    uint16_t m_magicWORD;                 // always seems to be 0x0000
    uint8_t m_unknownFlag;                // Observed values 0x00 - 0x02


I've found reference to the setup locations in memory in this block of data so I'm fair sure I'm on the right track, however there is a huge amount here that I still don't understand..


Last edited by K1lo on Thu Jul 20, 2017 12:08 am; edited 6 times in total
 
View user's profile Send private message
SubDrag
Administrator
Administrator


Joined: 16 Aug 2006
Posts: 6118

 PostPosted: Wed Jul 19, 2017 6:24 am    Post subject: Reply with quote Back to top

Zoinkity's documentation has a gigantic file on everything he found out.
 
View user's profile Send private message
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Wed Jul 19, 2017 6:31 am    Post subject: Reply with quote Back to top

Could you point me to where to find it? I did look but only found a download link to a site that no longer exists Confused
 
View user's profile Send private message
SubDrag
Administrator
Administrator


Joined: 16 Aug 2006
Posts: 6118

 PostPosted: Wed Jul 19, 2017 6:40 am    Post subject: Reply with quote Back to top

Looks like Zoinkity's media fire link to it is down. Hopefully he can post when he sees this latest link.
 
View user's profile Send private message
Wreck
Administrator
Administrator


Joined: 14 Dec 2005
Posts: 7197
Location: Ontario, Canada

 PostPosted: Wed Jul 19, 2017 10:42 pm    Post subject: Reply with quote Back to top

Or try sending him a PM in case he misses this thread.
 
View user's profile Send private message Visit poster's website
Trevor
007
007


Joined: 15 Jan 2010
Posts: 926
Location: UK, Friockheim OS:Win11-Dev PerfectGold:Latest

 PostPosted: Sat Jul 22, 2017 2:08 am    Post subject: Reply with quote Back to top

This is everything I have from zoink from a few years back.

In the parent directory I have modifications of some of his docs to correct display list stuff - mainly an updated ucode05.txt and because some branched into in-depth I split them off into their own files.

There is also Kholdfusions disassembly project that holds a copy of zoinks docs

Hope these help

Trev
_________________
 
View user's profile Send private message
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Sat Jul 22, 2017 5:45 am    Post subject: Reply with quote Back to top

I did PM Wreck but I guess he's not seen it yet.

Thanks for the link Trevor, was planning on spending some time today working more on Goldeneye so I'll start by reading through what you linked. Cool
 
View user's profile Send private message
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Sat Aug 26, 2017 6:45 am    Post subject: Reply with quote Back to top

Hey guys, I'm still confused after doing quite a bit of digging.

I am trying to identify where the setup offsets are within the ROM - specifically these values from rom-address-table.txt

Code:

solo setup
8004*number *word *ROM addy *word value @ pointer
7d40:00000269|8005b26c|008ab210   UsetuparchZ
7d4c:0000026a|8005b278|008af820   UsetuparkZ
7d58:0000026b|8005b284|008b33b0   UsetupaztZ
7d64:0000026c|8005b290|008b5cb0   UsetupcaveZ
7d70:0000026d|8005b29c|008b9b10   UsetupcontrolZ
7d7c:0000026e|8005b2ac|008bd610   UsetupcradZ
7d88:0000026f|8005b2b8|008bf240   UsetupcrypZ
7d94:00000270|8005b2c4|008c10d0   UsetupdamZ
7da0:00000271|8005b2d0|008c53a0   UsetupdepoZ
7dac:00000272|8005b2dc|008c8330   UsetupdestZ
7db8:00000273|8005b2e8|008ca680   UsetupjunZ
7dc4:00000274|8005b2f4|008cdd80   UsetuplenZ
7dd0:00000275|8005b300|008ce350   UsetuppeteZ
7ddc:00000276|8005b30c|008d12d0   UsetuprunZ
7de8:00000277|8005b318|008d2b30   UsetupsevbZ
7df4:00000278|8005b324|008d5190   UsetupsevbunkerZ
7e00:00000279|8005b338|008d6bc0   UsetupsevxZ
7e0c:0000027a|8005b344|008daed0   UsetupsevxbZ
7e18:0000027b|8005b354|008defc0   UsetupsiloZ
7e24:0000027c|8005b360|008e1a10   UsetupstatueZ
7e30:0000027d|8005b370|008e41e0   UsetuptraZ


To take the first entry - 0x008ab210 is the offset within the rom that the compressed setup for archives can be found. But where does this value 0x008ab210 come from? It must be hardcoded somewhere?

EDIT:

After much more digging I found that this table appears to occur within the decompressed 21990 block at offset 0x26FB0.

The structure appears to be exactly as described in ZoinkDocs:

Code:

uint32_t id;
uint32_t word; // not sure yet what this is
uint32_t offsetInRomToCompressedBlock; // Bingo
 
View user's profile Send private message
zoinkity
007
007


Joined: 24 Nov 2005
Posts: 1684

 PostPosted: Sat Aug 26, 2017 8:23 am    Post subject: Reply with quote Back to top

Middle value is a pointer to the filename string.
GE loads files based on filenames. This is why there's a minimum of two instances of every string in the game; one is from a table of all the files of a given type, corresponding to their IDs; the second is the one in this table, used for comparison when loading the file.

When a request is made, first it walks through a fixed number of entries in this table and compares the requested filename with the string assigned for each entry. If found, the file is loaded an an instance is filled into the corresponding slot in a table somewhere in the 8007-8008ish range. For the most part these entries are boring, unless...

If it isn't found, then it looks at a different table which doesn't exist, found at a memory address on the dev board. It continually looks at the list until it is found (since it presumes you will eventually load it), and once found fills in that 8007-8008ish fileentry with a pointer to the uploaded data and some of the features of it like filesize.

This might seem like a footnote, but it's also how saving and loading ramrom replay files works, how the Spectrum emulator really got its data, and how debug features like screenshots were taken.

+_+

Might as well mention that the setup editor makes some changes behind your back if you save a ROM in there--even if you make no changes to it. Most important is that it completely rewrites the image lookup table in a different, compacted format.
_________________
(\_/) Beware
(O.o) ze
(> <) Hoppentruppen!
 
View user's profile Send private message Send e-mail
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Wed Aug 30, 2017 11:57 am    Post subject: Reply with quote Back to top

Hmm so I ran in to a bit of a brick wall and took a break for a day or two.

This is what I've done so far:

- My code iterates over all the E and J data treating each E and J block as one 'entry' recording where the pointer to the data entry exists for both E and J in the 21990 block and taking a copy of the raw data referenced for E

- I then set all the bytes in the text block which contains both E and J data to 0x00

- Lastly I start at the end and write the copied data from step 1 from the end of the zero'd out space backwards, updating the pointers within the 21990 table setting both E and J to point to the same E data

This results in 33716 bytes saved, Japanese data gone and English data packed nicely at the end.

This all works perfectly (at least looking at how I change the ROM, layout the data and update the pointers), however when I start the ROM in PJ64 I get this:



.. so something's not right.

I suspect it relates to the next section described in http://fgfc.ddns.net/PerfectGold/ZoinkDocs/GameShark%20-%20GE%20specific/multi%20load%20information.txt :

Code:

-----------------------------------------------
pointers to replicate L~J/L~E table - RAM
800484dc-8004863c
[target list]
8005
b880|LameE   |b887   b888|LameJ   |b88f
b890|LarchE  |b897   b898|LarchJ  |b89f
b8a0|LarkE   |b8a7   b8a8|LarkJ   |b8af
b8b0|LashE   |b8b7   b8b8|LashJ   |b8bf
b8c0|LaztE   |b8c7   b8c8|LaztJ   |b8cf
b8d0|LcatE   |b8d7   b8d8|LcatJ   |b8df
b8e0|LcaveE  |b8e7   b8e8|LcaveJ  |b8ef
b8f0|LarecE  |b8f7   b8f8|LarecJ  |b8ff
b900|LcradE  |b907   b908|LcradJ  |b90f
b910|LcrypE  |b917   b918|LcrypJ  |b91f
b920|LdamE   |b927   b928|LdamJ   |b92f
...


I don't understand what is meant by these address ranges 800484dc-8004863c and this section at all Shocked

Quote:

Middle value is a pointer to the filename string.
GE loads files based on filenames. This is why there's a minimum of two instances of every string in the game; one is from a table of all the files of a given type, corresponding to their IDs; the second is the one in this table, used for comparison when loading the file.


Can you explain how I convert a 80xxxxxx address into an offset from the start of the ROM file?

If I understand you correctly (and I'm not sure if I do to be honest) I suspect the issue is this second table still has the old offsets which no longer match the ones I've updated in the 21990 table.
 
View user's profile Send private message
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Wed Aug 30, 2017 1:31 pm    Post subject: Reply with quote Back to top

Actually the problem appears to be in the way I write the 21990 block back in to the ROM. Just reading in the 21990 block, decompressing, recompressing and writing it back out seems to causing the infinite loop.

*sigh* I was sure this was working before but apparently not.. still investigating.. at least it gives me something to go on
 
View user's profile Send private message
pavarini
00 Agent
00 Agent


Joined: 07 May 2015
Posts: 479

 PostPosted: Wed Aug 30, 2017 1:55 pm    Post subject: Reply with quote Back to top

Not sure if you consider this, but try updating the ROM's CRC.
 
View user's profile Send private message
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Wed Aug 30, 2017 2:04 pm    Post subject: Reply with quote Back to top

pavarini wrote:
Not sure if you consider this, but try updating the ROM's CRC.


I don't think this is it as I haven't recomputed the CRC at all so far and not run in to problems. If the checksum needed to be correct I would have come across this by now. Razz

I'm rather stumped currently as to why my completely unaltered 21990 block when..

- decompressed (verified correct in content and length against http://fgfc.ddns.net/PerfectGold/ZoinkDocs/21990%20index%20extension.txt)
- recompressed
- the previous data from 0x21990 - 0x3358f set to 0x00
- newly compressed data written starting at offset 0x21990

... causes the ROM to go in to this infinite loop.

The newly compressed data is smaller than the original, so it's not exceeding the space available or overwriting any data later.

It suggests somewhere the emulated CPU encounters something it wasn't expecting or fails to decompress the new 21990. However if there was a problem with my decompression / compression I would have seen it elsewhere with the compressing of setup files that I change and re-inject into the ROM.

Really puzzling.. I don't think there's a length specified anywhere for the compressed 21990 block as the decompression handles null padded data.

SubDrag - did you have to do anything special when writing out updated 21990 blocks for the Goldeneye Setup Editor?
 
View user's profile Send private message
SubDrag
Administrator
Administrator


Joined: 16 Aug 2006
Posts: 6118

 PostPosted: Wed Aug 30, 2017 2:35 pm    Post subject: Reply with quote Back to top

If you edit anything < 0x100000, you need to redo the CRC. That includes 21990 file. Run rn64crc to redo the CRC.
 
View user's profile Send private message
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Wed Aug 30, 2017 2:57 pm    Post subject: Reply with quote Back to top

Gargh.. I used the tool in the SetupEditor and it works now.. can't believe I lost so much time trying to figure out what was happening. Thanks SubDrag and pavarini! I must have gotten lucky with the placement of the setup blocks in the ROM.

Now I'll put together a quick CRC calculator into Janus to do this automatically.

UPDATE:

Actually it was even easier than I thought since there was already a GPL'd implementation available here http://n64dev.org/n64crc.html . From studying that I was able to piece together what I needed quickly.

Right.. I think that's enough for one night. Hopefully tomorrow I can start making good progress again. Very Happy
 
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    ShootersForever.com Forum Index -> Q-Lab Hacking Department All times are GMT - 8 Hours
Goto page 1, 2, 3  Next
Page 1 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Cobalt 2.0 BB theme/template by Jakob Persson.
Copyright © 2002-2004 Jakob Persson


Powered by BB © 01, 02 BB Group

 


Please Visit My Other Sites: GoldenEyeForever.com | GrandTheftAutoForever.com

Got kids? Check out my Dora The Explorer site with games and coloring pages!

Our forums feature Nintendo 64 games, GoldenEye 007 N64 New Maps and Missions, GoldenEye Cheats, N64 Emulator, Gameshark, GoldenEye Multiplayer and more!

[ Privacy Policy ]