ShootersForever.com Forum Index

GoldenEye 007 Nintendo 64 Community, GoldenEye X, Nintendo 64 Games Discussion
GoldenEye Cheats, GoldenEye X Codes, Tips, Help, Nintendo 64 Gaming Community


Patching rom with no changes leads to a very different file

 
Post new topic   Reply to topic    ShootersForever.com Forum Index -> Q-Lab Hacking Department
View previous topic :: View next topic  
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Mon Jul 10, 2017 2:24 am    Post subject: Patching rom with no changes leads to a very different file Reply with quote Back to top

Hi guys,

So I have a project in mind to create a tool that will randomise weapons for characters within the Goldeneye ROM. For example, one mode might be change all KF7s to ZMGs. The idea would be to make small changes to mix things up slightly each time it is run.

To that end, I wanted to identify what changes are made to the ROM when I change one guard's weapon so I could work backwards from this to create a tool that changed the ROM's bytes safely.

I started by importing the default facility level in GSE and generating patching the ROM. This to my mind should have resulted in a 1:1 identical ROM to the original, however there are many many byte changes.

What gives?

Another question, I've started reading the excellent documentation at https://gitlab.com/kholdfuzion/Gold007/ am I right in thinking most of the ROM is therefore compressed data? This would explain some of the differences as even a single byte change could potentially have a cascading effect on the compressed data.

I also realised that there was a byteflipping issue with my original source ROM. When I byteflipped it the header looked right again barring what I assumed is a checksum area. However there were still a huge number of changes throughout the ROM file.
 
View user's profile Send private message
SubDrag
Administrator
Administrator


Joined: 16 Aug 2006
Posts: 6118

 PostPosted: Mon Jul 10, 2017 5:01 am    Post subject: Reply with quote Back to top

Yeah its the compression. I don't know which version of gzip or zlib Rare used, but this version I use produces different files. So even the identical uncompressed file yields vastly different compressed files. Zoinkity's documentation gives you the file list in rom which you may be after.

Also a big compressed file in rom contains file table, at 0x21990 which is updated whenever a file is inserted. I do some minor auto adjustments to rom to fix minor things.
 
View user's profile Send private message
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Mon Jul 10, 2017 6:20 am    Post subject: Reply with quote Back to top

Thanks SubDrag, it's slowly starting to make more sense. I'm going to write some code and start poking my way around these structures. I'll likely have more questions soon Rolling Eyes
 
View user's profile Send private message
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Wed Jul 12, 2017 12:46 pm    Post subject: Reply with quote Back to top

I've started writing my tool which can now memory map the file, parse the header and read arbitrary addresses from the ROM.

I'm guessing the next thing I need to do is parse the resources table for the locations of the level setups in the ROM, then decompress one, edit, recompress and re-inject.

One step at a time however - am I right in thinking these addresses are found in 21990-331E0 ? Additionally, can you give me a hint what the data structure of this block is?

EDIT:

Perhaps I'm overcomplicating by looking at the table at 21990. I looked more at rom-address-table.txt and found this address list:

solo setup
8004*number *word *ROM addy *word value @ pointer
7d40:00000269|8005b26c|008ab210 UsetuparchZ
7d4c:0000026a|8005b278|008af820 UsetuparkZ
7d58:0000026b|8005b284|008b33b0 UsetupaztZ
7d64:0000026c|8005b290|008b5cb0 UsetupcaveZ
7d70:0000026d|8005b29c|008b9b10 UsetupcontrolZ
7d7c:0000026e|8005b2ac|008bd610 UsetupcradZ
7d88:0000026f|8005b2b8|008bf240 UsetupcrypZ
7d94:00000270|8005b2c4|008c10d0 UsetupdamZ
7da0:00000271|8005b2d0|008c53a0 UsetupdepoZ
7dac:00000272|8005b2dc|008c8330 UsetupdestZ
7db8:00000273|8005b2e8|008ca680 UsetupjunZ
7dc4:00000274|8005b2f4|008cdd80 UsetuplenZ
7dd0:00000275|8005b300|008ce350 UsetuppeteZ
7ddc:00000276|8005b30c|008d12d0 UsetuprunZ
7de8:00000277|8005b318|008d2b30 UsetupsevbZ
7df4:00000278|8005b324|008d5190 UsetupsevbunkerZ
7e00:00000279|8005b338|008d6bc0 UsetupsevxZ
7e0c:0000027a|8005b344|008daed0 UsetupsevxbZ
7e18:0000027b|8005b354|008defc0 UsetupsiloZ
7e24:0000027c|8005b360|008e1a10 UsetupstatueZ
7e30:0000027d|8005b370|008e41e0 UsetuptraZ

however I'm not sure if I follow the format here - how do I get the absolute offset for UsetuparchZ for example?

EDIT 2: Nevermind, I looked at filelist.txt and this has the offsets I am looking for. Additionally I've noticed what looks like a header for each compressed block.. experimenting commences Very Happy


Last edited by K1lo on Wed Jul 12, 2017 1:10 pm; edited 1 time in total
 
View user's profile Send private message
SubDrag
Administrator
Administrator


Joined: 16 Aug 2006
Posts: 6118

 PostPosted: Wed Jul 12, 2017 1:05 pm    Post subject: Reply with quote Back to top

What are you trying to do? I mean overall. The editor handles all that stuff today already.

If you are trying to randomize weapons you can use the Setup Editor and make action blocks per level. Or write your own ASM for the ROM.
 
View user's profile Send private message
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Wed Jul 12, 2017 1:14 pm    Post subject: Reply with quote Back to top

In general I'm trying to create a tool that will take a ROM file and randomly mix the weapons for the guards in different levels and save a new ROM file. That's my goal.

I know it's possible with the setup editor, but I'd like to have a one stop 'randomiser' style tool. That's my challenge to myself.

In general, will my approach of..

1. decompress setup files
2. randomly change weapons bytes for existing guards
3. recompress setup files
4. reinject into ROM
5. recompute checksums

.. work or am I missing something critical?
 
View user's profile Send private message
SubDrag
Administrator
Administrator


Joined: 16 Aug 2006
Posts: 6118

 PostPosted: Wed Jul 12, 2017 2:58 pm    Post subject: Reply with quote Back to top

K1lo wrote:
In general I'm trying to create a tool that will take a ROM file and randomly mix the weapons for the guards in different levels and save a new ROM file. That's my goal.

I know it's possible with the setup editor, but I'd like to have a one stop 'randomiser' style tool. That's my challenge to myself.

In general, will my approach of..

1. decompress setup files
2. randomly change weapons bytes for existing guards
3. recompress setup files
4. reinject into ROM
5. recompute checksums

.. work or am I missing something critical?


That's fine, I guess. You'll have to make a mini setup parser. I was just mentioning in setup editor you could make an action block to do so for each guard instead of writing a tool. You must be trying to learn programming or GoldenEye format a bit.

Also the new files have to be smaller, but probably ok there.
 
View user's profile Send private message
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Thu Jul 13, 2017 12:38 am    Post subject: Reply with quote Back to top

Hmm could you expand on how it would work via action blocks?

I write C++ as my day job so it's more wanting to learn about the data structures within Goldeneye.

EDIT: So trying to decompress a setup block I am met with the following from zlib inflate:

"incorrect header check"

From https://stackoverflow.com/questions/9050260/what-does-a-zlib-header-look-like I can see zlib is expecting two nybbles with compression info which it doesn't seem like are present.

For reference this is what I believe to be the start of the compressed setup block for UsetuparchZ:



I'm simply feeding the 17935 bytes of this compressed setup into zlib.

Can you give me a pointer to what I'm doing incorrectly? I'm not sure if I follow what is said in https://gitlab.com/kholdfuzion/Gold007/blob/master/GE%20Documentation/note%20on%20compression.txt

EDIT 2:

Ok I understand bytes 0x11 0x72 are the header for the GE compressed blocks. But even stripping out these two bytes and passing the rest of the data in to inflate zlib is complaining about the header.

EDIT 3:

I read through www.zlib.net/manual.html#Advanced and solved the problem so please ignore my questions Cool

The solution in case anyone else searches about this in the future was to use a negative windowBits value of -8 to provoke a raw decode ignoring the GE specific header bytes.
 
View user's profile Send private message
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Thu Jul 13, 2017 4:45 am    Post subject: Reply with quote Back to top

Is there any documentation for the structure of the setup file?

From doing a little reverse engineering it appears that there is a repeating structure of 0x88 bytes ending with FF FF FF. The byte immediately before signifies the weapon, however I'm unsure how to interpret the rest.



Above the weapon byte is set to 0x08 which is KF7.


Any clues ? Smile

EDIT:

It's probably not the best way of doing it since I don't properly understand the structure, however the first weapon block seems to start with bytes

0x00, 0x00, 0x00, 0x01, 0x01, 0x00, 0x00, 0x08

followed by

0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x08

for each remaining weapon block.

They are all 0x88 bytes long and terminate 0xFF 0xFF 0xFF.

That's enough to go on for now but I'd prefer to understand more Shocked
 
View user's profile Send private message
Wreck
Administrator
Administrator


Joined: 14 Dec 2005
Posts: 7197
Location: Ontario, Canada

 PostPosted: Thu Jul 13, 2017 4:18 pm    Post subject: Reply with quote Back to top

There is probably documentation somewhere for the setup file. Zoinkity has pretty extensive docs on GoldenEye.

Size (0100)
Type of object (0008) (which also determines size of block)
Prop Model (00B8)
Pad / Guard ID Attached To (001C)
Special Flags (00004000 00000000) (this setting makes it held by guard)
...later
Health (03E8)
Weapon ID (08FFFFFF)

Those weapons actually end with the 00000000, and start with the size.
 
View user's profile Send private message Visit poster's website
K1lo
Agent
Agent


Joined: 10 Jun 2012
Posts: 112
Location: Albert Embankment, Vauxhall

 PostPosted: Fri Jul 14, 2017 5:47 am    Post subject: Reply with quote Back to top

Cool, thanks Wreck, I'll tweak my code
 
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    ShootersForever.com Forum Index -> Q-Lab Hacking Department All times are GMT - 8 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Cobalt 2.0 BB theme/template by Jakob Persson.
Copyright © 2002-2004 Jakob Persson


Powered by BB © 01, 02 BB Group

 


Please Visit My Other Sites: GoldenEyeForever.com | GrandTheftAutoForever.com

Got kids? Check out my Dora The Explorer site with games and coloring pages!

Our forums feature Nintendo 64 games, GoldenEye 007 N64 New Maps and Missions, GoldenEye Cheats, N64 Emulator, Gameshark, GoldenEye Multiplayer and more!

[ Privacy Policy ]