ShootersForever.com Forum Index

GoldenEye 007 Nintendo 64 Community, GoldenEye X, Nintendo 64 Games Discussion
GoldenEye Cheats, GoldenEye X Codes, Tips, Help, Nintendo 64 Gaming Community


Zoinkity - N64 Starcraft
Goto page Previous  1, 2, 3
 
Post new topic   Reply to topic    ShootersForever.com Forum Index -> Game-On!
View previous topic :: View next topic  
Gamma
Agent
Agent


Joined: 01 Aug 2009
Posts: 154

 PostPosted: Thu Jun 27, 2013 11:12 pm    Post subject: Reply with quote Back to top

I'm going to talk in SC1 lingo.

You could mention on teamliquid modders could use your file to change Starcraft 64 into any other game the want. A lot of total conversions changing buildings/units/animations are on Staredit.net. It is the only site left. I think UMS players would really enjoy playing particular genres on a n64 like SC RPG's. I think you can add your own music files and sound attacks. Maybe even SC2 sounds.

ALSO, regarding your comment sbout hacked tilesets- There are no such thing INGAMES of SC1. I'm not sure if hacks have been made for them (probably). With UMS you don't need to worry about hacked tilesets. On PC SC you could use 3rd party programs to put new stuff previously impossible into UMS games online. If you opened it in regular editor it would't work. But I think SPECIAL EDITOR UMS and Melee maps will work no problem in SC64. In the Iccup pack if you see computer Zerg creep where it shouldn't be naturally in map Colloseum your program works for 3rd party editors.

Sorry for text wall. I am enthusiastic. Hopeful some modder can make Dune 2 or SC2 for the N64!
 
View user's profile Send private message
Gamma
Agent
Agent


Joined: 01 Aug 2009
Posts: 154

 PostPosted: Thu Jun 27, 2013 11:34 pm    Post subject: Reply with quote Back to top

Here is the forum that Starcraft modmakers read where they may take interest in your program. There they almost exclusively change graphics and unit stats.
http://www.staredit.net/forums/14/

Here is most popular SC1 forum where you will get 2,000 views and a decent amount of replies:
http://www.teamliquid.net/forum/index.php?show_part=6
 
View user's profile Send private message
zoinkity
007
007


Joined: 24 Nov 2005
Posts: 1684

 PostPosted: Fri Jun 28, 2013 7:09 am    Post subject: Reply with quote Back to top

Oddly, I had used the SEN wiki as background for file identification. Somehow didn't realize there was a forum.

Have to admit, Dune 2 would be an awesome mod!

On the N64 side of things, sound is a bit of an issue. No idea what crazy custom setup they used, but it's some crazy custom setup. If they had used any of the standard libraries or raw sound it would already have been done.

The only sounds called by their "real name" are:
Code:
"staredit\wav\combeep0.wav"
"staredit\wav\combeep1.wav"
"staredit\wav\assault.wav"
"staredit\wav\coverop.wav"
"staredit\wav\demo.wav"
"staredit\wav\engineer.wav"
"staredit\wav\infantry.wav"
"staredit\wav\pyschic.wav"
"staredit\wav\pyro.wav"
"staredit\wav\survey.wav"
"staredit\wav\alert.wav"
"staredit\wav\endotfh.wav"
"staredit\wav\carbomb.wav"
"staredit\wav\sraw.wav"
"staredit\wav\flagtake.wav"
"staredit\wav\goal2.wav"
"staredit\wav\gottahurt.wav"
"staredit\wav\overtime.wav"
"staredit\wav\playball.wav"
"staredit\wav\pressstart.wav"
"staredit\wav\retrieve.wav"
"staredit\wav\yee haw.wav"
"staredit\wav\captured.wav"
"staredit\wav\yretrieve.wav"
"staredit\wav\yhasflag.wav"
"staredit\wav\ycapture.wav"
"staredit\wav\tpwrdown.wav"
"staredit\wav\intonydus.wav"
"staredit\wav\burrowup.wav"
"staredit\wav\nullsound.wav"

They do this in a messy case-by-case method, and they keep these separate from other sound banks. That needlessly complicates changing lookup to use a table.
All other sounds are called by a number, and without being able to extract the stupid things into, say, wav it isn't a simple matter to figure out their real-world correspondence.

So, sound will need some work ;*)
Apparently a simple empty map works if uncompressed. Compressor isn't playing nice yet, but already started on the offset calculator.
_________________
(\_/) Beware
(O.o) ze
(> <) Hoppentruppen!
 
View user's profile Send private message Send e-mail
zoinkity
007
007


Joined: 24 Nov 2005
Posts: 1684

 PostPosted: Sat Jun 29, 2013 9:09 am    Post subject: Reply with quote Back to top

Good news everyone!
Looks like I've got the stupid compressor working right now. Or, at least, it hasn't not worked yet ;*)

I freely admit this is not a very good implementation. It's not the prettiest code in the world either, so if anyone would like to clean it up, here it is in pythonese (v3.3):
Code:
    def cmpMass(tbl):
        """Converts table into binary format.  Returns bytearray()
        Table entries are tuples and expected in one of these formats:
            ('u', bytes, length)    uncompressed segment
                bytes is bytearray of data to write
                length should be len(bytes)

            ('c', back, length, cmdlen) compressed segment
                back is #bytes from cur.position backward to hit (actual)
                length is detected length of segment as to be written
                cmdlen is a count of #cmds required for entry, used as correction to length

            ('e',)  end of list

        Cascading backtracks from uncompressed to compressed segments is NOT supported yet!

        Note the resulting bytearray is not pre-aligned.  BOLTs require alignment to nearest 2-bytes.
        It is up to the user to apply any required alignment during insertion!
        """
        # Convert the table into a stream, doing some stuff to it.
        output = bytearray()
        if not tbl:
            return output
        for e in tbl:
            o = bytearray()
            if e[0]=='u':
                # Could also get length from len(bytes)-1
                d = e[1]
                l = e[2] - 1    # encoded length is 1 less
                # First four bits are accounted for in command
                o.append(0x80 | (l & 0xF))
                l>>=4
                if l:
                    v = l.bit_length()
                    for i in range(v//5):
                        o.append(0xA0 | (l & 0x1F))
                        l>>=5
                    if (v%5)>2:
                        o.append(0xA0 | (l & 0x1F))
                    else:
                        o.append(0x90 | (l&3))
                o.reverse()
                output.extend(o)
                output.extend(d)

            elif e[0]=='c':
                # First, deal with that backtrack value.
                o = bytearray()
                b, l, c = e[1], e[2], e[3]
                # Now write the commands!
                i = l&7
                i<<=4
                o.append(i | (b & 0xF))
                l>>=3
                b>>=4
                # Write all backtrack extensions
                v = b.bit_length()
                for i in range(v//6):
                    o.append(0xC0 | (b & 0x3F))
                    b>>=6
                    c-=1
                if (v%6)>2:
                    o.append(0xC0 | (b & 0x3F))
                    b>>=6
                    c-=1
                # Otherwise, b gets picked up below.
                j = l.bit_length()
                for i in range(j//5):
                    o.append(0xA0 | (l & 0x1F))
                    l>>=5
                    c-=1
                i = j%5
                if i>2 or c>1:
                    # Just use another big one otherwise
                    o.append(0xA0 | (l & 0x1F))
                    l>>=5
                    c-=1
                    i-=5
                if i>0 or b or c>0:
                    # One command if < 2
                    o.append(0x90 | (b<<2) | (l&3))
                o.reverse()
                output.extend(o)

        return output


Usage would be along the lines of:
cmpMass(cmpBackward(org))

I did it this way since you could really get away with different types of LZ schemes. All cmpMass() is (supposed) to do is encode the results.

Compression is fairly close to the original. So far I've gotten within +/- 1%, which itself is misleading since it's almost always within a few bytes regardless filesize. It's a bit slow, but that's primarily due to LZ lookup. The only fast difference detection scheme would be the one from BZdiff, and that's overkill to say the least.

That said, I haven't tested this against all the compressed files. There's 1129 of them after all.

Next step: fully-automated BOLT insert/replace script.

[edit]
Just updated the script above. It now works with all known samples without issues. There was an encoding issue with the previous thingy that would miss the (uncommon) situation of splitting a distance if you need a hanging 90 command to get a particular length encoded.

Takes forever, sure, but it does work.

Micro Practice does (mostly) work depending on how it's loaded. Still not 100% certain the conditions for triggering "Use Map Defaults", but one of them is in 2-player scenario mode. It works fine there, with various different side buildings under one of the two players' control.

BTW, you can set the stage# in the German proto via GS code:
810D1348

Really have half a mind to hack the BOLT subdirectory tables (idx in the list) so they use offsets expanded at load instead of physical offset from header. The difference would be changing one subdirectory versus changing all the ones following it as well any time you shove a new file in.
_________________
(\_/) Beware
(O.o) ze
(> <) Hoppentruppen!
 
View user's profile Send private message Send e-mail
Gamma
Agent
Agent


Joined: 01 Aug 2009
Posts: 154

 PostPosted: Mon Jul 01, 2013 10:16 am    Post subject: Reply with quote Back to top

I am clueless about computer programming and cannot understand any of your jargon haha. I understand it to be working though, so good work!

"Micro Practice does (mostly) work depending on how it's loaded. Still not 100% certain the conditions for triggering "Use Map Defaults", but one of them is in 2-player scenario mode. It works fine there, with various different side buildings under one of the two players' control."
I think you may be confused about how the map is supposed to work. It sounds like it is working with no problem. It can be played 1 player or 2 player. If in 1 player, some buildings are removed that the 2 player would have if there were 2 players in game.

When you are done can you post this in "program form" please? Very Happy Very cool project.
 
View user's profile Send private message
zoinkity
007
007


Joined: 24 Nov 2005
Posts: 1684

 PostPosted: Mon Jul 01, 2013 1:42 pm    Post subject: Reply with quote Back to top

Yeah, writing up some scripts to automate all the offset corrections, as well as others to rip maps out of SCX and SCM files.

I'll release a patch with a bunch of them already inserted along with the tools.

Of the ICCup maps, the only that might prove a problem are La Mancha, Nostalgia, Ride of Valkyries, and Vampire. These use custom audio which (at the moment) can't be handled. Probably work without the sounds, but well, it won't have them.
Didn't look at non-ladder maps yet.

Basically, I just did a couple tests manually inserting stuff and changing around map numbers.
_________________
(\_/) Beware
(O.o) ze
(> <) Hoppentruppen!
 
View user's profile Send private message Send e-mail
Gamma
Agent
Agent


Joined: 01 Aug 2009
Posts: 154

 PostPosted: Mon Jul 01, 2013 3:00 pm    Post subject: Reply with quote Back to top

Non ladder maps and obs maps are extra/unnecessary.
 
View user's profile Send private message
zoinkity
007
007


Joined: 24 Nov 2005
Posts: 1684

 PostPosted: Fri Jul 05, 2013 2:47 pm    Post subject: Reply with quote Back to top

Just to let you know, I've got map insertion working programmically.

The hiccup now is that the map select menu locks when reading some of the text embedded in those maps. I'm searching for the display rules and will make up something to either bypass it or automatically alter the maps themselves to play nicely. It probably requires strict ASCII or something.

The maps themselves run fine if you access them indirectly. The problem is exclusively due to menu access.

Oh, there's one I'm not going to put in: Icarus. They used something to "protect" their map, placing a giant wrapper around it and mixing stuff up. It could theoretically run, but the problem is that the thing is over 100kb. That's not just obscenely large for a StarCraft map, it's well over the allocated buffer on the N64.


Finally got Micro Practice working! Loaded it in place of one of the training missions since they are set to use map settings. That's pretty cool the way you direct units to add certain amounts of other units to the field.
_________________
(\_/) Beware
(O.o) ze
(> <) Hoppentruppen!
 
View user's profile Send private message Send e-mail
Gamma
Agent
Agent


Joined: 01 Aug 2009
Posts: 154

 PostPosted: Fri Jul 05, 2013 7:45 pm    Post subject: Reply with quote Back to top

Good job!

For future reference, there are map unprotection programs that will remove the problem you had.

I think the coolest door your work opened is that modders could essentially bring existing RTS games to SC64, albeit keeping the controls, money system and game engine.

Off topic: Do you do any Super Nintendo hacks? I love Genghis Khan/Liberty or Death (historical strategy games). :Drool: always wanted to change what general controlled what spot. Know anyone or any programs for this rype of thing? XD.
 
View user's profile Send private message
zoinkity
007
007


Joined: 24 Nov 2005
Posts: 1684

 PostPosted: Fri Jul 05, 2013 8:14 pm    Post subject: Reply with quote Back to top

Only SNES games I ever tampered with were Tactics Ogre and Bomberman II. That was ages ago though.

[edit]
My BS map unprotection wasn't enough. Well, it's enough to get them running, but not enough for certain types of protection. Unlike the PC game, it doesn't copy incomplete map data into preallocated buffers. Oh no. The N64 game reads it out of the chk directly, so if you don't have a complete file (aka. CPP Test Protector v2.0, etc) it's not going to work.

[Just verified the only problem is with incomplete map data tables.]

So, I'm guessing if post 1 at any of these SC sites read along the lines of:
"Insert new maps in SC64! First get SCM..."
I'd be banned within 30 seconds ;*)

[edit]
Made up a list of what data is even read by the N64 version. It ignores all the same keywords the PC version does, plus one or two. [edit: VCODs can be stubbed! Joy!]

VCOD's keyword is only tested for existence. Data isn't read. STR is a call-on-demand thing, so reduced sized tables work fine. TYPE is read if it exists but only returns a True/False result which may or may not be used. That's a lot of reductions.
Maps are sized precisely, so the masking trick doesn't work (protection method). Maps are read from the file directly and not placed in a buffer, so all terrain has to be fixed if protected. AI automaps if the mapping data is correct. Compression protection methods appear tied to compression within the MPQ itself and not internal so, well, it doesn't apply. Password protect was probably just zip/gzip/etc.'s own password protection system.
Screw testing them.
I'll just fix the output from SMC and shovel that in.

Currently testing each map and repairing terrain as required. All 35 maps. Aegh...

NSCE unlock cheats (via bitflags)
Code:
810D13D2 xxxx
   4000   Fog of War Off
   2000   Open Tech Tree
   1000   All Research
   0800   Unlimited Mana
   0010   Give Me Cash/Gas
   0008   All Upgrades
   0004   Special
   0002   Mega Build



[edit: bots work with proper mapping data]
Max #players is affected by remaining memory. Yes, it scales it based on available ram! So, Hunters_KeSPA is nerfed down to 5 players unless something interesting is done to the map itself.

Quick question: right now I haven't done anything to try to extend the maximum number of maps. (Pretty sure it's possible) If it isn't possible though, do you have any particular preference of which maps absolutely must be in without a doubt?
HBR and Python are (over)played enough on PC to merit them being in the N64 version. Any others?
_________________
(\_/) Beware
(O.o) ze
(> <) Hoppentruppen!
 
View user's profile Send private message Send e-mail
zoinkity
007
007


Joined: 24 Nov 2005
Posts: 1684

 PostPosted: Sun Jul 07, 2013 7:52 pm    Post subject: Reply with quote Back to top

I've edited that last post to death.
Here's a proof-of-concept patch. All non-scenario MP stages are replaced. It's not final! It's just a little demonstration and bugtesting it will help iron out further import problems.

http://www.mediafire.com/download/udiitbbyiqlzwkn/POC-ICCup-NSCE.zip

Patches provided are in xdelta and bps format. If you have trouble with one the other should work.

[note: For some insane reason I wrote 'SC' instead of 'SQ' for internal names. That will be fixed later.]


Read the readme for details on what's changed and known issues.

I haven't updated the stage images yet, and have a clever method of generating them in the works. Should be as simple as forcing the minimapper to load it on the menu screen ;*) (Once I trace it of course)

Trivia point: this patch shrinks the size of the BOLT! All unread keywords and unused data are either removed or stubbed.
_________________
(\_/) Beware
(O.o) ze
(> <) Hoppentruppen!
 
View user's profile Send private message Send e-mail
Gamma
Agent
Agent


Joined: 01 Aug 2009
Posts: 154

 PostPosted: Mon Jul 08, 2013 3:46 pm    Post subject: Reply with quote Back to top

I will test that as soon as possible.

The map preferences aren't too important but I would recommend leaving out 2 player maps if you run out if room.

I have a very moronic question. How does one replace files like maps on their own? Thanks.
 
View user's profile Send private message
zoinkity
007
007


Joined: 24 Nov 2005
Posts: 1684

 PostPosted: Mon Jul 08, 2013 4:59 pm    Post subject: Reply with quote Back to top

I'm still writing the program interface. Originally it was just replacing files, but now that I know how much conversion is needed it will have to be automated a little. I'll provide it as soon as it's done.

Map protections that do horrible things to the terrain need to be unprotected. So yes, SMC would be step one in those cases. Fake commands just waste space but can be parsed.


I'm doing three things right now.
1) tracing the minimap generator. That's going to generate the map previews so you won't need to fiddle around with image conversion, etc. to insert those.
2) tracing the hardcoded ranges for the map select.
3) rewriting the rebolter to automate certain kinds of file replacement. Basically, everything I know the format for or have a filename attached to it.


+_+

About the format:
BOLT files were used in Mass Media's four N64 titles, named after their development system.
Code:
BOLT file header format
0x0   4   b'BOLT'
0xB   0x1   #entries; if 0, 0x100
0x10   10ea.   entries
Entry format:
0x0   1   flags
   08   entire subdirectory uncompressed
0x3   1   #subtable entries; if 0, 0x100
0x4   4   size
0x8   4   offset
0xC   4   RESERVED

Subtables are all dedicated to a certain kind of file, possibly based on their original folders or something. For instance, all maps and map select images are in idx 8.
Code:
Subtable format:
0x0   1   compression flags
   00   LZ
   08   uncompressed
0x1   3   type
   0xA   No further conversion(?): text files, system menus, stages, scripts, etc.
   0xE   palette part of 16ci256 image
   0x12   image part of 16ci256 image
0x4   4   decompressed size
0x8   4   offset   (from BOLT header)
0xC   4   filename checksum(?)

I don't have a complete table of file types. Also didn't figure out the entire hash algorithm (just most of the last byte).

I've already posted a compression method.
LZ decompression routine, preferably with decompressed size known:
Code:
        l, b, cnt = 0, 0, 0
        # This catches the stream when dec_s isn't provided
        try:
            while True:
                if dec_s and len(output) >= dec_s:
                    break
                cmd = next(d)
                if cmd<0x80:
                    # (cmd&0xF) NOR (b<<4)
                    b<<=4
                    b|= (cmd&0xF)
                    b = len(output) + (~b)
                    l<<=3
                    l|= (cmd>>4)
                    l+=cnt
                    for i in range(l+2):
                        v = output[b+i]
                        output.append(v)
                    l, b, cnt = 0, 0, 0
                    continue
                elif cmd<0x90:
                    l<<=4
                    l+= (cmd&0xF)
                    for i in range(l+1):
                        output.append(next(d))
                    l, cnt = 0, 0
                    continue
                elif cmd<0xA0:
                    l = cmd&3
                    cmd>>=2
                    b = cmd&3
                elif cmd<0xC0:
                    l<<=5
                    l|= (cmd&0x1F)
                else:
                    b<<=6
                    b|= (cmd&0x3F)
                cnt+=1
        except StopIteration:
            pass
        return output


Actual changes are simply taking a .chk out of its .scm/.scx and replacing and existing file with it. If you ran out of IDs, you can add a few to the subsection. Maps are in idx 8.
Replacement requires looking up the offset to idx 8 in the header section. Goto BOLT+that offset. Look up the specific map's entry in the subtable. #69 is the first one in melee. Goto BOLT+that offset. Replace file. Update that entry's decompressed filesize, and if you have to move memory you'll need to update all the fileoffsets in all following subentries and header entries. (That's where the automation helps!)

I'll release the thing when I'm done with it though.

[Update]
Little update, since it's been over a week. Just worked out how minimaps are generated. It uses a set of four loaders, each tied to keyword lookup. The only real trouble is that MTXM segments call two routines specific to gameplay.asm as opposed to menus.asm. I'll have to relocate some code, in other words.
Manually forcing it to generate a map on the select screen (skipping a few things, settings a few variables manually, and a writing a few extra JALs) will work and has worked in trials. Now to just hardcode everything magically ;*)

In other news, you can completely remove VCOD dependency with 1 byte! As it was they didn't verify it, but with a single byte you don't even have to include a stub. Guess file protection was the least of their concerns ;*)

For how badly they did the graphical assets the cod is surprisingly sound and coherent. Unlike most companies they definitely looked at what their compiler was producing.
When it came to reducing bitdepth or shrinking images though, that was done just down-right awful. I think the reduction forced MS's default 16 color palette instead of a 16 color palette, if you catch my drift.

[update]
Found those hardcoded values for # melee and nonmelee maps. That means there's no map limit except ROM size now ;*)

Interestingly, they built in a doodlie that scans maps to determine if they have enough players for team play. [edit]TYPE is an optional catch, returning True or False if found. This makes determining which map to provide very, very easy.

[edit]A new bolt entry will be used for all new maps, with the option of spilling over into additional entries. This pushes the map cap into the ridiculous category. As it is 256 maps is crazy enough without lifting the limit entirely.

Automatic minimap generation is coming along. Some basic dependencies are handled, so now the only real problem is unhooking the existing display method. You'd think it would be simple, except the microcode for display differs significantly.

[UPDATE]
Redesigned the tool several times in light of the hacking side of things. The holdup now is on the hacking side of things, removing potential issues and hopefully automating some things.

Also, the sound file is known and added (or will be) to SubDrag's sound editor. Oh, and it's interesting to note the tilesets use 4x4 instead of 8x8 tiles but otherwise are identical.

As a side note, there's a way to quarter the size of a minimap in-game. It makes you wonder if they were working on 4-player support.
_________________
(\_/) Beware
(O.o) ze
(> <) Hoppentruppen!
 
View user's profile Send private message Send e-mail
zoinkity
007
007


Joined: 24 Nov 2005
Posts: 1684

 PostPosted: Mon Jul 29, 2013 8:52 am    Post subject: Reply with quote Back to top

New POC patch:
http://www.mediafire.com/?udiitbbyiqlzwkn

Has all the ICCup stages, using tool to do it. #stage limit has been altered, whereas final will remove it. It also relocates the BLTN sound bank.
Still uses a placeholder image for map preview, but have a scheme to do minimap gen. This still limits the new maps to ~188 or so, but final scheme uses a new entry to exceed this.

Note this changes the CRC, so if you're on an emu that cares about that add the new one in.

Anyway, for now enjoy the full map set. Hacking end is holding things up.

[edit]Known bug: any replay that happens to use an original MP game as a base...well, it may or may not work.
Did figure out replay format though. It's the same as a saved game, clumped with a bunch of padding, and a few map-specific values (specifics known) and more padding.
_________________
(\_/) Beware
(O.o) ze
(> <) Hoppentruppen!
 
View user's profile Send private message Send e-mail
Gamma
Agent
Agent


Joined: 01 Aug 2009
Posts: 154

 PostPosted: Sat Mar 07, 2020 8:49 am    Post subject: Reply with quote Back to top

Zoinkty this still interests me. You still active?
 
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    ShootersForever.com Forum Index -> Game-On! All times are GMT - 8 Hours
Goto page Previous  1, 2, 3
Page 3 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Cobalt 2.0 BB theme/template by Jakob Persson.
Copyright © 2002-2004 Jakob Persson


Powered by BB © 01, 02 BB Group

 


Please Visit My Other Sites: GoldenEyeForever.com | GrandTheftAutoForever.com

Got kids? Check out my Dora The Explorer site with games and coloring pages!

Our forums feature Nintendo 64 games, GoldenEye 007 N64 New Maps and Missions, GoldenEye Cheats, N64 Emulator, Gameshark, GoldenEye Multiplayer and more!

[ Privacy Policy ]