View unanswered posts | View active topics It is currently Sun May 27, 2018 11:02 am



Reply to topic  [ 3 posts ] 
 Mod Concatenator (v1.0 15:52 GMT 05/12/18) 
Author Message
happy carebear mom
User avatar

Joined: Tue Mar 04, 2008 1:40 am
Posts: 7088
Location: b8bbd5
Reply with quote
Post Mod Concatenator (v1.0 15:52 GMT 05/12/18)


Changelog:
  • v1.0: Release

Sup.
The illustrious CrazyMLC got me interested in concatenating script files in RTEs to make them load faster, so I made a little python script to do so. In my testing I found that load times are primarily held back by the actual load screen rendering itself, but the script does help a bit. You can check my data below and try it out yourself.

Attachment:
File comment: v1.0
concatenator.zip [1.33 KiB]
Downloaded 3 times


You will need at most the standard python distribution. I programmed it in 3.6 but I believe it should work fine in 2.7 as well, I don't use anything specific to either version as far as I'm aware. Run python from the command line to use it.

usage wrote:
python the_concatenator.py [-h] [--output OUTPUT] [--find-imports] filename

positional arguments:
filename The file to start concatenating to; will be backed up

optional arguments:
-h, --help show this help message and exit
--output OUTPUT Change the output; default Index.ini.new
--find-imports Instead of writing anything, just find the include lines
and print them out


Generally it looks like the following
Code:
cmd> python ./the_concatenator.py ./Base.rte/Index.ini
Writing to ./Base.rte\Index.ini.new
Successfully wrote file; traversed 310 includes and concatenated 97937 lines in 0:00:00.232592


Then you can move Index.ini to Index.ini.old (or delete it entirely) and rename Index.ini.new to Index.ini

I used DSTech.rte as my guinea pig, and then expanded to the normal RTEs included with the game. It works fine on all of them except when IncludeFile directives are inside block comments. The script does not handle block comments in any special way, so it will blindly attempt to follow any IncludeFile directives inside them. In particular, Missions.rte has a block-commented-out section in Missions.rte/Scenes.ini that includes some debug inis that do not exist, so the script will fall over.








Without DSTechWith Old DSTechWith New DSTechWith New All
Samples36.515.414.48.3
11.915.614.29.0
11.915.714.58.7
11.815.714.98.9
12.016.014.48.9
Median11.915.714.48.9
Speed (+ slower)31.93%21.01%-25.21%


In my testing, however, loading screen draw time was the biggest time sink. On my machine, loading from a 1TB 7200RPM hard drive, normal loading time at 1080p was around 12 seconds as shown in the table above. Changing the resolution Cortex drew at from 1080p to 640x480 improved load times to around 1.5-2s, too fast for me to accurately measure to 0.1s.


Sat May 12, 2018 5:15 pm
Profile
User avatar

Joined: Fri Dec 22, 2006 4:20 am
Posts: 4769
Location: Good news everyone!
Reply with quote
Post Re: Mod Concatenator (v1.0 15:52 GMT 05/12/18)
Here's a version that'll work just with a click, no python installation required.
It works differently than Duh102's version and I've seen load times almost halved when used.

The .zip only needs to be extracted into the same folder as Cortex Command, the same way you might with a .rte.

Download Here.
_________________________________________________________________

compile.exe - Compiles the code of each mod into one file with no comments or whitespace, decreasing load time. Can run the game after it finishes.
uncompile.exe - Conveniently uncompiles each mod, restoring them to their original state. Can run the game after it finishes.
benchmarker.exe - Will accurately measure the time it takes to load your current mods, and save a log file to the benchmarks subfolder. Use this to find out how much time you're saving by compiling! (Runs 5 times, and includes an average load time in the log)
_________________________________________________________________

compile.exe is intelligent and only does its thing when it has to; it'll skip up-to-date, already compiled mods. This means you can use it as a sort of launcher for the game, and it'll automatically make sure you're loading as fast as possible. Feel free to replace your Cortex Command shortcut with a shortcut to this exe.

Additionally, there's a file called CompileSettings.ini in the folder. This can be used to control aspects of compile.exe and uncompile.exe:
Code:
RECOMPILE - If this is True, already compiled mods will be recompiled. This is False by default, but you can turn it on if you want to test out how long it takes to compile all of your mods.

RECOMPILE_IF_ORIGINAL_MODIFIED - What it says on the tin. If this is set to True and the backup of the original Index.ini has been modified since the last compile, compile.exe will recognize it and recompile the code. True by default.

VERBOSE - 0 will show only critical information, 1 will show a bit more, 2 will show everything. 2 by default.

CONDENSE - Removes whitespace if True. True by default, but you can turn it off to potentially make the compiled code a little more readable. True is optimal for loading speed.

RUN_GAME - compile.exe and uncompile.exe will run Cortex Command after they finish if this is set to True. True by default.

CLOSE_DELAY - The time in seconds that compile.exe and uncompile.exe will wait after completion before they close. Set to 0 to get them out of your way, or set it to a high number to read at your own pace. 5 by default.

BLACKLIST - You can list rte names here. Listed rtes won't be compile, which is useful if you're working on a mod and you want the error messages to be meaningful. Names are comma separated, like so:
BLACKLIST = Base.rte,Coalition.rte

Python source files included.

Full release with all the trimmings on steam and mod releases section pending me remaining interested in this project. Let me know if you find any bugs.
If the compiler manages to break any of the mods somehow, delete the broken mod and reextract it from the steam workshop folder or have CC redownload it. If it's a base game rte, then try verifying the install through steam.
_________________________________________________________________

known issues: gdi.rte somehow manages to ruin everything. (????) and eventhorizon.rte breaks the comment handling code. All other mods I've tested seem to work fine, including every mod on the steam workshop. If you have these mods, or if any other mod breaks the compiler, add them to the blacklist.

With every mod on the steam workshop, my load times at 640x480 went from 43 seconds uncompiled to 32 seconds compiled. Each mod varies in how many lines of code it has, and how many files that code is in, and thus benefit to varying degrees from being compiled. Most large faction mods have their load times greatly improved by the compiler, while smaller mods blunt the effect.

For example, with just the base game and a couple faction mods (untitled, wh40k, saw) my load times at 640x480 went from 12 seconds to 7 seconds.

Higher resolutions definitely do increase load times, as Duh102 mentioned, because of flaws in CC's rendering. For me, load times at 1920x1080 are roughly tripled for both uncompiled and compiled.


Wed May 16, 2018 3:08 am
Profile WWW

Joined: Mon Dec 21, 2015 9:30 am
Posts: 79
Reply with quote
Post Re: Mod Concatenator (v1.0 15:52 GMT 05/12/18)
i have no damn clue what this is but i love y'all for still making ♥♥♥♥ for this game


Sun May 20, 2018 7:15 pm
Profile
Display posts from previous:  Sort by  
Reply to topic   [ 3 posts ] 

Who is online

Users browsing this forum: No registered users


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by STSoftware for PTF.
[ Time : 0.020s | 16 Queries | GZIP : Off ]