|Data Realms Fan Forums
|Mod Concatenator (v1.0 15:52 GMT 05/12/18)
|Page 1 of 1|
|Author:||Duh102 [ Sat May 12, 2018 5:15 pm ]|
|Post subject:||Mod Concatenator (v1.0 15:52 GMT 05/12/18)|
This work is licensed under a
Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The illustrious CrazyMLC got me interested in concatenating script files in RTEs to make them load faster, so I made a little python script to do so. In my testing I found that load times are primarily held back by the actual load screen rendering itself, but the script does help a bit. You can check my data below and try it out yourself.
File comment: v1.0
concatenator.zip [1.33 KiB]
Downloaded 51 times
You will need at most the standard python distribution. I programmed it in 3.6 but I believe it should work fine in 2.7 as well, I don't use anything specific to either version as far as I'm aware. Run python from the command line to use it.
python the_concatenator.py [-h] [--output OUTPUT] [--find-imports] filename
filename The file to start concatenating to; will be backed up
-h, --help show this help message and exit
--output OUTPUT Change the output; default Index.ini.new
--find-imports Instead of writing anything, just find the include lines
and print them out
Generally it looks like the following
cmd> python ./the_concatenator.py ./Base.rte/Index.ini
Writing to ./Base.rte\Index.ini.new
Successfully wrote file; traversed 310 includes and concatenated 97937 lines in 0:00:00.232592
Then you can move Index.ini to Index.ini.old (or delete it entirely) and rename Index.ini.new to Index.ini
I used DSTech.rte as my guinea pig, and then expanded to the normal RTEs included with the game. It works fine on all of them except when IncludeFile directives are inside block comments. The script does not handle block comments in any special way, so it will blindly attempt to follow any IncludeFile directives inside them. In particular, Missions.rte has a block-commented-out section in Missions.rte/Scenes.ini that includes some debug inis that do not exist, so the script will fall over.
In my testing, however, loading screen draw time was the biggest time sink. On my machine, loading from a 1TB 7200RPM hard drive, normal loading time at 1080p was around 12 seconds as shown in the table above. Changing the resolution Cortex drew at from 1080p to 640x480 improved load times to around 1.5-2s, too fast for me to accurately measure to 0.1s.
|Author:||CrazyMLC [ Wed May 16, 2018 3:08 am ]|
|Post subject:||Re: Mod Concatenator (v1.0 15:52 GMT 05/12/18)|
Here's a version that'll work just with a click, no python installation required.
It works differently than Duh102's version and I've seen load times almost halved when used.
The .zip only needs to be extracted into the same folder as Cortex Command, the same way you might with a .rte.
compile.exe - Compiles the code of each mod into one file with no comments or whitespace, decreasing load time. Can run the game after it finishes.
uncompile.exe - Conveniently uncompiles each mod, restoring them to their original state. Can run the game after it finishes.
benchmarker.exe - Will accurately measure the time it takes to load your current mods, and save a log file to the benchmarks subfolder. Use this to find out how much time you're saving by compiling! (Runs 5 times, and includes an average load time in the log)
compile.exe is intelligent and only does its thing when it has to; it'll skip up-to-date, already compiled mods. This means you can use it as a sort of launcher for the game, and it'll automatically make sure you're loading as fast as possible. Feel free to replace your Cortex Command shortcut with a shortcut to this exe.
Additionally, there's a file called CompileSettings.ini in the folder. This can be used to control aspects of compile.exe and uncompile.exe:
RECOMPILE - If this is True, already compiled mods will be recompiled. This is False by default, but you can turn it on if you want to test out how long it takes to compile all of your mods.
RECOMPILE_IF_ORIGINAL_MODIFIED - What it says on the tin. If this is set to True and the backup of the original Index.ini has been modified since the last compile, compile.exe will recognize it and recompile the code. True by default.
VERBOSE - 0 will show only critical information, 1 will show a bit more, 2 will show everything. 2 by default.
CONDENSE - Removes whitespace if True. True by default, but you can turn it off to potentially make the compiled code a little more readable. True is optimal for loading speed.
RUN_GAME - compile.exe and uncompile.exe will run Cortex Command after they finish if this is set to True. True by default.
CLOSE_DELAY - The time in seconds that compile.exe and uncompile.exe will wait after completion before they close. Set to 0 to get them out of your way, or set it to a high number to read at your own pace. 5 by default.
BLACKLIST - You can list rte names here. Listed rtes won't be compile, which is useful if you're working on a mod and you want the error messages to be meaningful. Names are comma separated, like so:
BLACKLIST = Base.rte,Coalition.rte
Python source files included.
Full release with all the trimmings on steam and mod releases section pending me remaining interested in this project. Let me know if you find any bugs.
If the compiler manages to break any of the mods somehow, delete the broken mod and reextract it from the steam workshop folder or have CC redownload it. If it's a base game rte, then try verifying the install through steam.
known issues: gdi.rte somehow manages to ruin everything. (????) and eventhorizon.rte breaks the comment handling code. All other mods I've tested seem to work fine, including every mod on the steam workshop. If you have these mods, or if any other mod breaks the compiler, add them to the blacklist.
With every mod on the steam workshop, my load times at 640x480 went from 43 seconds uncompiled to 32 seconds compiled. Each mod varies in how many lines of code it has, and how many files that code is in, and thus benefit to varying degrees from being compiled. Most large faction mods have their load times greatly improved by the compiler, while smaller mods blunt the effect.
For example, with just the base game and a couple faction mods (untitled, wh40k, saw) my load times at 640x480 went from 12 seconds to 7 seconds.
Higher resolutions definitely do increase load times, as Duh102 mentioned, because of flaws in CC's rendering. For me, load times at 1920x1080 are roughly tripled for both uncompiled and compiled.
|Author:||Hoovytaurus [ Sun May 20, 2018 7:15 pm ]|
|Post subject:||Re: Mod Concatenator (v1.0 15:52 GMT 05/12/18)|
i have no damn clue what this is but i love y'all for still making ♥♥♥♥ for this game
|Page 1 of 1||All times are UTC [ DST ]|
|Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group