Total War: Rome II Review

I'm really late to the party, but this game had such awful launch, I delayed it a bit, and then I was busy, so here I am, reviewing a 6 year old game.

It is not Rome I

At first I tried to play it like it's basically Rome I with better graphics and weird settlement system, and that really doesn't work.

Battles work more or less like in previous Total War games, but on campaign level it's definitely not so. The biggest difference is that you literally can't have any troops not attached to a general, and limit on number of armies you're allowed to have is quite low.

This breaks a lot of common patterns like recruiting more units and sending them to frontlines, leaving some units behind as extra garrison, or to protect settlement from rebels, splitting big army in half to clean up multiple leftover AI troops, or separating a single unit to send it ahead to scout.

There's a new system of provinces made out of (usually) 3-4 settlements. This really cuts on micromanagement. A very interesting thing they did is that province's capital settlement has walls, but none of its minor settlements do. This completely avoids the problem Medieval 2 had, where 80% of battles were assaults on walled settlements, which might have been historically accurate, but not terribly exciting.

Autoresolve all the things

A surprising and very welcome change is autoresolve not being ridiculously biased against the player, like it was in all previous Total War games. In Empire I'd sometimes try to autoresolve a trivial fight where I had 10:1 advantage and I'd likely wipe out the enemy completely without a single casualty, only to be told that my army lost. None of that here.

Even if win was guaranteed, manually fighting all trivial battles was necessary because reinforcing required getting back to high level settlement (in Rome I and Medieval 2), or paying ridiculous amounts of money (Empire).

Here, army losses are surprisingly inconsequential. Armies reinforce for free, and quite fast, so as long as none of the units in your army get completely wiped out. This was introduced back in Napoleon, but together with reasonable autoresolve, it means there's no point in fighting most one-sided battles. You'd mostly fight close battles, which are far more interesting.

Female generals drama

For my first campaign I took the obvious choice of Ptolemaic Egypt. I don't really give much shit about Western Rome, Constantinople the only true Rome etc. Back in Rome I, Egypt was infamous for being ridiculously ahistorical, with bronze age armies thousand years out of their time. In fact "Egypt" by that time was a Greek kingdom, with the usual Hellenistic armies of heavy phalanx supported by skirmishers and light cavalry.

Now Egypt has a reasonable unit roster. Except all the generals I can hire are female. I was quite baffled by that, and thought that maybe they're some family members, which would maybe be excusable, but nope, they're just some total unrelated randos. WTF?

So it turns out there was this big drama, about 5 years after release, Rome II silently pushed a patch which added female generals, at ridiculously high spawn rates, to all factions including those which had absolutely no business having them. And then instead of toning it down to reasonable levels, and just to factions where it would make some sense, or at least making this silliness optional, devs went full "fuck you all, don't like it, don't play the game" mode. They got very well deserved Steam review bombing for it, but did not learn their lesson.

It's shockingly different from how well a game like Crusader Kings 2 handles gender. Playing a king is quite different from playing a queen, different cultures and religions handle status of women differently, and when you start a new game you can choose a few options to expand state of women. Or if you reform a religion.

Immersion failures continue

One really annoying thing about historical Total War games is that they start by hiding the whole map except your country and its immediate neighbours. It's good that I remember what the map looked like, so I can play based on that. And it turns out Rome II map has very little to do with actual history. Seleucids are really tiny!

In 272 BCE Seleucids were basically half the map, a mega-Persia stretching from Western Anatolia through Mesopotamia, Persia, all the way to a chunk of Central Asia and Afghanistan and Western India. Instead they have 6 settlements on Syrian coast and a bunch of vassals. Wat?

Let's talk immersion. People play historical games for the same reason they watch 22nd Avengers movie. They have connection with historical countries or established characters. I've heard Shogun 2 is a good game, but I've never played it because I don't give a fuck about all the Shimazus, Takedas, and Uesugis. Who is that even? EU4 has about 400 countries, but it turns out over 60% of games are just top 15 nations. Only half of these are even that strong.

It's just so much more fun to immerse yourself in all those historical conflicts. Playing Byzantium in EU4 is borderline masochistic, and yet 1 in 40 of all games is someone trying to stop the kebab menace and restore the glory of Constantinople. People even make mods for restoring Byzantium in HoI4. It's great to also have Hins Kayfa and Tannu Tuva for people who are looking for their 100th campaign, but even these campaign feature mostly well known countries as key antagonists and NPCs.

What does it have to do with it all? When you start with historical setting, or established fictional setting for that matter, you have a budget for how much you can change before people go "fuck this shit, it's not the Harry Potter I love". And many things already demand a chunk of this budget. Better gameplay or technical issues will require breaks with history. Tiny Seleucids might very well be good for the game. Being historically inaccurate to increase the coolness factor is good use for the budget. Being true to people's perception of history rather than actual history (like Rome I Egypt) is fine use of the budget. Every MCU movie needs to spend some time introducing new characters viewers don't care for yet, that uses up part of that budget.

Go too far, and break immersion stupidly, and you get backlash. Even most beloved universes like Star Wars and Harry Potter have breaking point. For historical games this budget is a lot lower. Blowing up a big chunk of immersion budget on something as stupid as forcing female generals on Greek and Roman factions is just so fucking dumb. Not listening to the players is even dumber.

Interface prioritizing minimalistic style over functionality

Anyway, back to the game. Older Total War games had big interfaces where all relevant information was always easily accessible. Rome II instead uses minimalistic highly stylized interface, with completely meaningless icons without text, and where information is hidden behind multiple layers of tooltips, or requires alt tabbing to a wiki. Like, how do I know which buildings I can build in a settlement once I expand it? As far as I can tell, there's no in-game way at all.

From UX point of view that's just atrocious. If you play a lot and don't mind alt tabbing to wiki, you'll get over it eventually, but your first few campaigns it will be a constant pain.

In older games it was really easy to understand what's going on. When game gave me the choice which building to construct, or which unit to recruit, all relevant information was there. In Rome II it's just not there. What's the difference between those two units? Here's 10 sliders, have fun figuring out what they mean. How much money will this building generate compared to that other building? Can anyone even figure this out without alt tabbing to Excel?

This complexity doesn't make game deeper. On the contrary, without any clear information what which choice does, players will either pick at random, or just read somewhere what's the optimal choice, and in either case they'll make no meaningful choices during gameplay.

One interface issue that is highly problematic every single time is that routing units are basically invisible during battles, and chasing them is very important. Giving them white flags like in previous games would be such an obvious improvement.

Bugs 6 years after release

It really did not help my first impression of the game that during the first tutorial siege, AI army hit some invisible wall in the settlement, and just stood there stuck. I tried to attack them, but my armies were also staring at invisible wall in the middle of some street. I finally figured out that if I take my troops out of the settlement and walk in from same direction AI took, I can fight them. It was the only bug I encountered so far (not counting Greek female generals, which are arguably a bug), but wow, those were not good first impressions.

Politics stuff

Rome II has whole extra layer of managing politics of your faction, with other families, something like 40 interactions, civil wars, senate, and so on. It's not clear what all of that does, and so far I've been mostly ignoring it, and it seems to be fine to ignore it.

One baffling thing is that I can't find any options for getting my children married. Maybe all the women joined the army, so there's nobody left to marry?

Overall

Battles are great. Maybe comparing your best Medieval 2 battle to best Rome II battle, Medieval II still wins. But thanks to autoresolver and reinforcement changes getting rid of most one-sided battles, much better mix of walled sieges / unwalled sieges / field battles, and how well battles play, I'd say that a median Rome II battle is more fun than median battle in any previous Total War game.

Campaign changes reduce micromanagement, but consequences of the choices are much less clear, so it's a bit mixed.

Interface is just plain bad. It prioritizes style over functionality far more than is reasonable.

Immersion is mostly fine. I'm reasonably tolerant, but I'll get a mod to fix the biggest silliness for my next campaign.

Game performance is so far totally great. It runs better than Rome I on my hardware.

I had horrible first impressions of the game, but mostly positive second impressions.

Adventures with reverse engineering - Total War DB tables

Nearly five years after it all started, every single DB table for all Total War games is now decoded. I thought it would be a good idea to write about the process as reverse engineering case study.

If you find this interesting, you should probably also check my post about general principles of data archeology.

What are DB tables anyway?

Like most games these days, Total War games (Empire, Napoleon, Shogun 2, Rome 2, and various expansions for them) use virtual file system to store their data.

Within this virtual file system data is mostly of those kinds:

• standard formats for things like textures, sounds, text - no need to reverse engineer those
• ESF format - it's sort of like binary XML, and I documented it previously here
• Various one-off formats, usually very simple
• UI files - still not fully decoded, they control user interface
• DB tables - sort of SQL-like tables

There can be multiple files for same DB table - they are merged by virtual filesystem level. Generally one of the fields in a row is primary key, to decide if DB tables in later patch override or append to original patch's DB table.

DB table structure

DB tables contain bulk of the data. The format itself is as simple as possible:

• (optional) GUID
• schema version number
• number of rows
• first row
• second row
• ...

Every row within same table version has same schema. Whenever schema changed - usually to add extra rows - version number would be increased.

That sounds like pretty easy reverse engineering job. There are only a few problems:

• schema is not stored with the data, or anywhere else I'm aware of - the game code knows it, but we need to figure it out
• fields are stored as raw data, without any kind of type prefix
• rows might have same schema, but many fields are variable size, so we don't know where one row ends and another begins

What types are we working with anyway?

As is usually the case for data formats created for one format, they just go for the simplest thing that could possibly work, so data is mostly just a few things:

• int32
• float32
• boolean
• length-prefixed string
• nullable length-prefixed string

That's a very small set, and a few simpler DB tables can be solved entirely by eyeballing them in hex editor.

Sadly, the following 00 00 00 00, can mean any of:

• 1x int32 - 0
• 1x float32 - 0
• 4x boolean - false, false, false, false
• 2x unicode string - "", ""
• 4x nullable unicode string - NULL, NULL, NULL, NULL

That gets exponentially worse when there's a block of tons of NULLs. 12 consecutive 00 bytes can be any of 50559 schemas using just those 5 types, and we don't even know if we're still in the same row or another one started.

Is type T likely at offset N?

It was time to reduce it to a simpler question - can type T be at offset N? That's a stupid question, as the answer is generally yes. For example if next 4 bytes are: 01 00 59 00, that can mean any of: 

• int32 - number 5832705
• float32 - number 8.173360559359682e-39
• booleans - true, (and 3 more bytes)
• unicode string "Y"
• some 22784-character nullable unicode string (first one some kind of +Uxx00)

Most of them are pretty unlikely. So let's instead ask a question - is type T likely at offset N?

• integers over 100000 or any negative integers other than -1 are unlikely
• floats above 10000.0 or below 0.001 are unlikely
• all possible boolean values - 00 and 01 - are considered likely
• unicode strings with any non-ISO-8859-1 characters or above 256 characters are unlikely
• nullable string is likely if it's NULL (00) or 01followed by likely string.

That's a decent way to quickly evaluate a schema I guess. If I think the file may possibly be "int32, float32, nullable unicode string, boolean, boolean", and I know number of records from the header, I can quickly check if it parses, and contains sensible values.

Brute force time!

Now that I have a way to answer the question if a schema matches, why not just brute force every schema shorter than 10 columns or so? And so I did. And it decoded a lot of tables.

Unfortunately monstrosities with 20+ are not uncommon. The largest one had literally 184 columns (and there was no way to know how many it would end up with just raw data). That's definitely beyond brute force range.

It was time for a few more advanced techniques:

• the most common ambiguity was 00 00 00 00 which was either float 0.0 or integer 0. It was of course important, but it drastically reduces schema search space to treat them both as one type (int32_or_float32) - and guess in postprocessing which one it actually was.
• sometimes values I considered unlikely happened - like very large integers (for some IDs) and very long strings (help messages) - so for many tables I had to adjust likely heuristics based on eyeballing what was inside

The biggest problem with all that was that I pretty much had to get it all right. If I had row boundaries I would be able to make good guess for column 1, then column 2, and so on. Without row boundaries, I know where column 1 of row 1 starts, but column 1 of row 2 could be pretty much anywhere.

In a fortunate accident, a lot of tables started with primary key. Which was usually a non-nullable string. Which was really easy to spot. Even the 184-column monstrosity (models_artillery_tables) just so happened to start with a string. Followed by another string, and then constant number of bytes for rest of the row. To figure out exact mix of floats, integers, and booleans in rest of the row I had to write special script, but row boundaries solves most of the problem.

Quite often I had more complex situation - I could figure out where row starts (with non-null string primary key), but the rest of the row was not fixed size. In such cases I'd just guess the first few column types, and get my program to brute force the rest.

models_buildings

With this mix of eyeballing hexes and autodetection script I managed to figure out most schemas, but two really didn't work. As it turned out, both used schemas completely unlike any other DB table.

Table for building models was unique, but it relatively easy to figure out manually with hex editor, each row being:

• string
• string
• int32
• number of elements (as int32), each of them being:
• string
• int32
• 9x float32 (3x XYZ vectors)

models_naval

That left just one table which was both unique and insanely complicated. I tried a bunch of times, but it was just really difficult.

My entire approach was based on figuring out whole schema at once - so it was not useful that I could see bits and pieces of structure.

I tried a bit, but eventually I gave up.

Many years later, Creative Assembly released modding kit for Shogun 2, and with it an example ship as XML. It was fairly frustrating as it was just a single model and it didn't have any cannons, but it contained a lot of hints as to structure of the data.

Another great thing was that for some weird reason Shogun 2 contained main table with a lot of ship models, but also 11 DB files with one model each. That's amazing - instead of trying to figure out 2.2MB file Empire had, I could try matching individual 80kB or so models.

Even then, my approach of full schema match wouldn't work - I knew the format was too complex for it from my previous failed attempts.

Instead I started by fishing for strings, converting the file to output like:

• bytes 100..107 are string "foo"
• bytes 108..119 are something that's not a string
• bytes 120..127 are string "bar"
• ...

Well, that's a good start. Then I applied similar fishing to float32, int32, and everything that remained. This decoding contained a lot of errors (00 00 00 00 00 was sometimes "0, false", and something "false, 0" and the script completely messed that up), but it a lot of structure was visible, and there were loads of strings to work with.

Then came the breakthrough. I added custom matchers depending on value of a string. If a string started with word "cannon", try matching it with whatever I guessed to be cannon model. If it was "efline", go for guessed efline model. And so on.

Soon I'd get to:

• int32 4
• cannon: {cannon 1 data}
• cannon: {cannon 2 data}
• cannon: {cannon 3 data}
• cannon: {cannon 4 data} 
• int32 15
• efline: {efline 1 data}
• ...

Important thing was that those custom matchers operated on raw bytes (and falled back to lower level matchers if their match failed) - so they weren't at mercy of autodetector for raw 00s and such ambiguous data.

There were bits of weird stuff leftover, but surprisingly little.

Then I reran autodetector on data from Empire, Napoleon, and Rome 2, and it 90% fit, with fairly easy adjustments to make it fit completely.

And so it ends

With hindsight, naval models weren't quite as bad as I thought during my first attempt, and I probably could have figured them out even back then if I kept trying.

This still leaves a few things undecoded - mostly UI files for Shogun 2 and Rome 2. Perhaps I should give them another go as well.

Total War: Rome II or why professional game reviews are all worthless

A few years back I was really annoyed by how overhyped Empire Total War was:

In case you had some hopes they got better:

It doesn't surprise me that another Total War game is a disaster at release time - I hope they'll fix the worst issues over the next year, and I'll take a look at it then. But are these reviews intentional ad-driven dishonesty or just pure incompetence?

Highlights from CA modding summit

This is a very late post since the summit was three weeks ago, but better late than never.

I did some liveblogging during and immediately after the summit, if you want every detail, go to these three places:

• TWCenter thread about the summit (I'm linking to the comment where I started liveblogging, there's ton of worthless flamewars and other irrelevant stuff in the thread as well)
• Thread on /r/totalwar reddit.
• Daniu's liveblogging therad on TWCenter where I also blogged a bit.

Here I'll summarize the most important things. If you have any questions, ask away.

Modding Kit on Steam

There's now a bunch of modding tools released for Shogun 2 only, and available on Steam. Included is some way to build campaign maps, most of models, and some of minor formats etc.

There will also be Steam Workshop integration for easier distribution of mods.

It won't work on Empire or Napoleon, but it might be useful in reverse engineering them.

There now also official wiki. Only time will tell if it attracts much information about modding. The problem was too many wikis and fora, not too few of them. (relevant xkcd)

Still missing

There was nothing about any game before Empire, and very little about Empire/Napoleon.

Even for Shogun 2, a lot of formats are still missing.

There's no information about UI layout files (which we partially decoded), sources for .luac files (which we can partially decode), sound bank (again, partially decodable) etc.

One minor thing I asked for was .sav file format used by Rome and Medieval 2, but they don't have any documentation. I made some effort to decode it recently, but to be honest my hopes are not terribly high.

Another interesting thing CA guys said was that all damage calculations figures for all games community reverse engineered are wrong, and real calculations are far too complicated to release.

Rome 2

There was a short video of Rome 2 battle, short fragment of which later released to the public.

Judging from the video Rome 2 will have:

• ridiculously huge battle maps
• much more people per unit
• surprisingly good unit pathfinding in complex streets, serious weakness of previous games
• naval invasion as part of land battles, including sieges
• much more interesting siege equipment on both sides, there's even some partial defensive palisade attackers can hide behind
• full screen tactical map (bird eyes's view) - I'm surprised it took them so long, it was badly needed, especially in naval battles

Rome 2 will still be a 32-bit only game. They have 64-bit build, but don't plan releasing it. Feel free to spam them with requests of course. :-p

An interesting technical hint was that they're no longer using .lua for Rome 2, but who knows really.

There was also a mention of builtin console feature for debug build, I really wish they shipped with it enabled by some config flags, but my hopes aren't high.

Planned release date is a bit before Christmas 2013.