taw's blog

The best kittens, technology, and video games blog in the world.

Wednesday, August 01, 2018

Challenges for July 2018 SecTalks London

Cat sitting on a desk by freestocks.org from flickr (PD)

Last month I ran another round of London SecTalks CTF.

I only created 6/9 challenges this time, 3 Android challenges were created by imhotep.

Challenge files and code used to generate them available on github.

There are no answers below, but some serious hints which might make it too easy.

For previous rounds, see posts about September 2017, November 2017 CTFs, and May 2018.

Hidden treasure (5 points)

As per tradition, there was a bonus challenge with a zip bomb. This time at the end of 16-deep archive with 16 branches each there were pictures with a treasure, with EXIF tags containing either the flag or information that you failed.

Android 1 (10 points)

The simplest Android challenge wasn't even all that Android specific - as the flag was hidden directly inside the .apk.

RE1 (15 points)

Easy reverse engineering challenge was just a 32-bit Linux binary, which verified passed flag character by character, in random order.

Disassembling it, you'd get instructions like:

cmpb   $0x74,0x10(%ebx)
jne    80484b7

If you arranged them correctly, you'd see the flag. Or you could just grep for all characters, and use anagram solver - apparently that was an option too.

Android 2 (20 points)

This one required actually running the app, or fairly complex static analysis. The app saved a file with the flag to device storage. If you can find the file, the flag is yours.

RSA (25 points)

This challenge was actually quite realistic.
Bob sent a message we need to decrypt.

Probably due to bad RNG, it looks like Bob and Alice picked same N for their keys, and we managed to steal Alice's private key as well.

Perhaps there's a way to take advantage of this.
And there is a way. Having private key (n, e, d) there's algorithm to factor n into p, q. It's not completely obvious, but it's fairly short to implement. With p, q (shared between both), and Bob's e, you can trivially get Bob's d. Then you can decrypt the message.

RE2 (30 points)

It was a slightly harder version of reverse engineering challenge. The only difference was that flag was encoded into Base64 before checking, which could throw people off, but binary wasn't stripped so b64_encode method was a massive hint.

Android 3 (35 points)

The challenge app is making HTTP request, which needs to be intercepted and modified to get the flag from the server.

Once you get the request it's obvious what to do, and there were hints provided how to setup proxy on Android to get them.

CTR (40 points)

Another somewhat realistic challenge. A collection of Elon Musk's tweets and a flag was encrypted using AES-128 in CTR mode. Reusing same IV for all of them.

This turns CTR mode into XOR with random keystream, and breaking XOR cipher is fairly basic. It turns out nobody remembered mode names, so I had to give people some hints before they tried CBC or CFB attacks on it.

Croatian Monoalphabetic (50 points)

This challenge was worth the most points, but in principle it was very easy, just a bit time-consuming - and I ended up giving people hints how to approach that.

It's just statistical analysis of another Latin script language. So you can start by assuming space is the most common character, then use character frequency tables for Croatian, or lists of most common words, or take advantage of the fact that flag (very long word) is embedded inside.

Sunday, July 29, 2018

Which MCU movies are worth watching?

The World's Laziest Superhero by www.metaphoricalplatypus.com from flickr (CC-BY)

Not a day goes by that I'm not disappointed by what Wikipedia turned into. It went all the way from trying to be "sum of all human knowledge" under "neutral point of view" to cutting everything not "notable" enough and just mindlessly parroting establishment view.

That's tolerable when the establishment knows what they're saying, like in science articles. Well, mostly. Unfortunately this policy gets extended to domains without anything resembling genuine experts, and nowhere is it more ridiculous than with "film critics".

It's one thing to mindlessly parrot some social scientist who fudges statistics and uses questionable methodology, but is at least pretending to do science. Treating someone speaking out of their ass as a "reliable source" just because someone gave them a newspaper column to publish at is a mockery of what Wikipedia used to stand for in its early days.

There is an objective measure of film quality - enjoyment by the audience. And we have pretty good metrics of that - like IMDB and Rotten Tomatoes audience score. Such metrics occasionally suffer from issues like vote stuffing, demographic imbalance of voters compared with movie watching audience, less representative sampling for very old or foreign movies, and occasional overrating of most recent movies before it regresses to the mean, but it's insanity not to take these aggregates as starting points.

Even these issues are mostly overstated. IMDB filters handle vote spam quite well, voter demographics while imperfect tends to be far more representable than film critic demographics, and as for questionable early ratings due to hype, just check out how many completely forgettable movies managed to win an Oscars somehow. That reminds me, what was the Oscar list for the year of The Empire Strikes Back? It's hilarious in hindsight.

So what Wikipedia's doing? Ignoring all real data, and just parroting establishment of course!

MCU Movie Ratings

So here are the objective ratings, in chronological order:
I don't always agree with all them - for example I tend to enjoy superhero movies which don't take themselves too seriously more, so personally I find Doctor Strange overrated, and Iron Man 2-3 underrated.

But don't mind me, statistically speaking you're far more likely to enjoy what other people enjoy than what some cat blogger or film critic likes. All I can do is provide a bit of context for those numbers.

The best one

Pretty much everyone agrees that the best movie of them all is Avengers: Infinity War and it's not even close. If you plan to watch a lot of MCU movies, definitely include this one somewhere.

However, it might not be the best movie to start with. It's really good if you're invested in the story, and know at least half the characters. Otherwise, it will probably be too confusing.

Really good movies

There's a lot of options where to start. Movies establishing new characters like Iron Man and Guardians of the Galaxy, which are also really good on their own, are probably the best place. If you're completely new to MCU, these are just the two I'd recommend starting with. If you don't like them, I doubt you'd like the rest anyway.

Thor and Captain America have some good movies, unfortunately their first movies are rather mediocre. So you can either start with their first weak movie because it gets better eventually, or just read its plot summary and get straight to the better ones. Nobody will blame you for skipping early Thors, honest.

Watchable movies

These movies are still pretty good, and they mostly don't require watching anything before.

Notable here is Black Panther, which due to the nearly all-black cast became inevitable battleground of culture wars. It's decent movie, but it's massively overrated by the overwhelmingly liberal film critics, and this generates a bit of audience backlash. It's definitely far better than the trainwreck Wonder Woman was, but every single MCU movie is far better than Wonder Woman, even Thor 1.

Forgettable movies

Unless you're really invested in the MCU, it's probably best to skip them. Personally I quite enjoyed Iron Man 2-3 movies, but I definitely have preference for sillier movies that most people don't share, and I can totally see why they did not get universal aclaim.

None of them are key to anything, and a huge added benefit of skipping them all is avoiding Natalie Portman, who according to some is the most annoying actress in Hollywood [citation needed].

So that's it for today. If you know of anyone who's working on an encyclopedia which follows Neutral Point of View, please let me know.

Hearts of Iron IV Online Division Designer

Okay, I posed... now where's my treat?? by Lisa Zins from flickr (CC-BY)

There's division designer in the game, but it takes 10 minutes of console commands to ask basic questions such as "would this division be better with Superior Firepower or Mobile Warfare".

I've seen a few spreadsheet style division designers online, but they're not really able to answer such questions easily, and I don't have much confidence in their calculations.

So I wrote a command line tool to run such calculations based on game files. And then since I was halfway to something useful for other players, I added React.js frontend, then support for two most popular mods (Kaiserreich and Millennium Dawn).

Here it is.

It's first public release, so bugs are definitely possible, and UI could definitely use more polish.

Calculation engine got amount of decent testing to make sure it matches game data (disregarding Paradox rounding), but I could have missed something, especially if it's something only mods use.

It has extra features which are not exposed in the UI right now, like selecting any combinations of techs not just year + doctrine, and forcing old equipment, and I'd like to expose that somehow perhaps. I'd also like to make calculations more transparent with some extra tooltips explaining how those values got derived.

It doesn't even try to run in old browsers like IE11.

Best place for bug reports and feature requests is its github page, but I'm on all social media, so you can contact me whichever way you'd like to. Pull Requests welcome of course.

Over the years I wrote a huge number of various command line analysis scripts and modding tools for CK2, EU4, and HoI4. This is the first one that's available online, but I guess I could setup frontends for more of them. If you have any requests, send them over.

Saturday, June 30, 2018

UK was never much of a democracy

Your Queen by garykemble from flickr (CC-NC)

The concept of "democracy" is very loaded, but let's focus on its most basic part - people vote, and whoever gets majority of votes gets to run the government for the next few years. UK fails this standard miserably.

Here's percentage of votes in Parliamentary elections received by whoever got to run the government over last hundred years. And it wasn't any better before:
  • 1918 - 38.4%
  • 1922 - 38.5%
  • 1923 - 38.0% (minority government)
  • 1924 - 46.8%
  • 1929 - 37.1% (party which "won" didn't even get plurality)
  • 1931 - 55.0%
  • 1935 - 47.8%
  • most countries continued having regular elections during wartime, but UK didn't even bother
  • 1945 - 47.7%
  • 1950 - 46.1%
  • 1951 - 48.0% (party which "won" didn't even get plurality)
  • 1955 - 49.7%
  • 1959 - 49.4%
  • 1964 - 44.1%
  • 1966 - 48.0%
  • 1970 - 46.4%
  • Feb 1974 - 37.2% (party which "won" didn't even get plurality)
  • Oct 1974 - 39.2%
  • 1979 - 43.9%
  • 1983 - 42.4%
  • 1987 - 42.2%
  • 1992 - 41.9%
  • 1997 - 43.2%
  • 2001 - 40.7%
  • 2005 - 35.2%
  • 2010 - 59.1% (coalition)
  • 2015 - 36.9%
  • 2017 - 42.4% (minority government)
So in 27 elections, there were only 2 cases where government was actually backed by majority of the votes. And usually these weren't narrow margins just a bit under 50% (like George Bush in 2000 or Donald Trump in 2016) - it's routine for parties to govern with just 35%-40% support.

In fact it's more common for a party to "win" without even getting plurality than for someone to actually get genuine democratic mandate.

There's a lot of concern about democracy fading worldwide. When you look closer, maybe there wasn't that much of it in the first place.

Wednesday, June 13, 2018

Challenges for May 2018 SecTalks London

そんな目で見ちゃダメだねぇ by amika_san from flickr (CC-ND)

Last month I ran another round of London SecTalks CTF.

There were 8 challenges as before. And again, the winner only did 7 of 8 on time, the last one (Monoalphabetic) only after the event.

Challenge files and code used to generate them available on github.

There are no answers below, but some serious hints which might make it too easy.

For previous rounds, see posts about September 2017 and November 2017 CTFs.

Invisible Flag (5 points)

It's a third iteration of the zip challenge.

There's a zip file containing 16 zip files inside. And each contains 16 more zip files. And so on for 16 levels, until you get to the flag.

For a small additional complication, the flag is completely invisible, and made out of different types of Unicode spaces. Replacing special spaces with different characters will reveal it.

Perl (10 points)

It's a very simple password validator written in very straightforward Perl.

The validator is mostly a sequence of commands like

s/^(.{7})i(.*)$/\2\1/;

Which removes character "i" from 8th position, and swaps everything before and after it. If the answer is empty, the validation is successful.

The easy way to solve it is to work backwards from an empty answer.

Almost Invisible Flag (15 points)

The challenge is an image with 1s and 0s written onto it in a very faint color. Once you figure that out, and transcribe the numbers into a binary file, it's a zip. Unpacking it reveals the flag.

React.js (20 points)

It's a react.js based validator. Validator is build of component layers, first layer checks length of the password, then each layer checks one character and passes the data to next layer with some transformations applied.

React Developer Tools for Chrome can be quite helpful for this.

This can be solved in many ways. You could brute force each layer with a simple script, or turn React code into a recursive function, and solve in an old fashioned way.

RSA (25 points)

For this challenge p and q are very close to each other. In fact they differ by just 2. There's a very simple attack for such case.

Monoalphabetic Cipher (30 points)

This challenge is a bit of text with flag embedded in it, encrypted by monoalphabetic cipher - all lower case letters, upper case letters, digits, spaces, punctuation each getting separate character.

The trick is that symbols used by the cipher are all emoji, which doesn't make the challenge any harder, just more fun.

The slight difficulty is that the text is not actually in English, so statistical methods can suffer a small detour. It's actually in German.

This was the only challenge that wasn't solved in time, but I don't think it's really that hard or time consuming.

I really like challenges about classical ciphers, but I don't want the solution to be simply copypasting it on quipquip.

PDF (35 points)

The challenge is a PDF file encrypted by a XOR key. The content is just a result of Chrome's print to PDF function. For extra challenge the flag is censored with a black bar over it, but that's very easy to workaround.

PDFs have a lot of plaintext inside, so there's a ton to work with, however it's somewhat unpredictable where that plaintext is located.

The key is three lower case English words without spaces.

A surprise extra challenge is that I accidentally left newline character at end of the key, which I only noticed during the challenge and told the contestants.

Javascript (40 points)

It's a very simple Javascript validator, obfuscated with jsfuck to only use 6 charecters.

I expected that people would need to write a simple JSFuck decoder, and that's why it's worth so many points, but most people just used Chrome debugger instead to solve this challenge really quickly.

Monday, May 28, 2018

Preconstructed Magic Decks on mtg.wtf

2018-05-28-13 by RobynAnderson from flickr (CC-NC-ND)

There's now a number of search engines for Magic cards, but some information is still missing from them all.

I just fixed one of such areas. mtg.wtf now has searchable contents of all preconstructed decks ever released. They contain cards from correct set, so viewing a decklist like this you can see actual cards which are in the deck. Every other decklist visualizer I've ever seen just gives you the most recent scans (like this for example), which looks really weird, especially for older decks.

You can view the decks, download decklists, or mix deck queries with other queries. For example to see which cards from Feline Ferocity are Modern-legal, just use deck:"Feline Ferocity" f:modern.

All this information is available in simple machine readable format.

Raw data is in magic-preconstructed-decks, and compiled jsons are in magic-search-engine repository.

Future enhancements

There's still room for improvement - information which cards are foil is missing, but easy to add. In theory it should be possible to find exact card printing instead of just set it's from (how many copies of which basic land etc.), in practice it might be a bit too much work.

Decklist visualizer could get extra features like sample hand, mana curve statistics etc.

And inevitably like with every project, there's going to be some errors. If you find any, please report them on github.

Monday, May 07, 2018

Wonder Woman review

Superkitty! by swxxii from flickr (CC-NC)

The movie is bad. Everything about it is bad. Like Star Wars prequels tier bad, but there won't even be any memes.

It's not about Wonder Woman

First of all, it's not even about Wonder Woman, it's all about her boyfriend. He's driving the whole plot, and she just tags along, without any sense of what's going on.

Diana (the "Wonder Woman") doesn't make any choices in the whole movie. The closest she gets to doing something on her own is choosing an outfit in a pointless shopping scene, and then she decides to dress up like him. Just wow.

Every time she suggests something (usually "omg, we got to help those people"), he just shuts her down, and then she does things his way.

The world makes no sense

Obviously the movie shits at actual Greek mythology.

As far as I can tell, the Amazons on the island are just some mortals. They have no children, and seem to be aging, so between ancient Greek times and World War 1, they should all be dead, right? Or is it some Star Trek time bubble? And how do they even speak English and all the other modern languages if they've been isolated from the world for who knows how long?

Then there's a framing device with pointless email from Batman. Like, what the hell? Greek Gods and Batman do not belong in the same movie in any way whatsoever. The comics tend to be stupid like that, but it served no purpose whatsoever.

And of course there are unexplained black Amazons, Hollywood has racial quotas to fill.

Action scenes are few and bad

There are three action sequences in this 2h 21min "action" movie.

Early on there's German army landing on Amazons' island, where the Amazons dance-fight, doing bows trick shot, and combo moves with spears, leashes, and such. Wearing the kind of "female armor" everyone has been making fun of for last 30 years. It's all rather silly, but probably still the best part of the movie.

Then about an hour of bad acting later there's storming of No Man's Land. Wonder Woman runs into machine gun fire, and the movie gives up on any internal logic. I couldn't tell - is she bullet proof or not? If she is, then why the hell is she tying to block bullets one by one with whatever piece of token armor is on her forearms? It's ridiculous every time she does it. And if she's not bulletproof, then how isn't she dead exactly? There's a lot of bullets out there, and she's not exactly waving her arms to block.

Anyway, leaving extent of her powers aside, somehow here mere presence makes all bullets miss everyone in the whole British army, who all decide to run after her, without anyone getting even a scratch.

Another hour of bad acting later, there's final boss fight which is basically shitty CGI, and least entertaining of all three.

The villain

Invincible superheroes are inherently rather boring, and decent superhero movies rely on an interesting villain to work. Wonder Woman's is about the worst, as the main villain gets like a minute of screen time within the first two hours before an obvious surprise plot twist reveal.

Villain's two henchmen get a bit more screen time, one of which Wonder Woman decided to kill, the other one to spare, for pretty much no reason whatsoever.

So instead the fighting is mostly against random German and occasionally Turkish conscripts who have absolutely no idea what the hell is going on around them. Then again, there's not that much fighting in the movie.

Morality

I think the movie attempts to have some moral message about violence being bad, but delivery is so ridiculously poor I'm not even sure what they've been going for. In the end, she ignores her moral intuitions and does whatever her boyfriend asks for. Is that the message perhaps? Do what your boyfriend says?

Sidekicks

At some point the boyfriend and his sidekick Diana assemble a team. None of them does anything. Some Native American "chief" sends smoke signals at some point, it's not very clear why. They try to do some fighting, but Wonder Woman does all the work anyway.

Acting

The only person who got any kind of character development is the boyfriend with his heroic sacrifice. Diana starts the movie as Wonder Woman, ends the movie as Wonder Woman, and her character development is, I guess getting her first boyfriend?

All the acting from Gal Gadot playing Wonder Woman was bad, so maybe they tried to salvage the movie by giving somewhat more tolerable Chris Pine playing the boyfriend more to do, until he took over the whole movie?

Conclusion

The movie has are no redeeming qualities. I watched it yesterday, and it's already rapidly fading in my memory other than the silly dance fight scene.

The only thing you'll achieve by watching it is convince yourself that critics (who for some reason love it) are all idiots, but you should probably know that by now.

Tuesday, March 27, 2018

Let's make Lucca Mission Pack mod for Europa Universalis 4

In this post we'll create a very simple EU4 mission pack mod. I'll use Lucca as an example. It will be more fun if you pick another country, follow along these step, while customizing your mod a bit.

This is a beginner level tutorial. It doesn't require any skills beyond use of text editors. Because we'll need nonstandard encoding support to deal with localization files, I recommend Visual Studio Code, but other than this one step pretty much any editor will do.

For some more details on how mission system works, you might want to check my previous post. To just learn the new mission system from a player's perspective, watch DDRJake's videos.

There's more to the mission system than can be covered in one tutorial, so I want to write another one covering more advanced subject later.

If you have any questions about these steps, ask them here, or reach me on any social media.

Create mod descriptor

For more complex mods it's best to keep the files somewhere safe and version-controlled, like on github, but we're starting small now so I'm going to skip this step.

Open C:\Users\(YourName)\Documents\Paradox Interactive\Europa Universalis IV\mod or equivalent.

You need to create a descriptor for your mod, for this simple mod create lucca_mission_pack.mod. For very simple mod we'll only have three lines, like this:

name = "Lucca Mission Pack"
path = "mod/lucca_mission_pack"
supported_version = 1.25

name is what's going to appear in the launcher.

path is name of the folder where we'll create mod files. Create the lucca_mission_pack folder. It's best if it has the same name as mod descriptor (except for .mod extension).

supported_version tells the game which exact version the mod supports. It's used mainly to warn the user in game launcher, that the mod didn't get updated for the next version yet.

If you're following these steps in the future, change supported_version to whatever's appropriate.

At this point you can start game launcher, and see that your mod is on the list. If it's not, something went wrong with the mod descriptor and you should check it. No point running the game yet, as the mod does nothing.

Create empty mod files

Inside C:\Users\(YourName)\Documents\Paradox Interactive\Europa Universalis IV\mod\ create lucca_mission_pack folder, inside it lucca_mission_pack\missions subfolder, and inside that lucca_mission_pack\missions\Lucca_Missions.txt file.

The name of the file doesn't matter as long as it ends in .txt and doesn't conflict with any in the game. For example if you call it Hungary_Missions.txt you'll be overwriting Hungary's missions from base game - but if you call it MudkipIsBestPokemon.txt it's just fine.

Location of the file definitely matters, so if you make mistakes the mod will not load correctly.

Planning the missions

If you take a short break from modding and run the game, you'll see that Lucca's existing missions are:
  • column 1 - empty
  • column 2 - generic military missions
  • column 3 - generic diplomatic missions
  • column 4 - generic administrative missions
  • column 5 - regional missions for European countries and HRE minors
It's going to be similar for most countries, except sometimes column 5 will be empty as well.

For a small mission pack we'll create a single mission group, and put it into column 1.

For a bigger mission pack we might replace column 5 as well, add missions on the bottom, or replace some of the generic missions.

Column choice is not just for the interface, as multiple mission groups going into same columns might replace each other.

Create a mission group

Let's open lucca_mission_pack\missions\Lucca_Missions.txt in text editor and setup a group for the missions:

lucca_mission_group = {
  slot = 1
  generic = no
  ai = yes
  potential = {
    tag = LUC
  }
  has_country_shield = yes

  # missions will go here
}

What we're setting up here:
  • in column 1 (columns are called "slots")
  • not generic. This is used as priority if multiple mission groups try to use same column.
  • also available for the AI. Usually should be yes.
  • available if your tag at campaign start is LUC (Lucca). List of tags is on the wiki, usually it's first three letters of country name.
  • that we want to display country shield in the interface. This is purely graphical, but country-specific missions usually do, so let's be consistent here.
For potential you can choose based on other criteria like religion, culture, capital location etc., but they're only checked on campaign start, or when explicitly told so by an event or decision.

In vanilla game, they only change on some nation forming decisions. If you start as Lucca and form Italy, you'll keep your original missions since Italy decision doesn't change missions. If you form France, you'll get your mission pack replaced.

Creating out first mission

Time to create the first mission. We'll start with something very simple. The goal is to build a temple  in Lucca, and as a reward you'll get 200 admin points.

Open missions files, and in the place we left for missions, add this:

lucca_temple_in_capital = {
  icon = mission_european_church
  required_missions = {  }
  trigger = {
    2980 = {
      has_building = temple
    }
  }
  effect = {
    add_adm_power = 200
  }
}

What it does:
  • it sets mission's ID to lucca_temple_in_capital. This needs to be unique, and will be used for localization.
  • it uses icon mission_european_church. This is purely graphical, and I wouldn't worry too much about the icons until you're happy with how your missions work.
  • it sets lists of required previous missions to empty
  • it sets mission completion condition (trigger) - province 2980 (Lucca) has temple. You can find list of province IDs on the wiki.
  • it sets mission reward (effect) - 200 admin power points
At this point it makes sense to launch the game, and check that everything is indeed working.

Try writing something similar for country of your choice.

Start mission chain

OK, so that worked. Let's create some more missions. We'll put them all in same file, one under the other.

For design of a mission chain to be good, they should somehow connect with each other. Here's an idea for a small chain:
  • first we want to build up our capital. As a reward we get some permanent claims on nearby lands.
  • second we want to conquer claims we got in previous mission, as a reward we get some points of multiple types
  • third, we want Lucca to rival one of the big baddies, proving that we're ready. It will require completing previous mission too, so we don't need to check it again. As reward we get 25 year +10% discipline bonus for more enjoyable fights agains our new rivals.
This covers the most common cases - checking that we conquered something, getting rewarded by permanent claims, various types of points, or temporary bonuses. As well as a few extras.

Plan a similar chain for your own country, for now using similar kinds of effects.

Make Lucca Great Again

make_lucca_great_again = {
  icon = mission_high_income
  required_missions = {  }
  trigger = {
    capital_scope = {
      development = 25
      renaissance = 100
    }
  }
  effect = {
    tuscany_area = {
      limit = {
        NOT = { is_core = ROOT }
        NOT = { owned_by = ROOT }
      }
      add_permanent_claim = ROOT
    }
  }
}

Step by step:
  • it sets mission's ID to make_lucca_great_again. We'll use it as key to localization later.
  • it sets required missions to empty
  • to succeed it requires that our capital has at least 25 development and 100% Renaissance institution.  I could have used 2980 (specifically province of Lucca) instead of capital_scope (whichever province is our capital) here. We don't expect capital to move, so in this case it doesn't matter, but both selectors have their uses.
  • as reward we go through all provinces in tuscany_area and add permanent claim to them, unless they're already owned by us or have our core. Areas are defied in map\area.txt file. You don't need to use areas, you can just list provinces separately (the syntax is however a bit different).
So we can load the game, and see that it works.

Try something similar for your country. Pick some capital growth conditions, pick a relevant area from map\area.txt (most areas correspond to a state), and try it in the game.

Luccan Tuscany

We got our claims, so we ought to follow it with a mission checking that we actually conquered what we wanted.

As reward we'll just get some points, but of course you can chain a lot of conquer this area - get claims on next area missions. Making them branch in a way that looks good in the UI is a bit more advanced topic, but just making this chain as long as you'd like should be straightforward.

luccan_tuscany = {
  icon = mission_cannons_firing
  required_missions = { make_lucca_great_again }
  provinces_to_highlight = {
    area = tuscany_area
    NOT = {
      country_or_non_sovereign_subject_holds = ROOT
    }
  }
  trigger = {
    tuscany_area = {
      type = all
      country_or_non_sovereign_subject_holds = ROOT
    }
  }
  effect = {
    add_mil_power = 200
    add_legitimacy = 10
    add_republican_tradition = 10
  }
}

Let's go through it step by step:
  • it sets mission ID to luccan_tuscany
  • it sets mission icon to mission_canonns_firing
  • it requires completion of make_lucca_great_again first. If you arrange mission chain so that next mission depends on one just above it (or one row down and then sideways), the UI will draw pretty arrows to make it clearer how missions are related. Other dependencies still work, just won't look quite as nice.
  • it sets province highlight when mission is mouseovered to all provinces in  provinces_to_highlight which are not owned by us or a (nontributary) subject
  • it sets success condition of owning all provinces in tuscany_area by us or our (nontributary) subject. In addition you still need to do make_lucca_great_again first. 
  • as a reward, we get 200 mil power points, 10 legitimacy, and 10 republican tradition. Depending on government type of the country, the tooltip will only show relevant rewards, so if Lucca is a republic, the tooltip will show "200 mil points, 10 republican tradition", since legitimacy doesn't matter.
Try something similar for your country, customizing rewards, and using the same area you used in your previous mission.

You can also chain claims in a few missions.

Lucca in the Major League

So now that our Lucca conquered Tuscany, let's give it a more ambitious goal - rivalling one of big three regional baddies. As a reward we'll give it a bonus.

lucca_in_the_major_league = {
  icon = mission_cannons_firing
  required_missions = { luccan_tuscany }
  trigger = {
    OR = {
      is_rival = FRA
      is_rival = HAB
      is_rival = TUR
    }
  }
  effect = {
    add_country_modifier = {
      name = lucca_discipline
      duration = 9125 # 25 years
    }
  }
}

Step by step:
  • it sets mission ID to lucca_in_the_major_league
  • it sets mission icon to mission_cannons_firing, I just used the same one as before but feel free to pick something more creative. In the end it's just a minor graphical embellishment.
  • it requires completing luccan_tuscany first, which in turns requires make_lucca_great_again. Because we put those missions in proper order, they'll be connected by nice arrows in the UI.
  • to succeed we need to rival France (FRA), Austria (HAB), or Ottomans (TUR). You can see that in this case tags are not quite what one might guess, presumably as a leftover from EU2 or maybe even EU1 for all I know. There's no need to check that we also did our conquests, as previous mission in the chain checks that.
  • as a reward we get lucca_discipline modifier, for 9125 days (25 years). We just need to tell the game what that means in another file.

Creating modifiers

This is not strictly mission modding, but modifiers are about the most common mission reward, so it's useful to know how to create them.

In our mod create text file lucca_mission_pack\common\static_modifiers\lucca_missions.txt, and put this inside:

lucca_discipline = {
  discipline = 0.10
}

And that's it. You can check common\static_modifiers\00_static_modifiers.txt in the game for more examples.

You can put as many modifiers in single file as you want, just create your own instead of putting edited 00_static_modifiers.txt in your mod.

How EU4 localization works

EU4 localization files use .yml extension, but it's actually custom format somewhat inspired by YAML format, not the real thing.

All localization files live in localisation folder, with files sorted by language.

Each file has header like l_english, and a list of IDs and their localized strings. All the files need to start with the non-standard UTF-8 BOM. If you create a normal text file without BOM, it will be ignored by the game.

If you want your mod to support some other languages, it should be fairly obvious which file names and header to use instead.

As a bonus annoyance, Paradox can't decide between American and British spelling inside their files. Folder name is called "localisation" in all their games, but then HoI4 uses localization_key to choose entry from it. It's very easy to get confused.

Localization

So if you followed along, you have a working mod with 4 missions for your country, or however many you created. There's just one more issue - the missions all use internal development IDs instead of human readable names and descriptions.

There's 9 things we need to edit: 4 mission names, 4 mission descriptions, and name of our modifier.

Those IDs we gave the missions, they all need to be unique, so it makes sense to prefix it by name of the country or mission pack to avoid accidental conflicts, but we can localize them any way we want.

For every mission ID_title is its name key, and ID_desc is the description key. Static modifiers's name keys are just their IDs.

What we'd like to do is create lucca_mission_pack\localisation\lucca_missions_l_english.yml and put the following code there:

l_english:
 lucca_temple_in_capital_title: "Builde Temple in Lucca"
 lucca_temple_in_capital_desc: "As all good Italians, Luccans need a place to pray."
 make_lucca_great_again_title: "Make Lucca Great Again"
 make_lucca_great_again_desc: "Our capital should be the greatest city in Italy since ancient Rome"
 luccan_tuscany: "Conquest of Tuscany"
 luccan_tuscany: "Some Florentines, I assume, are good people, but they'll still be better off under Luccan rule."
 lucca_in_the_major_league_title: "Joining the Major League"
 lucca_in_the_major_league_desc: "Show the world that Lucca is now one of the great powers by rivaling a nearby Great Power."
 lucca_discipline: "Lucca's Dispciplined Army"

The game uses single space indentation, but it's not too strict about this.

What it is extremely strict about is the BOM, so if you use Visual Studio Code, choose "UTF-8 with BOM" as encoding for your new file on the bottom bar. For other editors find appropriate instructions, but many don't support UTF-8 BOM at all.

If you don't know how to get the BOM working, one think you can try is copying existing localization file, deleting everything except the first line (which might look like one broken character followed by l_english: tag) and just replacing the rest of the file with your content.

Great Success

If everything worked, you should see something like this:


Debugging

If you see image like above, but using internal IDs instead of English text, then mod is probably file , but localization is wrong. Most common issue is UTF-8 BOM.

If you see some total mess in first column, like wrong number of missions with silly names like icon, you probably made a typo in mod files, and game .

Open C:\Users\(YourName)\Documents\Paradox Interactive\Europa Universalis IV\logs or equivalent and search for hints. errors.log might give you line number where game got confused.

If mod was in the launcher, but when you play the game you see empty first column, mod probably didn't load at all. Or it did, but your potential tag is wrong, and game doesn't think these missions are appropriate for currently selected country. Double check errors.log and list of tags.

If you don't even see the mod is mod launcher, then mod descriptor file is probably wrong or in wrong place.

Or if none of that works, just ask, but remote debugging is pretty hard. I'll have easier time answering "I want mission to do X when Y" questions than "it doesn't work, what do?" questions.

Download the mod

It's best if you followed the tutorial and wrote it on your own, but if you want to check the files, all the files are on github.

Sunday, March 25, 2018

How to mod EU4 1.25 new mission system

https://www.flickr.com/photos/limowreck666/159234606/

One thing we lost in transition to the new 1.25 mission system were missions for idea groups, so I was wondering how they could be restored.

But first, I needed to do some investigating. This post is my notes, and it's primarily meant for other modders, but it could be of some use to people who just play and are interested in shenanigans.

If you want to check it from player's perspective, DDRJake recorded a series of quick presentations for some of the countries with more interesting missions.

How the old system worked

Until 1.24 every mission had separate potential check, and it could randomly appear for every country if its circumstances matched. Tag-specific missions usually had much higher weight than generic missions, so they usually appeared "in order".

On the upside, it was no trouble whatsoever for missions to be based on your rivals, idea groups, and so on.

On the downside, it was harder for this mission system to provide much guidance to players, and sometimes the RNG was pretty weird.

How HoI4 focus trees system works

The new system is clearly inspired by HoI4 focus trees, but there are some major differences too. Focuses are sort of like missions.

In HoI4 every major country has completely hardcoded tree of focuses. Focuses never change, so starting a game you'll already see Communist and Fascist focuses, even if you won't be able to ever do both same campaign (barring some major shenanigans). 

Minor countries all have the same generic focus tree, also with multiple mutually exclusive branches.

This system allows some interesting chains (see Kaiserreich for an example), and it works OKish with the full screen UI, but in the end it's basically one huge checklist per country, with nothing shared (except by copying and pasting), and no opportunity for anything dynamic - other than by making the checklist even larger.

Constraints behind design of the 1.25 mission system

The new mission system was obviously inspired by HoI4, but they faced different constraints.

People play a lot more countries in EU4. HoI4 after a few expansions has 22 focus trees, and while this isn't enough to stop demands for more, most players would be happy enough once about 30-40 or so top countries get national ideas. The kind of people who play Bhutan or Panama are a tiny minority, and generic ideas are powerful ideas that they don't have to be interesting.

In EU4 a lot more countries are played than in HoI4. There are already 353 national idea groups (counting both unique and regional groups) for 700 or so tags. There's no way in hell devs would be able to create HoI4 style trees for that many countries in a reasonable time.

Fortunately EU4 doesn't really need that kind of complexity. So instead they created modular system which allows wide range of mission trees from completely generic to completely unique.

How 1.25 mission system works

There are 5 column-shaped slots in mission pool, conveniently labelled 1 to 5 left to right.

Missions are all in groups. Each group has designated column, potential condition, and is marked as generic or not.

Each mission within a group can have a position (row), otherwise it will take the first free row instead.

When campaign starts or the mission pool is reinitialized (on some tag switching events), every mission group is checked. Every group for which potential condition is true is added to its column.

As an exception, if a generic missions wants a slot already taken by a non-generic mission, its whole group is removed. So if there's a group of generic missions for slot 4, positions 1-5, and a single non-generic mission for slot 4 position 1, then none of those generic missions will be available. Those in slots 2 and 3 won't be affected.

You can still have generic and non-generic missions in same slot, by explicitly giving them positions 6+.

If you're mixing missions this way, load order matters, as game doesn't like putting missions above previously placed missions. That's why generic missions are loaded first (file name starts with 00_).

Also putting generic missions under non-generic missions doesn't seem to work.

You could have multiple non-generic groups in same column (with or without explicit positions), but as far as I can tell, it's not used anywhere.

Possible mission pools

These combinations are used in the game:
  • countries Paradox really didn't care about like New World minors have only generic missions in columns 2, 3, 4, positions 1-5 each, for a total of 15 missions
  • slightly more important countries like Ulm get one or more regional generic missions groups in column 5. Ulm gets two groups there - one for all Europeans, and one specifically for HRE minors.
  • secondary powers like Bavaria get all that, and also tag-specific missions in column 1
  • as a slight variant, Coptic African countries get Coptic missions in column 1, African regional missions in column 5, and generic missions in columns 2, 3, 4
  • most big countries like Castile or Muscovy still get all generic missions in columns 2, 3, 4. They also get unique missions in columns 1 and 5 - and explicitly positioned unique missions in columns 2, 3, 4, under generic missions
  • a few countries like France do that, and they also replace some of generic missions. For France columns 2 and 4 include generic missions, and unique missions on the bottom; columns 1, 3, and 5 include unique missions
  • some countries like England replace all generic missions in all 5 columns
There are a few more things we could do.

How missions pools are reset

Mission pools are only changed explicitly if decision or event uses swap_non_generic_missions = yes

It is not triggered automatically, so if your Mazovia joins HRE, it won't get missions HRE minor get.

The change is included in some tag changing decisions. More or less those for which target tag has unique missions, but the lists don't seem to match exactly (like some decisions to form Japan reset missions, others don't). It's probably a few minor bugs, I don't think they're usefully exploitable.

This means that if you form England (which includes the reset) as France you get English missions, but if you form Shan (which doesn't), you keep French missions. They'll have Shan shield on them, the shield is purely decorative.

What can be easily modded?

First, the good news. If you want to create missions in format similar to what already exists, it's very easy.

So your Trebizond Mission Pack mod, or Indonesian Minor Mission Pack mod will require no shenanigans.

For this kind of mods simply check existing files, they're very easy to understand.

What requires manual resets

Now, the bad news. Anything else will require messing with the system.

For example if you religion-specific mission packs like Coptic do, let's say Sunni Mission Pack, you could simply add them to column 1 or 5, and just accept that countries which override it (like Ottomans and Mamluks) won't get your missions.

Then you could try writing mission packs for Catholics, Protestant etc., putting missions in column 1, and you'll run into a big problem - your newly Protestant Anhalt will still keep its Catholic missions, as it started the campaign Catholic.

From testing so far it seems that triggering swap_non_generic_missions = yes is completely harmless, so if you see good event for it (on_actions - on_religion_change), you could simply stuff it there. 

For things like missions based on idea groups, there's no good place to put the trigger, so perhaps it needs a regularly triggered maintenance event which will periodically reset mission pool regardless of them being needed. You can probably live with a small delay.

You could also create a decision to manually reset the mission pool. It feels a bit awkward, but works just fine from the testing. I'm not sure how to tell AI to click it the right times, so it could be in addition to other mechanisms.

Can we have more columns?

For extra missions it would be nice to add extra columns 6+. This way we could for example add religious missions in column 6, culture specific missions in column 7, idea group missions in columns 8-10, and not worry about overlap.

With vanilla UI it doesn't display at all, and a quick look at defines and interface files don't show any obvious fix. It would be nice if someone did some testing here. I'm not seeing any obvious reason why it shouldn't work.

Dev Diary once said "The grid can be as wide and tall as you like, it will automatically expand to fit all missions" - testing shows it's false, and expansion is only rows.

Idea group missions

And finally let's get back to where we started - missions for idea groups. Let's say 2 missions per full group, which is not amazing, but it's a decent start.

If we had columns, we could just put generic missions in those extra columns, somehow reset mission pool when ideas are taken (or on some timer if that's not possible).

As far as testing goes, what actually works is this:
  • create a file with some late name like missions/ZZZ_Idea_Group_Missions.txt.
  • create some mission groups, let's say all administrative idea missions in column 2, diplomatic idea missions in column 3, and military idea missions in column 4
  • even though they'll be the most generic missions possible, you need to mark them as generic = no, so they display even if nation already has non-generic missions in given column.
  • don't make them display nation's shield, it's not necessary.
  • you need to explicitly mark position, lower than any exiting missions. Portuguese and English missions use positions up to 11, so you can start at 12 unless I missed something. Or a bit below, the only difference will be extra scrolling
  • those missions will display for every nation, and simply use checks like full_idea_group = administrative_ideas in their fulfilment criteria.
Now that the investigation is complete, the only thing to do is creating a bunch of interesting missions and uploading them as a mod.

Saturday, March 24, 2018

Realistic solutions to the Israeli-Palestinian conflict

myst and menorah (1) by isfullofcrap from flickr (CC-BY)

Eight years ago I suggested a more creative solution to the Israeli-Palestitian problem. Sadly, my advice was not followed.

The most likely scenario is that the conflict will just keep on going indefinitely. The world is full of such frozen conflicts, with more getting generated - like the recent Crimea occupation. Very rarely they get successfully resolved by either force (like Sri Lanka) or negotiations (like Northern Ireland).

It seems very unlikely that either side will be able to solve this issue by force. Israel could try expelling some number of Palestinians, but they'd just move to refugee camps elsewhere. Arab armies didn't win any major war in about a thousand years (even their Medieval caliphate was mostly ran by Turks, Circassians, and other non-Arabs), and even if they suddenly became competent, they won't be able to do much against explicit US guarantees for Israel.

So let's consider for a moment a range of negotiated solutions which could possibly work, from easiest to hardest.

"Palestinians" living in Arab states

A huge obstacle to solving the problem is the official policy of Arab states, which all agreed to not grant citizenship or any other regular legal status to people born in them who are descendants of "Palestinians". This sometimes includes people with mixed "Palestinian" and non-Palestinian Arab ancestry.

Calling these people "Palestinians" is just a political ploy. They've never seen "Palestine", and back when their ancestors lived there there was no "Palestinian" national identity - it was only in the 1970s that Pan-Arabism faded away enough for any "Palestinian" nationality to start developing.

The policy of using millions of people this way, keeping them in a legal limbo to put pressure on Israel, is definitely nasty, but even worse - so far it did absolutely nothing to achieve its goals.

So as a necessary step all "Palestinians" who were born in Egypt, Jordan, Lebanon, Syria (there's an awkward civil war here, but it will likely be over long before anything moves on Palestinian issue), Saudi Arabia, Qatar, and various Arab minor states need to be granted full citizenship or equivalent, as soon as possible.

There's no possible solution without most Arab states cooperating on this. If some of them don't, the rest pretty much need to accept large number of the "Palestinians" in. Non-Arab countries could accept some number of them (like Chile and US did), but these people already share culture, language, and (generally) religion of their country of birth. It's ridiculous to make them move to another continent because of petty politics.

This can only be done unilaterally by each Arab country, so if you want to pressure someone to do something useful to get this problem towards a solution, that's the most obvious place to start.

Gaza Strip

This is really simple. Nobody has any issues with borders or population living there. There used to be some tiny Israeli settlements in the Gaza Strip, but they're all gone since 2005.

So in the end, some Arab entity will end up running Gaza. Israel and Egypt don't really care about the details, as long as security is maintained. Unfortunately so far it's not, and not only Gaza exports rockets to Israel, it also exports Jihadists to Egypt.

Apparently such terrorism is approved by 80% of the locals. As a result of this, Gaza ended up blockaded from both Israeli and Egyptian side. It's even worse, as Gaza under Hamas is in conflict with West Bank under Palestinian Authority, so even Palestinian Authority joined the blockade. Attempts to break it failed quite miserably, and probably won't be repeated.

This foolishness could easily end if attacks stopped, which would likely result in economic blockade getting gradually lifted, and everybody moving on with their lives.

Israel can't do much about it, other than building an even bigger wall, and occasionally dropping some bombs, possibly even on someone related to the attacks.

One plausible solution would be for a third party like Egypt or Arab League or the UN to take control over security in Gaza for some time, sort of like happened in Bosnia.

Israel definitely doesn't want the job, and so far nobody else volunteered. This is something Palestinians living in Gaza could in principle fix unilaterally, but in practice it's difficult as even a fairly small group of assholes could still export rockets and Jihadists even if they didn't have majority support.

As a minimum if Gazans antagonized only one of their neighbours, not both, perhaps the blockade could be loosen up a bit.

Jerusalem

It's going to remain the capital of Israel. Short of a total military defeat, there's no way in hell Israel will abandon it into some kind of international status, or just give away to the Arabs the part which happened to be on Jordanian side of 1949 armistice line.

There's some margin for negotiations of exact borders of Jerusalem, status of Palestinians living there, arrangement for the holy sites, and so on, but it's going to remain Israeli.

Trump's unilateral recognition of reality was a small step in right direction. Other countries should do the same, so we can all move on.

Money

Whichever entity ends up running Palestine will need a lot of funding, and the US, Europe, rich Arab states, and many other foreign sources will need to provide a huge deal of it to make it economically viable.

Not every problem can be solved by throwing money at it, but it works reasonably well for rebuilding economies after destructive conflicts.

As long as conflict is ongoing, Palestinian economy can't rebuild, so foreign money spent will achieve little more than temporary humanitarian relief.

One nice thing Palestinians could do to convince foreigners to be more generous would be ending payments to terrorists and their families.

There's also issue of enormous levels of corruption, but let's focus on the easier problem for now.

West Bank

There's some disagreement, but it's actually surprisingly small.

Palestinians would like to follow something like 1949 armistice lines, with as many of the Jewish settlement being dismantled as possible. Israel would like its border to follow something like the barrier wall, including most of the major settlements, annexing about 9% of territory beyond armistice lines.

Both sides are open to some degree of land swaps, and once allegedly the negotiations even reached the point where border differences were down to 3% vs 6% of the West Bank.

Security arrangements

Israel would also like some kind of strong security guarantees, which would invariably restrict Palestinian sovereignty, at least for some time. Palestinians are understandably not terribly happy about that.

Depending on how it's all designed, such arrangements could even be in Palestinian interest. The absolutely worst outcome for the Palestinians would be getting their own state, getting overran in a civil war by some local Islamists, who start exporting terrorists and rockets in every direction, and getting occupied again. Which isn't exactly unlikely.

It's hard to talk about details, as this part has the widest range of possible outcomes.

Most of these would be temporary, and once long term peace and security is achieved, such restrictions can go away. That can still take very long time, like with Bosnia.

It's possible that both sides would be happy to involve third parties in these arrangements, but third parties are not terribly happy to volunteer.

Oh wait, that was exactly the offer

What I described above was pretty much the Israeli offer in 2000 Camp David Summit. 100% of the Gaza Strip, 92% of the West Bank, $30bln resettlement fund, and some rather awkward security arrangement, many but not all temporary.

There were still significant differences on Jerusalem's holy sites, security arrangements and other such issues, but these are really tiny relative to the whole issue.

There's space for some minor tweaks around the margins, like a bit more generous land swaps, making security arrangements temporary, and throwing more money at rebuilding Palestinian economy, but any possible negotiated solution will look roughly like this, and everyone knows it.

So how is this not solved yet?

There's just no desire for serious peace negotiations on the Arab side. Arab dictators of nearby countries don't care about Palestinians, and find it more convenient to just keep the conflict frozen.

Palestinian Authority doesn't want peace, as any kind of peace would immediately halve number of "Palestinians", with the rest getting naturalized into countries of their birth. They'd rather have fake authority over all the "refugees" than a real country.

Recent American attempts at solving the problem rightly focused on ignoring Palestinian Authority completely, and getting direct agreement between Arab countries and Israel, which then PA without any foreign support will be forced to accept.

This doesn't seem terribly likely to succeed, especially with both Trump and Netanyahu so politically weak, but it's probably the best approach.

Perhaps president Winfrey will try again, and it will look almost exactly like Bill Clinton's Camp David plan from 2000, except Palestinians will be forced to take it seriously by lack of Arab backing.