The best kittens, technology, and video games blog in the world.

Friday, December 11, 2020

What if all EU4 idea groups were good

Cuddle Kittens by C_Greengrass from flickr (CC-ND)

Many EU4 idea groups are bad. There are two sides of this badness - power level and identity.

Power level is relatively easy to analyze objectively. I once created an algorithm to rank all EU4 idea groups. This was 5 years ago, patches completely changing game balance happened, so those specific values are outdated. Not to mention original algorithm had some minor issues.

I try to keep the scoring algorithm more or less up to date, so assuming I didn't miss any important changes, here's the 1.30 latest ranking.

Everything in this post is about single player only, multiplayer meta is drastically different.

Basic Ideas Ranking

  • 9.09 Expansion Ideas
  • 7.65 Diplomatic Ideas
  • 6.00 Exploration Ideas
  • 5.66 Religious Ideas
  • 5.58 Administrative Ideas
  • 5.35 Influence Ideas
  • 4.67 Humanist Ideas
  • 4.33 Plutocratic Ideas
  • 3.89 Trade Ideas
  • 3.89 Aristocratic Ideas
  • 3.80 Quantity Ideas
  • 3.33 Offensive Ideas
  • 3.13 Innovative Ideas
  • 2.92 Quality Ideas
  • 2.75 Espionage Ideas
  • 2.26 Defensive Ideas
  • 1.58 Economic Ideas
  • 1.12 Maritime Ideas
  • 0.75 Naval Ideas

National Ideas Ranking

  • 10.75 Holy Roman Ideas
  • 10.27 Asturian Ideas
  • 10.17 Great Yuan Ideas
  • 9.60 Imperial German Ideas
  • 9.50 Great Qing Ideas
  • 9.17 Roman Ideas
  • 9.12 Kiwi Ideas
  • 8.93 Andalusian Ideas
  • 8.83 Russian Ideas
  • 8.25 Synthetic Ideas
  • 8.04 Icelandic Ideas
  • 7.98 Castilian Ideas
  • 7.92 American Ideas
  • 7.85 Oirat Ideas
  • 7.67 West Indies Ideas
  • 7.54 Jerusalem Ideas
  • 7.24 Oda Ideas
  • 7.19 French Ideas
  • 7.18 Herzegovinian Ideas
  • 7.09 Rûmi Ideas
  • 6.96 Yamana Ideas
  • 6.85 Lotharingian Ideas
  • 6.85 Italian Ideas
  • 6.79 Carib Ideas
  • 6.74 Spanish Ideas
  • 6.69 Norwegian Ideas
  • 6.66 Mongolian Ideas
  • 6.65 Mughal Ideas
  • 6.56 Austrian Ideas
  • 6.55 Leonese Ideas
  • 6.54 Mysorean Ideas
  • 6.37 Hindustani Ideas
  • 6.37 Kazani Ideas
  • 6.36 Hadramhi Ideas
  • 6.34 Golden Horde Ideas
  • 6.34 Ottoman Ideas
  • 6.24 Iroquois Ideas
  • 6.19 Japanese Ideas
  • 6.18 Sardinian Ideas
  • 6.10 Australian Ideas
  • 6.10 Crimean Ideas
  • 6.07 Rajputana ideas
  • 6.03 Aq Qoyunlu Ideas
  • 5.97 Canadian Ideas
  • 5.93 Irish Ideas
  • 5.92 Bharathi Ideas
  • 5.86 Deccani Ideas
  • 5.86 Prussian Ideas
  • 5.84 Saluzzo Ideas
  • 5.83 Anatolian Ideas
  • 5.81 Najdi Ideas
  • 5.77 Candarid Ideas
  • 5.74 Chernihiv Ideas
  • 5.72 Karamanid Ideas
  • 5.70 Nagpuri Ideas
  • 5.67 Burmese Ideas
  • 5.65 Imagawa Ideas
  • 5.63 Nepali Ideas
  • 5.58 Sardinian-Piedmontese Ideas
  • 5.56 Ayyubid Ideas
  • 5.54 Delhian Ideas
  • 5.54 Qara Qoyunlu Ideas
  • 5.54 Desmondian Ideas
  • 5.51 Bosnian Ideas
  • 5.46 Chagatai Ideas
  • 5.44 Trent Ideas
  • 5.42 Divine Ideas
  • 5.41 Manipur Ideas
  • 5.40 Knights Hospitaller Ideas
  • 5.39 Khorasani Ideas
  • 5.39 Greek Ideas
  • 5.36 Native Ideas
  • 5.36 Manchu Ideas
  • 5.33 Antemoro Ideas
  • 5.30 Colognian Ideas
  • 5.29 Three Leagues Ideas
  • 5.25 Tarascan Ideas
  • 5.25 Wallachian Ideas
  • 5.22 Tibetan Ideas
  • 5.22 Polish Ideas
  • 5.21 Date Ideas
  • 5.19 Mantuan Ideas
  • 5.19 Mushasha Ideas
  • 5.17 Dalmatian Ideas
  • 5.15 Muscovite Ideas
  • 5.13 Swedish Ideas
  • 5.13 Moluccan Ideas
  • 5.11 Ryazan Ideas
  • 5.08 Byzantine Ideas
  • 5.08 Pegu Ideas
  • 5.05 Bremen Ideas
  • 4.95 Bulgarian Ideas
  • 4.94 Saxon Ideas
  • 4.92 Asakura Ideas
  • 4.92 Shoni Ideas
  • 4.91 Münster Ideas
  • 4.90 Teutonic Ideas
  • 4.89 Kievan Ideas
  • 4.88 Epirote Ideas
  • 4.83 Timurid Ideas
  • 4.80 British ideas
  • 4.77 Provençal Ideas
  • 4.76 Sukhothai Ideas
  • 4.75 Otomo Ideas
  • 4.75 Hungarian Ideas
  • 4.74 Kaffan Ideas
  • 4.73 Westphalian Ideas
  • 4.72 Rigan Ideas
  • 4.71 Perugian Ideas
  • 4.71 Tokugawa Ideas
  • 4.70 Gond Ideas
  • 4.69 Romanian Ideas
  • 4.67 Jan Mayen Ideas
  • 4.66 Utrecht Ideas
  • 4.66 Great Shun Ideas
  • 4.65 Kitabatake Ideas
  • 4.63 Assamese Ideas
  • 4.63 Ajami Ideas
  • 4.62 Rassid Ideas
  • 4.62 Manx Ideas
  • 4.62 Ethiopian Ideas
  • 4.61 Württemberger Ideas
  • 4.60 Ryukyuan Ideas
  • 4.59 Bohemian Ideas
  • 4.56 Cilli Ideas
  • 4.56 Sicilian Ideas
  • 4.55 Papal Ideas
  • 4.55 Holstein Ideas
  • 4.55 Palatinate Ideas
  • 4.54 Hojo Ideas
  • 4.53 Nanbu Ideas
  • 4.53 Trierian Ideas
  • 4.53 Dai Viet Ideas
  • 4.52 Dithmarscher Ideas
  • 4.52 Liège Ideas
  • 4.51 Aachen Ideas
  • 4.51 Kanem Bornuan Ideas
  • 4.51 Mapuche Ideas
  • 4.50 Brandenburg Ideas
  • 4.50 Georgian Ideas
  • 4.49 Cossack Ideas
  • 4.48 Bengal Sultanate Ideas
  • 4.48 Rostov Ideas
  • 4.47 Burgundian Ideas
  • 4.46 German Ideas
  • 4.43 Alaskan Ideas
  • 4.43 Nizhny Novgorod Ideas
  • 4.43 Daimyo Ideas
  • 4.42 Bamberger Ideas
  • 4.42 Lithuanian Ideas
  • 4.41 Shimazu Ideas
  • 4.40 Maravi Ideas
  • 4.38 Satake Ideas
  • 4.37 Kongo Ideas
  • 4.37 Athenian Ideas
  • 4.37 Tumbuka Ideas
  • 4.36 Medri Bahri Ideas
  • 4.35 Butua Ideas
  • 4.34 Uesugi Ideas
  • 4.33 Rajput Ideas
  • 4.32 Moravian Ideas
  • 4.31 Kildarean Ideas
  • 4.31 Granada Ideas
  • 4.30 Khmer Ideas
  • 4.30 Thüringian Ideas
  • 4.29 Malian Ideas
  • 4.29 Swabian City-State Ideas
  • 4.28 Ansbach Ideas
  • 4.25 Swabian Ideas
  • 4.25 Yarkandi Ideas
  • 4.23 Franconian Ideas
  • 4.20 Novgorod Ideas
  • 4.19 Butuan Ideas
  • 4.19 Colonial Ideas
  • 4.18 Ogasawara Ideas
  • 4.16 Sindhi Ideas
  • 4.15 Montferrat Ideas
  • 4.14 Mossi Ideas
  • 4.13 Ito Ideas
  • 4.13 Client State Ideas
  • 4.12 Kikuchi Ideas
  • 4.11 Brunswicker Ideas
  • 4.10 Lan Na Ideas
  • 4.09 Scottish Ideas
  • 4.09 Hanoverian Ideas
  • 4.08 Jurchen Ideas
  • 4.08 Lur Ideas
  • 4.06 Trebizond Ideas
  • 4.06 Bregenzer Ideas
  • 4.06 Arakanese Ideas
  • 4.06 Dortmund Ideas
  • 4.04 Ilkhanid Ideas
  • 4.04 Naxian Ideas
  • 4.04 Ouchi Ideas
  • 4.03 Chosokabe Ideas
  • 4.03 Québécois Ideas
  • 4.02 Tsutsui Ideas
  • 4.02 Luxembourg Ideas
  • 4.01 Isshiki Ideas
  • 4.01 English Ideas
  • 4.00 Farsi Ideas
  • 3.98 Aztec Ideas
  • 3.96 Ormond Ideas
  • 3.93 Ajuuraan Ideas
  • 3.93 Shan Ideas
  • 3.92 Luban Ideas
  • 3.91 Semien Ideas
  • 3.91 Tuscan Ideas
  • 3.91 Afghan Ideas
  • 3.90 Chinese Ideas
  • 3.90 Texan Ideas
  • 3.88 Finnish Ideas
  • 3.88 Savoyard Ideas
  • 3.86 Armenian Ideas
  • 3.86 Bahmani Ideas
  • 3.86 Rothenburg Ideas
  • 3.85 Great Ming Ideas
  • 3.85 Hormuz Ideas
  • 3.84 Beninese Ideas
  • 3.82 Offaly Ideas
  • 3.81 Huron Ideas
  • 3.81 Berg Ideas
  • 3.80 Cornish Ideas
  • 3.79 Songhai Ideas
  • 3.79 Pskovian Ideas
  • 3.78 Hatakeyama Ideas
  • 3.78 Maratha Ideas
  • 3.77 Breton Ideas
  • 3.76 Punjabi Ideas
  • 3.76 Brazilian Ideas
  • 3.76 Amago Ideas
  • 3.75 Sonoran Ideas
  • 3.75 Persian Ideas
  • 3.74 Odoyev Ideas
  • 3.74 Thomondian Ideas
  • 3.74 Samtskhe Ideas
  • 3.73 Mainzian Ideas
  • 3.73 Tirhuti Ideas
  • 3.72 Ando Ideas
  • 3.72 Dhundhari Ideas
  • 3.71 Ruthenian Ideas
  • 3.70 Yemeni Ideas
  • 3.70 Chiba Ideas
  • 3.69 Mayan Ideas
  • 3.68 Clevian Ideas
  • 3.67 Gujarati Princedom Ideas
  • 3.66 Shawnee Ideas
  • 3.66 Estonian Ideas
  • 3.65 Krakowian Ideas
  • 3.64 Piratical Ideas
  • 3.64 Javan Ideas
  • 3.63 Garhwali Ideas
  • 3.63 Moldavian Ideas
  • 3.63 Catalan Ideas
  • 3.62 Bayreuther Ideas
  • 3.60 Mexican Ideas
  • 3.59 K'iche Ideas
  • 3.58 Vermont Ideas
  • 3.58 Anhalt Ideas
  • 3.58 Milanese Ideas
  • 3.58 Air Ideas
  • 3.58 Utsunomiya Ideas
  • 3.55 Hessian Ideas
  • 3.55 Mesoamerican Ideas
  • 3.54 Ashanti Ideas
  • 3.54 Shirvani Ideas
  • 3.54 Tyrconnell Ideas
  • 3.53 Malabari Ideas
  • 3.53 Goslar Ideas
  • 3.53 Lan Xang Ideas
  • 3.51 Yi Ideas
  • 3.50 Pagarruyung Ideas
  • 3.50 Garjati Ideas
  • 3.49 Leinster Ideas
  • 3.49 Theodoro Ideas
  • 3.48 Mori Ideas
  • 3.48 Genevan Ideas
  • 3.48 Circassian Ideas
  • 3.48 Ladakh Ideas
  • 3.48 Orleanaise Ideas
  • 3.47 Nepalese Princedom Ideas
  • 3.47 Gutnish Ideas
  • 3.46 Miao Ideas
  • 3.45 Muiscan Ideas
  • 3.45 Nuremberger Ideas
  • 3.44 Siddi Ideas
  • 3.44 Laotian Ideas
  • 3.44 Shiba Ideas
  • 3.44 Ardabili Ideas
  • 3.43 Neapolitan Ideas
  • 3.42 Sinhalese Ideas
  • 3.42 Bavarian Ideas
  • 3.41 Portuguese Ideas
  • 3.41 Yaroslavlyian Ideas
  • 3.41 Sadiyan Ideas
  • 3.41 Northumbrian Ideas
  • 3.40 Transylvanian Ideas
  • 3.39 Kutai Ideas
  • 3.39 Takeda Ideas
  • 3.39 Creek Ideas
  • 3.39 Sami Ideas
  • 3.37 Mewari Ideas
  • 3.36 Bolognese Ideas
  • 3.35 Fulani Jihad Ideas
  • 3.34 Welsh Ideas
  • 3.34 Munster Ideas
  • 3.34 Imerina Ideas
  • 3.34 Polotskian Ideas
  • 3.34 Pacific Northwest Ideas
  • 3.33 Kurdish Ideas
  • 3.33 Ava Ideas
  • 3.31 Tverian Ideas
  • 3.31 Slovak Ideas
  • 3.31 Kangra Ideas
  • 3.31 Tyrone Ideas
  • 3.30 Croatian Ideas
  • 3.29 Deccani Sultanate Ideas
  • 3.27 Interlacustrine Ideas
  • 3.23 Cherokee Ideas
  • 3.20 Wurzburgian Ideas
  • 3.18 Vijayanagar Ideas
  • 3.17 Mutapan Ideas
  • 3.16 Frankfurter Ideas
  • 3.16 Bengali Ideas
  • 3.16 Meath Ideas
  • 3.16 Barbary Corsair Ideas
  • 3.15 Sumatran Ideas
  • 3.14 Arabian Ideas
  • 3.14 Dutch Ideas
  • 3.12 Guarani Ideas
  • 3.12 Lunda Ideas
  • 3.12 Vindhyan Ideas
  • 3.11 Beloozero Ideas
  • 3.10 Swahili Ideas
  • 3.10 Gelre Ideas
  • 3.10 Nubian Ideas
  • 3.09 Hosokawa Ideas
  • 3.09 Highlander Ideas
  • 3.08 Korean Ideas
  • 3.08 Danish Ideas
  • 3.08 Zambezi Ideas
  • 3.08 Couronian Ideas
  • 3.07 Mindanao Ideas
  • 3.06 Evenk Ideas
  • 3.03 Incan Ideas
  • 3.03 Saxe-Lauenburg Ideas
  • 3.02 Serbian Ideas
  • 3.00 Arawak Ideas
  • 3.00 Tripuran Ideas
  • 3.00 Galician Ideas
  • 2.99 French Ducal Ideas
  • 2.98 Kuban Ideas
  • 2.97 Veronese Ideas
  • 2.97 Ulmer Ideas
  • 2.91 Baluch Ideas
  • 2.91 Ainu Ideas
  • 2.88 Aragonese Ideas
  • 2.88 Mamluk Ideas
  • 2.87 Corsican Ideas
  • 2.87 Toki Ideas
  • 2.87 Swiss Ideas
  • 2.85 Hejazi Ideas
  • 2.84 Pisan Ideas
  • 2.83 Orissan Ideas
  • 2.83 Lusatian Ideas
  • 2.80 South Indian Ideas
  • 2.80 Caspian Ideas
  • 2.79 Mazovian Ideas
  • 2.79 Lüneburger Ideas
  • 2.78 Cypriot Ideas
  • 2.78 Urbinate Ideas
  • 2.78 Montenegrin Ideas
  • 2.78 Tapuian Ideas
  • 2.76 Jaunpuri Ideas
  • 2.75 Ulster Ideas
  • 2.72 Silesian Ideas
  • 2.72 Zaporozhian Ideas
  • 2.70 Hausan Ideas
  • 2.68 Norman Ideas
  • 2.67 Tunisian Ideas
  • 2.66 Hamburger Ideas
  • 2.64 Sligonian Ideas
  • 2.64 Malayan Sultanate Ideas
  • 2.64 West African Ideas
  • 2.63 Gujarat Sultanate Ideas
  • 2.63 Ayutthayan Ideas
  • 2.62 So Ideas
  • 2.61 Caucasian Ideas
  • 2.59 Siberian Ideas
  • 2.58 Danziger Ideas
  • 2.57 Lorraine Ideas
  • 2.57 Ragusan Ideas
  • 2.57 Sistani Ideas
  • 2.56 Malagasy Ideas
  • 2.56 Indian Sultanate Ideas
  • 2.56 Malvi Ideas
  • 2.54 Clanricarde Ideas
  • 2.54 Ionian Ideas
  • 2.52 Betsimisaraka Ideas
  • 2.52 Mogadishan Ideas
  • 2.51 Dahomey Ideas
  • 2.49 Brabant Ideas
  • 2.49 Pomeranian Ideas
  • 2.47 Dali Ideas
  • 2.47 Permian Ideas
  • 2.46 Pueblo Ideas
  • 2.44 Al-Haasa Ideas
  • 2.44 Omani Ideas
  • 2.43 Cham Ideas
  • 2.43 Smolenskian Ideas
  • 2.42 Ferraran Ideas
  • 2.42 Hanseatic Ideas
  • 2.40 Berber Ideas
  • 2.40 Italian Ideas
  • 2.39 Chimu Ideas
  • 2.39 Baden Ideas
  • 2.38 Frisian Ideas
  • 2.37 Cebu Ideas
  • 2.37 Chachapoyan Ideas
  • 2.37 Moroccan Ideas
  • 2.36 Nivernais Ideas
  • 2.35 Kongolese Ideas
  • 2.35 Telugu Ideas
  • 2.33 Albanian Ideas
  • 2.31 Sienese Ideas
  • 2.30 Pattani Ideas
  • 2.26 Ashikaga Ideas
  • 2.25 Somali Ideas
  • 2.24 Transoxianian Ideas
  • 2.23 Flemish Ideas
  • 2.14 Mahri Ideas
  • 2.06 Cascadian Ideas
  • 2.01 Horde Ideas
  • 2.00 Aymaran Ideas
  • 1.99 Khivan Ideas
  • 1.98 Luzon Ideas
  • 1.93 Luccan Ideas
  • 1.91 Tupi Ideas
  • 1.78 East Frisian Ideas
  • 1.77 Navarran Ideas
  • 1.76 Charruan Ideas
  • 1.74 Andean Ideas
  • 1.69 National Ideas
  • 1.68 Chickasaw Ideas
  • 1.55 Venetian Ideas
  • 1.08 Genoese Ideas
  • 1.07 Kono Ideas
  • 1.04 Madyas Ideas
  • 1.03 Sulawesi Ideas

Identity Issue

So anyway, power level issues can be tweaked by just changing some numbers, no big deal. A much more interesting issue is lack of identity, or what playstyle should you be taking this idea group for?

The algorithm scores ideas for some generic playstyle assumptions. The further your playstyle is from generic, the more it makes sense to pick some idea group that's not generically best.

Some idea groups have great identity. For example it's really clear what Religious is about. It's about spreading your religion, by fighting infidels, and missionary work in your country. Or for a weaker group Trade - it has perfectly fine identity - it's all about making money from trading, and it does exactly what it says.

But for of the ideas have no such identity, and their name usually corresponds to nothing. Like what's "Defensive" ideas about? How is it about defending specifically? Not much at all.

Or how about Diplomatic and Influence? They both give some bonuses to diplomatic stuff, but does it make any sense when you should pick which one? Not really. Or even worse, how about Espionage? It's supposed to be about spies, but spies are not a meaningful EU4 playstyle, it's not CK2. So it's a bag of unrelated bonuses nobody's ever happy about taking. And now that AE reduction got moved from Influence to Espionage, none of Diplomatic/Influence/Espionage trio actually has much identity left.

And the worst of all are idea groups like Maritime, which both lack any sort of identity, and which are painfully weak in every playstyle.

So here's some of my thoughts. What if every idea group was buffed to be playable, and also had some strong identity, so you know that for certain playstyle it actually makes sense to pick it. I grouped them by how well defined they already are.

Idea groups that already work

Administrative

With governing capacity change, it's pretty much a mandatory pick for most playstyles, and doesn't need help.

1.30 weakened its identity. Before that, it was mostly just had coring cost reduction, so if you wanted to play wide (a default playstyle) and expanded mostly by coring, administrative was very good, but not necessary. For tall play, it just didn't do much. Or for expanding by integrating subjects.

It also had state limit increase. Due to the way pre-1.30 trade companies worked, it wasn't really as big a deal as devs wanted, since most of you 0% autonomy land would just belong to a trade company anyway. But state limit also said "take this if you want to play wide".

Now with governing capacity change, wide play makes administrative nearly mandatory, whichever way you expand, but tall play and developing your medium-sized country a lot is also going to run into a lot of governing capacity issues, so administrative are a good idea either way. But they're still especially useful for wide play.

Since administrative ideas are so strong, the best way to improve its identity is not strengthen it, but to give other idea groups some ways to save on governing capacity.

Exploration and Expansion

Back when EU4 was released these two used to have very clear identity. Exploration was about colonizing, especially Americas. Expansion was bout colonizing, especially Africa and Asia.

Back then Exploration had 2 Colonists, perfect for mostly empty America, and gave very good CB on New World pagans, so it was about colonizing there. And Expansion had just 1 Colonist but gave very good CB on Asia and Africa (or whichever parts of Old World you're not based in), so it was about conquering Old World with some colonizing on the side - also perfect match as those lands had a lot fewer uncolonized provinces and a lot more easy pickings of technologically backwards nations.

Somehow second colonist got moved to Expansion, both special CBs got lost (and even the shitty replacement of claim fabrication in trade company regions got removed in 1.30), so they're both really mainly about colonizing the New World, with side serving of the Old.

I think it's best to treat it as a single idea group that just so happens to take two idea group slots. Usually you'll take Exploration as your 1st, Expansion as your 2nd or 3rd.

Taking just one of them really isn't advisable. Maybe Expansion only at 4th or 5th if you want to join colonization late. Or taking Expansion only if you're in Asia and really want to get free Colonialism institution and don't care much about colonists.

So while it would be nice to restore old system where each of them corresponded to part of the world you might want to focus your expansion into, they're good enough as is. And they definitely don't need any buffs, colonists are already one of the best bonuses you can get.

Humanist

Its main selling point is that you don't need to care about peasants' religions, peasants' cultures, and rebellions in general.

From power level point of view, I think it's strictly worse than Religious, since rebels in EU4 are ridiculously weak, so it's no big deal to just crush them whenever they show up. It also lacks amazing Deus Vult from Religious, and you usually only want to take one of the two.

But it definitely saves a lot of rebel fighting micro, so a lot of players love it, and that's clear enough identity.

Religious

It's extremely strong, and has very strong identity - spread your faith by fighting infidels, spread your faith by converting peasants, and have a stable country as result of its religious unity. It doesn't need any help.

Idea groups that just need a little buff

Aristocratic

It's a fairly unfocused bag of bonuses, but I think it can be really good with just a small tweak.

Aristocratic sort of focuses on helping cavalry, but on base level EU4 cavalry is just so awful such bonuses just don't mean much. With no bonuses you should never bother with cavalry. Even with fairly big bonuses cavalry is just barely worth it, and only huge bonus stacking (like going Aristocratic + Tengri + Commonwealth) make it actually good.

If Aristocratic just about tripled its cavalry bonuses, it would gain strong identity without actually making it all that strong. If you take Aristocratic, it would incentivize going from 16inf 4cav 10art armies to 12inf 8cav 10art, or something along those lines, and perhaps even keep that cavalry into late game.

Economic

In principle an idea group saying "play tall" and "earn more money" is fine, but those bonuses are just too low. Especially changing construction costs from -10% to let's say -30% could allow countries to afford all those temples, workshops, and manufactories more easily.

It should probably get bigger development discount as well for playing tall (like -20% to -30%). Various administrative groups are among the strongest, so it needs a big buff to be competitive.

Offensive

Offense primarily means wanting to siege opponent's forts faster, and offensive already does that, but its bonus could be a lot stronger. Doubling down on this, like by adding +1 Leader siege, or +2 max artillery bonus (so you can siege faster, but only if you bring tons of artillery there and pay attrition price) would really help.

I think that paradoxically giving some siege defense would also really help offensively focused countries, as siege racing is how you win fast offensive wars in EU4.

Trade

It's about trade, and gives decent trade bonuses, but it could definitely be stronger.

It could also give big bonuses to trade company provinces - like lower minimum autonomy, lower governing capacity cost, lower trade company investment cost, or such.

If your trade companies had lower governing capacity cost, and were overall more useful, then you could play wide without going administrative - at least as long as you mostly expanded with trade companies, not states.

It could also add some small bonus to trade fleets, like let's say +25% Naval force limit modifier, as trade oriented nations need naval force limit more.

Quality

It has some kind of identity - it wants to make your army stronger per soldier / ship. It just doesn't do enough.

Increasing +0.5 yearly army tradition to like +2 yearly army and +2 yearly navy tradition would greatly round up this idea group, by making not just common soldiers but also army leadership much stronger.

Quantity

It has strong identity - it lets you have bigger army. It's just not clear who would want that. Is it meant for a small country which wants to make its army bigger; or for an already big country like Russia or Ming which wants to make its big army even more enormous?

It needs a buff, and it's actually clear what kind of buff - your huge army will never have enough generals, so like +2 leader limit and maybe a -20% leader recruitment cost discount, will really round it up.

Idea groups that need stronger identity

Defensive

The group claims it's about defending better, but what does that actually mean? EU4 attrition is capped at such a low level, and manpower is so abundant, that there's no hope of ever running the attacker out of manpower by attrition, the way that's totally viable in CK2.

Small bonus to fort defense would actually be far better if you're on the offense. If you're defending, it just delays the inevitable loss slightly. And even if you successfully defend your territory you're still at nearly 0% warscore and with AI unwilling to give you anything for your trouble.

Really the only way this idea group makes any sense is by massively buffing your combat abilities as long as you're on your core territory. It's not exactly an original idea. CK2 "Defensive Pagans" have it. Civ5 Ethiopia has it. So many games have this. I'm surprised EU4 doesn't have anything like that.

Some nations already get +1 naval combat bonus off owned coasts, but there are no bonuses of any kind for land combat on own territory.

-20% Local hostile movement speed would probably do, especially since it stacks with Scortched Earth so well, but it would need to be applied to every province owned by nation with Defensive ideas by an event, so it would be fairly messy to mod.

Without heavy modding, I think +20% movement speed would actually do nicely. It sounds offensive, but actually in EU4 offense is about sieges, and defense is about getting into a advantageous battles to bleed out the attackers. Buffing this weak group with more siege defense and enemy attrition wouldn't hurt either, especially if attrition cap got increased from 5% to something more meaningful like 10%.

Diplomatic

When EU4 came out, Diplomatic used to be a stupidly good idea group. Then it got split into Diplomatic and Influence, nerfed a bunch, and finally some of their bonuses got moved to Espionage to shore that up.

So what Diplomatic / Influence / Espionage each do? They all make your diplomacy betters with a fairly random mix of bonuses each. Fortunately with some simple rearrangement we could give them solid identity.

Let's start with Diplomatic - it will be an idea group for making other countries tolerate your aggressive expansion.

It already has +25% Improve relations, −20% Province warscore cost, Lowered impact on stability from diplomatic actions, and +2 diplomats. All it needs is getting that sweet −20% Aggressive expansion impact from Espionage, and −50% Unjustified demands from Influence.

It could then lose +1 Diplomatic relation or +2 Diplomatic reputation, as these are very strong but don't really fit in its new core identity of helping expand aggressively.

Influence

If Diplomatic is about smoothing out diplomacy for aggressively expanding countries, what should Influence do? Expanding through subjects of course!

First it absolutely needs May fabricate claims for subjects from Espionage, and it could get bigger Diplomatic reputation bonus, and possibly +1 Diplomatic relation if Diplomatic loses it.

−25% Diplomatic annexation cost could be increased to maybe even as much as -50%. Expansion through subjects used to be really effective but now just so much worse than expansion through coring, that we could definitely use a bonus bigger than Administrative ideas' −25% Core-creation cost.

It would be a great pick for a country that has a lot of PUs through missions and no free annexation by button, like Bohemia.

Naval ideas actually have decent identity - they're about making your navies win battles. It's just that in EU4 this is completely irrelevant 99% of the time, and maybe worth throwing some money at, but definitely not a whole idea group slot.

At the very least it needs to give an extra leader slot or two, so you can actually hire some admirals without starving your land army of leadership (I vaguely recall that next patch is supposed to have separate admiral limit, so that problem might go away).

Even throwing in every single modifier from Naval and Maritime into a single idea group, and then a bit more, would really make a completely underwhelming idea group nobody has any reason to take.

One way to help it would be by increasing importance of naval combat, and removing all straits could do just that, but that would really mess up with AI and overall game balance.

Plutocratic

Plutocratic is a fairly decent but not very coherent group of bonuses for republics (except Noble republics, but they're rare and bad) and also somehow plutocratic monarchies (basically a rare Indian Ocean thing).

Plutocratic could easily become "play a republic" idea group if it had some modest republican tradition bonus like +0.25.

That wouldn't help plutocratic monarchies, but they're weird hybrid already, so that's fine.

Idea groups I don't know what to do about

Espionage

Third "make diplomacy better" group after Diplomatic and Influence, and by far the worst. And I want to make it even worse by moving some of its bonuses to Diplomatic and Influence, as they're close enough to fixable.

The worst thing about Espionage ideas is that EU4's espionage system is just terrible.

Most covert actions unlock really late, and most of them - like Slander Merchants, Support Rebels, or Sabotage Recruitment, are a total waste of time.

Somehow EU3 had a lot better spy system - spies were separate from diplomats, and you needed ideas to get any spies so they could be more powerful. They had abilities such as:

  • Infiltrate Administration - removes fog of war for a while (it's in EU4 but at adm tech 30 so game never gets there)
  • Bribe defenders - free siege progress
  • Commission privateer - creates pirate naval rebels
  • Support revolt - actually generates rebellion (in 5 different variants) - in early EU4 patches it still sort of worked, by now it's completely useless

And a bunch more.

EU4 really needs drastic changes to spy system for Espionage ideas to do their job. Some ideas:

  • make Espionage ideas unlock espionage abilities like Infiltrate Administration early
  • or just make those abilities happen early for everyone
  • make Support rebels just actually spawn the damn rebels

And it could use some new spy options. Most obvious one would be CK2 style ruler/heir assassinations, but balancing that one looks like hell. Especially if AI starts using it on you.

Some other ideas for new actions:

  • create royal marriage regardless of target's opinion
  • if you have royal marriage, give them heir of your own dynasty (or give yourself heir of their dynasty)
  • steal money (scaling that properly would be difficult)
  • bribe enemy general to switch to your side
  • reduce AE with target at price of some gold (like 100 gold and 25 spy points for -10 AE or something)
  • fabricate other CBs, like Subjugation CB

Unfortunately implementing them in balanced way, especially if they're meant for both player and AI, is not going to be easy.

The easiest way to make Espionage at least a little good would be to make Support rebels actually usable (like +50% monthly rebel tick chance, not +10%, and at far lower money cost, maybe zero, as it's currently ridiculous relative to what you get), make late game covert actions like Infiltrate Administration available earlier, and then increase bonuses for this group anyway.

Innovative

By name this group wants to be about being ahead of everyone else in technology. Except that is just plain impossible in EU4. Especially since they introduced Institutions, the whole world - from Western Europe all the way to African jungle - has the same military technology plus or minus maybe 2 techs (25 years difference), and maybe up to 4 techs (50 years) in adm/dip tech, but those barely make an impact on anything. In reality technology difference between Europe and rest of the world was between centuries (Asia) to thousands of years (Africa and best parts of Americas) to literally tens of thousands of years (Australia and rest of Americas). Especially in ocean going naval technology, Europe was like in a completely different universe compared with what everyone else was doing.

You have no way of getting ahead of the pack, as 10% per year ahead of time penalty is so brutal, even throwing away all awful you can't go more than a few years ahead of everyone else.

Recent patch introduced "innovativeness" mechanic, which is basically a bonus for being ahead of tech, because techs themselves are quite irrelevant. It helps with strategy of going for very high (or even 100%) innovativeness, which is quite achievable for the player as most Western European countries, but totally impossible for the AI.

The whole world has only 302% innovativeness to share between each other, from all techs and ideas, so that's like 1-2% per country. Maybe a bit more as there's events that give innovativeness, and it's possible for multiple countries to gain bonus for the same tech if they get it within 90 days of the leader, but really it's a weird race.

If you don't try to go for max innovativeness, the idea group is really unfocused.

One thing that could work would be reducing ahead of time penalty for techs if you have innovative, so you only pay 5% per year not 10% per year. This cannot be modded directly, so it would need to be a huge list of triggered modifiers based on current year, current tech, and that tech's year. Or the whole "ahead of time" system could be replaced with something less punitive and more dynamic, but the game heavily relies on current system.

Innovative cannot really be saved by just increasing its bonuses. Technology is gained by monarch points, so if we just give Innovative a lot more monarch points - like by huge tech/idea discounts, and advisor cost discounts, then it will be overpowered for everything - while not actually making you take technologies any faster.

Maybe it could increase government reform speed? It's currently painfully slow to get to the good stuff at the end, so it would be one option to do so faster.

Maritime

It's a total dupster fire, even more so than Naval.

It sort of has one use - if you go over force limit on light ships protecting trade, and stack all other naval tradition modifiers to get permanent 100%, you get really huge trade steering bonuses, so your long chain of dominated trade nodes gan give you ridiculous amount of income. Maritime's +100% Naval tradition from protecting trade can actually be quite useful to achieve this, but really if you ever get to this point, then you control half of the world already, so the game is over regardless.

Making it all about trade companies could be a thing, but really Trade already wants those trade company bonuses, and making Maritime "second Trade ideas, just worse" really doesn't seem that appealing.

One way to help it would be to make it "second Trade ideas, except not worse", throwing in some big trade steering and light ship trade power bonuses, maybe an extra merchant, and 5% trade efficiency.

It would probably be better to just remove it from the game completely, and divide its flimsy bonuses between Naval and Trade.

What could have idea group

If we drop Maritime, or just wanted to introduce a completely new group, either as replacement, or an extra, what could that be? Here are a few ideas.

Industry Ideas - give bonuses to goods produced, production efficiency, and development cost. As finisher it lets you change trade goods in a province you own if you develop it enough. So that Napoli at 22dev making crappy Grain? Maybe my throwing some money or points you could change it to Wine or Paper. What about Grain producing mountain in Avellino next door? Maybe deving it up to 20 would flip it to Iron or Copper. The whole system needs to consider terrain, climate, and such things to not do anything crazy, but modest trade good changes wouldn't break the game.

We could even do some limited terrain changes, replacing Forest or Marsh with Farmland, but right now there's not terribly much incentive to do so - it would save you some dev cost, but mostly just remove your defense bonus.

I'm not sure what would be other good idea groups that would fit within framework of the game.

Game changes it all requires

So to recap, to make it work I'd need a few game changes beyond idea group bonuses:

  • remove most or all strait crossings (for Naval)
  • higher attrition cap (for Defensive matters)
  • redo spy system (for Espionage)

That brings 17/18 idea groups into fairly good state. I really don't know how to salvage Maritime.

If you have any fun ideas about rebalancing existing ideas differently or adding new ones, let me know in the comments.

Saturday, December 05, 2020

Tier Zoo, or why Gorilla is a F Tier Trash

Howler Monkey by Terry Kearney from flickr (PUBLIC-DOMAIN)

Tier Zoo is a popular YouTube channel with analyses of animal metagame, placing each build in one of S, A, B, C, D, or F tiers.

It's highly entertaining and has high production values, however its analyses are often just as bad as Binkov's tier lists for another popular game.

Notable thing almost always missing from Tier Zoo's analyses are metagame share statistics maps (which I'll quote extensively). Using hard data is always preferred to relying exclusively on purely subjective criteria.

Another problem is that tiers are never clearly defined. But what really inspired this is ridiculous rankings like in this video, which claims that Gorilla - critically endangered jank build that was never even remotely playable - is somehow S tier, same as the most ridiculous build of last billion years Human.

To put metagame discussion on firmer settings, here's my suggestion what those tiers should be. But whichever tiers you choose, under no definition of tiers is Gorilla trash of any value whatsoever.

I'll mostly use Mammal builds as examples, due to likely reader's familiarity, as my statistics show that non-Mammal players very rarely visit this blog.

There's a related issue that there's often disagreement on what should be counted as same or separate build. Some analyses count every tiniest variation for every server separately, I don't think that's all that helpful, and I'd rather count all closely related variants with about the same stats and gameplay as single build. Reasonable disagreement is definitely possible here, and same data can be re-analyzed differently.

S+ Tier

S+ Tier warps the whole metagame around itself.

By definition there can't be more than one such build at a time.

There's just one S+ Tier build and that's Human. There's a reason the current version is called Anthropocene Patch.

To find any place not affected by Human build, you'd need to run to Deep Sea meta, but not for much longer.

The build is so ridiculous, devs are about to double the number of servers to provide some challenging environments to Human players, as every existing server is too damn easy.

Looking back, the only other S+ tier build to ever exist were the first Oxygen Producing Bacteria, but some predict robots might be the next S+ tiers.

S Tier

S Tier are builds that are great across many different metas, and very often take over any server to which they're introduced

Anthropocene has many S Tier builds, and if you're looking for an easy game, IUCN created a nice list of most builds that easily take over new metas.

  • Cat - pretty much the only build that's insanely successful in both Human Support and Wild metagames and can almost freely switch back and forth between them
  • Pig - Human Support builds often go Wild, but never back
  • Rat
  • Fox
  • Mouse
  • Red Deer
  • Goat

I only disagree with one thing about their list - invading Australian or New Zealand meta hardly makes a build S tier, those metas were pretty much defined by low tier trash with some mid tiers mixed in, and anything half-decent could have dominated them.

Before Anthropocene there weren't really that many S tier builds, as slower overall pace of metagame change meant that even successful build usually had different sub-builds on different servers.

For example Small Cat build's metagame share before Anthoropocene patch was already enormous, even if nitpickers point out that technically these aren't all quite identical builds, and some vary quite meaningfully from base build.

The most notable pre-Anthopocene S Tiers, even without counting sub-builds:

Really, nothing should ever be described as S Tier until it has track record of dominating server after server. Calling total trash like Gorilla an S Tier is just laughable.

This tier is quite well defined, and there's rarely much doubt if something belongs here or not.

A Tier

A Tiers are builds that are strong across wide range of metas, and resilient to meta change, even if they might struggle to expand to very different metas, or drastic patches like Anthropocene Patch, or coming Global Warming Patch.

Some notable pre-patch A-Tiers which are:

They were never as successful as Grey Wolf or (counted together) Small Cat builds, but it's still extremely impressive performance.

Build like Brown Bear is still very clearly A tier post-patch, Lion and Tiger would arguably be more B tiers if evaluated just by their post-patch performance.

Some A Tiers which are great as Human Support, and also do fine in Wild metas, but not really to the point of being easily invasive. Notable example is:

B Tier

B Tiers are a builds that are really damn good in its meta, and that's about it. They're often very vulnerable to metagame change, and unlikely to expand beyond.

Let's face it - this is the highest any non-Human Ape ever got.

Some notable builds:

  • Chimpanzee
  • Bonobo
  • Various Macaque builds - taken together they do quite well, but most sub-builds really struggle
  • Capybara

Anthropocene is also full of B Tiers that are super successful as Human support classes, but really struggle otherwise like:

  • Cow

It's also best to describe as B tier builds that are fairly widespread, but only on low competition servers like:

C Tier

C Tiers are builds which survive in its meta, but doesn't even do anything special there.

This is actually where most builds belong you've probably never heard of. They're neither good enough nor bad enough to be notable.

Some examples include:

  • Red Panda - not doing great, but definitely far better build than Giant Panda
  • Hamster - mediocre as Human Support, mediocre in the Wild

D Tier

D Tiers are builds which survive in its meta, but are just plain bad even there.

Notable example:

  • Koala - eating highly toxic food with extremely little nutritional value, having tiny brain so without any ability to adapt, and being riddled with STDs to boot. The only reason this build isn't extinct is that nobody else wants to get anywhere this trashy lifestyle. Their one and only strong point keeping them above F tier is that nobody else wants to get anywhere this trashy lifestyle.

F Tier

F Tiers are builds which are trash even in metagames where they somehow manage to survive; and which completely miss the point by having abilities completely mismatched to their gameplay

Notable examples:

  • Giant Panda - infamous F tier trash - only ever existed in one narrow meta, carnivore digestive system and build, and lifestyle based on eating bamboo which they digest poorly; can't even reproduce properly, in Wild or as shitty Human Support
  • Gorilla - another F tier trash and inspiration for this post - build features potential for powerful brain, but it doesn't use it for anything at all! Instead of hunting in pack or alone, or somehow accessing high value food, or even raiding Human trash piles, it never hunts anything bigger than insects, and eats very low quality food like some kind of Cow. Except it lacks Cow's specialized digestive system. To compare just how trash Gorillas' XP sources are relative to other Apes, they eat estimated ~18kg food per day, compared with Chimp's ~2kg, in spite of being only 3x heavier (those estimates not hard data, but they definitely don't eat alike). They're very strong, and they just fight each other, never using their strength for anything else. It's a really just a meme build. It's like some Ape player once decided "I wish I was more like a Cow", and did this extremely poorly. And to add insult to injury, they're close enough to Human build that they can get Human diseases like Ebola, but without benefiting from any of Human abilities - the worst combination ever. Of course Gorilla was never a popular build, and now the only thing standing between it and extinction are some Human weirdos which really want this trash to survive for their entertainment.

I hope this post introduces some clarity to the world of Animal Tier Lists, and helps you choose a better build.

I totally understand that people might differ in their judgement by one tier, or argue if group of related builds should be treated as one build or not - this happens in every game. But I really hope nobody ever calls F tier trash like Gorilla an S tier ever again. If you do, you're just making a fool of yourself.

Thursday, November 26, 2020

Empires are not about economic exploitation of periphery by the core

Pacificats Floris by Pacificat Ragdolls from flickr (CC-NC-ND)

I recently discovered a fun blog "A Collection of Unmitigated Pedantry", and it's been very enjoyable reading, but one of the posts is so wrong I need to write a rebuttal.

The author in Why Are There No Empires in Age of Empires? comes up with the following thesis:

An empire is a state where the core ruling population exercises control and extracts resources from a periphery which is composed of people other than the core group (linguistically/culturally/ethnically/religiously distinct). So an empire is a state where one set of people (the core) extract resources (typically by force) from another set of people (the periphery).

And so...

And that's how we know that "United States" is really a Puerto Rican Empire, as Puerto Rico is sure sucking up vastly greater flow of resources than it's ever getting back. The ruling Puerto Rican elite is literally not even paying income tax like those exploited mainlanders!

The so called "Russia" is really Great Chechen Empire, they sneakily conquered it by losing the war.

The so called "China" is really Greater Tibetan Empire.

The "United Kingdom" is really Scottish Empire, just look at all the money they're extracting from England!

And the earlier so called "British" Empire was really ran by the Americans - the exploited Brits were paying 10x as much in tax per capita than the ruling Americans.

And during Scramble for Africa, Africans managed to colonize Europe, as sure as hell average European was paying for all that elite map painting, and didn't benefit in any way whatsoever.

And so on.

This is all nonsense

If you look at any actual empires, especially recently, the overwhelming pattern is that burden of maintaining the empire - in terms of both money and manpower - falls disproportionately on core population, and peripheral populations are mostly expected to not cause trouble, and are otherwise net economic drain.

Exceptions happen, and occasionally core areas find some place extremely exploitable, but they're just that - rare exceptions. Like in 1700s' French colonial empire, tiny Guadeloupe was somehow economically worth far more than a quarter of North American continent.

Most of the time, those peripheral taxes are far less than core population pays, and far less than costs of holding those places in the Empire.

Foreign elites

There's a distinct and far more common situation where foreigners conquer some place, displace local elite, and rule as a dynasty or nobility over far more numerous locals.

This would actually fit in the original definition, except as a general rule the conquerors move to their conquered lands, so it feels a bit silly to talk about "Norman Empire" which lost Normandy rather quicky, or "Manchu Empire", which was absolutely ruled from within China, and is generally labelled "China" in history books, or such.

Exceptions proving the rule

Perhaps if one keeps looking, it's possible to find some "empires" where core successfully and sustainably extracts resources from periphery, especially in more ancient times.

Roman Empire is actually halfway there, as it managed to exploit places like Greece and Egypt pretty damn well, but even Romans admitted that their periphery like Britain were a net drain. Eventually the ruling center of Roman Empire moved to its richest province, and the usual pattern was restored.

In any case, such exceptional cases cannot possibly be used as definition of an empire.

Monday, November 16, 2020

Mechs vs Tanks

Miisa by andymiccone from flickr (PUBLIC-DOMAIN)

Sci-fi and anime loves mechs, but there's a widespread belief that mechs are just absolute trash idea, and tanks are superior in every way.

Here's example of such claims: part one, part two.

This is completely wrong.

Cube Square Law

Part of the problem is conflating the idea of a mech, with the idea of a giant mech. Mechs can come in all sizes.

And in fact military is already actively researching mechs! Just among many such projects - DARPA's TALOS project is an bulletproof, weaponized, AI-enhanced exoskeleton - basically a small mech.

It might take a few decades before such mechs become deployed by actual armed forced, but it's ridiculous to call something crazy when prototypes have been in development for years now.

Weapons vs Armor

History of warfare is an endless race between weapons and armor, with either being on top at different times.

During the Middle Ages, armor was king, and a fully armored knight was nearly invulnerable to whatever peasants could throw at them. But just a few short centuries later, no amount of armor humanly possible to carry could stop a musket bullet, so weapons were king, and armor got abandoned completely. Ironically the Peak Armor era was time when armor was already on verge of obsolescence, but it was easier to double down on old strategy instead of rethinking everything.

When tanks were introduced, a big reason they were quite successful was protection offered by their armor. Common infantry weapons just couldn't do anything to them, but of course that didn't last long.

As weapons got better, tank armor got heavier, and fast. First tanks to see combat in World War I had 12mm of armor.

By end of World War II, armor reached 250mm. To still be functional, it had to have far more compact shape.

It wasn't really possible to keep increasing the weight, and modern tanks aren't really much heavier than WW2 tanks, but using advanced materials to increase protection, the race between weapons and armor continued for a while longer.

Passive vs Active Protection

Right now it looks like the weapons are getting ahead. Armor is by no means useless, but in 2006 Lebanon War, Iranian anti-tank guided missiles were able to penetrate the most modern tanks just fine.

Weapons keep getting better, and it doesn't seem like armor has much space left to improve.

All militaries can see that, and they're equipping their tanks and other vehicles with active protection systems, which attempt to shoot down incoming missiles before they hit.

It's tempting to think that active protection is just another phase in evolution of tanks, but that's just wrong.

Tanks are the form they are because that's the only way to have thick armor while keeping weight reasonable.

Active protection has no such limitations. It doesn't matter if you install it on a tank, a light vehicle, or for that matter on an exoskeleton.

If active protection ends up being the winner, then mechs work just as well as tanks. And then liberated from just one form factor, we'll likely see a wide variety of different shapes of military vehicles - some might still look like current tanks, but others might very well look like mechs of various sizes, or something completely different.

And if active protection ends up being a big loser? Then giant mechs make no sense, but then neither do tanks.

Power Considerations

Another issues with mechs of all sizes is difficulty providing enough power and energy storage for them. Right now tanks are just more efficient, and mechs and exoskeletons are not very practical.

Fortunately for mechs, battery and engine technology is improving at very rapid pace, so it's entirely reasonable to expect this issue will go away soon.

There are even ways to recharge from air or space without any physical contact.

Future Prediction

In our world, it's very likely that exoskeletons will become widespread over next few decades, just like drones before them.

It's also very likely that active protection systems will increasingly become main protection. This will likely mean future tanks becoming lighter, as all that heavy armor just won't matter all that much. Wider variety of (mostly autonomous) military vehicles will come into widespread use, with very few relying on heavy armor, and most on active protection, stealth, or just being really cheap and expendable like most types of current drones.

Giant mechs? That's more farfetched for now, but if we have this discussion in a few decades, at time when most armies consist of exoskeleton-clad infantry supported by drone swarms, this really won't sound that far outside realm of possibility.

So if even our world is on verge of having mechs, it's just ridiculous to think there's no place for them in any sci-fi world, where technologies might have developed along different lines, and trade-offs are different?

The "tanks are always superior" crowd is going to sound just as ridiculous as believers in eternal superiority of heavily armored cavalry were a few centuries ago.

Thursday, November 05, 2020

Prediction Markets were Batshit Insane this Election

. by Runs With Scissors from flickr (CC-NC-ND)

The elections are over. The biggest loser are the prediction markets.

As I'm writing this, Arizona has already been called for Biden. And PredictIt has it as 30% chance of Trump.

Trump has no mathematically possible way of winning. PredictIt has it him at 20%.

I talked shit about 538, and I was absolutely right, but even they in the end had Trump at just 10%. Models I'd consider more reasonable like The Economist's had Trump at 3%. Prediction markets had him at completely batshit insane 40%.

And as votes started coming, when Trump did a tiny bit better than expected (but at no point was anywhere near having any chance of winning), Trump peaked at completely ridiculous 72%! It was known long in advantage that election day vote will be a lot more Republican than mail in vote, and states were clear how much of each kind outstanding there is. Prediction markets completely missed that, and had Trump as a favourite just because Republicans mostly went there in person while Democrats mostly sent their votes by mail, and it was too hard for them to understand.

Everything I said turned out to be right. If polls were right, elections would have been very easy Biden victory, and the "normal polling error" benefiting Biden would turn them into a Reagan style landslide.

But not only that - "normal polling error" towards Trump, would still mean easy Biden victory. Even unusually high polling error - what we had this year - didn't bring Trump particularly close to competitive territory. Everything that possible could go right for Republicans were perfectly for them - and Trump lost anyway, because his chances were nonexistent.

Polls weren't great, but they were fine. They predicted Biden to win by about +8%. By the time counting will end, he'll end up closer to +4 or +5%. We won't know for a while, as California takes forever to count their votes, so popular votes always ends up more Democratic than it seems on election day.

It turns out Trump was indeed closer in tipping point states, so Electoral College gave me a few points, but it never managed to overcome that kind of lead. There's no reason to predict which party will be at an advantage in 2024.

Real markets work because they're dominated by sophisticated institutional investors, whose job it is to figure things out, and whose predictions are tested every day. We hoped that small prediction markets dominated by small scale amateurs tested once every 4 years would be just as good, but it turns out they failed utterly.

I'm definitely disappointed as for a while it seemed that we might get great rationalist tool for very little money. It turns out it takes more than that.

EDIT: It's November 13th now, Trump lost nearly two weeks ago, every state has been called long ago, most betting sites paid out on Biden (including one where I bet), and Trump is still 12% on PredictIt. Some people must really hate their own money.

Wednesday, August 26, 2020

Why 538 election model is wrong and Biden has an easy win ahead

#TrumpYourCat by jillccarlson from flickr (CC-BY)

538 releases a model predicting what they think is going to happen every elections, and this time they released a crazy one, where "Biden is only slightly favored to win the election", and Trump has insanely high 30% chance of  second term.

This is in stark contrast to everyone else's predictions. For example The Economist gives Trump just 9% chance of winning, and this is a much more sensible number.

In this post I'll explain why 538 believes elections are quite close, and why it's completely wrong.

It's not a terribly sophisticated analysis, but it's too long for a tweet, so I thought I'd put it here.

If you're interested, I wrote "A Guide to 2020 Elections for non-Americans" a while back, and I dislike both Trump and Biden very much.

Why 538 model believes what it does

Their model is very sophisticated, but its basic assumptions leading to their crazy conclusions are fairly straighforward:

  • Biden is far ahead in polls, however:
  • there's still a lot of time until elections
  • things are likely to change until then
  • especially since pandemic year is full of events
  • Electoral College benefits Trump, so he will likely win even if he losses popular vote narrowly
  • the economy went to hell earlier this year, but they think it will improve, and that will somehow benefit Trump

Why 538 is completely wrong - not much time

The only thing they got right is that Biden is indeed far ahead in the polls. They're wrong about everything else.

The elections are a lot closer than they think. In normal times, most people vote on Election Day. This time due to the pandemic a lot more people will vote by mail early, and mail voting opens about a month before Election Day, depending on the state. So for a lot of voters, time until elections is only half of what they think. And people might be voting even earlier as there's a lot of talk about Post Office having trouble processing votes in time.

Even if Biden says the dumbest thing he ever said one week until elections, or Trump single-handedly invents coronavirus vaccine, enough votes will be cast by then that it probably won't matter.

Why 538 is completely wrong - pandemic prevents most campaigning

People don't change political beliefs based on clocks ticking, they do based on events and campaigning. This time due to pandemic there's going to be a lot less campaiging, and people's attention will be largely on other issues.

It's not even certain if the usual debates will happen or not. And even if they do, only the first of the three is supposed to happen before mail voting starts.

Why 538 is completely wrong - people already made up their mind

If you compare Trump's approval ratings with all past presidents, Trump's are the least volatile.

People love Trump, or hate Trump, and there's a lot fewer people in between that could be convinced by the campaigns. So no matter how good Trump's campaign is, it will likely not matter. And likewise, while Biden will almost certainly win, he's also very unlikely to win by 1984 style landslide.

Why 538 is completely wrong - Electoral College advantage can go either way

In 2000 and 2016 elections, Democrats won the popular vote, but Republicans won the presidency. While this might give an impression that Electoral College system inherently benefits Republicans (the way Senate arguably does), if you look back both parties benefited from the system about equally often.

Right now it might look like 2020 map will be similar to 2016 map, and seems to benefit Trump. This thinking is completely mistaken.

If very little changes, Trump will lose terribly, and while he might indeed lose Electoral College by less than popular vote, it will be completely inconsequential.

For elections to be even competitive, fairly big changes must happen. But while it's possible, there's no reason to expect that they'll uniformly apply to all states (in proportion to their "swinginess").

For example if Biden says something that will offend a million Californians and makes them stay at home or vote third party, this will significantly reduce his popular vote share, but it won't matter for Electoral College one bit, since California will go Democratic regardless.

Trump needs not only a really big swing - he needs a swing that's either eerily uniform, or happen in just the right states.

In reality, if a major swing happens, it throws all assumptions about Electoral College advantage out of the window. Even if much bigger swing than could be reasonably expected happens, and Trump somehow manages to reach vote share similar to what he got in 2016 (losing popular vote by 2%), it's actually still quite unlikely that this swing will leave his Electoral College advantage intact, and get him another victory.

Why 538 is completely wrong - the economy will not favor Trump

A popular way to predict elections is by looking at "economic fundamentals". The thinking goes that whenever economy is doing poorly, people blame whoever is in charge for it, and vote for someone else. This is generally true, and after every major economic crisis like 1929 or 2008, governments all across the world got a lot less popular, regardless of which part of the political spectrum they were on, and how much or how little they had to do with causes of the crisis. There's no reason to expect 2020 to be any different.

538 really hates fundamentals models, and for a good reason. While bad "economy" affects people's voting preferences, there's no obvious way to pick a single number for what is the "economy". You can use unemployment, or GDP, or disposable income, or stock market, or so many other indicators, of combinations of indicators. So in year were economy is doing so-so, you can't really use those models to predict much.

Instead of stopping there, 538 somehow decided to ask experts where the economy will go just before elections, and since most expect some recovery from the worst of the lockdowns, they count this as benefiting Trump. While there's a room for reasonable disagreement how to model this, there's no way in hell 2020 economy could possibly benefit Trump, people's memories are longer than a few weeks, and US is still and will likely be in state of mass unemployment.

Are 2020 Elections a foregone conclusion?

Democrats are overwhelmingly likely to win the House - they'll win the popular vote, and there's not even a possibility of Electoral College stopping them. Republican House is about as likely as Nicola Sturgeon becoming the next Prime Minister.

Biden is extremely likely to win presidency - things would need to go really damn well for Trump.

Senate is another matter. Only 1/3 of it is reelected each time, and Republicans do have a meaningful structural advantage there, so it seems fairly close to even who's going to win. And it probably matters more than anything else this year.

If Democrats control the Senate, they'll be able to abolish filibuster, pass pretty much whatever they want, and then possibly also pack Supreme Court with ten new liberal judges that will pronounce all of that Constitutional. They probably won't go that far, but in principle they could.

In Democrats keep the House, and Republicans keep the Senate, then it will most likely be four more years of gridlock regardless of who's the president. The parties hate each other so much, they won't compromise on anything beyond the most routine operations of the government. Biden's policies will be different than Trump's, but presidential policies don't matter as much as people thing, an the next president in 2024 might very well reverse most of them.

And yes, I obviously put money where my mouth is.

Friday, August 14, 2020

Trying out Minecraft again after 10 years

Dolphin who followed me to my base, jumped onto the sand, and died tragically.

Some time later its buddy made the same bad life choice.

Like everyone else, I played Minecraft really long time ago. I recently discovered that the game is still receiving regular updates after all those years. Thanks to corona, I have a bit more free time than usual, so I decided to give it another go!

Here's my thoughts. Little of this will be new to you if you played recently.

My favourite changes

I really like the new villager system. This is possibly the biggest change - trading with villagers at first seems pointless, as they don't sell anything useful, but it turns out they level up, and offer more and more interesting items - while stuff they buy you can build semi-automated system to collect.

It's also possible to train villagers to choose new profession, breed villagers, and they have some protection with iron golems they summon, and some enemies with pillagers.

Treasure ships and treasure maps found in them are great. You'll randomly run into them as you explore, and they have nice stuff inside, and map where to get even more nice stuff. I really didn't love how I got two copies of same treasure map once, but such small side quests are great fun.

Boats no longer take damage when sailing around! Looks like they changed that back in 2012, that makes boats a great transportation system, and canals for them are quite easy to build.

The game now shows you recipes when you open crafting table, and that's a great change! No need to alt tab for rarely used things.

The game needs a damn map!

The game now has maps, and I thought that's great until I tried actually using them, and omg they're truly terrible.

This was one thing that finally convinced me to go through the hassle of installing Forge modding system (make sure you get one that matches your Minecraft's version), and then both Xaero's map mods (world map, minimap). It seems like such a small thing, but honestly it makes bigger difference for player enjoyment than everything they changed in Minecraft over last 10 years put together.

Game still hates the player sometimes

The game has its share of "fuck you player" moments. Falling into lava and losing all you stuff is a classic, but the game has more.

Like when you get hit by a Wither Skeleton, that not only damages you when hit, and poisons you so you keep losing your health (which is far enough so far...), but it also hides your health bar, which is just Minecraft devs being assholes.

For another "fuck you player moment", I was just wandering around exploring, setting up temporary bed whenever night fell and then taking it with me afterwards. I was in the boat during daytime, and somehow tridents started flying my way from the water, and I died out of nowhere. I lost all my stuff as usual, but most annoyingly game decided to spawn me at 0, 0 instead of my base or last sleeping place since last bed I slept in was no longer there.

The problem was that in my seed, there were no villages within over a 1000 blocks from my spawn point, and the closest one I finally found was abandoned, so I finally setup my base at about -2000, -1000 or so next to the first inhabited village I found. So by spawning at 0, 0 I wasted like 15 minutes of my life walking there. Just because the devs hate me.

And then there's all the other stuff, like all the sand updating because I placed a torch, resulting in sudden fall into a ravine.

There are games where mechanics like that make sense, but Minecraft is mainly about exploring and building, and last thing it should do is punishing exploration.

Other things that suck

Inexplicably there's still no builtin modding system, and you need to install third party mod manager to even download and install mods. That is truly baffling.

Combat in Minecraft was always bad, and it's still bad. Turning it on Easy doesn't seem to do anything. At some point they changed combat system, and before I watched some youtube tutorials explaining how it really works, it seemed just ridiculous.

Placing item when building a bridge is really obnoxious. And it looks like other versions of Minecraft fixed it, but somehow Java version still uses the old and really tedious system.

With single exception of crafting table recipes, the game is really bad at explaining anything. There's so many things that are completely unintuitive, and which are not explained, and not realistically discoverable. It's as if the game expects players to watch youtube tutorials for half of its mechanics. And sometimes I watched a tutorial, tried something, and it absolutely didn't work, because Bedrock and Java are subtly different, or they updated things at some point, and it no longer works.

I always felt like player's inventory was too small, and it wanted me to keep running back to base too often. It makes sense that I can only carry so much, but why can't my boat or minecart have some extra inventory space, that I'd then bring home with me? Big Boat With Extra Chest - as well as Big Boat With Space For An Extra Animal would be great additions to the game.

Things I didn't care much for

I really didn't care for the Nether and End changes - they're far too combat oriented, and combat in Minecraft is just not very fun, before or after the big combat update.

It feels like there's far too many blocks that just don't do anything different. There's naturally occurring stone in like 10 colors now, and a lot of cosmetic variation that feels largely pointless.

There's a lot of new mobs too, which seem to do nothing interesting.

I don't hate such changes, and I guess they add some extra flavour.

Is it worth playing again?

To be honest compared with other games that had 10+ years of updates, Minecraft changed surprisingly little. Something like Factorio or Crusader Kings II are such drastically different games than they were at release, but Minecraft now and 10 years ago, pretty much the same thing.

I might be worth giving it a go. Then again, if you never played Factorio, you should try that first instead. There, your buildings actually do things.

Sunday, May 03, 2020

Fun New Conspiracy Theories

p1240692 by generalising from flickr (CC-SA)

For your entertainment. There's no hard evidence for any of them, and I'm definitely like actually endorsing any of them, but they all sound quite plausible, right?

Corbyn was a Tory plant

UK is not even close to being a democracy.

It's also trying its best to be a Tory single party state. Starting the timer when the last Liberal Party government ended in 1922, and ending it when Tony Blair was elected with New Labour in 1997, Tories were in government 68% of the time - despite having actual majority of the vote only once.

They achieve this by combination of heavy gerrymandering, radicalising the Labour Party, and splitting the opposition.

UK redistricting is technically performed by "independent" Boundary commissions, but rules under which they're operating resulted in very consistent Tory bias.

Cheating a few percent through gerrymandering is however not enough. Tory Deep State figured out a far better way - by radicalizing Labour Party in ensures moderates are too scared to vote for them, and most of the time manages to split the vote between Labour and Liberals/LibDems/SDP/such.

Political science predicts that in FPTP systems both parties will drift to where the center of the electorate is. Somehow UK is the only exception to this pattern.

All those Tory shenanigans were ended when Tony Blair created New Labour - which was exactly what the political science predicted. Modern pro-European center-left party without any of the openly Communist baggage.

New Labour was mortal enemy of the Tory One Party State project, so they did their best to undermine it. The 2008 Great Recession let them get back to power as part of a coalition government, but that was living on borrowed time.

So what did they do? Tory Deep State dug up Comrade Corbyn, and managed to put him in charge of Labour Party to destroy it. Once in charge, Tory plant Comrade Corbyn somehow managed to get as many open Communists and open Anti-Semites as possible to join to make Labour completely unacceptable to an average voter.

Under control of Tory puppet-masters, Labour adopted the most extremist plans possible on basically every issue. It did not take pro-European position, in spite of overwhelming majority of Labour voters being pro-European.

Normally when a party loses elections, its leader goes, but somehow the Tory puppet managed to stay and lose elections after elections, and only got kicked out this year. Damage Tories managed to do to Labour Party was so extensive, it's not clear if Keir Starmer will ever be able to fix it.

Boris faked coronavirus infection

It is very well established that Boris is willing to lie about literally anything to get the power, and doesn't care about how much damage his actions cause.

So why do people suddenly believe that he wouldn't lie about getting coronavirus and needing ICU?

Let's get some facts.

Boris allegedly got moved to ICU on April 7th. According to data I could find for two days later, only 3883 people in all of UK got COVID-19 ICU.

There's 0.005% chance that a random person would need COVID-19 ICU.

Also, only 20% of patients admitted to ICU for COVID19 recovered.

So the chance of Boris getting COVID-19, needing ICU at this point, and promptly recovering, is about 0.0001%. Do you seriously believe that the chance that habitual liar Boris Johnson lying about this is less than 0.0001%? If so, I have a great EU trade deal to sell you.

And it would be far from the first case of head of state lying about their health.  It sounds totally implausible, but Roosevelt managed to hide his paralysis from the whole public and the press cooperated with that!

Secret Service actively interfered with photographers who tried to take pictures of Roosevelt in a wheelchair or being moved about by others. The Secret Service commonly destroyed photographs they caught being taken in this manner
The list of cases of politicians hiding their illness which only comes out after their death is very long, including nearly half of recent US presidents.

How many faked instead of hiding their illness? That's harder to tell, as there's usually no advantage to do so, but it's sure as hell higher than 0.0001%.

In all likelihood Boris was either totally fine, or just got common cold, and requested ICU admittance to fake COVID-19, as he thought that would be politically beneficial to him.

Democratic governors are destroying their states' economies on purpose

"Fundamentals" models of election forecasting have been accused of overfitting, but their basic message is still true - when economy is doing awfully, people vote against the ruling party. The Great Recession in most countries showed drastic drop in support for whoever happened to be in the government - left wing, right wing, center, or extreme, regardless of how responsible they were, and how well they were doing.

None of that mattered, voters always turned away from the ruling party - either towards the opposition, or towards new usually extremist parties.

Notably there was no such backlash in any subnational elections. Republican states stayed relatively Republican, and Democratic states stayed relatively Democratic, there was no grand flipping.

During corona crisis, Democratic governors have generally issued far more strict lockdown orders than Republican governors, pretty much regardless of how affected their states are. Outside New York, most states aren't particularly affected, and none of that is even close to rational policy.

So here's a fun conspiracy theory to think about - what if they're trying to damage their economies on purpose, because they know Trump is going to get blamed for it anyway?

During normal times it's not something they could get away with, but this is a great unique opportunity to do just that.

Notice this theory doesn't work backwards - if you try to flip that and speculate that Republican governors are trying to make things better to help Trump, well can't blame people for trying to make things better.

Wednesday, December 18, 2019

Challenges for November 2019 SecTalks London

Selfie by SethBahl from flickr (CC-NC)


October 2019 London SecTalks CTF was jointly won by me and Laurence, so we wrote challenges for November together.

There was a total of 9 challenges, 5 written by me and 4 by Laurence. It was a bit much, as one of the challenges got zero solves until after the event. In spite of very generous hints given halfway through the event.

Challenge files and code used to generate them are available on github.

There are no answers below, but some serious hints which might make it too easy.

For previous rounds, see posts about September 2017, November 2017, May 2018, July 2018, October 2018, February 2019, April 2019, and July 2019 CTFs.

Rockstar (5 points)

The puzzle was a rock song / program in Rockstar programming language. Just running the program would print ASCII codes, which could be easily turned into a flag.

Think Outside The Box (10 points)

The challenge is a BMP image file with flag hidden by incorrect image size. Changing the header reveals the image.

Dots (15 points)

This was Unicode Braille, but dots and no-dots were simply 1s and 0s to be decoded, ignoring any "Braille" structure. To my surprise nobody solved it even when given basically full explanations of what they're supposed to do.

CRC32 MAC (20 points)

A signed cookie server, which uses CRC32 with secret key as MAC. CRC32 is extremely unsuitable in this role.

Decryption Engine (25 points)

A small Python program which uses default password based on current date, and encoded file. Assuming password wasn't passed, the space of possible keys is tiny.

Salted MAC (30 points)

A signed cookie server, which uses SipHash with secret key, and then salt just to be extra sure.

In principle this should be difficult to break. The problem is a trivial implementation mistake.

PHPain (40 points)

It was a separately hosted PHP application vulnerable to recently published CVE.

Poly-DIY MAC (50 points)

Another signed cookie server, using a simplified variant of Poly1305. The variant is reasonable, however the server isn't very careful about nonce handling, and such schemes tend to fail catastrophically when nonce is reused.

GeoCaptcha (60 points)

It's a web "Captcha" which asks the user to locate 20 major cities on the map, with any mistake failing the Captcha.

You can play it here (for a while).

It's worth so many points mostly due to its very high entertainment value, not due to difficulty.

I was curious how people would approach it - only 2 players solved it with code (as robots), 6 solved it with tedious manual process (as humans). So I guess it works as a Captcha.

Thursday, October 10, 2019

How to configure OSX 10.14 Mojave for software development

Kitten by Magnus Bråth from flickr (CC-BY)

With every new Macbook, I'm updating the guide, previous version is here.

And I get it that it's ironic that I'm posting this just as 10.15 came out, but I started the setup a few days ago, and in general it's a poor idea to update OS as soon as it comes out.

Basics

  • Go to Settings > Security > FireVault, turn on FireVault. This will restart your computer.
  • Install some sensible browser like Chrome or Firefox.
  • Afterwards either sign up into your account on which you hopefully have your ad blocker setup, or install some. Most popular seems to be uBlock Origin these days, but pretty much any of them will do just fine.
  • Install whichever cloud sync service you're using like Dropbox etc. And start syncing your stuff.
  • Install iTerm2 for sensible terminal emulator.
  • Clean up all crap from dock. Other than Launchpad and System Settings, everything else should be gone. Add iTerm2, your browser, and your text editor, and any application you wish to install there instead of stock Apple crap.

Editor

Install some sensible text editor. These days most people use Visual Studio Code.

If that's your choice, run it, open Command Palette, and choose: "Shell Command: Install 'code' command in PATH".

Settings

Like every other operating system, OSX has a lot of bad default settings. Here are some obvious fixes:
  • Settings > Mouse > increase speed
  • Settings > Keyboard > Key Repeat > Fast
  • Settings > Keyboard > Delay Until Repeat > Short
  • Settings > Sound > Disable "Play user interface sound effects"
  • Settings > Sound > Alert volume > 0% (for Terminal ping)
  • Settings > Sound > Show volume in menu bar
  • Settings > Trackpad > Scroll & Zoom > Disable "Scroll direction: natural"
  • Settings > Displays > Arrangement > drag and drop your external monitors into desired order
  • Settings > Displays > Max out brightness
  • Settings > Displays > Turn off "Automatically adjust brightness"
  • Settings > Dock > enable "Automatically hide and show the Dock"
  • Menu bar > Battery icon in task bar > Enable "Show Percentage"
  • Menu bar > Time icon in task bar > Open Date & Time Preferences > Use a 24-hour clock
  • Menu bar > Time icon in task bar > Open Date & Time Preferences > Show date
  • iTerm > Preferences... > Profiles > Terminal > Unlimited Scrollback
Press Ctrl-Up arrow, add a few desktops (or "spaces" as they were used to know), then go to Settings > Keyboard > Shortcuts > Mission Control - and enable their keyboard shortcuts Ctrl-1 to Ctrl-6 or however many you have there.

Open Screenshot app, choose options, then disable "Show Floating Thumbnail".

Drivers

OSX already includes drivers for laptop itself, but you might need some for peripheral hardware.

In particular, external PC keyboard need a tweak to work properly, as left and left Windows keys are in reverse order from Mac keyboard.

Go to Settings, Keyboard, Modifier Keys..., and swap positions of Option and Command keys. Feel free to change functionality of Caps Lock key as well, it's a huge easily accessible key with no useful function people love to remap, usually to extra Control.

If you need any special keyboard layouts, get them too.

Standard paths

OSX renames a lot of directories. While in theory scripts could just use env vars to find proper paths, it's more reliable to symlink all the things:

  sudo ln -s /Volumes /mnt
  sudo ln -s /Volumes /media
  sudo mv /home /home-old
  sudo ln -s /Users /home

Development tools

You'll need a package manager, and the only one anyone uses is homebrew, MacPorts and the rest being basically dead. You need to tell homebrew to not spy on you with brew analytics off command.

You'll need Xcode. Fortunately homebrew installer does it for you automatically. If you don't use homebrew for some reason, you can install Xcode manually by running xcode-select --install from command line.

Create new SSH key pair

Before you do that, name your computer something memorable with sudo scutil --set HostName your_host_name command.

Open Terminal and run ssh-keygen to create ~/.ssh/id_rsa, then upload the generated key to any place that needs to know about it like github, bitbucket, or whatever else you use.

Checkout your dotfiles

Hopefully you're storing your dotfiles somewhere. If it's a git repository, or your Dropbox account, get them now and symlink them all properly.

If there are any other repositories you might need, checkout them too.

Install homebrew packages

Your list might vary. Here's a few obvious suggestions:

brew install rbenv ruby-build mc wget p7zip unrar trash git htop bash zsh youtube-dl jq imagemagick coreutils bash-completion zsh-completion

Then enable all services you installed, unless you want to start them manually:

  ln -sfv /usr/local/opt/*/*.plist ~/Library/LaunchAgents/

And install non-system ruby, so you can install gems without sudo. Currently latest is:

  rbenv install 2.7.0-preview1
  rbenv global 2.7.0-preview1

To make that actually work, you need to make sure ~/.rbenv/shims is in your $PATH. If you type rbenv init, it will tell you what to do.

Due to OSX limitations you'll need to run sudo htop if you want to use htop.

Install gems

Again, your list my vary. These days most of the software will have its own Gemfile so long list of gems are generally unnecessary. But some global utilities are still useful:

gem install bundler rak pry pry-rescue

Sane bash or zsh

bash version shipped with OSX is ancient and BSD utilities are all awful.

You have two choices. Either install modern version of bash from homebrew, or use zsh from homebrow.

First, edit /etc/shells as admin and add the following lines at the end of it:
/usr/local/bin/bash
/usr/local/bin/zsh

Then set it as your shell, with whichever one you prefer:
  chsh -s /usr/local/bin/bash $USER
  chsh -s /usr/local/bin/zsh $USER

Either way, run to touch ~/.hushlogin to prevent some worthless spam on every open terminal tab.

If zsh is your choice, the first time you start a new terminal, it will ask you some questions on how to configure it.

Coreutils

This is optional step. OSX coreutils are awful, and generally GNU versions are a lot better, but if you switch there could be occasional minor incompatibilities.

If you want to do so, add GNU coreutils to your PATH:

  export PATH="/usr/local/opt/coreutils/libexec/gnubin:$PATH"
  export MANPATH="/usr/local/opt/coreutils/libexec/gnuman:$MANPATH"

Better window manager controls

Sadly OSX window manager is extremely dubious for keyboard use. Fortunately programs to make it usable exist. I recommend installing HyperSwitch for sane alt-tab window switching, and Spectacle for managing windows.

I previously used HyperSwitch, but it no longer works with latest version of OSX. Spectacle shortcuts are different than HyperSwitch, and I like HyperSwitch's more, so I adjusted shortcuts to match.

You'll need to give them necessary access. To do so:
  • Settings > Security & Privacy > Privacy > Allow the apps below to control your computer > enable HyperSwitch and Spectacle
Also open its preferences, and set it to run in the background, and start at login.

All other software

There's a lot of other software you might want. The most obvious one is the VLC media player.

You might also want some kind of Git UI program, like GitUp or (another for of) GitX.

Enjoy

Once you go through this list, and successfully get everything going, I'd recommend modifying it to your liking and reposting your version on your blog. Everybody's needs are different, so guide like this is just a starting point.