taw's blog

The best kittens, technology, and video games blog in the world.

Sunday, January 27, 2019


Cat (OOF!) by Long Road Photography (formerly Aff) from flickr (CC-NC-ND)

I was playing a bit with Electron, building a small image viewer, as Xee is weirdly crashy nowadays. A small aspect of any image viewer is sorting images to show them in order. As far as I know there's no programming language with any special support for sorting file names, and default ASCII sort is just atrocious - going cat1.jpg,  cat10.jpg,  cat2.jpg etc.

It's not uncommon to find file manager or image viewer which uses this completely unsuitable sort order.

OSX Finder is notable for very much not doing so, and using human-friendly ordering. Exact details are unfortunately not properly documented.

I created something which works more or less the same, and published it as npm package finder-sort.

There could be some differences, especially for non-ASCII locales. In the end, it's trying to solve the same problem, not necessarily match OSX Finder exactly.

The source code is on github.

About the only interesting thing about it from code point of view is that I used ava testing package this time. Javascript has over 9000 different testing packages, and it's not uncommon for a single program to duct tape together 10+ of them. I'm surprised there's no clear winner yet. Some of the new ones like ava and cypress seem mostly decent.

Sunday, January 13, 2019

What I learned from Awair Air Quality Monitor

Unimpressed kitten by Magnus Bråth from flickr (CC-BY)

After watching DHH's video about air quality I decided to get myself AWAIR 2nd Edition air quality monitor.

Here's what I found.


Awair setup was fairly awkward. It couldn't connect with my WiFi until I changed my router to some compatibility mode, there were weird error messages about firmware update and so on. I finally got it running, but it wasn't smooth. At least there were no further issues after that.

Air circulation inside the flat is poor

I originally assumed that air in the flat would rapidly be fully mixed up, as it's a fairly small London flat, but that's not what I found. If I open all windows, fresh air gets to the sensor real quick, but as soon as I close them, the stale air is back just as fast from parts of the flat not on air flow path between open windows. I guess it's all the awkwardly placed walls and doors. Keeping windows open much longer eventually works, at least for a while.

How much CO2 per day do I need to get rid of?

One person burns about 2000 kcal a day, so let's say that's 500g carbohydrates getting burned. That's 200g carbon, or 730g CO2 per day, or 17 moles. 17 moles of CO2 takes volume of a bit over 400 liters. Plus a bit more for the cat, and for gas cooker.

Let's say the flat is 50 square meters, at height of 250 cm, so 125 cubic meters. Atmospheric CO2 is 400ppm, and let's say I want to keep it under 1000ppm, so 600ppm difference. 600ppm of 125 cubic meters is just 75 litres.

These are all heavily rounded calculations, but unless I forgot all the high school chemistry already, I need to somehow fully replace air at home 5-6 times a day to keep it in the happy range. Either by opening windows until fully replaced, or by having some small continuous air "leaks".

This number doubles with a second person, to 11 times a day.

That's a far higher number than I expected.

This number also changes depending on carb vs fat based diet, actual caloric expenditure, and so on, but in any case, it's very high.

Cooking anything generates massive PM2.5 spike

The sensor is placed in the computer room, about as far from the kitchen as it gets. And it looks like cooking anything whatsoever without windows open generates a massive PM2.5 spike for a fairly long time. This can fortunately be easily fixed by only cooking with windows open.

It's difficult to keep Awair happy

Awair never had any serious problems with chemicals, and PM2.5 spikes were only caused by cooking.

The problem were the other three readings. If windows are closed, CO2 levels creep up. If windows are open, CO2 levels go down, but temperature and somehow humidity go into unhappy zone. Pretty much no matter what I do, Awair will be unhappy. Only briefly after closing the windows when temperature goes back up but CO2 is still low Awair becomes briefly satisfied.

Perhaps I should move it farther away from airflow to make it more stable, then again it's fairly close to my computer chair, so it should be more representative of air I breathe this way.

What should I do?

I've been definitely opening windows more now, especially when cooking, but that's not too great - I like my place real warm (24-26C), and the outside is quite cold, especially in the winter.

I don't want to keep windows open too long not just because of higher heating bills, but also because of all the sound coming from outside. I'd have serious trouble falling asleep with bedroom windows open, as I'm extremely sensitive to light and sound when falling asleep. And it's distracting when I'm trying to focus on coding or gaming or just about anything.

I could open some other windows, not those in the bedroom, but honestly I don't think air would circulate much - bedroom door is always just slightly ajar for cat's sake, to keep bedroom as dark and quiet as possible. Closing them fully would generate cat scratching on door sound, which also negatively affects sleep.


So an obvious idea is to get some houseplants. It's a bit of a hassle, and the cat would probably damage them a bit. But mostly I have no idea how much CO2 would they really remove. I suspect the numbers would be really low for any realistic amount of plants, making it not worth it. Especially at night, when the light is understandably off.

Or is there any other trick I'm missing?

Sunday, November 11, 2018

Webpack boilerplate package for Imba and SCSS

Fluffball by Rum Bucolic Ape from flickr (CC-ND)

Imba looks extremely interesting - seriously, just check out the code examples on their website.

I wanted to give it a go, there was just one tiny problem - in Javascript world you can't just gem install a few things and run them like that. Nope, painless setup is just a crazy ruby idea that never got much traction anywhere else. In Javascript universe everything always requires painfully complicated setup.

I found some boilerplate example for Imba, but it was broken on so many levels, I had to start pretty much from scratch.

So here's a working webpack boilerplate for Imba with SCSS support. Feel free to fork it into your project.

What's in it:
  • Latest Webpack
  • Imba
  • SCSS (as plain old compile to CSS, intentionally no CSS-in-JS shenanigans)
  • CSS normalize to avoid cross browser pain
  • standard npm commands for development and production builds.
Everything uses sane 2 space indentation, and tries to avoid doing anything weird.

What's obviously missing is some kind of testing framework, so PRs wanted.

I haven't used it for anything more complicated than just another TODO app yet, so I don't know if there are any issues. Just report them on github.

Thanks to all the brave souls who answered webpack questions on Stack Overflow - somehow I managed to duct tape working boilerplate out of all that.

Sunday, October 28, 2018

Challenges for October 2018 SecTalks London

She loves the computer by glennwilliamspdx from flickr (CC-BY)

Last month I ran another round of London SecTalks CTF.

There were 10 challenges, and the winner got 9/10 of them during the event (and last one on the following weekend), so difficulty level was about right.

Challenge files and code used to generate them available on github.

There are no answers below, but some serious hints which might make it too easy.

For previous rounds, see posts about September 2017, November 2017, May 2018, and July 2018 CTFs.

Archive (5 points)

A small variant of the 16-level 16-way nested archive, to test for basic Unix scripting skills. This time using RAR.

MonoRSA (10 points)

It's RSA-encrypted message, but it uses only one prime, not two. This is extremely insecure, and can be trivially broken, but you still need to do some math.

BCRYPT (15 points)

Each letter of the flag was encrypted with bcrypt. It's easy to break, as long as you know how bcrypt works, which isn't quite the same as plain hashes, so it was causing a bit of confusion.

RSA RNG (20 points)

It's Debian weak RSA key attack all over. We have target's public key and encrypted message. Also a lot of other people's public keys, all generated using same bad RNG. If two keys share a prime it's easy to break them, even if direct factoring isn't viable.

Python (25 points)

Small bit of reverse engineering - small Python password validator obfuscated with one of online tools for it.

MultiRSA (30 points)

A little known fact about RSA is that it works just fine with more than two primes. So this challenge uses 16 - which is fine, except key size was not adjusted appropriately, so each of those primes is small enough to break it.

Binary (35 points)

Binary password validator, provided in two versions (Linux and OSX) for convenience. It was compiled with -O3 which made its encrypting loops unroll, and what was very trivial code turned into hard to understand vectorized mess. Then again, actually running the program might reveal something interesting.

SVG XOR (40 points)

The flag is written in SVG flag, which was encrypted with a XOR cipherer. This turned out to be quite easy, as SVG files have a lot of structure which can be used to attack this.

FPGA (45 points)

Probably the most original challenge this time. A netlist of NAND gates which can validate the flag and tiny emulator were provided. Can you figure out the inputs necessary to get the validator to accept?

This was the only challenge without anyone solving it during the event, but there were some solutions afterwards.

Tweets (50 points)

A flag was hidden in collection of Donald Trump's tweets, encrypted with monoalphabetic cipher. Great exercise for frequency analysis.

Sunday, October 21, 2018

My current GTD system

She works hard for the money... by Tamer Akça from flickr (CC-NC)

I don't know if there's any way to do GTD properly. I've been trying so many tools, and it always feels like no setup works properly, but if I don't do some kind of GTD then my life instantly falls apart and I never achieve anything, or even stay on top of everyday responsibilities.

In all likelihood if I write this post again in a few years, it will be a fairly different list. Anyway, for other people who try to do GTD, here's my current setup.

Core of the system

The most important part is private git repository which also happens to be mirrored on Dropbox using magic of symlinks.


One recent complication is how often I'm with just a phone, and often offline on a train to make it even worse. The least shitty solution I found is Google Keep. In the past I carried a tiny pen and a stash of post-it notes with me, but that's a bit less practical. Most other software I tried really doesn't like working with limited connectivity.

A big downside of Google Keep is that it's hard to copy a list from the web UI and paste in somewhere, without going through the hassle of exporting to temporary Google Docs document and copy&pasting from there. Someone should seriously write a Chrome extension to improve that part, and well, that someone might end up being me.

I probably should give Google Keep alternatives another try, since Google products don't have very high half-life.

Big section of the core system is inbox folder with anything that hasn't been processed yet.

I have a physical wicker basket at home for letter and related physical stuff to check.

I think on paper a lot, so I often generate a lot of paper mindmaps and lists. Once I'm done with them, they land in the inbox.

Incoming emails which requires some further actions get a star. I never delete any emails, so those stars are the only indication that it's not done. Some people delete or archive stuff and treat their email inbox as a TODO list, and that just feels really weird, but if it works for you.

Usual GTD lists

The core system contains the usual GTD lists like Projects and Next Actions.

I don't divide Next Actions by context, since there's no meaningful context for most of them.

Reference System

I have a physical reference system consisting of a bunch of ring binders with contents inside organized alphabetically by tag. That's mostly for things like bank statements, bills, and other boring paperwork which I might refer to every now and then.

I also have a ref folder on Dropbox - not related to that git repository - which contains all the digital stuff.

Emails stay in Gmail, as they're very easily searchable there.


This is a bit awkward, as I use a mix of Google Calendar and old text file based system.

Calendars I use are:
  • upcoming events
  • saved dates for potential upcoming events
  • any periodic actions I want to do every N weeks or months - mostly boring cleanup, backups, reviews etc.
  • birthday calendar
Possibly it would make sense to move to just purely Google Calendar system.


This might be the most interesting part, as I found I need multiple different kinds of planning to make things work:
  • the usual lists of GTD next actions
  • ad-hoc planning like at start of the day usually happens on paper - by the end of the day what's left of that list goes into inbox
  • weekly goals list to keep me focused - it's typically about 10-15 goals, and I'm aiming at 80%+ Partial Success or better rating. Those lists are not meant to be modified once created. If something fails for a good reason, it fails. Any unachieved goals (including Partial Success) go into inbox.
  • rolling goals list for next 12 months - I keep it as Google Docs document and update what's in progress, done, or definitely failed with some color coding. This documents gets updated whenever needed. Every 3 months or so I archive old document, clean up done or failed things, and create a new one.

Weekly Goals List

The biggest risk of GTD is that a lot of things will get done (as far as failure modes go, it's not the worst one), but whole areas of life where progress is most difficult get neglected.

Because the most important function of those lists is helping to balance different aspects of life, anything where I achieved meaningful progress towards stated goal counts as Partial Success, even if it's still very far from being finished.

Of course since Partial Success is still not finished, it still needs to go onto the next list.

I'm aiming at 80%+ completion rate because typically a few things will just not work around due to external circumstances.

If same thing fails multiple times, then it's a very strong indicator that it needs a lot more planning.

Rolling Annual Goals List

You know how people are best behaving in January after they make their New Year's resolution, but then usually give up by March? People completely miss the point thinking that such resolutions are ineffective - they're extremely effective, you just need to refresh it often.

The list is about 4 pages of Google Docs, and lists many highly specific goals, hopefully covering every aspect of life. This includes many more meaningful goals, but also lists movies, games, and books I'd like to enjoy over the next 12 months.

Most goals on the list are very specific and measurable, but it's not always possible, so some vague entries, and some refer to ongoing practices. If the goal itself is not specific enough, I try to have some more specific subgoals.

12 months feel like about the right perspective for this list. It's really difficult to think in longer term perspective in concrete enough terms, and for shorter perspective it would be guaranteed that many life aspects will go unaddressed.

Whenever I update this list, I try to have a chat about it with certain special people.


The system is private and difficult to share even if I wanted. Occasionally I want to discuss some plans with others, and for this I usually use Google Docs and Google Calendar - or talk about that in person, which also works.

Other lists

I keep Waiting For lists, mostly for things I ordered.

I keep Someday Maybe lists, for things which are not really actionable, but I might get there someday.

I keep Social lists for people I'd like to keep in my life - I check it every now and then, and if I'm at risk of losing contact because everyone is too busy, I try to arrange something.

I used to have Shopping list, but since I do overwhelming majority of my shopping in Tesco online and Amazon, I just throw whatever I need into relevant basket, and every now and then order what's in those baskets.


And they're not really part of the GTD flow, but I keep a lot of different logs, measuring and writing things every day.

And everything else

And I also have beeminder setup, but it only tracks things like exercise, so it's fairly peripheral to the system. I tried to use it more, but most interesting things might be specific, but not necessarily quantifiable in the way beeminder wants, and usually giving something long term goal with weekly commitment is not actually the best idea.

That's the rough outline of my system.

Big 5 Bonus

And that's what a person who's a few standard deviation high on Conscientiousness is like. Also about as high on Openness to Experience, extremely low on Neuroticism, and somewhere halfway on Extraversion and Agreeableness.

Music I like

Piano Cat I by Nina A. J. G. from flickr (CC-ND)

I mostly listen to music on youtube - or songs I downloaded from youtube with the glorious youtube-dl to dumb devices for listening while offline.


I listen to them at 140% speed. I've been watching everything at high speed so much, 140% is my neutral speed, that's the lowest I can go without anything seeming to be artificially slow.

Nowadays my speeds are:
  • 140% - neutral speed - music, movies, some denser TV shows like Game of Thrones
  • 180% - medium fast - most TV shows like Family Guy
  • 200% - fast - podcasts, audiobooks, let's plays, conference videos, nearly everything on youtube that's not music
  • 220% - very fast - some particularly slow sources like Tolarian Community College
This means I'm really limited to youtube as my only place for music, and to "unofficial" download sites as my only source for shows and movies. I tried Spotify and Amazon Prime, but they have no speed control, and that's insane in this day and age.

I guess I can watch things at 100% in cinema or when watching it with someone, but that's an exception, and it still feels artificially slow.

By the way, if you know of any source of music or streaming that has speed control, good quality, and good selection (not just in US), I'm definitely interested.

Youtube for music

The great things about youtube is that it has basically everything, it's free, and ad blockers work perfectly on it, at least on desktop. I guess they recently added some sort of subscription service for people who feel sad about using ad blockers, and I'm not philosophically opposed to that, but I didn't have any time to investigate that. 

The worst thing about youtube is total lack of metadata. Also people thanking their Patreons for a minute after each song while I'm trying to listen on shuffle. And it's basically useless on the phone with no or limited connectivity.

The so-so thing about youtube is its recommendation algorithm. It's not completely useless, but it keeps suggesting stuff to me a one line python script would know not to (if vocalist.gender == "male": return False).

Their fancy subscription service solves none of the problems I have, just a problem ad blockers already solve.


But seriously, what really annoys me is that I youtube never bothered to provide any metadata for its song.

There's a plugin which tries to parse song titles with regular expressions, but it's not very good. It's sort of OK for "official" songs, but it's failing for almost every independent artist who mostly do cover songs.

So mostly for my own future reference, here's a list of artists I've been recently listening to a lot, in alphabetical order, manually extracted from my youtube watch history. I tried to categorize them into mainstream and independent artists, but it was fairly futile.

The List

If by any chance you're enjoying similar music, and you have any fun recommendation, send me the links.

Saturday, September 22, 2018

Fun and Balance mod for EU4 1.26.1

MIDORI by Marco Mosti from flickr (CC-ND)

Fun and Balance is a mod which tries to make Europa Universalis IV a better version of itself. The mod has very limited goals:
  • fix any issues where poor balancing makes gameplay worse - making more options viable, and occasionally toning down anything that's overpowered enough to make alternatives irrelevant
  • let people have fun in any way they choose, removing arbitrary prohibitions penalties
  • reduce AI cheating, as game is more fun when everyone plays by the same rules
  • reduce forced historical railroading via events or restrictions on player actions
These goals mean that every patch the right thing to do changes, and I need to go through list of fixed I've made and decide if they're still applicable. Many of previous changes I've made become obsolete because most problematic areas are likely to be addressed by future patches.

Playing with this mod shouldn't feel like you're playing a mod, it should feel like you're playing vanilla which finally patched silly things right.

It doesn't try to significantly affect game difficulty - it might increase it slightly by reducing cheesy tactics, or maybe slightly reduce it if you're trying to play naturally.

So here's the full list of changes, ordered roughly by impact, with reasoning behind them explained in detail.

Download links

Base diplomatic relations increased from 4 to 8

In vanilla diplomatic slots are always exhausted by every nation. You absolutely need a few strong allies, a few vassals to expand, and that prevents you from having any diplomacy beyond that. All options such as royal marriages, guarantees, marches, local alliances, supporting independence of others etc. tend to get mostly unused as they take precious slots you don't have.

So the mod just doubles base limit, and this opens a new world of diplomacy.

This bumps up difficulty, as allies are much more useful defensively than offensively. It leads to much denser alliance networks, and it's much less likely to be able to get a free attack on unprotected minor. At least as long as you play with Cossacks DLC enabled.

Mercenary limit reduced to half

Both players and AI have access to infinite manpower pool of mercenaries, and close to unlimited very cheap loans.

This means defeats have minor consequences. It doesn't matter than you just killed every military age man in a country, they'll just spam loans and mercs next day. Attrition, manpower buildings, manpower bonuses, all of that matters a lot less when you can just loan and merc.

Unfortunately it's a bad idea to do any drastic mercenary or loan nerfs, as AI often loses all its manpower to self-inflicted stupidity such as parking a doomstack in a low supply province during peace time.

With some testing I found out that the best balance can be achieved by halving merc limit (both base and increase from force limit), which is unreasonably high.

This doesn't affect reasonable use mercs to supplement your manpower. You can even go vanilla style merc spam if you stack available mercenaries (administrative, quantity, aristocratic) and force limit (quantity, offensive) bonuses. AI can handle it reasonably well.

Small AI countries are mostly unaffected as mercenary limit was already higher than their force limit, but for big ones like Ottomans or Ming you can now actually defeat their army and not have army just as big show up next day.

Fort upkeep reduced to half

Forts are unreasonably expensive. Most good players just delete all or nearly all of their forts, and AIs tend to suffer from having far too many crappy forts in wrong places breaking their economy.

Reducing upkeep to half of vanilla values makes keeping or even building forts more reasonable option for the player, and helps AI economies.

Unfortunately there's no way to mod in more understandable zone of control system.

AI cheats reduction

Call for Peace and naval attrition are removed, as they're mechanics which are exclusively applied against the player. AI no longer gets extra free leader.

It doesn't make a huge difference in terms of difficulty, it's just better if everyone plays fair.

Tweaked subject settings to match wider diplomacy

Vassal annexation minimum year increased to 20 years.

Vassal annexation is just half the vanilla cost - it's unreasonably expensive considering you have to pay full core cost for something which will likely be just a territorial core.

Liberty desire from development a bit lower, and vassals don't count your marches strength in their calculations.

Colonies actually care about relative power of themselves and their supporters, but have negative base LD to balance that out.

Diplovassalization max cap increased, but penalty from their development is still quadratic.

Big tributaries care a bit more rebellious.

There's now -100 cap from annexed vassal opinion, so you're not going to accidentally stack it too high by poor annexation timing.

Liberty desire from historical friend or rival toned down.

All of this generally works well for player and AI, and doesn't require unnatural play.

More building slots

Building slots in vanilla are very restrictive, so many building see zero play, and especially AI wastes its slots on useless buildings a lot. The mod increases extra slot from +1 every 10 development to +1 every 5 development.

Improve awful idea groups

Some idea groups are better than others, and it's totally reasonable but two are so ridiculously useless people only ever take them as a joke.

So mod gives maritime ideas +50% light ship trade power and +1 merchant - so you can actually do some trading - so you can actually get some trading; and it gives naval ideas +1 free leader - so you can hire that admiral without taking a general slot from your armies.

This should hopefully move them from joke tier to situational tier.

You can convert in territories

This is controversial change in 1.26 patch. It's not a completely bad idea, as conversion was really fast and really easy - but until some outs are added (like religious ideas giving you ability to convert in territories, or replacement of outright ban with just slower speed), it needs to go.

Everybody can claim states

This feature is locked to Russian Tsardom government in vanilla, but there's no good reason for it, so I just made it available to everyone.

I thought about letting any empire-tier government do it, but game didn't like this idea (without creating a lot of government types), and there's really little downside to just giving everyone this feature.

Rival and Power Projection changes

I'd love to be able to restore rival system from early patches where everyone could rival everyone else - or at least for every great power to be able to rival every other great power.

Unfortunately that's not moddable, and it's very common that you get nobody to rival late game (and therefore very little power projection), or extremely limited choice of countries to rival early game.

The mod therefore increases power projection for great power status, for eclipsing rival, and slows down decay from actions against rivals. It also increases max rival range slightly, so early game you have more choices.

Religious Shift Decision

You can now freely switch religion to one of your capital at cost of some stability. It's disabled for Papal States, as that messes up with the game.

Disable End Game Tag checks for player

End game tag checks are an egregious case of stopping people from playing the way they want for no good reason.

I left these checks in place for AI to avoid checking them one by one if they make sense, but they're disabled completely for the player.

More formable countries

It's fairly arbitrary which countries are formable and which aren't. If you want to become Norway or Portugal and managed to shift the culture (which is admittedly very easy), why shouldn't that be possible?

Right now it keeps your original missions. It probably should ask if you want old or new missions with popup similar to one for ideas.

All those decision follow similar pattern - you need to have fully unified that culture, be big enough, and have admin tech 10. Excluded from this is anyone with existing form nation decision. Also excluded are Japanese, Russian, and Chinese culture groups, as they already have different mechanic for tag progression (forming Japan, Russia, and becoming emperor of China).

Coalition CB changes

Games sometimes need "invisible wall" style mechanics - fairly brutal means to limit where player can go. They don't have to be fun, but they should be very difficult to trigger accidentally. Normal in game mechanic, even negative ones, should be enjoyable.

Coalition system in EU4 fails this completely. It's really easy to trigger - so easy that AI minors in the HRE often have coalitions against them by 1450s - and it's completely miserable.

Most experienced players learn to play around coalition system by juggling truces, or attacking any country which joins coalition day one, or by using cheesy tactics like offering ally land as soon as possible once coalition war triggers (or in previous patches, offering 10000 gold).

If you actually try to fight and win coalition war, game makes it miserable. You can't separate white peace anyone even if you 100% them, so you'll have to keep going back then to swat their rebels. If you had any allies, they'll peace out leaving you with -40% warscore from battles and -25% ticking warscore - which somehow still counts against you even after they leave the war (that should seriously be fixed regardless of coalition issues). After a few tries everyone learns to never even attempt this unfun fight as just cheese it.

Not to mention just how ahistorical and immersion breaking it all is.

We have somewhat limited possibilities to mod our way around it. We could try to rebalance AE and tone it down a bit so you're less likely to hit the invisible wall. Or we could make fighting coalition more enjoyable.

A small modification of changing coalition CB from superiority in battles to defending capital goes very far towards making it a regular challenging war. That's how it used to be in early patches.

Burgundy event chain removed

EU4 is a sandbox game, and it's more fun when different outcomes happen in different campaigns. It's unfortunately been leaning towards very heavy railroading - Ottomans become second GP after player in nearly every campaign, England forms Great Britain, Muscovy forms Russia, Castile forms Spain, all almost every time unless player stops that, Ming never collapses or expands much etc.

It would be nice if we could make things more dynamic, so sometimes Aq Qoyunlu grows into the Middle Eastern menace, or Scotland sometimes won the British struggle, or some German minor seriously attempted unification. Unfortunately there's no straightforward system for modding in this kind of unpredictability without huge gameplay changes.

One piece of such historical railroading is going way too far - partition of Bugundy event chain. It's a totally nonsensical system where major European country gets divided completely disregarding its situation. The mod just kills it with fire.

I'm open to suggestions how to create higher diversity of outcomes.

Defender Aggressive Expansion discount increased

Being defender in EU4 sucks, as you can't use any of your CBs (during the war, or for duration of truce afterwards), can't declare anyone cobeligerent, and don't get CK2-style reparation for winning.

To make this slightly less miserable, mod increases defender AE discount from 25% to 50%. Remember that extra AE for non-cobeligerent attackers still applies.

Rebalanced Religious Conversion Rates

It's really silly that it's easier to turn a Catholic into a Sikh than turn an Sunni into a Shia.

The mod rebalances completely arbitrary conversion penalties (+4 against pagans, +2 normally, +1 or +0 sometimes) into consistent +4 against pagans, +2 against heretics, +1 against heathens.

Trade Map Tweaks

EU4 doesn't support dynamic trade map, and doesn't allow cycles, so any trade map will require compromises.

The mod adds Panama to Mexico and Patagonia to Lima link, and (to prevent cycles) removes Philippines to Panama and Mexico to Panama links.

This lets Asian powers enjoy New World trade from at least Pacific parts of the New World. The tiny downside is that Spain can't transfar trade from Philippines through Mexico into Europe, which historically happened, but that never happens in game anyway.

I'm considering much more aggressive changes, something like Better Tradenodes and Tradeflows mod, but it would require a bit more testing first.

You can use subject's religious CBs

You can declare religious war on your enemies which you only neighbour through subject.

For this both you and your subject need to have religious, and you must have same religious group as your subject (for holy war), or same religion (for cleansing of heresy).

This is mostly to reduce bordergore required to maintain CBs.

Doubled tradition gain from battles

EU4 made a strange change at some point that nerfed tradition gain from actual fighting to very low values, and it became based more on idea group choice than actual fighting. With numbers (land and naval) both doubled, it's a bit less silly, and if you're constantly fighting you should now have decent tradition level.

Religious Leagues as any Christian religion

In unlikely event that HRE will be split between some other denominations than Catholic / Protestant, a league war can happen in a different way. So you could have Reformed, Orthodox, Coptic, or Anglican challengers.

First victory by the challengers will just flip the dominant religion, you need second victory to lock the new religion.

No war exhaustion reduction while still at war

This button is only available for countries at peace, so you can actually make other countries suffer, or be forced to suffer yourself. It's no longer just diplomatic monarch point cost.

To match this, AI willingness to peace out based on War Exhaustion doubled.

Corruption Slider goes twice as far

If you're interested in rooting out twice as much corruption, at twice the cost, you can move the slider all the way to root out -2 a year. This is a necessary change to deal with corruption from too many territories.

It's obviously very expensive to go that far.

China nerf

Rebels in EU4 are too weak, and we all love seeing a Mingsplosion every now and then. So unrest from zero mandate doubled from 5 to 10.

Custom Nations Improvements

Most custom ideas get levels all the way up to 10 at higher cost. There's no penalty for taking too many of same kind of ideas - there's no power level reason for it.

Base monarch stats changed from weird 2/2/2 to 3/3/3 which actually matches average in-game monarch. Limit on distance between provinces of your nation increased to more reasonable values (so you can recreate something like in-game Genoa).

Merchant republics 20 statified province limit removed

It's a pointless nerf to a weak and unique government type.

Also for people without Dharma, Adopt Plutocratic Administration decision has province limit lifted. It's only useful for roleplaying anyway.

Longer CB on Backstabbers

EU4 has CB against allies who betrayed you which last 3 years, so you can only use it if you're willing to truce break. It's as pointless as things get, a relic from times when breaking alliance didn't create truce. The mod increases it to 10 years, but it's a very weak CB, so it's probably just there for roleplaying.

Some arbitrary decisions limits removed

A few decisions like moving capital to Constantinople and recreating Byzantium have a bit less strict limits, so you can use them even if you're doing something unusual like Serbian culture Coptic Ottomans.

Imperial Ban CB

The game has CB to take provinces from non-HRE owners, but it takes 100% AE with HRE penalty, so it's basically useless. Changed it to just 25% AE penalty, for affected provinces only.

Faster peace out

AI willingness to fight a losing war just because it's not been going long enough reduced slightly.

Some overly expensive action cost reduced

Moving capital, moving trade port, and culture conversion are mostly useful for doing weird things, and are quite overcosted, so all of their costs halved.

Everything is optional

As much as possible I tried to make every change optional, and keep it in separate files, unfortunately it's not always possible. It should be fairly compatible with most minor mods. For some popular total big mods (Extended Timeline, 1356) I can just offer separate builds.

There even used to be in-game menu for some of that, but people had performance complaints, so I got rid of it just to be sure.

I'm quite ruthless at killing off any feature which is no longer necessary. If there's balancing issue with relatively simple fix, I'm happy to include it in future versions.

If you think some of the changes cause more problems than they solve, definitely tell me about it too.

I don't always publish this kind of long explanatory blog post, but it's usually updated week or two after new major patch codes out.

Wednesday, August 01, 2018

Challenges for July 2018 SecTalks London

Cat sitting on a desk by freestocks.org from flickr (PD)

Last month I ran another round of London SecTalks CTF.

I only created 6/9 challenges this time, 3 Android challenges were created by imhotep.

Challenge files and code used to generate them available on github.

There are no answers below, but some serious hints which might make it too easy.

For previous rounds, see posts about September 2017, November 2017 CTFs, and May 2018.

Hidden treasure (5 points)

As per tradition, there was a bonus challenge with a zip bomb. This time at the end of 16-deep archive with 16 branches each there were pictures with a treasure, with EXIF tags containing either the flag or information that you failed.

Android 1 (10 points)

The simplest Android challenge wasn't even all that Android specific - as the flag was hidden directly inside the .apk.

RE1 (15 points)

Easy reverse engineering challenge was just a 32-bit Linux binary, which verified passed flag character by character, in random order.

Disassembling it, you'd get instructions like:

cmpb   $0x74,0x10(%ebx)
jne    80484b7

If you arranged them correctly, you'd see the flag. Or you could just grep for all characters, and use anagram solver - apparently that was an option too.

Android 2 (20 points)

This one required actually running the app, or fairly complex static analysis. The app saved a file with the flag to device storage. If you can find the file, the flag is yours.

RSA (25 points)

This challenge was actually quite realistic.
Bob sent a message we need to decrypt.

Probably due to bad RNG, it looks like Bob and Alice picked same N for their keys, and we managed to steal Alice's private key as well.

Perhaps there's a way to take advantage of this.
And there is a way. Having private key (n, e, d) there's algorithm to factor n into p, q. It's not completely obvious, but it's fairly short to implement. With p, q (shared between both), and Bob's e, you can trivially get Bob's d. Then you can decrypt the message.

RE2 (30 points)

It was a slightly harder version of reverse engineering challenge. The only difference was that flag was encoded into Base64 before checking, which could throw people off, but binary wasn't stripped so b64_encode method was a massive hint.

Android 3 (35 points)

The challenge app is making HTTP request, which needs to be intercepted and modified to get the flag from the server.

Once you get the request it's obvious what to do, and there were hints provided how to setup proxy on Android to get them.

CTR (40 points)

Another somewhat realistic challenge. A collection of Elon Musk's tweets and a flag was encrypted using AES-128 in CTR mode. Reusing same IV for all of them.

This turns CTR mode into XOR with random keystream, and breaking XOR cipher is fairly basic. It turns out nobody remembered mode names, so I had to give people some hints before they tried CBC or CFB attacks on it.

Croatian Monoalphabetic (50 points)

This challenge was worth the most points, but in principle it was very easy, just a bit time-consuming - and I ended up giving people hints how to approach that.

It's just statistical analysis of another Latin script language. So you can start by assuming space is the most common character, then use character frequency tables for Croatian, or lists of most common words, or take advantage of the fact that flag (very long word) is embedded inside.

Sunday, July 29, 2018

Which MCU movies are worth watching?

The World's Laziest Superhero by www.metaphoricalplatypus.com from flickr (CC-BY)

Not a day goes by that I'm not disappointed by what Wikipedia turned into. It went all the way from trying to be "sum of all human knowledge" under "neutral point of view" to cutting everything not "notable" enough and just mindlessly parroting establishment view.

That's tolerable when the establishment knows what they're saying, like in science articles. Well, mostly. Unfortunately this policy gets extended to domains without anything resembling genuine experts, and nowhere is it more ridiculous than with "film critics".

It's one thing to mindlessly parrot some social scientist who fudges statistics and uses questionable methodology, but is at least pretending to do science. Treating someone speaking out of their ass as a "reliable source" just because someone gave them a newspaper column to publish at is a mockery of what Wikipedia used to stand for in its early days.

There is an objective measure of film quality - enjoyment by the audience. And we have pretty good metrics of that - like IMDB and Rotten Tomatoes audience score. Such metrics occasionally suffer from issues like vote stuffing, demographic imbalance of voters compared with movie watching audience, less representative sampling for very old or foreign movies, and occasional overrating of most recent movies before it regresses to the mean, but it's insanity not to take these aggregates as starting points.

Even these issues are mostly overstated. IMDB filters handle vote spam quite well, voter demographics while imperfect tends to be far more representable than film critic demographics, and as for questionable early ratings due to hype, just check out how many completely forgettable movies managed to win an Oscars somehow. That reminds me, what was the Oscar list for the year of The Empire Strikes Back? It's hilarious in hindsight.

So what Wikipedia's doing? Ignoring all real data, and just parroting establishment of course!

MCU Movie Ratings

So here are the objective ratings, in chronological order:
I don't always agree with all them - for example I tend to enjoy superhero movies which don't take themselves too seriously more, so personally I find Doctor Strange overrated, and Iron Man 2-3 underrated.

But don't mind me, statistically speaking you're far more likely to enjoy what other people enjoy than what some cat blogger or film critic likes. All I can do is provide a bit of context for those numbers.

The best one

Pretty much everyone agrees that the best movie of them all is Avengers: Infinity War and it's not even close. If you plan to watch a lot of MCU movies, definitely include this one somewhere.

However, it might not be the best movie to start with. It's really good if you're invested in the story, and know at least half the characters. Otherwise, it will probably be too confusing.

Really good movies

There's a lot of options where to start. Movies establishing new characters like Iron Man and Guardians of the Galaxy, which are also really good on their own, are probably the best place. If you're completely new to MCU, these are just the two I'd recommend starting with. If you don't like them, I doubt you'd like the rest anyway.

Thor and Captain America have some good movies, unfortunately their first movies are rather mediocre. So you can either start with their first weak movie because it gets better eventually, or just read its plot summary and get straight to the better ones. Nobody will blame you for skipping early Thors, honest.

Watchable movies

These movies are still pretty good, and they mostly don't require watching anything before.

Notable here is Black Panther, which due to the nearly all-black cast became inevitable battleground of culture wars. It's decent movie, but it's massively overrated by the overwhelmingly liberal film critics, and this generates a bit of audience backlash. It's definitely far better than the trainwreck Wonder Woman was, but every single MCU movie is far better than Wonder Woman, even Thor 1.

Forgettable movies

Unless you're really invested in the MCU, it's probably best to skip them. Personally I quite enjoyed Iron Man 2-3 movies, but I definitely have preference for sillier movies that most people don't share, and I can totally see why they did not get universal aclaim.

None of them are key to anything, and a huge added benefit of skipping them all is avoiding Natalie Portman, who according to some is the most annoying actress in Hollywood [citation needed].

So that's it for today. If you know of anyone who's working on an encyclopedia which follows Neutral Point of View, please let me know.

Hearts of Iron IV Online Division Designer

Okay, I posed... now where's my treat?? by Lisa Zins from flickr (CC-BY)

There's division designer in the game, but it takes 10 minutes of console commands to ask basic questions such as "would this division be better with Superior Firepower or Mobile Warfare".

I've seen a few spreadsheet style division designers online, but they're not really able to answer such questions easily, and I don't have much confidence in their calculations.

So I wrote a command line tool to run such calculations based on game files. And then since I was halfway to something useful for other players, I added React.js frontend, then support for two most popular mods (Kaiserreich and Millennium Dawn).

Here it is.

It's first public release, so bugs are definitely possible, and UI could definitely use more polish.

Calculation engine got amount of decent testing to make sure it matches game data (disregarding Paradox rounding), but I could have missed something, especially if it's something only mods use.

It has extra features which are not exposed in the UI right now, like selecting any combinations of techs not just year + doctrine, and forcing old equipment, and I'd like to expose that somehow perhaps. I'd also like to make calculations more transparent with some extra tooltips explaining how those values got derived.

It doesn't even try to run in old browsers like IE11.

Best place for bug reports and feature requests is its github page, but I'm on all social media, so you can contact me whichever way you'd like to. Pull Requests welcome of course.

Over the years I wrote a huge number of various command line analysis scripts and modding tools for CK2, EU4, and HoI4. This is the first one that's available online, but I guess I could setup frontends for more of them. If you have any requests, send them over.