taw's blog

The best kittens, technology, and video games blog in the world.

Friday, March 01, 2019

Challenges for February 2019 SecTalks London

Snow Leopard Cub Looking Curious by Eric Kilby from flickr (CC-SA)

I ran another round of London SecTalks CTF.

There were 12 regular challenges, and 1 super-hard bonus challenge. Only 10/13 got at least one solve during the event, so maybe difficulty or number of challenges were a bit too high.

Challenge files and code used to generate them available on github.

There are no answers below, but some serious hints which might make it too easy.

For previous rounds, see posts about September 2017, November 2017, May 2018, July 2018, and October 2018 CTFs.

from_future_import_flag (5 points)

It was a simple program which could tell you the flag if you run it in year 2020.

Censored SVG (10 points)

A simple HTML file with SVG image on it. The image contains flag covered with some censoring square. SVG image is base64 encoded for one small layer.

Hello RISC-V (15 points)

A simple binary which will give you the flag if you run it. It only runs on RISC-V, so you'll need to figure out a way to do so.

Crystal (20 points)

Password validator binary compiled from Crystal. Binary contains a lot of extra stuff, but validating function shouldn't look all that strange.

Powerpoint (25 points)

PowerPoint PPTX file encrypted with XOR. Flag is on the slides.

RISC-V Verify (30 points)

RISC-V binary validating password. It might be easier to run static analysis, but you should still understand basics of RISC-V assembly. Or you could run gdb on RISC-V emulator.

OCaml (35 points)

Password validator binary compiled from OCaml. OCaml uses unusual integer representation, so it's extra layer of confusion.

It had zero solves during the event.

LOLHEX (40 points)

English text encoded by hex encoding, but which of 16 digits is which is scrambled. Simple statistical analysis should do here.

Imba (45 points)

Password validator written in Imba. It will tell you if you get your flag right. It can be solved by static analysis (data driving logic is easy to see), or in-browser, possibly by by attaching DOM breakpoints.

It had zero solves during the event.

Catzip (50 points)

For mandatory meme challenge. It's a zip format made by gzipping something, then turning it into a PNG containing cat emojis. There's two cat emojis, 8 per row, so you can probably guess where it's going.

FPGA (55 points)

A slight variant of challenge from previous time. It's a small circuit, and you need to find inputs which give positive result. This time circuit has NAND, AND, NOR, OR, and XOR gates.

It's highly advisable to use Z3 or similar for it.

Monoalphabetische Chiffrierung (60 points)

German text encoded with monoalphabetic cipher, and then turned into Unicode Fraktur.

It's actually very easy, as punctuation, capitalization, digits etc. were not really scrambled.

LOL64 (100 points)

For super hard bonus challenge, it's English text encoded by Base64 variant, but which of 64 digits is which is scrambled.

Understandably, it had zero solves during the event.

It's definitely solvable in principle, but I'm not really sure how to write such solver. It can be seen as breaking a key of 64 6-bit parts, so 512 bits total. Knowing that it's all ASCII and simple statistical methods get a lot of those bits, but from that point on it would take some creative statistical analysis. Big difficulty is just size of the key, and how a single error in decoding can mess up all following statistical analysis, and cost a lot of time.

I'm sure there are also some ways to circumvent this whole process, and find exact encoded text from some simple statistical properties, but I didn't try it this way as it's less fun.

Sunday, February 24, 2019

Parable of the Sharks

Art show for sharks at Spoke Art by Steve Rhodes from flickr (CC-NC-ND)

Alice believes sharks are the biggest problem facing humanity.

If you ask her, she'll tell you about all documented cases of shark attack. She'll tell you how it's definitely undercounted - random disappearances will be counted as drownings even if they were shark attacks. Or someone could actually drown but only because they were running away from sharks.

She'll tell you that population increases every year, mostly in warmer and so presumably more shark-infested areas, that most people live in coastal areas, so we can expect more shark attacks in the future.

She'll tell you that as people get wealthier, tourism increases, and more tourists means more possibility of shark attacks.

She'll tell you that even living far away from the coast doesn't guarantee safety, as some sharks live in rivers. And what stops sharks from swimming into canals and sewer systems really?

She'll tell you that direct sharks attacks are just start of the problem. Sharks will eat fish, causing famines or mass unemployment in fishing dependent regions. Panics caused by shark attacks might cause mass migration, and all that together increases likelihood of armed conflicts. The more you talk to her, the most every problem turns out to be related to sharks.

She'll tell you how we need global solution to shark problem. Armed guards on every beach. Shark radars. Maybe even security bars in every toilet so shark won't swim up to bite you in your most vulnerable moment. The more you talk to her, the more elaborate her solutions become.

And of course she'll happily provide links with evidence for all those claims.

Bob doesn't believe sharks are even real.

Which of them is less wrong?

Sunday, January 27, 2019

finder-sort

Cat (OOF!) by Long Road Photography (formerly Aff) from flickr (CC-NC-ND)

I was playing a bit with Electron, building a small image viewer, as Xee is weirdly crashy nowadays. A small aspect of any image viewer is sorting images to show them in order. As far as I know there's no programming language with any special support for sorting file names, and default ASCII sort is just atrocious - going cat1.jpg,  cat10.jpg,  cat2.jpg etc.

It's not uncommon to find file manager or image viewer which uses this completely unsuitable sort order.

OSX Finder is notable for very much not doing so, and using human-friendly ordering. Exact details are unfortunately not properly documented.

I created something which works more or less the same, and published it as npm package finder-sort.

There could be some differences, especially for non-ASCII locales. In the end, it's trying to solve the same problem, not necessarily match OSX Finder exactly.

The source code is on github.

About the only interesting thing about it from code point of view is that I used ava testing package this time. Javascript has over 9000 different testing packages, and it's not uncommon for a single program to duct tape together 10+ of them. I'm surprised there's no clear winner yet. Some of the new ones like ava and cypress seem mostly decent.

Sunday, January 13, 2019

What I learned from Awair Air Quality Monitor

Unimpressed kitten by Magnus Bråth from flickr (CC-BY)

After watching DHH's video about air quality I decided to get myself AWAIR 2nd Edition air quality monitor.

Here's what I found.

Setup

Awair setup was fairly awkward. It couldn't connect with my WiFi until I changed my router to some compatibility mode, there were weird error messages about firmware update and so on. I finally got it running, but it wasn't smooth. At least there were no further issues after that.

Air circulation inside the flat is poor

I originally assumed that air in the flat would rapidly be fully mixed up, as it's a fairly small London flat, but that's not what I found. If I open all windows, fresh air gets to the sensor real quick, but as soon as I close them, the stale air is back just as fast from parts of the flat not on air flow path between open windows. I guess it's all the awkwardly placed walls and doors. Keeping windows open much longer eventually works, at least for a while.

How much CO2 per day do I need to get rid of?

One person burns about 2000 kcal a day, so let's say that's 500g carbohydrates getting burned. That's 200g carbon, or 730g CO2 per day, or 17 moles. 17 moles of CO2 takes volume of a bit over 400 liters. Plus a bit more for the cat, and for gas cooker.

Let's say the flat is 50 square meters, at height of 250 cm, so 125 cubic meters. Atmospheric CO2 is 400ppm, and let's say I want to keep it under 1000ppm, so 600ppm difference. 600ppm of 125 cubic meters is just 75 litres.

These are all heavily rounded calculations, but unless I forgot all the high school chemistry already, I need to somehow fully replace air at home 5-6 times a day to keep it in the happy range. Either by opening windows until fully replaced, or by having some small continuous air "leaks".

This number doubles with a second person, to 11 times a day.

That's a far higher number than I expected.

This number also changes depending on carb vs fat based diet, actual caloric expenditure, and so on, but in any case, it's very high.

Cooking anything generates massive PM2.5 spike

The sensor is placed in the computer room, about as far from the kitchen as it gets. And it looks like cooking anything whatsoever without windows open generates a massive PM2.5 spike for a fairly long time. This can fortunately be easily fixed by only cooking with windows open.

It's difficult to keep Awair happy

Awair never had any serious problems with chemicals, and PM2.5 spikes were only caused by cooking.

The problem were the other three readings. If windows are closed, CO2 levels creep up. If windows are open, CO2 levels go down, but temperature and somehow humidity go into unhappy zone. Pretty much no matter what I do, Awair will be unhappy. Only briefly after closing the windows when temperature goes back up but CO2 is still low Awair becomes briefly satisfied.

Perhaps I should move it farther away from airflow to make it more stable, then again it's fairly close to my computer chair, so it should be more representative of air I breathe this way.

What should I do?

I've been definitely opening windows more now, especially when cooking, but that's not too great - I like my place real warm (24-26C), and the outside is quite cold, especially in the winter.

I don't want to keep windows open too long not just because of higher heating bills, but also because of all the sound coming from outside. I'd have serious trouble falling asleep with bedroom windows open, as I'm extremely sensitive to light and sound when falling asleep. And it's distracting when I'm trying to focus on coding or gaming or just about anything.

I could open some other windows, not those in the bedroom, but honestly I don't think air would circulate much - bedroom door is always just slightly ajar for cat's sake, to keep bedroom as dark and quiet as possible. Closing them fully would generate cat scratching on door sound, which also negatively affects sleep.

Plants?

So an obvious idea is to get some houseplants. It's a bit of a hassle, and the cat would probably damage them a bit. But mostly I have no idea how much CO2 would they really remove. I suspect the numbers would be really low for any realistic amount of plants, making it not worth it. Especially at night, when the light is understandably off.

Or is there any other trick I'm missing?

Sunday, November 11, 2018

Webpack boilerplate package for Imba and SCSS

Fluffball by Rum Bucolic Ape from flickr (CC-ND)

Imba looks extremely interesting - seriously, just check out the code examples on their website.

I wanted to give it a go, there was just one tiny problem - in Javascript world you can't just gem install a few things and run them like that. Nope, painless setup is just a crazy ruby idea that never got much traction anywhere else. In Javascript universe everything always requires painfully complicated setup.

I found some boilerplate example for Imba, but it was broken on so many levels, I had to start pretty much from scratch.

So here's a working webpack boilerplate for Imba with SCSS support. Feel free to fork it into your project.

What's in it:
  • Latest Webpack
  • Imba
  • SCSS (as plain old compile to CSS, intentionally no CSS-in-JS shenanigans)
  • CSS normalize to avoid cross browser pain
  • standard npm commands for development and production builds.
Everything uses sane 2 space indentation, and tries to avoid doing anything weird.

What's obviously missing is some kind of testing framework, so PRs wanted.

I haven't used it for anything more complicated than just another TODO app yet, so I don't know if there are any issues. Just report them on github.

Thanks to all the brave souls who answered webpack questions on Stack Overflow - somehow I managed to duct tape working boilerplate out of all that.

Sunday, October 28, 2018

Challenges for October 2018 SecTalks London

She loves the computer by glennwilliamspdx from flickr (CC-BY)

Last month I ran another round of London SecTalks CTF.

There were 10 challenges, and the winner got 9/10 of them during the event (and last one on the following weekend), so difficulty level was about right.

Challenge files and code used to generate them available on github.

There are no answers below, but some serious hints which might make it too easy.

For previous rounds, see posts about September 2017, November 2017, May 2018, and July 2018 CTFs.

Archive (5 points)

A small variant of the 16-level 16-way nested archive, to test for basic Unix scripting skills. This time using RAR.

MonoRSA (10 points)

It's RSA-encrypted message, but it uses only one prime, not two. This is extremely insecure, and can be trivially broken, but you still need to do some math.

BCRYPT (15 points)

Each letter of the flag was encrypted with bcrypt. It's easy to break, as long as you know how bcrypt works, which isn't quite the same as plain hashes, so it was causing a bit of confusion.

RSA RNG (20 points)

It's Debian weak RSA key attack all over. We have target's public key and encrypted message. Also a lot of other people's public keys, all generated using same bad RNG. If two keys share a prime it's easy to break them, even if direct factoring isn't viable.

Python (25 points)

Small bit of reverse engineering - small Python password validator obfuscated with one of online tools for it.

MultiRSA (30 points)

A little known fact about RSA is that it works just fine with more than two primes. So this challenge uses 16 - which is fine, except key size was not adjusted appropriately, so each of those primes is small enough to break it.

Binary (35 points)

Binary password validator, provided in two versions (Linux and OSX) for convenience. It was compiled with -O3 which made its encrypting loops unroll, and what was very trivial code turned into hard to understand vectorized mess. Then again, actually running the program might reveal something interesting.

SVG XOR (40 points)

The flag is written in SVG flag, which was encrypted with a XOR cipherer. This turned out to be quite easy, as SVG files have a lot of structure which can be used to attack this.

FPGA (45 points)

Probably the most original challenge this time. A netlist of NAND gates which can validate the flag and tiny emulator were provided. Can you figure out the inputs necessary to get the validator to accept?

This was the only challenge without anyone solving it during the event, but there were some solutions afterwards.

Tweets (50 points)

A flag was hidden in collection of Donald Trump's tweets, encrypted with monoalphabetic cipher. Great exercise for frequency analysis.

Sunday, October 21, 2018

My current GTD system

She works hard for the money... by Tamer Akça from flickr (CC-NC)

I don't know if there's any way to do GTD properly. I've been trying so many tools, and it always feels like no setup works properly, but if I don't do some kind of GTD then my life instantly falls apart and I never achieve anything, or even stay on top of everyday responsibilities.

In all likelihood if I write this post again in a few years, it will be a fairly different list. Anyway, for other people who try to do GTD, here's my current setup.

Core of the system

The most important part is private git repository which also happens to be mirrored on Dropbox using magic of symlinks.

Inboxes

One recent complication is how often I'm with just a phone, and often offline on a train to make it even worse. The least shitty solution I found is Google Keep. In the past I carried a tiny pen and a stash of post-it notes with me, but that's a bit less practical. Most other software I tried really doesn't like working with limited connectivity.

A big downside of Google Keep is that it's hard to copy a list from the web UI and paste in somewhere, without going through the hassle of exporting to temporary Google Docs document and copy&pasting from there. Someone should seriously write a Chrome extension to improve that part, and well, that someone might end up being me.

I probably should give Google Keep alternatives another try, since Google products don't have very high half-life.

Big section of the core system is inbox folder with anything that hasn't been processed yet.

I have a physical wicker basket at home for letter and related physical stuff to check.

I think on paper a lot, so I often generate a lot of paper mindmaps and lists. Once I'm done with them, they land in the inbox.

Incoming emails which requires some further actions get a star. I never delete any emails, so those stars are the only indication that it's not done. Some people delete or archive stuff and treat their email inbox as a TODO list, and that just feels really weird, but if it works for you.

Usual GTD lists

The core system contains the usual GTD lists like Projects and Next Actions.

I don't divide Next Actions by context, since there's no meaningful context for most of them.

Reference System

I have a physical reference system consisting of a bunch of ring binders with contents inside organized alphabetically by tag. That's mostly for things like bank statements, bills, and other boring paperwork which I might refer to every now and then.

I also have a ref folder on Dropbox - not related to that git repository - which contains all the digital stuff.

Emails stay in Gmail, as they're very easily searchable there.

Calendar

This is a bit awkward, as I use a mix of Google Calendar and old text file based system.

Calendars I use are:
  • upcoming events
  • saved dates for potential upcoming events
  • any periodic actions I want to do every N weeks or months - mostly boring cleanup, backups, reviews etc.
  • birthday calendar
Possibly it would make sense to move to just purely Google Calendar system.

Planning

This might be the most interesting part, as I found I need multiple different kinds of planning to make things work:
  • the usual lists of GTD next actions
  • ad-hoc planning like at start of the day usually happens on paper - by the end of the day what's left of that list goes into inbox
  • weekly goals list to keep me focused - it's typically about 10-15 goals, and I'm aiming at 80%+ Partial Success or better rating. Those lists are not meant to be modified once created. If something fails for a good reason, it fails. Any unachieved goals (including Partial Success) go into inbox.
  • rolling goals list for next 12 months - I keep it as Google Docs document and update what's in progress, done, or definitely failed with some color coding. This documents gets updated whenever needed. Every 3 months or so I archive old document, clean up done or failed things, and create a new one.

Weekly Goals List

The biggest risk of GTD is that a lot of things will get done (as far as failure modes go, it's not the worst one), but whole areas of life where progress is most difficult get neglected.

Because the most important function of those lists is helping to balance different aspects of life, anything where I achieved meaningful progress towards stated goal counts as Partial Success, even if it's still very far from being finished.

Of course since Partial Success is still not finished, it still needs to go onto the next list.

I'm aiming at 80%+ completion rate because typically a few things will just not work around due to external circumstances.

If same thing fails multiple times, then it's a very strong indicator that it needs a lot more planning.

Rolling Annual Goals List

You know how people are best behaving in January after they make their New Year's resolution, but then usually give up by March? People completely miss the point thinking that such resolutions are ineffective - they're extremely effective, you just need to refresh it often.

The list is about 4 pages of Google Docs, and lists many highly specific goals, hopefully covering every aspect of life. This includes many more meaningful goals, but also lists movies, games, and books I'd like to enjoy over the next 12 months.

Most goals on the list are very specific and measurable, but it's not always possible, so some vague entries, and some refer to ongoing practices. If the goal itself is not specific enough, I try to have some more specific subgoals.

12 months feel like about the right perspective for this list. It's really difficult to think in longer term perspective in concrete enough terms, and for shorter perspective it would be guaranteed that many life aspects will go unaddressed.

Whenever I update this list, I try to have a chat about it with certain special people.

Cooperation

The system is private and difficult to share even if I wanted. Occasionally I want to discuss some plans with others, and for this I usually use Google Docs and Google Calendar - or talk about that in person, which also works.

Other lists

I keep Waiting For lists, mostly for things I ordered.

I keep Someday Maybe lists, for things which are not really actionable, but I might get there someday.

I keep Social lists for people I'd like to keep in my life - I check it every now and then, and if I'm at risk of losing contact because everyone is too busy, I try to arrange something.

I used to have Shopping list, but since I do overwhelming majority of my shopping in Tesco online and Amazon, I just throw whatever I need into relevant basket, and every now and then order what's in those baskets.

Logs

And they're not really part of the GTD flow, but I keep a lot of different logs, measuring and writing things every day.

And everything else

And I also have beeminder setup, but it only tracks things like exercise, so it's fairly peripheral to the system. I tried to use it more, but most interesting things might be specific, but not necessarily quantifiable in the way beeminder wants, and usually giving something long term goal with weekly commitment is not actually the best idea.

That's the rough outline of my system.

Big 5 Bonus

And that's what a person who's a few standard deviation high on Conscientiousness is like. Also about as high on Openness to Experience, extremely low on Neuroticism, and somewhere halfway on Extraversion and Agreeableness.

Music I like

Piano Cat I by Nina A. J. G. from flickr (CC-ND)

I mostly listen to music on youtube - or songs I downloaded from youtube with the glorious youtube-dl to dumb devices for listening while offline.

140%

I listen to them at 140% speed. I've been watching everything at high speed so much, 140% is my neutral speed, that's the lowest I can go without anything seeming to be artificially slow.

Nowadays my speeds are:
  • 140% - neutral speed - music, movies, some denser TV shows like Game of Thrones
  • 180% - medium fast - most TV shows like Family Guy
  • 200% - fast - podcasts, audiobooks, let's plays, conference videos, nearly everything on youtube that's not music
  • 220% - very fast - some particularly slow sources like Tolarian Community College
This means I'm really limited to youtube as my only place for music, and to "unofficial" download sites as my only source for shows and movies. I tried Spotify and Amazon Prime, but they have no speed control, and that's insane in this day and age.

I guess I can watch things at 100% in cinema or when watching it with someone, but that's an exception, and it still feels artificially slow.

By the way, if you know of any source of music or streaming that has speed control, good quality, and good selection (not just in US), I'm definitely interested.

Youtube for music

The great things about youtube is that it has basically everything, it's free, and ad blockers work perfectly on it, at least on desktop. I guess they recently added some sort of subscription service for people who feel sad about using ad blockers, and I'm not philosophically opposed to that, but I didn't have any time to investigate that. 

The worst thing about youtube is total lack of metadata. Also people thanking their Patreons for a minute after each song while I'm trying to listen on shuffle. And it's basically useless on the phone with no or limited connectivity.

The so-so thing about youtube is its recommendation algorithm. It's not completely useless, but it keeps suggesting stuff to me a one line python script would know not to (if vocalist.gender == "male": return False).

Their fancy subscription service solves none of the problems I have, just a problem ad blockers already solve.

Metadata

But seriously, what really annoys me is that I youtube never bothered to provide any metadata for its song.

There's a plugin which tries to parse song titles with regular expressions, but it's not very good. It's sort of OK for "official" songs, but it's failing for almost every independent artist who mostly do cover songs.

So mostly for my own future reference, here's a list of artists I've been recently listening to a lot, in alphabetical order, manually extracted from my youtube watch history. I tried to categorize them into mainstream and independent artists, but it was fairly futile.

The List

If by any chance you're enjoying similar music, and you have any fun recommendation, send me the links.

Saturday, September 22, 2018

Fun and Balance mod for EU4 1.26.1

MIDORI by Marco Mosti from flickr (CC-ND)

Fun and Balance is a mod which tries to make Europa Universalis IV a better version of itself. The mod has very limited goals:
  • fix any issues where poor balancing makes gameplay worse - making more options viable, and occasionally toning down anything that's overpowered enough to make alternatives irrelevant
  • let people have fun in any way they choose, removing arbitrary prohibitions penalties
  • reduce AI cheating, as game is more fun when everyone plays by the same rules
  • reduce forced historical railroading via events or restrictions on player actions
These goals mean that every patch the right thing to do changes, and I need to go through list of fixed I've made and decide if they're still applicable. Many of previous changes I've made become obsolete because most problematic areas are likely to be addressed by future patches.

Playing with this mod shouldn't feel like you're playing a mod, it should feel like you're playing vanilla which finally patched silly things right.

It doesn't try to significantly affect game difficulty - it might increase it slightly by reducing cheesy tactics, or maybe slightly reduce it if you're trying to play naturally.

So here's the full list of changes, ordered roughly by impact, with reasoning behind them explained in detail.

Download links

Base diplomatic relations increased from 4 to 8

In vanilla diplomatic slots are always exhausted by every nation. You absolutely need a few strong allies, a few vassals to expand, and that prevents you from having any diplomacy beyond that. All options such as royal marriages, guarantees, marches, local alliances, supporting independence of others etc. tend to get mostly unused as they take precious slots you don't have.

So the mod just doubles base limit, and this opens a new world of diplomacy.

This bumps up difficulty, as allies are much more useful defensively than offensively. It leads to much denser alliance networks, and it's much less likely to be able to get a free attack on unprotected minor. At least as long as you play with Cossacks DLC enabled.

Mercenary limit reduced to half

Both players and AI have access to infinite manpower pool of mercenaries, and close to unlimited very cheap loans.

This means defeats have minor consequences. It doesn't matter than you just killed every military age man in a country, they'll just spam loans and mercs next day. Attrition, manpower buildings, manpower bonuses, all of that matters a lot less when you can just loan and merc.

Unfortunately it's a bad idea to do any drastic mercenary or loan nerfs, as AI often loses all its manpower to self-inflicted stupidity such as parking a doomstack in a low supply province during peace time.

With some testing I found out that the best balance can be achieved by halving merc limit (both base and increase from force limit), which is unreasonably high.

This doesn't affect reasonable use mercs to supplement your manpower. You can even go vanilla style merc spam if you stack available mercenaries (administrative, quantity, aristocratic) and force limit (quantity, offensive) bonuses. AI can handle it reasonably well.

Small AI countries are mostly unaffected as mercenary limit was already higher than their force limit, but for big ones like Ottomans or Ming you can now actually defeat their army and not have army just as big show up next day.

Fort upkeep reduced to half

Forts are unreasonably expensive. Most good players just delete all or nearly all of their forts, and AIs tend to suffer from having far too many crappy forts in wrong places breaking their economy.

Reducing upkeep to half of vanilla values makes keeping or even building forts more reasonable option for the player, and helps AI economies.

Unfortunately there's no way to mod in more understandable zone of control system.

AI cheats reduction

Call for Peace and naval attrition are removed, as they're mechanics which are exclusively applied against the player. AI no longer gets extra free leader.

It doesn't make a huge difference in terms of difficulty, it's just better if everyone plays fair.

Tweaked subject settings to match wider diplomacy

Vassal annexation minimum year increased to 20 years.

Vassal annexation is just half the vanilla cost - it's unreasonably expensive considering you have to pay full core cost for something which will likely be just a territorial core.

Liberty desire from development a bit lower, and vassals don't count your marches strength in their calculations.

Colonies actually care about relative power of themselves and their supporters, but have negative base LD to balance that out.

Diplovassalization max cap increased, but penalty from their development is still quadratic.

Big tributaries care a bit more rebellious.

There's now -100 cap from annexed vassal opinion, so you're not going to accidentally stack it too high by poor annexation timing.

Liberty desire from historical friend or rival toned down.

All of this generally works well for player and AI, and doesn't require unnatural play.

More building slots

Building slots in vanilla are very restrictive, so many building see zero play, and especially AI wastes its slots on useless buildings a lot. The mod increases extra slot from +1 every 10 development to +1 every 5 development.

Improve awful idea groups

Some idea groups are better than others, and it's totally reasonable but two are so ridiculously useless people only ever take them as a joke.

So mod gives maritime ideas +50% light ship trade power and +1 merchant - so you can actually do some trading - so you can actually get some trading; and it gives naval ideas +1 free leader - so you can hire that admiral without taking a general slot from your armies.

This should hopefully move them from joke tier to situational tier.

You can convert in territories

This is controversial change in 1.26 patch. It's not a completely bad idea, as conversion was really fast and really easy - but until some outs are added (like religious ideas giving you ability to convert in territories, or replacement of outright ban with just slower speed), it needs to go.

Everybody can claim states

This feature is locked to Russian Tsardom government in vanilla, but there's no good reason for it, so I just made it available to everyone.

I thought about letting any empire-tier government do it, but game didn't like this idea (without creating a lot of government types), and there's really little downside to just giving everyone this feature.

Rival and Power Projection changes

I'd love to be able to restore rival system from early patches where everyone could rival everyone else - or at least for every great power to be able to rival every other great power.

Unfortunately that's not moddable, and it's very common that you get nobody to rival late game (and therefore very little power projection), or extremely limited choice of countries to rival early game.

The mod therefore increases power projection for great power status, for eclipsing rival, and slows down decay from actions against rivals. It also increases max rival range slightly, so early game you have more choices.

Religious Shift Decision

You can now freely switch religion to one of your capital at cost of some stability. It's disabled for Papal States, as that messes up with the game.

Disable End Game Tag checks for player

End game tag checks are an egregious case of stopping people from playing the way they want for no good reason.

I left these checks in place for AI to avoid checking them one by one if they make sense, but they're disabled completely for the player.

More formable countries

It's fairly arbitrary which countries are formable and which aren't. If you want to become Norway or Portugal and managed to shift the culture (which is admittedly very easy), why shouldn't that be possible?

Right now it keeps your original missions. It probably should ask if you want old or new missions with popup similar to one for ideas.

All those decision follow similar pattern - you need to have fully unified that culture, be big enough, and have admin tech 10. Excluded from this is anyone with existing form nation decision. Also excluded are Japanese, Russian, and Chinese culture groups, as they already have different mechanic for tag progression (forming Japan, Russia, and becoming emperor of China).

Coalition CB changes

Games sometimes need "invisible wall" style mechanics - fairly brutal means to limit where player can go. They don't have to be fun, but they should be very difficult to trigger accidentally. Normal in game mechanic, even negative ones, should be enjoyable.

Coalition system in EU4 fails this completely. It's really easy to trigger - so easy that AI minors in the HRE often have coalitions against them by 1450s - and it's completely miserable.

Most experienced players learn to play around coalition system by juggling truces, or attacking any country which joins coalition day one, or by using cheesy tactics like offering ally land as soon as possible once coalition war triggers (or in previous patches, offering 10000 gold).

If you actually try to fight and win coalition war, game makes it miserable. You can't separate white peace anyone even if you 100% them, so you'll have to keep going back then to swat their rebels. If you had any allies, they'll peace out leaving you with -40% warscore from battles and -25% ticking warscore - which somehow still counts against you even after they leave the war (that should seriously be fixed regardless of coalition issues). After a few tries everyone learns to never even attempt this unfun fight as just cheese it.

Not to mention just how ahistorical and immersion breaking it all is.

We have somewhat limited possibilities to mod our way around it. We could try to rebalance AE and tone it down a bit so you're less likely to hit the invisible wall. Or we could make fighting coalition more enjoyable.

A small modification of changing coalition CB from superiority in battles to defending capital goes very far towards making it a regular challenging war. That's how it used to be in early patches.

Burgundy event chain removed

EU4 is a sandbox game, and it's more fun when different outcomes happen in different campaigns. It's unfortunately been leaning towards very heavy railroading - Ottomans become second GP after player in nearly every campaign, England forms Great Britain, Muscovy forms Russia, Castile forms Spain, all almost every time unless player stops that, Ming never collapses or expands much etc.

It would be nice if we could make things more dynamic, so sometimes Aq Qoyunlu grows into the Middle Eastern menace, or Scotland sometimes won the British struggle, or some German minor seriously attempted unification. Unfortunately there's no straightforward system for modding in this kind of unpredictability without huge gameplay changes.

One piece of such historical railroading is going way too far - partition of Bugundy event chain. It's a totally nonsensical system where major European country gets divided completely disregarding its situation. The mod just kills it with fire.

I'm open to suggestions how to create higher diversity of outcomes.

Defender Aggressive Expansion discount increased

Being defender in EU4 sucks, as you can't use any of your CBs (during the war, or for duration of truce afterwards), can't declare anyone cobeligerent, and don't get CK2-style reparation for winning.

To make this slightly less miserable, mod increases defender AE discount from 25% to 50%. Remember that extra AE for non-cobeligerent attackers still applies.

Rebalanced Religious Conversion Rates

It's really silly that it's easier to turn a Catholic into a Sikh than turn an Sunni into a Shia.

The mod rebalances completely arbitrary conversion penalties (+4 against pagans, +2 normally, +1 or +0 sometimes) into consistent +4 against pagans, +2 against heretics, +1 against heathens.

Trade Map Tweaks

EU4 doesn't support dynamic trade map, and doesn't allow cycles, so any trade map will require compromises.

The mod adds Panama to Mexico and Patagonia to Lima link, and (to prevent cycles) removes Philippines to Panama and Mexico to Panama links.

This lets Asian powers enjoy New World trade from at least Pacific parts of the New World. The tiny downside is that Spain can't transfar trade from Philippines through Mexico into Europe, which historically happened, but that never happens in game anyway.

I'm considering much more aggressive changes, something like Better Tradenodes and Tradeflows mod, but it would require a bit more testing first.

You can use subject's religious CBs

You can declare religious war on your enemies which you only neighbour through subject.

For this both you and your subject need to have religious, and you must have same religious group as your subject (for holy war), or same religion (for cleansing of heresy).

This is mostly to reduce bordergore required to maintain CBs.

Doubled tradition gain from battles

EU4 made a strange change at some point that nerfed tradition gain from actual fighting to very low values, and it became based more on idea group choice than actual fighting. With numbers (land and naval) both doubled, it's a bit less silly, and if you're constantly fighting you should now have decent tradition level.

Religious Leagues as any Christian religion

In unlikely event that HRE will be split between some other denominations than Catholic / Protestant, a league war can happen in a different way. So you could have Reformed, Orthodox, Coptic, or Anglican challengers.

First victory by the challengers will just flip the dominant religion, you need second victory to lock the new religion.

No war exhaustion reduction while still at war

This button is only available for countries at peace, so you can actually make other countries suffer, or be forced to suffer yourself. It's no longer just diplomatic monarch point cost.

To match this, AI willingness to peace out based on War Exhaustion doubled.

Corruption Slider goes twice as far

If you're interested in rooting out twice as much corruption, at twice the cost, you can move the slider all the way to root out -2 a year. This is a necessary change to deal with corruption from too many territories.

It's obviously very expensive to go that far.

China nerf

Rebels in EU4 are too weak, and we all love seeing a Mingsplosion every now and then. So unrest from zero mandate doubled from 5 to 10.

Custom Nations Improvements

Most custom ideas get levels all the way up to 10 at higher cost. There's no penalty for taking too many of same kind of ideas - there's no power level reason for it.

Base monarch stats changed from weird 2/2/2 to 3/3/3 which actually matches average in-game monarch. Limit on distance between provinces of your nation increased to more reasonable values (so you can recreate something like in-game Genoa).

Merchant republics 20 statified province limit removed

It's a pointless nerf to a weak and unique government type.

Also for people without Dharma, Adopt Plutocratic Administration decision has province limit lifted. It's only useful for roleplaying anyway.

Longer CB on Backstabbers

EU4 has CB against allies who betrayed you which last 3 years, so you can only use it if you're willing to truce break. It's as pointless as things get, a relic from times when breaking alliance didn't create truce. The mod increases it to 10 years, but it's a very weak CB, so it's probably just there for roleplaying.

Some arbitrary decisions limits removed

A few decisions like moving capital to Constantinople and recreating Byzantium have a bit less strict limits, so you can use them even if you're doing something unusual like Serbian culture Coptic Ottomans.

Imperial Ban CB

The game has CB to take provinces from non-HRE owners, but it takes 100% AE with HRE penalty, so it's basically useless. Changed it to just 25% AE penalty, for affected provinces only.

Faster peace out

AI willingness to fight a losing war just because it's not been going long enough reduced slightly.

Some overly expensive action cost reduced

Moving capital, moving trade port, and culture conversion are mostly useful for doing weird things, and are quite overcosted, so all of their costs halved.

Everything is optional

As much as possible I tried to make every change optional, and keep it in separate files, unfortunately it's not always possible. It should be fairly compatible with most minor mods. For some popular total big mods (Extended Timeline, 1356) I can just offer separate builds.

There even used to be in-game menu for some of that, but people had performance complaints, so I got rid of it just to be sure.

I'm quite ruthless at killing off any feature which is no longer necessary. If there's balancing issue with relatively simple fix, I'm happy to include it in future versions.

If you think some of the changes cause more problems than they solve, definitely tell me about it too.

I don't always publish this kind of long explanatory blog post, but it's usually updated week or two after new major patch codes out.

Wednesday, August 01, 2018

Challenges for July 2018 SecTalks London

Cat sitting on a desk by freestocks.org from flickr (PD)

Last month I ran another round of London SecTalks CTF.

I only created 6/9 challenges this time, 3 Android challenges were created by imhotep.

Challenge files and code used to generate them available on github.

There are no answers below, but some serious hints which might make it too easy.

For previous rounds, see posts about September 2017, November 2017 CTFs, and May 2018.

Hidden treasure (5 points)

As per tradition, there was a bonus challenge with a zip bomb. This time at the end of 16-deep archive with 16 branches each there were pictures with a treasure, with EXIF tags containing either the flag or information that you failed.

Android 1 (10 points)

The simplest Android challenge wasn't even all that Android specific - as the flag was hidden directly inside the .apk.

RE1 (15 points)

Easy reverse engineering challenge was just a 32-bit Linux binary, which verified passed flag character by character, in random order.

Disassembling it, you'd get instructions like:

cmpb   $0x74,0x10(%ebx)
jne    80484b7

If you arranged them correctly, you'd see the flag. Or you could just grep for all characters, and use anagram solver - apparently that was an option too.

Android 2 (20 points)

This one required actually running the app, or fairly complex static analysis. The app saved a file with the flag to device storage. If you can find the file, the flag is yours.

RSA (25 points)

This challenge was actually quite realistic.
Bob sent a message we need to decrypt.

Probably due to bad RNG, it looks like Bob and Alice picked same N for their keys, and we managed to steal Alice's private key as well.

Perhaps there's a way to take advantage of this.
And there is a way. Having private key (n, e, d) there's algorithm to factor n into p, q. It's not completely obvious, but it's fairly short to implement. With p, q (shared between both), and Bob's e, you can trivially get Bob's d. Then you can decrypt the message.

RE2 (30 points)

It was a slightly harder version of reverse engineering challenge. The only difference was that flag was encoded into Base64 before checking, which could throw people off, but binary wasn't stripped so b64_encode method was a massive hint.

Android 3 (35 points)

The challenge app is making HTTP request, which needs to be intercepted and modified to get the flag from the server.

Once you get the request it's obvious what to do, and there were hints provided how to setup proxy on Android to get them.

CTR (40 points)

Another somewhat realistic challenge. A collection of Elon Musk's tweets and a flag was encrypted using AES-128 in CTR mode. Reusing same IV for all of them.

This turns CTR mode into XOR with random keystream, and breaking XOR cipher is fairly basic. It turns out nobody remembered mode names, so I had to give people some hints before they tried CBC or CFB attacks on it.

Croatian Monoalphabetic (50 points)

This challenge was worth the most points, but in principle it was very easy, just a bit time-consuming - and I ended up giving people hints how to approach that.

It's just statistical analysis of another Latin script language. So you can start by assuming space is the most common character, then use character frequency tables for Croatian, or lists of most common words, or take advantage of the fact that flag (very long word) is embedded inside.