The best kittens, technology, and video games blog in the world.

Friday, October 28, 2016

Review of scryfall - mtg.wtf competitor

♥  My Girl ♥ by Trish Hamme from flickr (CC-BY)

After magiccards.info got borderline abandoned, I wrote mtg.wtf as a much better search engine for Magic cards.

There have been a few other attempts by other people, but they generally had extremely simple search functionality, so nothing worth writing about. A few days ago a new site came out - scryfall - which is at least seriously trying to support MCI-style syntax, so let's take a look.

Basic design choices

Let's compare MCI, MW, and SF:
  • All search engines treat every printing and every card part as a separate object
  • SF follows MW, and avoids polluting search results with non-English cards
  • SF unfortunately follows MCI and pollutes o: search with remainder text
  • SF includes nonstandard cards (like Planes) in all searches. MCI completely skips them. MW only searches for them if requested explicitly (by t:* or by specific search like t:plane).
  • SF decided to skip un-cards. MCI has them, and MW not only has them, it even has a lot of special queries (like for fractional cmc) which only makes sense for un-cards.
  • I have no idea why the hell SF included this. Is that some joke, or a phantom card?
I feel fairly ambiguous about including nonstandard cards by default, and perhaps SF's simple approach is better. As for everything else, they'd probably be better off matching MW.

Search engine issues

I tried to run searches from their syntax example page and compare them with what mtg.wtf returns.

Their results had a lot of bugs, such as:
  • wrong cmc for all meld cards
  • wrong cmc for some DFC cards
  • many cards wrongly tagged as is:commander, like Brisela, Voice of Nightmares, Kenzo the Hardhearted, or Ormendahl, Profane Prince.
They also made some really weird design choices, which I'd honestly just call bugs, as all of them make searching less useful:
  • color identity search being >=, while what you want pretty much every time is <= - you're generally filtering for cards fitting specific commander, not the other way around
  • mana search being >= on symbols, so if you search m:4 it matches {4}{G} but not {5}. That's weird as hell, as the natural search would be equality, and failing that at least treating {4} as {1}{1}{1}{1}.
  • Weirdly they decided that cards with changeling should be returned for searches for all creature/tribal types. So if you're searching for monkeys, you get 1 monkey and 30 changelings.
  • cards like conspiracies are listed as "banned" instead of simply excluded from all formats
Some of their choices are probably fine either way, but MW generally follows MCI in such borderline cases:
  • is:timeshifted changed its meaning from MCI, and SF is:colorshifted does what MCI is:timeshifted would.
  • color search being "and" instead of "or" like MCI/MW. So c:rg searches for cards which are red and green, as well as any additional colors. This is easily achieved on MCI/MW with search for c:r c:g.
  • they treat 2015 frame update as different frame type, which MCI/MW doesn't
SF doesn't have any of MW's extensions. First such limitation I ran into is that there's no way to search for split cards like Fire // Ice or t:creature // t:planeswalker.

They added a bunch of search aliases like color:red for c:r - I looked through the list, and I added some of them to MW as well, because there's pretty much no cost.

mtg.wtf's codebase is all open source, so I'd recommend them to just take my test suite, and try to make theirs work better.

Extra features

The most ambitious feature they have is that they want to instantly include cards from upcoming sets as they are revealed, not only after set appears on Gatherer. This is sweet, and definitely seems like way too much work for me at mtg.wtf.

They have card price integration, which I still haven't added, as all card pricing information seems to be hidden behind private APIs.

They have online API, which I guess I could add as well, but you can just download the code from github, and run it all locally, without even bothering with the API. I could add API to MW quite easily too, but it would be useful to do some performance testing first, as APIs often end up having order of magnitude more requests than real users.

SF has much better feedback for search query syntax errors than MW. That's definitely a good area of improvement. On the other hand their spelling correction is awful - compare SF with MW.

MW has been rather conservative with website design, and SF is trying something much more ambitious.

3 comments:

Scryfall Team said...

We appreciate the thorough review and going to use it as inspiration to make a few updates. Definitely some bugs to fix too.

Two quick things:

We do have Un-cards, they're hidden here:
https://scryfall.com/search?q=is%3Afunny

And here's the backstory on Gleemox:
http://mtgsalvation.gamepedia.com/Moxen#Magic_Online
https://www.cardhoarder.com/cards?data%5Bsearch%5D=gleemox

Hans Nowak said...

So how do I find, say, all creatures with flying? :)

(Searching for "flying" in the oracle text is not the same, you'll get many false positives that way...)

taw said...

Hans Nowak: True, "o:flying" is not quite that, but getting rid of reminder text cuts vast majority of false positives. Search engine could in principle split card text on permanent into individual abilities, then it could have something like "ability:flying".

Actually, it shouldn't be too difficult, I did such thing once for mana generation analysis.