Playtesting is king. It’s a mantra among game designers, who often know all too well the pain of carefully designing a mechanic and then finding that it’s absolutely awful to play with or too hard to understand when playing in a real group. On the other hand, it’s a huge relief whenever a mechanic or idea that looks iffy on paper happens to test really well among players.

We’ve been pretty damn insistent that we want playtest feedback for Pokémon Journeys over untested theorycrafting, so it’s only fair we explain a bit of what we’ve learned through playtesting and what makes for a useful playtest for us. I’m going to be jumping through a number of sub-topics here, but I’ll try to keep each of them anchored to actual playtest experiences we’ve had in PTU or Pokémon Journeys.

If you haven’t gotten a chance to look at our draft materials for Journeys, you can find them at this link.

What makes a good playtest?

Ideally, you want to strike a good balance between designing the playtest to test certain aspects of the system and keeping the playtest organic. That’s a bit of a hand-wave-y answer, so I’ll try to give some specific examples of what we’ve done in testing Pokémon Journeys.

• The first playtests we ran in the new system were at low but not starting levels, so that there was enough progression to have builds diverge without us needing to create a ton of high level content. We had a curated list of Pokémon we had converted move lists for, but we didn’t prescribe which specific mons a given player had to take or how players statted them using Roles and Builds. These were very combat-centric tests with what we saw as typical encounters for those levels, which gave us a fairly targeted test while preserving the player-side statting process rather than handing out pre-mades which might have been too well designed for the playtest encounters.
• Later on we would do high level playtests with no Trainer involvement at all, just to see how the Pokémon battle math worked out in a pure and evenly balanced battlefield. We had to know whether the game functioned at that level before returning to tackling Trainers.
• One of the most influential pieces of feedback we got early on was from Domovoi saying he felt that his choice of starting Class (at that time called Specialization) did very little to affect his character’s playstyle or make them feel distinctive. Since then, we’ve been trying to run (or encourage others to run) Level 1 tests with known problematic Classes to see if we can make their base kit fun to use. This is also why taking a Class gives you more now than it used to.
• I’m currently running a playtest mini-campaign where the players will level from 8 to 13 over the course of 6 sessions of play. It’s pretty accelerated, but the idea is to give the players a taste of the end of Tier 2 through the beginning of Tier 4 and see how it feels to develop a Trainer and a team of mons through those levels.
• At some point, we’ll be running tests with no combat at all, purely driven by Challenge Checks. Combat isn’t necessary in every playtest, and if we want to focus on how the non-combat mechanics work, it’s best to cut it out. That said, we may have to make the penalty for failing certain rolls be similar to the attrition that a party might experience from a fight in order to keep it feeling more authentic.

In general, it’s good to pick a goal (I want to test starting trainer kits, pokémon combat math at high levels, non-combat mechanics and travel, etc) and design the test session to hone in on that while leaving as much freedom as possible for players to build characters and play in the way they’d naturally play.

Players are usually very good about identifying when something feels bad. They’re not always very good at prescribing a solution.

You have to dig into why a player is feeling a particular way when you get negative feedback. Is the mechanic fundamentally broken? Or perhaps merely hard to understand? Maybe it has too many moving parts for what it’s trying to accomplish. Or it could simply be that what the player is looking for is not what your game is aiming to do.

For this one, I’m going to jump WAY back in time to when I started playing in zoofman’s Tales of Visiwa campaign, which was among the first full-length PTU campaigns to run and conclude. We started off the campaign with a party that slanted pretty heavily towards full contact fights, and we quickly noticed that it felt pretty bad to take our Trainer turns. Even building offensively, we would be easily out-damaged by even our weaker starting Pokémon, and we weren’t any bulkier for the most part. If anything, we were even squishier in the cases where we tried to shore up our offenses more.

So we buffed Trainer Stats because the players (including me) thought that was the most direct and straightforward thing to do. And in retrospect, that kind of approach to combat Trainers likely set the stage for the severe imbalance between combat and pokésupport Trainers throughout the rest of PTU’s lifetime. #oops

Thinking about it with a bit of distance, none of the other players or I were wrong about how combat Trainers felt to play. Having your main avatar in a game be bad at their prescribed role doesn’t feel good, even if, as in the case of combat Trainers, having them be that weak was more balanced. Giving combat Trainers more Stats made them feel better to play but also meant making them way more powerful than their pokésupport counterparts, who couldn’t possibly match the contribution of a second damaging Standard Action every round doing the same amount of damage as an average Pokémon.

It really wasn’t until Pokémon Journeys that we finally found a good solution to making combat Trainers feel good to play while keeping them balanced. Which brings me to…

Test radical ideas and let yourself be surprised.

There was a good deal of skepticism when I proposed giving Trainer-Pokémon pairs one shared Standard Action each round as well as sharing an initiative value based on the Pokémon’s initiative alone.

There’s not much to say here other than that they both ended up testing pretty well. Having both Pokémon and Trainer on the same initiative tick made rounds go by a lot faster, and while new Combat Classes are still a WIP, we’ve found that the basic rhythm of alternating attacks between the Trainer and Pokémon has been well-received in our tests and gave us a way to keep Combat Classes feeling strong without overwhelming Pokésupport Classes.

On the other hand, Approaches fell pretty flat with some players, due to a combination of the small number of different Approaches and their relatively low values. Players were split on the idea of having Approaches shared between Trainer and Pokémon, but ultimately we decided that didn’t test well enough.

Players try to play optimally and tend towards convenient and easy options.

This isn’t a bad thing, by the way! It’s really helpful to see how players actually play the game, as opposed to how you designed for them to play, so that you can unify the two better. It doesn’t do anyone any good to work against the natural inclinations of their players.

Something I’ve realized already in running my mini-campaign, which was explicitly advertised as being a combat focused game with heavily involved boss encounters each session, is that it’s probably really hard as a player in games like this to want to take anything but healing items with Supply Points.

Lacking a Potion when you need it is much worse than lacking a Pester Ball in most circumstances or lacking adventuring items when they could help you get past a non-combat obstacle more easily. And besides which, predicting when other items would be useful is hard while Potions and Full Heals are almost always useful.

A haunted house session Doxy ran surfaced a similar dynamic around Supply Points. I played an Electric Ace // Taskmaster with a Life Orb on my Chinchou and spent the whole session blowing through my own HP every battle to do massive damage and healing up with Potions afterwards. I absolutely could not afford to have spent my Supply Points anywhere else, but I had just enough to get by through the adventure.

The glaring exception to this observation was when Masterly, without even knowing what the playtest scenario was going to be, chose to take Cleanse Tags with his Supply Points cause he’s too used to Doxy’s GMing by now. He insisted I mention how he felt very rewarded for his choices there.

Another interesting case happened the haunted house session. 13thsyndicate played a Fairy Ace but didn’t really make use of its abilities. We designed Fairy Ace’s base kit around using Fairy Moves to create Fairy Lights that impeded foes’ movement and ranged attacks, which she didn’t use because having her Mimikyu use Shadow Claw and Feint Attack in a haunted house adventure with spooky ghosts turned out, surprise surprise, much more effective than using Fairy Moves. Fairy Lights were also very difficult to place effectively, so it definitely wasn’t enough to make using a Fairy Move more attractive.

Masterly played a Duelist in that session which revealed a similar problem with the Class. The base kit just didn’t have effects for spending Momentum that were useful to low level characters, and it was far more effective to keep Momentum high for the Talents we wrote that had minimum Momentum requirements than to actually ever spend it.

Finally, in that same session (it was a very informative playtest!), we discovered the way we wrote Rogue made it a very good tank, forcing bosses to focus on it and punishing them for trying to swap targets. Aori, who was piloting the Rogue, really leaned into that since it was the effective way to play the Class. We promptly took the offending Talent and gave it to Martial Artist instead.

And that’s all I’ve got for now. Hope this gave you some insight into how we approach playtesting Pokémon Journeys. There’s definitely a lot that we wish we could do with playtests that we don’t have the resources for – giving the rules to groups for blind playtests, running enough tests to collect lots of numerical data we can crunch, etc. A fair chunk of material goes through less tested due to being a small team with limited time to work on the game. Alas, the difficulties of hobbyist game development.

Next time: Not really sure! If there’s something you’d be interesting in hearing about, let me know on the forum thread here or over Discord. I may simply post more about my playtest mini-campaign because I plan on releasing that as a pre-made adventure when it’s done.