Disclaimer: I have not been provided any inside or NDA’d information on any gaming platform in any form.  All of this is from public information and my own conjecture and professional experiences.

I’m telling this story for a few reasons.  First, I was able to be a little clever in my approach and since I can share it, I thought that I would.  Second, it’s a really good example of how I approach problems and also an example of how some highly competent people who are very close to problems that they have worked really hard on behave and, at times, an external take on it can be revolutionary.  I got super excited about doing this job.  I thought: “All my usual nerd skills plus risk management for in game economies? Too cool.  Let me look into prep for this.”  This is some of what I came up with.

Enter The Ninja

Some time ago, I was interviewing with one of the serious ninjas of the gaming industry.  I gather that most people in games has heard of him and everyone who plays games knows his work.  The level of success in the teams that he was a leading member demonstrated in yielding profitable game franchises is nearly unparalleled.

I mentioned to him that I’ve played his games, praised his previous work, and that I admired one of his presentation that I was able to review earlier last year in which he was presenting ways that he had worked in the past reviewing attacks and deterrents use difference code base cheating and other attacks used mostly in massive multiplayer online games.  I hoped that I didn’t sound like I was sucking up overly much.

Having researched his presentations and LinkedIn references, I felt that I had some idea of his type of skill base and a vague sense of their perspective when it came to preventing and mitigating attacks to these gaming platforms.  After all, it is pretty hard to give useful advice or an interesting proposal if you don’t have any idea of the interests and nature of your audience.

I surmised from the position description that the rockstar hiring manager was looking to offload responsibility from himself so that he could focus on other matters as a visionary for his game that was coming into focus.  He had previously been performing these deterrence and mitigation tasks himself and was likely to be looking for someone similar to him to carry on his efforts in this area with a comparable methodology of pure code audit.  My proposal, instead of focusing on a pure code-based strategy as he would, focused on analytics,  information gathering, basic deterrence in reducing the easy attack surface and classical targets, and attacking the bread-and-butter of the profitability of the attack itself; the sale transaction.  The premise that I was arguing was that if you can make the trade situation unprofitable for gold farmers and thieves that are drawn to large popular online economies, they will go elsewhere with less protection and easier profits; just like the rest of the internets criminals. You don’t always have to outrun the wolves, but just to be faster than most of the herd.  If you are fortunate enough to be wildly successful, additional resources can be found to meet these latter challenges to cover exposures when revenue makes it possible.

Global organized crime, corporate espionage, and endless complexity of layered third party solutions is often present, expected, and accounted for in enterprise environments.  I figured that the challenges of a nearly pure in-house development, fat client, and all server side controlled environment to be an easier one than most I’ve been tasked to find solutions and was upbeat about it.

When you are able, gathering intelligence on the behavior of your professional adversaries is an advantage that should not be missed.  Sun Tsu and all that.

Virtual Economies

Virtual economies are becoming more and more interesting to people other than fun-seeking gamers as they are beginning to be considered as real world assets instead of worthless imaginary online trinkets.  Since they might be treated as normal assets, the IRS is becoming increasingly interested in them.  More and more, these online realms are starting to be oddly considered as commodity and property houses.  It makes the kind of work that ninja and I were speaking about all the more interesting after you understand this detail.  What are large highly valued gaming environments today?  What will they be tomorrow and what should these companies be planning to handle in managing them?

So how can two people who are clearly doing pretty good work respectively come to different conclusions about how best to meet these challenges?  It’s really just about different experience and insights lead to different approaches.  I haven’t come from a comparable hardcore development background as have many who I talk to and work with in common environments and shared stakeholders.  Most of my work has been in systems management, managing operating systems, information security implementations, tuning, management and maintenance strategies.  In short, I bring things together and make them work.  I had a different perspective than he when it came to focusing on at the attacks that he was most interested interested in deterring.  He went to his core competence; code audit, and I went after mine; a systematic approach with the intent to derail the opponent’s ability to accomplish their goals.

Naturally those who run these gaming operations aspire to write perfect code and prevent or deter misuse that takes away from the profitability or enjoyable experience of the game for themselves and their users.  They, like everyone else, don’t generally speak about the specifics of how they try to accomplish these goals; secrecy has value here.  However, one can guess based on the actions of the most successful MMO, World of Warcraft, that the same lessons will likely apply to other MMO efforts.

The consequences of neglecting the in-game economic factors of the sale of gold in game seems to yield inflation if not addressed.

Blizzard implemented dual factor authentication and very nearly gifted the hardware tokens for their users.  They subsidized the cost of a pseudorandom two factor authentication to their subscribers was required because the online assets of blizzard were valued at over $12 billion dollars (at the time a couple of years ago and sure to be a greater number now). It only makes sense that that amount of value would be worth protecting with an investment in authentication threat mitigation.  For a game that is just about to be launched however, this would be a difficult investment justify immediately as the total in-game assets are still close to zero.

Back in the good old days when massive multiplayer online games were new, offerings like Ultima Online had interesting problems that existed in their game environments. Things like duping, which was possible by analysis of network stream or attacks on the application memory itself.  In the 90s, people were able to duplicate the best items in the game without having to earn them after having seen the item picked up or dropped and identifying that the network traffic could be easily replayed to the advantage of the player.  So just by injecting traffic into their appstream during play, it would be accepted as valid and the items generated out of nothing.  Add in the ability to macro up a massive amount of these items during the lifetime of the bug and caching them away for after the bug window closes and prices re-adjust can have serious implications to the game economy.  I make the assumption that hand-picked game devs will know and understand game problems such as duping and why they should be avoided by not trusting clients and having trust perimeters and countermeasures to keep the game honest.

When real money and recognition come to games like this, and I think they will some day, there will be APTs in games just as there are in other areas of more contested electronic mediums; pharmaceuticals, trade and state secrets, illicit markets, and everyone’s personal favorite, anarchist cyberpunk types who don’t care and do what they want.  These corporate espionage people may take advanced measures like infiltrating the businesses that feed off of theirs via lawyers or more clandestine methods.  Most of the world seems to employ a very can-do attitude and lose interpretation of business ethics.  I’ve heard off the record stories from a couple of people telling of the havoc that disgruntled and motivated individuals have thrown down in some game environments, basically trashing the whole operation.  You don’t want to be in that situation.

The Competition

There has been some academic inquiry into the behaviors of these black/grey market commerce cabals and the results point to these groups exhibiting behaviors that would normally be seen in drug trafficking organizations and money launderers.  The statistics seem clearly point to a few key points (and their charts and graphs are pretty amazing):

• The common stereotype that most Gold Farmers are Chinese is correct. In fact in EverQuest 2 more than three quarters of all Gold Farmers are Chinese.
• It is possible to identify and construct a set of attributes of Gold Farmers which can be used to build machine learning for automatically detecting Gold Farmers.
• Social Network Analysis plays a crucial role in identifying Gold Farmers.
• Criminal network in the online world (Gold Farming Networks) behave in a manner similar to criminal networks (Drug Trafficking Networks) in the offline world.

Here is an example of a programmable, customizable, and flexible framework for gold farming and TOS violating advantage.  I grabbed it randomly as an example and removed the name as there are always more than a few out there at any given time. Let’s look at its features:

Multi-Boxing
One of $PROGRAM’s biggest strengths is it’s ability to control multiple programs on multiple computers. This means you an automate other game characters on other computers very easily. $PROGRAM can control the programs in a fully automatic or improvised way, depending on how you’ve configured your file. $PROGRAM controls the other computers by sending them commands and information directly through your Local Area Network.

Macro Recordings
The easiest way for a macro user to create a routine that is specific to them is for them to simply record it. You can prompt a user to record a certain routine and then you can use that routine in your macro. For instance, you can prompt a user to record the keys they should press when their game character should heal. Then you can use that recording in a healing bot.

Pixel Monitoring
The current line of monitors were made with the gamer in mind. Want a healer to heal your group when needed? Well then, simply monitor pixels on your group’s health bars, using pixel monitors, and when a certain pixel turns from green to black, configure the pixel monitor to start a routine that will heal the person.

Text File Monitors
Want to navigate your MMO character to a certain location? Well, you can either navigate by finding your current location through your log file (using a Text File Monitor) or you can smoothly monitor your in-game location by grabbing it from the your game’s memory (using a Memory Monitor)! Log file monitoring can provide some of the best means of automation as well- if your game uses live log files. By monitoring your game’s log file you can create damage-per-second statistics readouts, notifications of messages to you, notifications of when you need to recast a spell, etc.

Memory Monitoring
Using Memory Monitors is probably the most difficult part of $PROGRAM, yet potentially most powerful. $PROGRAM can monitor your game’s memory and look for certain information like health, in-game location, etc. This information can then be used when creating your routines. With that information you could potentially create navigating bots, harvesting bots, etc.

Purchase Information
Once you purchase the $PROGRAM program it’s yours to use forever. Your purchase also comes with a 1 year subscription to available MMO Bots, $PROGRAM updates, upgrades and forum access. More subscription time can be purchased at a later date.

Many high quality MMORPG bots are available here as well. You can find these in the forums. These are available to use while you have an active $PROGRAM subscription.

$PROGRAM does not require a constant connection to our servers while you use it. You only need to connect to $PROGRAM’s servers when you’d like to upgrade the program by downloading the newer version. Make sure to frequently check back with $PROGRAM to see what new features and new bots have been added!

So really what this offering is allowing for a very low amount of dollars is the ability to perform idealized and programmed tasks in a way that mimics typical user behavior with a low cost subscription model that allows farmers to farm in complex scenarios and maintain profitability.

If these games were p2p and treated every join as a trusted resource (see people playing CoD on modded xboxes with aimbots for example), there wouldn’t be a source for analytic data.

The Way Out

The only effective way [IMHO] to address threats in an environment where anti-behavioral countermeasures are directly ineffective is by identifying statistically suspicious behavior, investigating to determine what percentage is false positives, and after an acceptable tuning phase, impose penalties.  The online poker communities have been all over this in their ability to detect statistical outliers and automated behaviors on their players consoles and sometimes even pushing the envelope to the point where the online poker venues get accused of using spyware on their players. When a player is caught cheating in online poker, they simply take all of their money.  Real money.  Not perceived value or arguable dollars that a rare item, in-game gold, or other server-side bits of data with a street value, but cash-you-can-spend-anywhere money.  Mortgage payments.  Rent money.  Way more serious dollars than EULA-signed things defined as having no value.

This is just a simple example of what happens without the principals of something that security experts have trying have been trying to teach developers for a long time; that trust modeling and threat modeling are differently distinct and important design concepts.  In short and as a rule, client provided data cannot be trusted.  By extension, you can not trust the workstation network, memory, or obscurity without controls or at least replability at a minimum.  The developers in days of yore thought that there be no way that any player would analyze their game network traffic and be clued enough to analyze it and inject it into their own appstream.  In short, they trusted application layer when there wasn’t anything there to make it a difficult challenge to beat.  They also trusted that the player character could not drop something that the game environment should know they didn’t possess.  However, this was not the case, a player could drop anything that they had previously captured were able to find capture data for an inject back into the the stream for their benefit.

Because for some, hacking the game is more fun than playing the game.  Real world money is just a way to keep score.

All this is an example of things which should already be known in a hand-picked staff of awesomes that the ninja had working for him.  I felt pretty silly making suggestions knowing that the simple concepts I was throwing down was likely worked in first-hand intricate detail previously by the people with whom I was speaking.

Mindset Methodology

I try to look at of the issue and what once it is the for challenge, easiest, and by easiest I mean, what with the least amount of time and monetary expense to address or get into the issue.  In short, I want to solve the easiest problems first and then retool and approach the longer term more difficult challenges thereafter when there’s no discovery or planning stage available.  Often we don’t get ideal circumstances to make our contributions and letting perfect be the enemy of good can produce paralysis and catastrophe.  It’s all about taking your best shot and doing the most good you can with what you have.

A risk manager should always be pragmatic, but if there isn’t time to properly analyze risk previous to working on a small budget or smaller time schedule, I usually find it most effective just to address the easy things first and the harder things later.  The latter may require larger changes than many will expect to address the issues than may have been initially understood as an acceptable tradeoff.

The Times Have Changed

The days of irrevocable cash transfers are over after the Secret Service raided e-gold many years ago.  People in the world don’t like ransomware and 419 schemes, and they will reach out and touch you. Since everything in meta-cash transactions mostly follows credit card rules with the exception of some very limited use gambling money hiding platforms, which are fairly uncommonly used, credit-type transactions can be reversed for 30 days, what the credit industry calls a chargeback. If the objective is to stop the sale of any game currency or assets, so-called gold farming, or in limiting account hijacking, the most effective mechanism seemed to be to attack the profit stream directly and roll back account changes if data indicates that violations or unauthorized account changes have taken place.

So my suggestion if this was to be my gig was a metrics and data program analyzing transactions to find offenders and then roll back the transactions destroying the profit motive for the people lessening the value and game experience for both the game developer and userbase in general.  After that, look then to find what areas were actually being attacked and exploited in ways that damage the platform.  With that knowledge, focus remedial efforts in those areas.  Add into this scoring methodology to gauge the successfulness of the program and then reviewing those scorers to determine if a refocusing efforts would be appropriate.

One should always keep score to know were the best decisions were made.

Something like this I thought would be in keeping with some of the better thought patterns that I encountered in my professional life; recurring self-review, willingness to change, doing what works, the perfect not being allowed to be the enemy of good, and knowing ones opponent. I thought this would be a fairly compelling and successful method for addressing these challenges.

I didn’t get the project this time.  I would like to think it was because my approach was too contextually unconventional.

I was not able to put any of these thoughts into practice for this particular client, but it was a interesting theoretical exercise for me and left me with a lot to think about and to think about how portable some of these concepts can be.  I’m sure that there’s a few people out there with this job at various companies and are having a great time with the challenge of putting it into practice.

My favorite concept from this conversation is that I like the idea of the best coder wins.  “May the best code win!” like some kind of divine right settled by combat concept.  Sadly, often it’s much easier to be an attacker than a defender.  In the case of an MMO, the entire gaming environment is hosted server-side, so really there is no limit the amount of analytics and detection methods that can be employed without any awareness by the attackers.  I would think this would dovetail into a discussion of appropriate application instrumentation, but that’s pretty far outside my realm.

Parting mentions
(because I took so long to publish this draft)

• It appears that Blizzard is beginning to employ the tactics of targeting the transactions by using lawyers lately.  I guess someone else thought that was a good idea.
• I still don’t plan on reading any Doctorow.  He doesn’t write for me.
• If you would like to learn more about Paypal and how they deal with fraud challenges now that they realized that they needed to care about it, give Ohad Samet’s talk a look: