Simulated Game Reasoner

Simulated Games is one of Grail’s AI techniques, which is based on the Monte Carlo Tree Search algorithm. You can use manually operating directly on the SimulatedGame object or through reasoner - SimulatedGameReasoner - that stores SimulatedGame internally.

The former approach gives your more flexibility in intepreting the results of simulations. The reasoner approach assumes that the game is created for a certain AIEntity and is generaly integrated with the rest of the Grail architecture.

First, take a look at the main public methods of SimulatedGameReasoner that aren’t inherited from base Reasoner and not related to debugging process:

//constructor
SimulatedGameReasoner(std::unique_ptr<ISimulatedGameStateTranslator> worldStateTranslator,
            std::unique_ptr<ISimulatedActionTranslator> actionTranslator,
            std::size_t iterationsPerFrame, std::size_t maxIterations = 1000,
            std::size_t maxRecalculationIterations = 1000);

void RequestRecalculation(AIEntity& entity);

//fallback behavior - used when Simulated Game produces null or illegal behavior
void SetFallbackBehavior(std::unique_ptr<Behavior> _fallbackBehavior);

//constructor
public SimulatedGameReasoner(ISimulatedGameStateTranslator gameStateTranslator,
               ISimulatedActionTranslator actionTranslator,
               uint iterationsPerFrame,
               uint maxIterations = 1000,
               uint maxRecalculationIterations = 1000)

public void RequestRecalculation(AIEntity entity);

//fallback behavior - used when Simulated Game produces null or illegal behavior
public Behavior FallbackBehavior { private get; set; }

The constructor provides configuration of the reasoner. We will now go over its parameters.

Game state translator

The goal of this interface is create a SimulatedGame object - which is the definition of your simulated game. For more information, please consult Simulated Games article and Simulated Games - defining a game in 10 steps

To create the game object use the CreateGame function. This function also returns a unit defined in the simulated game that is paired with the entity the reasoner will be attached to. Such a unit will be used to translate its simulated game actions onto entity’s behavior. This is one of the differences between using the reasoner and SimulatedGame directly. If you use the game object directly, you may translate units' actions onto any kind of behaviors of any entities.

This interface consists of two functions:

class ISimulatedGameStateTranslator
{
  public:
    virtual ~ISimulatedGameStateTranslator() = default;

    /// <summary>
    /// Return the SimulatedGame object an the unit paired with it.
    /// The paired unit is a virtual representation of the entity Reasoner is attached to.
    /// </summary>
    virtual std::pair<std::unique_ptr<SimulatedGame>, std::shared_ptr<class ISimulatedGameUnit>> CreateGame() = 0;

    /// <summary>
    /// This method is called when SimulatedGameReasoner requests for a new start state.
    /// When you are done with preparing the proper start state (all properties of units etc.) call the @refreshingFinishedCallback.
    /// </summary>
    virtual void StartRefreshingGameState(const class AIEntity& entity, std::function<void()> refreshingFinishedCallback) = 0;
};

public interface ISimulatedGameStateTranslator
{
  /// <summary>
  /// Return the SimulatedGame object an the unit paired with it.
  /// The paired unit is a virtual representation of the entity Reasoner is attached to.
  /// </summary>
  (SimulatedGame, ISimulatedGameUnit) CreateGame();

  /// <summary>
  /// This method is called when SimulatedGameReasoner requests for a new start state.
  /// When you are done with preparing the proper start state (all properties of units etc.) call the @refreshingFinishedCallback.
  /// </summary>
  void StartRefreshingGameState(in AIEntity entity, System.Action refreshingFinishedCallback);
}

The first one has already been discussed. In the second function - StartRefreshingGameState - provide a callback that will update state of units defined in the simulated game (starting game) based on the actual state of the (real) game.

Action translator

Used to translate actions planned by Simulated Game for the unit onto one or more behaviors of the AIEntity. Just take the first (or more) element from the actions collection and add them to the plan variable. The added behaviors will be queued for execution.

The AddBehaviors method will be called after all the simulations required to take the action are completed.

  /// <summary>
    /// Translates actions defined in SimulatedGame onto Behaviors used in Grail.
    /// </summary>
    class ISimulatedActionTranslator
    {
    public:
      ISimulatedActionTranslator() = default;
      ISimulatedActionTranslator(const ISimulatedActionTranslator& other) = default;
      virtual ~ISimulatedActionTranslator() = default;
      ISimulatedActionTranslator& operator = (const ISimulatedActionTranslator& other) = default;

      /// <summary>
      /// Translate actions from SimulatedGames (@actions) into any number of corresponing behaviors and add push them to the @plan object.
      /// </summary>
      /// <param name="entity">The entity that performs the behaviors.</param>
      /// <param name="SimulatedGameUnit">The unit from SimulatedGame that performed the actions.</param>
      /// <param name="actions">The actions to be translated to behaviors.</param>
      /// <param name="plan">A plan object which is a container for behaviors this method has to generate.</param>
      virtual void AddBehaviors(AIEntity& entity, const ISimulatedGameUnit& SimulatedGameUnit, std::vector<const ISimulatedGameAction*> actions, Plan& plan) = 0;

 public interface ISimulatedActionTranslator
  {
    /// <summary>
    /// Translate actions from SimulatedGames (@actions) into any number of corresponing behaviors and add push them to the @plan object.
    /// </summary>
    /// <param name="entity">The entity that performs the behaviors.</param>
    /// <param name="SimulatedGameUnit">The unit from SimulatedGame that performed the actions.</param>
    /// <param name="actions">The actions to be translated to behaviors.</param>
    /// <param name="plan">A plan object which is a container for behaviors this method has to generate.</param>
    void AddBehaviors(in AIEntity entity, in ISimulatedGameUnit SimulatedGameUnit, in IEnumerable<ISimulatedGameAction> actions, ref Plan plan);
  }

In simpler cases, when the actual game is similar to simulated game, it is possible to use the same object for actions and behaviors. Just make it derive both from Behavior and ISimulatedGameAction.

iterationsPerFrame

The maximum number of iterations that can be performed in a single render frame. The higher the value, the more likely the game will stutter (due to low FPS).

maxIterations

The total number of iterations planned for the action-selection. Once this number is reached, the simulated game actions are determined. Those actions are then translated onto behaviors. The behaviors are chosen. Once all the planned behaviors are executed, the process starts again.

RequestRecalculation

As state above, the ISimulatedActionTranslator translates the actions computed by SimulatedGame used inside SimulatedGameReasoner to behaviors (represented by the class Behavior). It constructs a plan (of such behaviors). The plan is accordingly executed, i.e. the first behavior is started and when it terminates, the next one starts etc. until the plan is empty.

However, the Simulated Game may be just approximation of the actual game. The application of behaviors may result in a different effect than intended. If you can check it, then you have the option to intervene. Call the RequestRecalculation() method at anytime if you want to stop applying behaviors (it will terminate the current one and clear all future ones). The method will make the reasoner perform additional simulations if computational budget allows for it.

You can include the expected effect in your Behavior class to monitor whether it has been achieved.

maxRecalculationIterations

The maximum number of iterations available to be executed after RequestRecalculation() was called. The consecutive calls to RequestRecalculation() will make less and less iterations:

Example with maxRecalculationIterations = 6000

3000
1500
750
375
187
94
...

This budget is reset as soon as SimulatedGameReasoner performs regular iterations. The regular iterations are performed when the plan is empty, i.e. either for the first time or all behaviors from the previous plan have been already applied.

Debugging SimulatedGameReasoner

In addition to regular debugging in your IDE, you can use the GUI-based tool dedicated to debugging Simulated Games: Simulated Game Debugger

Now we show the main methods related to the debugging process. However, the best place to learn about the debugging in Grail is: Debugging Setup: Tutorial. In a typical scenario shown in the tutorial, you will attach a debugging object GrailDegugger to a reasoner and calling the functions below will be automatized for you.

void SetSnapshotProduction(bool shouldProduce);
SimulatedGameReasonerSnapshot ProduceDebugSnapshot();

void SetDebugSnapshotFirstIteration(size_t iterationNumber);
size_t GetDebugSnapshotFirstIteration() const;

void SetDebugSnapshotLastIteration(size_t iterationNumber);
size_t GetDebugSnapshotLastIteration() const;

public void SetSnapshotProduction(bool shouldProduceData)
public SimulatedGamesReasonerSnapshot ProduceDebugSnapshot()

public uint DebugSnapshotFirstIteration;
public uint DebugSnapshotLastIteration

SetSnapshotProduction() tells the reasoner to gather debug data. This will cause significant performance overhead. The ProduceDebugSnapshot() returns the snapshot containing history of iterations (with details) if such a snapshot is already available.

With the code above, you also can limit gathering the debug data to only certain iterations between [DebugSnapshotFirstIteration, DebugSnapshotLastIteration].

If you use the global GrailDebugger, then it will set snapshot production and will be producing snapshots automatically for you.

The only thing that requires manual intervention is to set the [DebugSnapshotFirstIteration, DebugSnapshotLastIteration] interval. By default - if nothing is done - all iterations will be included.

API Reference

SimulatedGameReasoner API