Implementing atomic transactions using data stores

Help and Feedback Scripting Support
#1

I need a safe procedure for modifying two datastore keys. The problem is, this can’t be done in a single API call, so it can fail half-way, causing data loss or duplication.

It is possible to do this safely using mutex locks, but this means many calls to UpdateAsync, which might be a burden on the request budget. I want to find the fewest calls necessary to be able to securely complete a transaction.

My first concept requires 6 UpdateAsync calls. The intent is for the transaction to be able to be resumed at any point in-between these calls, so imagine the server shuts down and a player has to rejoin in-between each request.

  • The first 2 calls establish a lock on each key, which lives in the DataStoreKeyInfo metadata and contains some metadata of its own to store details about the transaction and its state.
    • If an outage occurs here, then when a player joins and their data is locked, it resumes the transaction by requesting to lock the other key.
  • Once both keys are locked, the data needs to be validated to ensure the transaction is legal. If it passes, 2 more UpdateAsync calls update the data and change the metadata to indicate the transaction is partially complete. If it doesn’t pass then the data isn’t changed and the metadata indicates that the transaction failed.
    • If the transaction is resumed here where only 1 key is partially complete/failed and the other is still locked then it updates the other.
  • A key can be unlocked when it is partially complete/failed if the other is also partially complete/failed or is no longer locked. The lock has to uniquely identify transactions so that it doesn’t get confused if the other key is locked again but as part of a different transaction.

in other terms:

  1. lock player 1’s data
  2. lock player 2’s data
  3. update player 1’s data once both are locked, mark as partially complete/failed
  4. update player 2’s data once both are locked, mark as partially complete/failed
  5. unlock player 1’s data once player 2’s data is marked/unlocked
  6. unlock player 2’s data once player 1’s data is marked/unlocked

Is there another solution which uses less of the request budget? Ideally some day there will be a built-in feature to modify multiple datastore keys in unison as a single API call, but until then, some version of this is needed.

#2

You could start with 2 calls, and if either one fails, revert and rollback the one which went through.

#3

Setting an arbitrary limit is probably your best solution.

#4

The rollback can also fail, an outage can occur before the second call which prevents any further changes. Retry until it works also isn’t a solution because the server can end and there’s nothing externally tracking the partial success to tell it to continue retrying in the next server.

#5

I have a new idea which only requires 4 UpdateAsync calls, the last of which can be saved for later if the budget is low. Player1 is initiating a transaction with player2.

The first call updates and locks player1’s data, storing relevant details about the transaction.
The second call validates the transaction against player2’s data, updating it if valid, and leaving a success/failure note.
The third call unlocks player1’s data, reverting it if player2’s data has a failure note.
The fourth call, which can be saved for later, cleans up the success/failure note from player2’s data after player1’s data is unlocked.

Player1’s data has to be locked in case a rollback is necessary. Player2’s data doesn’t need to be locked and only needs to store a tag indicating the result of the transaction, and player1’s data is unlocked after getting the result.

2 Likes