Promises and Why You Should Use Them

The following is an introduction to and justification for using my Promise implementation! If you just want to check out the library, click that link!

Threads

When writing programs, it’s possible to divide functions into two groups: “synchronous” and “asynchronous”. A “synchronous operation” is one that can run to completion and generate any necessary return values with only the information available to your code at the time the operation begins. For example, a function that takes two Parts and returns the distance between them would be synchronous, because all information needed to compute that value is available when you call the function.

But sometimes situations arise where we call a function that needs access to a value that doesn’t exist at call time. This could be because it requires a network request to get the data, or the user needs to input some text, or we’re waiting for another process to finish computation and give us the value. In any case, we refer to this as an “asynchronous operation”.

The simplest way to deal with this is to just stop execution of the thread, or “block”. This means that when you call a function that needs some data that doesn’t exist yet, the entire thread stops running and waits for the data to be ready before returning and continuing. This is actually how many low-level languages typically model asynchronous operations. To allow tasks to run at the same time, programs will create new threads that branch from parent threads and jump back on when they’re finished blocking. However, this presents challenges with sharing memory and synchronizing data across threads, because at the operating system level threads truly are running in parallel.

Coroutines

To simplify sharing memory and potentially reduce overhead, many programs will emulate a multi-threaded environment using green threads or coroutines, which are run concurrently inside of one OS thread. The key difference between OS threads and coroutines is that coroutines do not actually run in parallel – only one coroutine is ever executing at a time. In the context of Lua, the term “thread” is used to refer to a coroutine, but they are not the same thing as OS threads.

To facilitate this emulation, a thread scheduler is introduced to keep track of the emulated threads and decide which thread to run next when the current thread yields. Yielding is similar to blocking, except when a coroutine yields, it signals to the thread scheduler that it can run other code and resume the thread at a later time.

When the game starts, each Script and LocalScript in your game becomes its own Lua thread in the thread scheduler and each script is run either to completion or until it yields. Once all of the scripts have gone through this process, Roblox does other things like updating humanoids and running physics. After all that’s done, the next frame begins and this process repeats until the game closes.

So, what really happens when we call an asynchronous function like Player:IsInGroup? Well, the current Lua thread yields (letting other Lua code start running elsewhere in your game), and Roblox makes a new OS thread which blocks on an HTTP request to their internal group APIs in the background. Sometime in the future when that request comes back, the value jumps back onto the main Roblox thread and your Lua thread is scheduled to be resumed with the given arguments on the next step.

Problems with the Coroutine Model

Coroutines fix the memory sharing problem of OS threads, but they still inherit other problems when used on their own:

  • It’s impossible to know if a function that you call is going to yield or not unless you look at the documentation or strictly abide by a naming convention (which is not realistic). Unintentionally yielding the thread is the source of a large class of bugs and race conditions that Roblox developers run into.
  • When an asynchronous operation fails or an error is encountered, Lua functions usually either raise an error or return a success value followed by the actual value. Both of these methods lead to repeating the same tired patterns many times over for checking if the operation was successful, and make composing multiple asynchronous operations difficult.
  • It is difficult to deal with running multiple asynchronous operations concurrently and then retrieve all of their values at the end without extraneous machinery.
  • Coroutines lack easy access to introspection without manual work to enable it at the call site.
  • Coroutines lack the ability to cancel an operation if the value is no longer needed without extraneous manual work at both the call site and the function implementation.

Enter Promises

In Lua, Promises are an abstraction over coroutines. A “Promise” is just an object which we can use to represent a value that exists in the future, but doesn’t right now. Promises are first-class citizens in other languages like JavaScript, which doesn’t have coroutines and facilitates all asynchronous code through callbacks alone.

When calling an asynchronous function, instead of yielding, the function returns a Promise synchronously. The Promise object allows you to then attach a callback function which will be run later when the Promise resolves. The function you called is in charge of resolving the Promise with your value when it is done working.

Promises also have built-in error handling. In addition to resolving, a Promise can reject, which means that something went wrong when getting the future value we asked for. You can attach a different callback to be run when the Promise rejects so you can handle any error cases.

Let’s take a look at this in action. We will make a function which wraps HttpService:GetAsync and instead of yielding, it will return a Promise.

local HttpService = game:GetService("HttpService")
local function httpGet(url)
	return Promise.async(function(resolve, reject)
		local ok, result = pcall(HttpService.GetAsync, HttpService, url)

		if ok then
			resolve(result)
		else
			reject(result)
		end
	end)
end

Let’s break this down. The Promise.async function accepts a function, called an executor, which receives a resolve function and a reject function. Promise.async calls the executor on the next Lua step. Inside it, we have created a safe space to safely call yielding functions, which has no possibility of unintentionally delaying other parts of your code. Since the Promise value itself was already returned from the httpGet function, we aren’t delaying the return by yielding with GetAsync.

Let’s use the value now:

local promise = httpGet("https://google.com")

promise:andThen(function(body)
	print("Here's the Google homepage:", body)
end)

promise:catch(function(err)
	warn("We failed to get the Google homepage!", err)
end)

So, we call the andThen method on the Promise returned from httpGet. If the Promise resolved, the handler we passed into andThen is called and given the resolved values as parameters (body in this example).

Likewise, we attach a failure handler with catch to be run if the Promise rejects.

But wait! In addition to attaching a callback, andThen and catch also return new Promises themselves! If the original Promise rejects, then the Promise returned from andThen will also reject with the same error, allowing is to rewrite our code like this:

httpGet("https://google.com")
	:andThen(function(body)
		print("Here's the Google homepage:", body)
	end)
	:catch(function(err)
		warn("We failed to get the Google homepage!", err)
	end)

The Promise returned from andThen will resolve with whatever value you return from the callback.

And if that value returned from the andThen handler is itself a Promise, it is automatically chained onto and the Promise returned from andThen won’t resolve until that Promise resolves.

httpGet("https://google.com")
	:andThen(function(body) -- not doing anything with body for this example
		return httpGet("https://eryn.io") -- returning a new Promise here!
	end)
	:andThen(function(body) -- Doesn't get called until the above Promise resolves!
		print("Here's the eryn.io homepage:", body)
	end)
	:catch(warn) -- Still catches errors from both Promises!

Composing Promises

Promises are composable. This means that Promises can easily be used, interact with, and consume one another without manually threading values between them. We already saw above how returning a Promise from the andThen handler will chain onto it. Let’s expand that idea by diving into some more ways you can compose Promises with each other:

Let’s assume that we have a number of asynchronous functions which all return Promises, async1, async2, async3, async3, etc. Calling one of these functions will return a Promise. But what if we want to call all of them in sequence, each one after the one before it finishes? It’s as simple as this:

async1()
	:andThen(async2)
	:andThen(async3)
	:andThen(async4)
	:andThen(async5)
	:catch(function(err)
		warn("Oh no! This went wrong somewhere along the line:", err)
	end)

In this sample, we first call async1, then we chain the rest of the functions together with andThen. If any of the Promises returned from these functions reject, then all remaining andThen'd functions are skipped and it will jump instantly to the catch handler.

And as a side note, if you forget to add a catch to a long chain of Promises and one of them errors, the Promise library is smart enough to emit a warning in the console. Always catch your Promises!

Let’s think of another situation. What if we want to run all of the functions concurrently, and wait for all of them to be done? We don’t want to run them one after another, because sometimes that can be wasteful. We want them all to run at once! We can do this with the static method Promise.all:

Promise.all({
	async1(),
	async2(),
	async3(),
	async4()
}):andThen(function(arrayOfResolvedValues)
	print("Done running all 4 functions!")
end):catch(function(err)
	warn("Uh oh, one of the Promises rejected! Abort mission!")
end)

Promise.all accepts an array of Promise objects, and returns a new Promise. The new Promise will resolve with an array of resolved values in the same places as the Promises were in the array. The new Promise will reject if any of the Promises that were passed in rejects.

Promise.race is similar to Promise.all, except it will resolve or reject as soon as one of the Promises resolves or rejects.

We can call functions that return Promises from inside a Promise and safely yield for their result by using the await method of Promises. This is akin to the await keyword in languages like JavaScript. Sometimes it might be easier to just directly resolve with a Promise though, in which case that Promise is chained onto and the outer Promise won’t resolve until the inner one does.

local function async1()
	return Promise.async(function(resolve, reject)
		local ok, value = async2():await()
		if not ok then
			return reject(value)
		end
		
		resolve(value + 1)
	end)
end

Wait, nevermind.

Sometimes, we no longer need a value that we previously asked for (or we just want to stop a sequence of events). This could be for a variety of reasons: perhaps the user closed a menu that was loading, or a player’s ability gets interrupted, or a player skips a cutscene.

When situations like these come up, we can cancel a Promise. Cancelling a Promise in its simplest form prevents the andThen or catch handlers from running. But we can also optionally attach a hook inside of the Promise executor so we know when the Promise has been cancelled, and stop doing work.

There is a third parameter sent to Promise executors, in addition to resolve and reject, called onCancel. onCancel allows you to register a callback which will be called whenever the Promise is cancelled. For example:

local function tween(obj, tweenInfo, props)
	return Promise.new(function(resolve, reject, onCancel)
		local tween = TweenService:Create(obj, tweenInfo, props)
			
		-- Register a callback to be called if the Promise is cancelled.
		onCancel(function()
			tween:Cancel()
		end) 
			
		tween.Completed:Connect(resolve)
		tween:Play()
	end)
end

-- Begin tweening immediately
local promise = tween(workspace.Part, TweenInfo.new(2), { Transparency = 0.5 }):andThen(function()
	print("This is never printed.")
end):catch(function()
	print("This is never printed.")
end):finally(function()
	print("But this *is* printed!")
end)
wait(1)
promise:cancel() -- Cancel the Promise, which cancels the tween.

(Why we are using Promise.new: If your Promise executor needs to yield, then you should use Promise.async. If it doesn’t, such as in this case, you should use Promise.new. They are the same, except that Promise.async begins executing on the next Lua step instead of immediately, and it allows you to yield inside the executor. Promise.new doesn’t allow yielding inside the executor.)

If we didn’t register an onCancel callback, the Promise returned from the tween would never resolve or reject (so the andThen and catch handlers would never get called), but the tween would still finish.

For times when we need to do something no matter the fate of the Promise, whether it gets resolved, rejected, or cancelled, we can use finally. finally is like andThen and catch, except it always runs whenever the Promise is done running.

Propagation

Cancelling a Promise will propagate upwards and cancel the entire chain of Promises. So to revisit our sequence example:

local promise = async1()
	:andThen(async2)
	:andThen(async3)
	:andThen(async4)
	:andThen(async5)
	:catch(function(err)
		warn("Oh no! This went wrong somewhere along the line:", err)
	end)

promise:cancel()

Cancelling promise (which is the Promise that catch returns here) will end up cancelling every Promise in the chain, all the way up to the Promise returned by async1. The reason this happens is because if we cancel the bottom-most Promise, we are no longer doing anything with the value, which means that no one is doing anything with the value from the Promise above it either, and so on all the way to the top. However, Promises will not be cancelled if they have more than one andThen handler attached to them, unless all of those are also cancelled.

Cancellation also propagates downwards. If a Promise is cancelled, and other Promises are dependent on that Promise, there’s no way they could resolve or reject anymore, so they are cancelled as well.

So, now we understand the four possible states a Promise can be in: Started (running), Resolved, Rejected, and Cancelled. It’s possible to read what state a Promise is in by calling promise:getStatus().

But I want to be able to use pre-existing functions that yield!

You can easily turn a yielding function into a Promise-returning one by calling Promise.promisify on it:

-- Assuming myFunctionAsync is a function that yields.
local myFunction = Promise.promisify(myFunctionAsync)

myFunction("some", "arguments"):andThen(print):catch(warn)

Problems, revisited

Now, let’s revisit the problems we laid about before and see if we’ve solved them by using Promises:

  • It’s impossible to know if a function that you call is going to yield or not.
    • Calling a function that returns a Promise will never yield! To use the value, we must call andThen or await, so we are sure that the caller knows that this is an asynchronous operation.
  • When an asynchronous operation fails or an error is encountered, Lua functions usually either raise an error or return a success value followed by the actual value. Both of these methods lead to repeating the same patterns.
    • We have Promise:catch to allow catching errors that will cascade down a Promise chain and jump to the nearst catch handler.
  • It is difficult to deal with running multiple asynchronous operations concurrently and then retrieve all of their values at the end without extraneous machinery.
    • We have Promise.all, Promise.race, or other utilities to make this a breeze.
  • Coroutines lack easy access to introspection without manual work to enable it at the call site.
    • We can just call :getStatus on the returned Promise!
  • Coroutines lack the ability to cancel an operation if the value is no longer needed without extraneous manual work at both the call site and the function implementation.
    • promise:cancel() is all we need!

Another point that’s important to drive home is that you can do all of these things without Promises, but they require duplicated work each time you do them, which makes them incompatible with each other and that allows for slight differences between implementations which can lead to usage mistakes. Centralizing and abstracting all of this logic by using Promises ensures that all of your asynchronous APIs will be consistent and composable with one another.

Examples

More examples can be found on this page. Specifically, I recommend checking out the “Cancellable animation sequence” example to see a common problem that’s solved quite easily with Promises!

The library

Now that you know all about our good friend Promises, you should check out the API reference to see all of the cool bits and bobs that weren’t mentioned here!

This library was originally written by LPGhatguy, but I have since taken over as the maintainer of the library and made several additions to the API, including adding cancellation.

This isn’t the only Promise implementation for Lua on Roblox, but it’s probably the most fully-featured one. The main alternative I’m aware of is NevermoreEngine’s implementation, but it doesn’t support cancellation.

If you have any questions about using Promises, feel free to ask them down below!

– evaera

98 Likes

3 posts were merged into an existing topic: Off-topic and bump posts

I want to second how useful promises are.

Hi! I want to touch on my implementation, NevermoreEngine’s implementation and how I handle cancellation as a mechanic.

So my promises are complaint from a Promises/A+ standard. This means there are only three states in which they can be in: Pending, Fulfilled, and Rejected. There is no cancellation method.

Unfortunately, the tweening thing is a bit convoluted, but my method sort of looks like this:

local function promisePlayTween(tween)
	local promise = Promise.new()

	-- Couple promise state to the tween
	local conn = tween.Completed:Connect(function(playbackState)
		if playbackState == Enum.PlaybackState.Completed then
			promise:Resolve()
		elseif playbackState == Enum.PlaybackState.Cancelled then
			promise:Reject()
		end
	end)

	promise:Finally(function()
		conn:Disconnect()
		tween:Cancel()
	end)

	tween:Play()

	return promise
end


local tweenAppear = TweenService:Create(workspace.Part, TweenInfo.new(2), { Transparency = 0.5 })
local tweenDisappear = TweenService:Create(workspace.Part, TweenInfo.new(2), { Transparency = 0 })

local promise = promisePlayTween(tweenAppear)
promise:Then(function()
	-- Doesn't play because we cancel, but it would!
	return promisePlayTween(tweenDisappear)
end)

wait(1)
promise:Reject() -- Reject the Promise, which cancels the tween!

However, I really don’t like coupling the state of my promise to the tween. In this case, I’d do something more like this:

promisePlayTween(tweenAppear):Then(function()
	return promisePlayTween(tweenDisappear)
end)

wait(1)
-- Cancel the appear tween directly from the API, which then results in the promise cancelling.
tweenAppear:Cancel()

Generally, I want my side effects to be explicit. Oftentimes, I try to keep promises immutable after being returned, i.e. there is no external reject/resolve. However, sometime it is too useful to have an API to reject and resolve.

In the future, I plan to make promises immutable, and then create a Deferred class to handle this sort of cancellation code.

Overall, I highly recommend promises.

5 Likes

I didn’t even know I needed this until you did it. But wow, I definitely needed this. Thanks!

3 Likes

I’ve been recommended Promises in the past and I overlooked it primarily because I hadn’t a clue of what they were or how to use them. Some time of reading material and it didn’t click.

This article is a treasure. I’ll be holding onto this and seeing if I can incorporate Promises into my code sometime - of course, given necessity.

Much thanked for the explanation on Promises.

6 Likes

Maybe it’s just me, but I feel like your promise implementation is a bit confusing. I feel like cancelling a single promise would be simpler than cancelling multiple asynchronous calls.

1 Like

If I understood Promise correctly, we should use it for asynchronous functions that returns values we want use in the future. Values we don’t know at the time of calling. Some questions:

  1. I don’t understand the benefits of wrapping the tween as shown in the example. I can just replace :andThen with Tween.Completed event and call Tween:Cancel() directly. Also received an error when using the example ServerScriptService.Script2:17: attempt to index a function value for the line local promise = tween( ...

  2. Couldn’t I have achieved the cancellable animation sequence example without Promise with much less code? I can make it composable with a series of Tween.Completed. If I cancel it, I can change the properties manually for the end state.

  3. It says in the documentation to never use spawn and replace it with promise.spawn instead. But I don’t understand the benefits. To avoid throttling and consistent timing? How would that benefit my uses:

.

for _, AI in pairs(workspace.Zombies:GetChildren()) do
    spawn(function()
    	while wait(rng:NextInteger(10, 20)) and AI.Parent do
            AI.Humanoid.MoveTo(randomPos)
    	end
    end)
end

function Cooldown()
	spawn(function()
		Tool.Enabled = false
		wait(0.6)
		Tool.Enabled = true
	end)
end

function LoadPlayer()
     spawn(function()
          wait(60)
          ShowGui()
     end)
     Do Other Stuff
end
  1. It’s impossible to know if a function that you call is going to yield or not.
    If I call a function, I can just check if there’a a wait() or async method in it beforehand? I know you wrote “unless you look at the documentation or strictly abide by a naming convention” but I still don’t get it. How is this a problem?

  2. When an asynchronous operation fails or an error is encountered, Lua functions usually either raise an error or return a success value followed by the actual value. Both of these methods lead to repeating the same patterns.
    By pattern do you mean local success, message = pcall(function() do something end) if success then do something else end. Wouldn’t replacing that with Promise also be using a pattern?

  3. It is difficult to deal with running multiple asynchronous operations concurrently and then retrieve all of their values at the end without extraneous machinery.
    This one I understand. Can’t see how I would achieve that without Promise easily.

  4. What would be the benefit of calling the composable async functions in a Promise chain, as opposed to just calling them after one another like normally, function() async1() async2() async3() end

  5. Coroutines lack the ability to cancel an operation if the value is no longer needed without extraneous manual work at both the call site and the function implementation.
    Same as my first two points. What kind of operations would I want to cancel that I can’t already?

I guess all my points and questions boils down to me still not understanding the use case.

4 Likes

A post was merged into an existing topic: Off-topic and bump posts

  1. There are multiple benefits.
    • Another point that’s important to drive home is that you can do all of these things without Promises, but they require duplicated work each time you do them, which makes them incompatible with each other and that allows for slight differences between implementations which can lead to usage mistakes. Centralizing and abstracting all of this logic by using Promises ensures that all of your asynchronous APIs will be consistent and composable with one another.

    • Composing multiple tweens at once would require more work than chaining Promises together.
    • The API would not be interoperable with other asynchronous code in your game, which would require writing superfluous connective code. Remember, Promises do not claim that these things are impossible without them (otherwise how would Promises be implemented in the first place?). The purpose of a Promise is to provide a consistent, compatible way to model all asynchronous operations in your game. This enables you to write generic code that can deal with future code that you might not know about yet. Promises are robust, so they can model any kind of asynchronous operation.
  2. Yes, but again you lose the consistency of the API if you do this. Tweens have an explicit cancel method, but not every asynchronous operation has something like this. The Promise object wraps everything up into a neat box so you always know what you’re working with.
  3. spawn is evil. It’s still on the 30hz pipeline (which makes it slower in general), and if there are too many threads being resumed at once your spawn can be deferred for several seconds. When you have lots of code stacked on top of each other, this can cause cascading delays and inconsistent results. Using spawn in your game will seem like it works fine at first but then once your game gets more complex with lots of different processes happening at once, they’ll start to interfere with each other.
  4. No, this is unrealistic. Not only is it a massive time waste to sift through source code to find the definition of a function (especially if it wasn’t written by you), function implementations are seldom that simple. If you’re building proper abstractions, that means the function you use might be calling several functions inside of it. Then you have to check all of those functions to see if they yield. And those functions might be calling even more functions! This is not a solution if you care about your time. Further, even testing it might yield misleading results, because functions don’t always fit into the categories of “yields” or “doesn’t yield”. Some functions only yield sometimes (see: WaitForChild) and it’s near impossible to account for that if you don’t know what triggers the possible yield.
    • Why calling a yielding function unknowingly is a problem: This can lead to race conditions and code that you think will execute all at once but it instead ends up taking several seconds. Imagine you’re writing a function that kills a player, and before you set their health to zero you want to run some animation. If the function you call to run the animation yields, and you don’t realize it, this can cause their health to remain untouched until the animation is done playing. Now imagine you want to kick off several things before killing a player; an animation, a log, a network event, anything. If any of these yield, you’re going to have that delay. These mistakes are very common and you might not even notice them in a vacuum, but they can lead to cascading problems because your code isn’t running in the order that you think it is. Promises ensure that when you call a function, it’s not going to delay the code that follows that call.
  5. Errors that occur in a Promise chain will collect at the next catch. This means that if an error occurs, you don’t have to manually thread it through to code to handle. This means you can compose Promise-returning functions and only catch them at the call site (letting the user handle it) instead of forcing the user to do it. And this might just be an opinion thing, but I feel like using :catch is much nicer and more accepting to refactors than creating a new closure and calling pcall on it every time. Regardless, wrapping in a pcall means that you’re yielding again, which is the whole problem we’re trying to avoid by using Promises in the first place. You can’t use pcall to call an asynchronous function and get the result without yielding the thread that’s calling pcall.
  6. Yes.
  7. As hinted at above, the benefit is twofold. First, using a Promise chain and catching at the end lets you handle errors without yielding. The second benefit is that by creating a Promise chain, you create yet another Promise that you can pass around to other code and let it consume. It’s like building with Legos.
  8. Any asynchronous operation might want to be cancelled. Cancelling a Promise only prevents its resolve and reject callbacks from running. Adding additional abort semantics where possible is great for operations that you can do so for. One prime benefit of cancellation via Promises is that if you cancel a Promise that’s part of a chain, it will also cancel all of the other pending Promises in that chain. So you can quite easily stop the entire process, whereas cancelling without using Promises requires you to handle all of that by hand, which is cumbersome.
3 Likes

And to everyone who thinks it makes them look smart by spewing vulgarities at me and this post or saying “Why would I want Lua to be like JavaScript”: You’re totally missing the point. The purpose of this library is not to produce some kind of nostalgia for JavaScript by using one of its features in Lua. Promises/Futures/Tasks/etc are present in all of the following languages: C#, Python, Hack, Dart, Kotlin, JavaScript, Scala, Rust, C++, and probably more that I just don’t know about.

The Promise pattern exists in these languages because it solves real problems (that I outlined in the OP). Promises are not only a different way to handle asynchronous code, they provably prevent mistakes and centralize the model so disparate parts of our code can all be on the same page.

It’s not funny, and being so dismissive about ideas that are new to you actively harms yourself and deters others from being open to learn. This idea that Lua should “remain pure” and resist outside influence from other languages is absolutely misguided.

We should all work to build each other up and learn from each other. This kind of attitude is toxic to the community and toxic to yourselves.

18 Likes

Thanks for the thorough answers. Quick question, I see that promise.spawn was removed from the latest version. Am I supposed to use promise.async now to replace my spawns?

Also, am I supposed to use it like this or add a resolve() in the end or something? Thinking it could cause memory leaks.

function Cooldown()
	Promise.async(function()
		Tool.Enabled = false
		wait(0.6)
		Tool.Enabled = true
	end)
end

Check out the Guide. The way you’re using Promise.async isn’t correct. You need to resolve or reject your Promises! I removed Promise.spawn because it was exposing an implementation detail that I decided I didn’t want to encourage people to use outside of in conjunction with Promises. Promise.async always just used Promise.spawn implicitly anyways, so the behavior is still the same.

1 Like

How would I build a recursive Promise that runs itself when it rejects until it successfully resolves?

Promises are eager which means that the instant you create a Promise it’s already running. Thus there’s no way to re-run a Promise per se. To retry the same operation, you’d need to have a new Promise for each attempt. That usually boils down to calling the function multiple times.

Something like this:

local function retry(callback, times, ...)
  local args, length = {...}, select("#", ...)

  return callback(...):catch(function(...)
    if times > 0 then
      return retry(callback, times - 1, unpack(args, 1, length)) 
    else
      return Promise.reject(...)
    end
  end)
end

local function test(value)
	return Promise.new(function(resolve, reject)
		(math.random() >= 0.95 and resolve or reject)(value)
	end)
end


print(retry(test, 10, "hello world"):awaitStatus())
1 Like