HashLib - Cryptographic hashes in pure Lua

boatbomber · December 25, 2019, 11:26pm

Introduction

Module was originally written by Egor Skriptunoff and distributed under an MIT license.
It can be found here: pure_lua_SHA/sha2.lua at master · Egor-Skriptunoff/pure_lua_SHA · GitHub

That version was around 3000 lines long, and supported Lua versions 5.1, 5.2, 5.3, 5.4, and LuaJIT.
Although that is super cool, Roblox only uses Lua 5.1, so that was extreme overkill.

I worked (with @howmanysmaII’s guidance) to port it to Roblox in a way that doesn’t overcomplicate it with support of unreachable cases. then, howmanysmall came in to do some final optimizations that really squeeze out the most performance possible.

After quite a bit of work and benchmarking, this is what we were left with.

Enjoy!

Features

This module contains functions to calculate SHA digest:

MD5, SHA-1,
SHA-224, SHA-256, SHA-512/224, SHA-512/256, SHA-384, SHA-512,
SHA3-224, SHA3-256, SHA3-384, SHA3-512, SHAKE128, SHAKE256,
HMAC

Additionally, it has a few extra utility functions:

hex_to_bin
base64_to_bin
bin_to_base64

Usage

Input data should be a string
Result (SHA digest) is returned in hexadecimal representation as a string of lowercase hex digits.

Simplest usage example:

local HashLib = require(script.HashLib)
local your_hash = HashLib.sha256("your string")

API

HashLib.md5
HashLib.sha1

SHA2 hash functions:
HashLib.sha224
HashLib.sha256
HashLib.sha512_224
HashLib.sha512_256
HashLib.sha384
HashLib.sha512

SHA3 hash functions:
HashLib.sha3_224
HashLib.sha3_256
HashLib.sha3_384
HashLib.sha3_512
HashLib.shake128
HashLib.shake256

Misc utilities:
HashLib.hmac (Applicable to any hash function from this module except SHAKE*)
HashLib.hex_to_bin
HashLib.base64_to_bin
HashLib.bin_to_base64

Benchmarking and Comparison

For the benchmarking test, I could only do 350 hashes per test, because with higher amounts the unoptimized version caused a script timeout. So, that’s a sign that I did a really good job.

Benchmark Script

--[=[
	
	Messy code to run some tests
	
	Note that CPU usage goes through the roof while doing this
	
--]=]


local HashLib = require(script.HashLib)
local HashLib_Unoptimized = require(script.HashLib_Unoptimized)

local ipairs = ipairs


local TestAmount = 350
-------------------------------------------------------------------------------------------------

local function TestCompare(FunctionName, Arg)
	
	wait(0.1)
	
	print("\n### Testing",FunctionName)
	
	
	-- test my version
	
	local Times = {}
	for i=1, TestAmount do
		
		local t = tick()
		
		local hash = HashLib[FunctionName](Arg)
		
		local TimeTaken = tick()-t
		
		Times[#Times+1] = TimeTaken
		
	end
	
	local Total = 0
	for _,Time in ipairs(Times) do
		Total = Total+Time
	end
	
	print("* Optimized HashLib took `".. string.format("%.10f", Total).. "` to do ".. TestAmount .." "..FunctionName.." hashes with each one taking `".. string.format("%.10f", Total/#Times) .. "` on average")

	-- test original

	local UnoptTimes = {}
	for i=1, TestAmount do
		
		local t = tick()
		
		local hash = HashLib_Unoptimized[FunctionName]("This is my input")
		
		UnoptTimes[#UnoptTimes+1] = tick()-t
		
	end
	
	local UnoptTotal = 0
	for _,UnoptTime in ipairs(UnoptTimes) do
		UnoptTotal = UnoptTotal+UnoptTime
	end
	
	print("* Unoptimized HashLib took `".. string.format("%.10f", UnoptTotal).. "` to do ".. TestAmount .." "..FunctionName.." hashes with each one taking `".. string.format("%.10f", UnoptTotal/#UnoptTimes) .. "` on average")

end


wait(2) -- Let game state finish initializing before we eat CPU

TestCompare("md5", "hash this string please")
TestCompare("sha1", "hash this string please")
TestCompare("sha224", "hash this string please")
TestCompare("sha256", "hash this string please")
TestCompare("sha512_224", "hash this string please")
TestCompare("sha512_256", "hash this string please")
TestCompare("sha384", "hash this string please")
TestCompare("sha512", "hash this string please")
TestCompare("sha3_224", "hash this string please")
TestCompare("sha3_256", "hash this string please")
TestCompare("sha3_384", "hash this string please")
TestCompare("sha3_512", "hash this string please")

Benchmark Results

Testing md5

Optimized HashLib took 0.0990054607 to do 350 md5 hashes with each one taking 0.0002828727 on average
Unoptimized HashLib took 0.4256963730 to do 350 md5 hashes with each one taking 0.0012162754 on average

Testing sha1

Optimized HashLib took 0.1464135647 to do 350 sha1 hashes with each one taking 0.0004183245 on average
Unoptimized HashLib took 1.1473040581 to do 350 sha1 hashes with each one taking 0.0032780116 on average

Testing sha224

Optimized HashLib took 0.2236335278 to do 350 sha224 hashes with each one taking 0.0006389529 on average
Unoptimized HashLib took 2.3714523315 to do 350 sha224 hashes with each one taking 0.0067755781 on average

Testing sha256

Optimized HashLib took 0.2243692875 to do 350 sha256 hashes with each one taking 0.0006410551 on average
Unoptimized HashLib took 2.3787267208 to do 350 sha256 hashes with each one taking 0.0067963621 on average

Testing sha512_224

Optimized HashLib took 0.7043654919 to do 350 sha512_224 hashes with each one taking 0.0020124728 on average
Unoptimized HashLib took 6.3255631924 to do 350 sha512_224 hashes with each one taking 0.0180730377 on average

Testing sha512_256

Optimized HashLib took 0.7125756741 to do 350 sha512_256 hashes with each one taking 0.0020359305 on average
Unoptimized HashLib took 6.3216791153 to do 350 sha512_256 hashes with each one taking 0.0180619403 on average

Testing sha384

Optimized HashLib took 0.7102406025 to do 350 sha384 hashes with each one taking 0.0020292589 on average
Unoptimized HashLib took 6.3122458458 to do 350 sha384 hashes with each one taking 0.0180349881 on average

Testing sha512

Optimized HashLib took 0.7212450504 to do 350 sha512 hashes with each one taking 0.0020607001 on average
Unoptimized HashLib took 6.3489837646 to do 350 sha512 hashes with each one taking 0.0181399536 on average

Testing sha3_224

Optimized HashLib took 0.8299937248 to do 350 sha3_224 hashes with each one taking 0.0023714106 on average
Unoptimized HashLib took 13.0032684803 to do 350 sha3_224 hashes with each one taking 0.0371521957 on average

Testing sha3_256

Optimized HashLib took 0.8421859741 to do 350 sha3_256 hashes with each one taking 0.0024062456 on average
Unoptimized HashLib took 12.9854648113 to do 350 sha3_256 hashes with each one taking 0.0371013280 on average

Testing sha3_384

Optimized HashLib took 0.8211791515 to do 350 sha3_384 hashes with each one taking 0.0023462261 on average
Unoptimized HashLib took 12.9778954983 to do 350 sha3_384 hashes with each one taking 0.0370797014 on average

Testing sha3_512

Optimized HashLib took 0.8204553127 to do 350 sha3_512 hashes with each one taking 0.0023441580 on average
Unoptimized HashLib took 12.8966257572 to do 350 sha3_512 hashes with each one taking 0.0368475022 on average

The optimized version is many times faster in almost all cases.

Module:

Edit: Disclaimer! I didn’t alter the actual functionality of the hashing functions. If one of them produces non-standardized results, that’s on Egor! However, I did look into it, and it does all seem to perform exactly as expected.

Enjoying my work? I love to create and share with the community, for free.

If you’d like to help fund my work, consider sponsoring me on GitHub or donating on BuyMeACoffee!

boatbomber · December 25, 2019, 11:30pm

Just a quick addendum:

I made this for fun after attending a wonderful cryptography lecture earlier today. I don’t know how useful it’ll be for most Roblox developers, but it was something I wanted to do and decided I might as well share it!

grilme99 · December 25, 2019, 11:32pm

This is awesome! This is going in my bookmarks for later, I’ll have a use for it very soon with my current project no doubt! Keep these awesome modules coming

boatbomber · December 25, 2019, 11:34pm

If you don’t mind sharing, what’s your use case? If it’s not something you want to post publicly, I’d really appreciate a DM.

~~I don’t have a usage myself, so I’m pretty curious.~~
Edit: I realize I have a very important use for this. Read my later reply.

grilme99 · December 25, 2019, 11:39pm

I’m not too deep into planning yet, but off the top of my head, I will probably need either a hashing algorithm or an encryption algorithm for a part of my current project. I’m not going to go into too much detail yet, though. It’s to do with communicating to and from the game and an external server.

Conejin_Alt · December 26, 2019, 3:19am

Hello! Maybe this is off topic and I’m sorry about that but what is HashLib and how I can use it, I mean what I can use and achieve with it…

boatbomber · December 26, 2019, 3:44am

Its a library of cryptographic hashing algorithms written as pure Lua functions, specifically tuned to run well in the latest Roblox Lua VM.

This isn’t a thread to explain fundamental cryptography, so you should do some research and feel free to DM me if you need additional explanations!

boatbomber · December 27, 2019, 3:09am

Made a rather simple module for handling a login system. Made in an hour or two, and lightly tested. Please do try to find flaws or holes in it!
I doubt anyone has a use for it, but I do!

My rather uncommon use case

I plan on having a team of moderators accept/decline tutorial submissions in Lua Learning. However, if their account is stolen, that bad actor would then be able to allow inappropriate content into my game, and it would promptly be [Content Deleted].
Therefore, having another layer prevents this. That bad actor would need to steal the mod’s Roblox account, and the mod’s Lua Learning login! I included a setting in the module that only allows the creator of the login account to be permitted to login (others will be told username or password is incorrect even if it’s correct), which I’d use to ensure that the bad actor can’t login without first managing to steal the mod’s Roblox account.

This was a fun project in an area I’ve studied but never had a chance to implement in anything! Even if I don’t end up using this exact system, I still had a fun time making this.

Module features

Settings

bool LOGIN_MUST_BE_CREATOR
Sets if only the creator of the account can login (This is what I mentioned earlier)
bool PASSWORD_REQUIREMENTS
Sets if passwords must be 5+ chars, contain uppercase and lowercase, etc
string HASH_ALGORITHM
Chooses which hash function to use from HashLib
number SALT_SIZE
Sets how many characters the hash salt is (Salt possibilities are 93^SALT_SIZE so size being 10 gives over 48 quintillion possible salts)
number RATE_LIMIT_MULTIPLIER
Sets the rate at which the rate limit multiplies after each failed attempt (helps prevent brute force)

API

function LoginSystem:Register(Player,Username,Password)

Returns two values:

bool Success
string Message

Message will say things like “Username is taken” or “Password must contain uppercase letter(s)” or that sort of thing.

function LoginSystem:Login(Player,Username,Password)

Returns two values:

bool Success
string Message

Message will say things like “User not found” or “Username or password is incorrect” or that sort of thing.

Module

https://www.roblox.com/library/4547866920/LoginSystem

MagikTheWizard · December 27, 2019, 4:28pm

I mean I would of used this if I didn’t have my custom b64 encoding and I currently have no use for hashing though I can see myself using this in other projects!

Thanks ~

pobammer · December 28, 2019, 2:50am

I actually got it quite a bit faster now.

github.com

howmanysmall/Rewrites/blob/master/src/HashLib.lua

--[=[------------------------------------------------------------------------------------------------------------------------
-- HashLib by Egor Skriptunoff, boatbomber, and howmanysmall
-- Boatbomber did the majority of the work, I just helped him with initial optimizations and did the final speed boost.

Documentation here: https://devforum.roblox.com/t/open-source-hashlib/416732/1

--------------------------------------------------------------------------------------------------------------------------

Module was originally written by Egor Skriptunoff and distributed under an MIT license.
It can be found here: https://github.com/Egor-Skriptunoff/pure_lua_SHA/blob/master/sha2.lua

That version was around 3000 lines long, and supported Lua versions 5.1, 5.2, 5.3, and 5.4.
Although that is super cool, Roblox only uses Lua 5.1, so that was extreme overkill.

I, boatbomber, worked (with howmanysmall's guidance) to port it to Roblox in a way that
doesn't overcomplicate it with support of unreachable cases.

After quite a bit of work and benchmarking, this is what I was left with.

Enjoy!

This file has been truncated. show original

boatbomber · December 28, 2019, 3:16am

Thank you so much for your help and contribution!

~~When I have a chance, I’ll update OP and the module.~~ Done.

You’re a freaking speed demon. Your version of HashLib is scary fast.

sloss2003 · December 29, 2019, 12:40am

This is so cool! Great job, and thank you for posting this! Probably will end up using this down the line.

builderboy256 · December 29, 2019, 1:12am

b64 and hashing are two completely different things… I don’t see what you mean wally

NeoInversion · December 29, 2019, 1:13am

Have to agree with builderboy here, bird boy, Base64 is encoding, something like MD5 is hashing.

MagikTheWizard · December 29, 2019, 10:29am

I have no idea why you assumed that I said “base64 encoding is hashing”, I know what hashing is…
Also I’m not wally… @NeoInversion

and I currently have no use for hashing

RussellSchutle · December 30, 2019, 12:25am

Can this module be used to encrypt then decrypt string messages or just hashing?

buildthomas · December 30, 2019, 12:41am

First post only lists hashing algorithms, not any encryption ones.

boatbomber · December 30, 2019, 5:01am

It’s HashLib, not EncryptLib.

Cryptographic hashing, by definition, has to be a one way function. No decryption possible. If you could decrypt a hash, then it’s a cracked algorithm and is totally insecure.

boatbomber · February 16, 2020, 9:25pm

Update:

@howmanysmaII went ahead and restructured the module to be easier to read for those of you who are curious, as well as getting it even faster than it already was. She’s truly incredible. Achieving outstanding speed while somehow improving readability is not common.

The latest version is published to the Roblox model.

https://www.roblox.com/library/4544052033/HashLib

Ewanophobia · March 22, 2020, 4:36pm

Pretty epic module, I’m excited to see what will happen with this however a question have arisen:

Would it be reasonable to hash leaderstats (using this module) and send them to an external Database?