Are the UDim2 in pixels?

If I set a the following piece of code:

Frame.Size = UDim2.new(0, 500, 0, 500)

will it be the same as a 500px by 500px frame or are they different?

Yes.
Reference: GuiObject | Documentation - Roblox Creator Hub