Here is a simple function to convert a number to its hexadecimal string representation:

function num2hex(number)
	local base = 16
	local result = {}
	local resultstr = ""

	local digits = "0123456789abcdef"
	local quotient = flr(number / base)
	local remainder = number % base

	add(result, sub(digits, remainder + 1, remainder + 1))

  while (quotient > 0) do
  	local old = quotient
  	quotient /= base
  	quotient = flr(quotient)
  	remainder = old % base

		 add(result, sub(digits, remainder + 1, remainder + 1))
  end

  for i = #result, 1, -1 do
  	resultstr = resultstr..result[i]
  end

  return resultstr
end

Usage:

	print(num2hex(255)) -- ff
	print(num2hex(10)) -- a
	print(num2hex(1050)) -- 41a

The built-in tostr() function can do hex if you pass a second arg of 'true':

> i=255
> print(tostr(i))
255
> print(tostr(i,true))
0x00ff.0000

If you want something that isn't the full 32-bit value, you can strip off the '0x' and the leading/trailing '0's pretty easily by checking individual characters. This is the function I use:

-- note that ord('0') is 48
function hex(v) 
  local s,l,r=tostr(v,true),3,11
  while(ord(s,l)==48) l+=1
  while(ord(s,r)==48) r-=1
  return sub(s,min(l,6),r>7 and r or 6)
end

> print(hex(123))
7a
> print(hex(15.5))
f.8
> print(hex(3/7))
0.6db6

And here's a slightly-tweaked version that doesn't insist on at least one integer digit:

-- note that ord('0') is 48
function hex(v) 
  local s,l,r=tostr(v,true),3,11
  while(ord(s,l)==48) l+=1
  while(ord(s,r)==48) r-=1
  return sub(s,l<r and l or 6,r>7 and r or 6)
end

> print(hex(3/7))
.6db6

Thanks Felice.

Really nice. I didnt know that (my lua knowledge is pretty weak)...

Guess I will remove some tokens...

It's less about Lua and more about knowing PICO-8's tiny API. It's based on the Lua API, but only some very basic parts of it. It's simplified in some ways, and a few things are a little different.

I find it's helpful to re-read the manual now and then. When I started, I would skim/skip parts of the manual. I either didn't use the functions at the time, or I didn't really understand why they were useful. After you've written for PICO-8 for a while, you start seeing what you need, and what's useful. When you re-read the manual, you find yourself saying, "AH! So that's what that is for!"

People like to say "RTFM!" I think they need to say "RTFMA!" too. :)