I've been thinking about how PICO-8 could provide very primitive 3D software rasterization capabilities that remain within the spirit of the system. These are just goofy ideas, please feel free to bury them if they're too far outside the project scope, but I had to at least write it down once.

In any case, it should be possible to implement nearly all of the features below in a cartridge.

Following things need to be implemented in order for this to work:

• Mesh ROM
• Mesh editor
• API functions
• Z-buffer

Mesh ROM specs:

• 128 meshes (or less? animated meshes are considered here)
• per mesh: 16 vertices, 16 quads (96 bytes)
• index of 8x8 sprite to use as texture / colormap
• Vertex format: 3x(0..7) position
• Quad format: 4x(0..15) vertex index, 2x2x(0..7) uv rect (x1,y1,x2,y2)
  0 = no quad

The editor would get a new page with an editing view (64x64) that supports two modes: mesh editing and uv editing.

mesh editing:

• The view supports free (orthogonal) rotation to preview the result and select obstructed vertices/faces.
• An extra label displays the index of the selected vertex or face.
• A toggle switches the view mode: textured / wireframe.
• left-double-click creates a new vertex.
• left-click selects a vertex or a face (depending on proximity of cursor to edge of face).
• right-click on a vertex begins to build a face; 4 vertices of which at least 3 are unique need to be right-clicked in succession to complete the face; right-clicks that don't hit vertices are ignored. a left-click will abort. If the 4 vertices already comprise a face, the operation has no effect.
• left-drag moves the vertex or face (all vertices of the face) along the orthogonal plane that's best facing the camera (dot product of view normal and plane normal closest to -1). That way no explicit orthogonal view is necessary. If a vertex is moved onto the position of another vertex, the vertex will be reset to its original position. That way, overlapping vertices are avoided.
• right-drag orbits the camera around the center, and implicitly selects the plane along which vertices and faces will be moved when editing.
• DELKEY removes the selected vertex or face. face removal does not remove vertices.
• the mesh can be copied/pasted to other slots in order to facilitate the creation of animations.

uv editing (allowed when a face is selected):

• The view shows the default 8x8 texture for this mesh, with a rectangle indicating the region that the active face will be mapped to. If only three vertices of the face are unique, the face will be a triangle (x1,y1 - x1,y2 - x2,y1).
• Per default, the face is mapped to the entire texture.
• x1,y1 are rounded to the upper left edge of the texel, x2,y2 are rounded to the lower right edge. That way, a single texel can color the entire quad.
• An extra panel allows selecting the default texture this mesh will use.
• left-drag draws a new rectangular bounding box for the quad. A simple left-click selects a single pixel as the quads texture.

The API gets 6 new functions:

clsz()

• clears the z-buffer to 0 (see Z-buffer notes further below)

zget(x,y)
zset(x,y,z)

• get or set the z-value of a z-buffer pixel.

camera3d(<x>, <y>, <z>, <m00>, <m01>, <m02>, <m10>, <m11>, <m12>, <m20>, <m21>, <m22>, [<orthogonal>])

• configures the camera (view) matrix for subsequent mesh drawing calls.
• x y z: origin of camera
• m00-m22: orientation of camera as a 3x3 matrix; (1,0,0,0,1,0,0,0,1) would describe an identity matrix.
• orthogonal: if true, an orthogonal projection will be applied instead of the default perspective projection, which is analog to an infinite far plane projection matrix.

model(<x>, <y>, <z>, <m00>, <m01>, <m02>, <m10>, <m11>, <m12>, <m20>, <m21>, <m22>)

• configures the model matrix for subsequent mesh drawing calls
• x y z: origin of model
• m00-m22: orientation of model as a 3x3 matrix; (1,0,0,0,1,0,0,0,1) would describe an identity matrix.

mesh(<mesh index>, [<texture>], [<flags>], [<face mask>], [<mesh 2>, <blend>])

• rasterizes the triangles of a mesh with model, camera, projection transformations applied.
• mesh index: the index of the mesh to rasterize (0..128).
• texture: index of the sprite to use as the meshes texture (0..128). If omitted or nil, the default texture will be used. If texture is -1, the mesh will be rendered as wireframe.
• flags: Sets various render flags: bit 0 = read & test depth; bit 1 = write color; bit 2 = write depth; bit 3 = cull backfaces. The default flags are 0xF.
• face mask: a set of 16 bitflags, indicating which faces should be rasterized/skipped. If omitted, all faces will be rasterized (face mask = 0xFFFF)
• mesh 2, blend: if specified, blends the vertices between this mesh and another by factor <blend> (0 = 100% mesh A, 0.5 = 50% mesh A, 50% mesh B, 1 = 100% mesh B). This way,
  smooth keyframed animations can be rendered.

Z-buffer:

To facilitate order-independent rasterization (depth culling), a 128x128 Z-buffer must be available. The minimum required precision is unclear, but I assume half-float or 16bit precision is needed at least (= 32k). The Z-buffer ranges from 0 to 1, where 0 is infinitely far away and 1 is the near plane. This configuration is appropriate when transforming triangles with an infinite far plane projection matrix. In this entire setup, the near plane is fixed to a system default, e.g. 0.1 or 1.

Here is exemplary GLSL code that describes how to do the projection transform:

// transform a vector v by an infinite projection matrix so
// that z is projected to the range 1..0 after the vertex shader.
// aspect is usually vec2(1, w/h) or vec2(h/w, 1)
vec4 projection_transform(vec2 aspect, float near, vec4 v) {
return vec4(v.xyaspect, v.wnear, v.z);
}

// do the inverse of projection_transform(), with baked-in divide by w
vec4 inverse_projection_transform(vec2 aspect, float near, vec4 v) {
float w = near / v.z;
return vec4(v.xyw/aspect, v.ww, 1);
}

wow, exciting. a+

I'm not really sold on 3d support in pico-8, maybe in pico-16, or pico-32 :)

Pico-8 is roughly a NES level "console", 3d support only started to appear around like two generations later, in PS1 for example.

Ofcourse, since this is a fantasy console, there is no real limitation if zep wants to include 3d support. :)
But even than, I would say the support should stay on the level of something like mode-7 was.

Harrumph, kids these days. There was plenty of 3D in the 8 bit era, just not on the earliest machines or with anything in the way of textures.

While I like the idea of Pico-8 remaining small and limited to reduce the decision overhead of its programmers, I'd also love to bring its accessibility to programming simple 3D games too.

Anyway, watch Elite in 3D on 'NES-level hardware': http://www.youtube.com/watch?v=zoBIOi00sEI

Support could be as simple as adding a trifill() command that renders a filled triangle between the defined points. A lot of 3D can be done from that.

I think this is available for PICO-8 users already, it's called "Voxatron". Unfortunately I haven't looked into Voxatron very much because when I tried it before it wasn't well-documented, so it seemed rather daunting at the time. I might have a look at it again, though, since I'm making progress as a programmer and maybe I can figure it out. :1

Or you can look at demos/cast.p8, that's pretty interesting. I bet you could add objects drawn as flat sprites and make some Wolfenstein 3D or Doom kind of thing.

I would also prefer to just have a builtin "trifill" function that'd work cleanly (e.g. no seams if drawing triangles with touching vertices), since it's hard to make a good and fast rasterizer. The rest is a matter of math.

A very well written post which you have obviously put some thought into. :)

However, I am totally against putting this into Pico-8. I think there is a reason why zep wanted a separate Voxatron project which expands into 3D space which is one of the selling points for Voxatron. Another poster mentioned that Voxatron is more complex and daunting and lacks documentation - and that's even an off-putting point. With Pico-8, almost everything is self-explanatory and you only need to look through a small manual to utilize it fully.

The more major features you add like this, the more you "weaken" the original concept of a simple "2D Fantasy Console" and you add another decision point to your process. With the extra dimension and meshes, you also drastically increase the art/asset space as well by a lot of pixels. For me as a non-artist - the current limitations helps me tremendously. There are just a certain number of pixels that actually fit - and that's that. You don't need to break more sweat on it.

If you want 3D in Pico-8 at this point - you know about the limitations and make a very deliberate decision to make something by pushing its limits and do some extra work - that's also a fun challenge in itself for many people. I really like to see all the clever stuff people do in order to achieve som results.

I have also seen some mentions off making money etc. off Pico-8 carts as well. Personally, I don't really think that should be a goal - and TBH I doubt that there is a market for it... I would rather use Pico-8 to nail down core concepts, gameplay and enjoy it as a tool to help me focus. If I someday find something that works really well - I can just port everything to Unity, Luxe, OpenFL, Gamemaker or whatever and continue expanding from there.

Just my $0.02