I've been looking at a few carts and most of the closest to being actual games are right on the Lua code limit (but they have plenty of graphical / map space.) It seems that the real major limitation is the code size and not the sprite limit.
The code size limit is causing hard to read code and ugly shortcuts. I absolutely understand the reason for the limit, but I think it should be raised considering the code isn't compiled or compressed and a lot of carts have vast sprite spaces but no more code space. In most applications the code text doesn't cause any appreciable size increase. It's always the graphics and sound assets.
That said, I think there might be some interesting ways to fit more code in the size of that space but at what point does it circumvent the spirit of the Pico-8? Cutting variable names and comments doesn't make a game easier to write or eliminate design fatigue. It just makes it more difficult for difficulty's sake.
I'm not sure if the text size limit has anything to do with the .png cart format though.
What about, instead of a text byte limit, it ignores whitespace and does a word count limit instead? At least that way you're not penalized for cleanly formatting your code and making reasonable variable names.
while i do enjoy the constraint, it may be a little too limiting.
esp if the completeness of a game is going to suffer.
maybe multiple code pages?
I'm not delighted with the idea of playing artificial games with the size limitations (do we count tokens like = or { as words, or fractions of words?) Do we put constant strings in a different segment with a different coefficient?
The whole point of pico-8 is that platforms, and particularly limited platforms, influence style. Some platforms (such as interactive fiction's z-machine) have keyboard inputs. Pico-8 does not. That influences what kinds of games are made on the z-machine versus the Pico-8.
The restricted-platforms-are-interesting idea suggests that if the games are currently bottlenecked by code size, we need to develop new styles, possibly ones that lean on the underutilized sprite and map memory - for example, can we write code that reads the sprite and map memory to control behavior and/or rules?
On the other hand, if the default distribution method is open source (for example, javascript-in-the-browser), then it is easier to grow a community. People can learn from examples and borrow techniques from one another.
Maybe a reasonable compromise would be to have a source segment to the pico-8 cartridge format, which is ignored when the cart is loaded - instead the bytecode segment controls the behavior of the cart. But when the cart is saved, the bytecode segment is generated (compiled) from the source segment. Then the size limits could apply to the bytecode rather than the source code.
Even if zep does nothing, we could start using external minimizers, like http://luasrcdiet.luaforge.net/
Yeah, I'm looking at the bottom-right numbers in the editor and kinda hating me for using long_but_expressive_function_names().
May I suggest not counting comments in the code size? This alleviates the problem of short but potentially confusing variables and function names, encourages heavy commenting (good for code people may read to learn), and allows the use of autodoc tools for those who are into them.
I like the idea of counting var names as 1char, not counting whitespace and not counting comments. The "local" keyword also seems to be a huge sink, but leaving it out means that your program becomes global variable hell.
What if something like "var"could be used as an alias for "local"? It would save 2 characters per instance, which is not a ton, but could add up over a large file.
I do think the idea of comments not being counted could make sense if you pretend the code has to be compiled, which could fit in with the fantasy console mythos :) I keep cringing when I take out comments to make the size smaller at the cost of readability XD. But I do appreciate the reasoning for the limitation, and the limit has to be set somewhere, so I'm a little torn about it.
If you consider collision detection and other must-haves for games, 15k is actually even less.
I don't think source code size has ever been a consideration for consoles. I think it's a consideration for the demo scene and for "see what you can create in 4k" (or whatever) challenges.
I think it should have a limit, but maybe scaled in such a way that the restriction is in line with the other restrictions. 128 sprites and 128x32 map is absolutely do-able.
With the restriction currently, you can render a Wolfenstein game, but no way it's going to have anything but the most basic gameplay.
I think you can easily mimic Atari 2600 and Intellivision (maybe) gameplay, but that's probably it. I'm seeing inspiring graphics demos but they're all crippled by the code limit.
> I think it should have a limit, but maybe scaled in such a way that the
> restriction is in line with the other restrictions. 128 sprites and
> 128x32 map is absolutely do-able.
Yeah, it certainly is the case for the current thing I'm working on that the source code limit is the bottleneck, not the graphics limit, and it sounds like you have been running into the same thing. But I don't know whether these examples are outliers or they are a sign that the source code limitation is disproportionate to the graphics limit. It seems like a hard thing to gauge objectively.
The thing I am working on is a video poker game, which is a very simple game, so it surprised me that I started running into the limit! Now it may have something to do with the fact that I'm in the habit of using function and variable names that are on the longer, more descriptive side; it just feels "wrong" to me to use one-letter variable names, but it feels almost like this limitation is encouraging me to do so.
That said, the challenge of refactoring stuff until it fits into the limit can be sort of fun! So again, I'm of two minds :)
| With the restriction currently, you can render a Wolfenstein game, but no way it's going to have anything but the most basic gameplay. |
Generally waiting for 0.11 to introduce auto-reload on run and/or command-line arguments - that would permit to make a small tool to automatically minify the code.
My general concerns are around:
- While cart sizes were limited, their layout wasn't - developer could split the available ROM memory pretty much as they please, permitting to sacrifice one section in favour of another.
- 15K is not too bad, but 15K of Lua is not comparable with 15K of compiled assembly - what would a 1 or 2 byte instruction on a CPU, is 3+ bytes in Lua (in some cases, like jumps, Lua looses a lot).
- There were different-sized ROMs - for example, while most NES ROMs were around 64...128K, 384K was also not uncommon, and certain ROMs would reach 1M. Different games need different amounts of data.
+1 for applying the code size limit to the bytecode.
I think that would still be a challenge (with a good size chosen), but not force us to do things like artifically making very hard to read code.
I'm not sure it's doable though - I don't exactly know how the cartridge is encoded in the png, but I'm sure there is a limit, which could be "hidden" in the picture, and I'm not sure if compiled "bytecode only" cartridges are a fit for the pico-8 philosophy...
Probably just classic PNG metadata... Adobe Fireworks stores literally the world in the PNG metadata (entire images, paths, layers), so I don't believe there is a limit to how much you can store in them.
Great thread!
I'm working on a scheme based on token counting which I think will play nicely with pico-8's constraints and as a side effect will raise the effective code capacity slightly. Until now, I haven't made much of a distinction between the 'developer console' (pico-8 binaries with the editor, .p8 files) and 'consumer console' (the html5 player, .p8.png files) -- but I think it will be useful here.
New scheme:
-
.p8 files have a ascii character code limit of around 64k, that most users will normally never reach unless they're doing something weird.
-
Code is also limited to 8192 tokens. Roughly -- each word ("function") or operator ("]") counts as a token. Newlines and comments are ignored.
- When a .p8.png cartridge is saved, the existing cartridge data layout is used. But if the ascii goes over 15k, an lz4-compressed version is saved instead (this is transparent to the user and doesn't remove any information). In the vast majority of cases, the 8k token limit will be reached before the 15k compressed data limit, but if it goes over 15k then the save is declined.
The largest existing cart I could find token-wise is Tower of Archeos, which is around 6418 tokens and maxed out ascii. So with automatic minification, I'd guess the current capacity is ~7k tokens. If compression works out ok, this means capacity with the new scheme would be 1/7th larger.
This feels about right to me -- on one hand I'm digging the current level of complexity of the most elaborate carts like Archeos, but also agree that there is a slight mismatch in code vs. data size. At present it's pretty hard to write enough code to meaningfully utilize all of the spritesheet, even with large sprites. This doesn't bother me as much as allowing carts to be too complex though -- complexity increases exponentially, and you can always dump remaining graphics and sound into a fancy title screen!
Counting tokens is a somewhat artificial thing to do. Pairing it with a hard limit on the compressed data makes it feel more plausible to me though -- especially as many old languages did in fact store code as tokens + the size caused by each string. And I want to avoid storing any kind of binary data in order to keep the property of cartridges that by default anyone can view and toy with the source code.
There are still cases where making variable names shorter might be used, but they are rare -- for example, to store extra data in the code area. Most of the time, you'd reach your 8k token limit and be done with it, or optimize code in more interesting ways.
If this plan goes ahead, it would be for 0.2.0 -- after the current round of features and subsequent bug fixes. Any suggestions or criticism welcome, of course.
On a related note -- I'm also looking at adding compress() and decompress() functions to the 0.2.0 api. For example, it would be possible to write a simple tool to compress a bunch of maps to one cartridge, and then decompress them into the map memory space as needed during runtime.
It's already possible to do this anyway by writing custom compression tools and decompression code in the main cart. But it's something that would eat away code size for carts that need it most, and probably falls into the annoying technical challenges basket.
So now code's limited by tokens and not characters it will depend on complexity and not actual length? Woo, my rogue cart only had like 109 characters left because I'm used to objectiveCLongAssDescriptiveNamesWithElaborate:Variables;
On previous ludumdare I reached the code limit in roughly 24h. I used the next 24h to make gfx and sfx, but also to optimize code size for new features and fx.
The interesting part was getting rid of the uncessary effects or feature to write new ones.
The boring part was renaming variables and deleting comments until my code was unreadable to myself.
With the 0.2.0 I will only have the good part so this solution seems perfect to me !
That sounds great Zep!
I was thinking to write a routine that converted and compressed my code into a custom binary format then on the actual program have a decompress and convert function.
I'm not sure how that would work memory-wise though. If I had, say 20k of source code compressed into 10k and it decompressed back into 20k at run-time.
I love the token switch! That sounds great. I haven't made anything real in pico-8, but even my quick tests, that code countdown timer was def pushing me towards shorter function names.
The one potential fantasy problem that it brings up is strings would count as a single token regardless of length, and could be used for storing arbitrary data. Since there is no way (currently) of extracting the keys from a non integer indexed table, I think that is the only point of exposure for exploiting the token limit for storing more data than allowed.
so, maybe characters in strings could count for tokens (or 4 string chars per token or something like this).
Small update: the code compression and token counting is going well. I've bumped this up to 0.1.1 (the next update) as I know there are at least a few half-finished projects hanging out for it.
It looks like the new ascii code limit will be 32k characters. But normally the token limit would be reached first (I'm still guessing a bit here). There is a 3rd limit, which is that the code can fit into the 15360 byte section of the cartridge when compressed, but it looks like the vast majority of users won't ever need to deal with or know about this. This limit is only enforced when saving to .p8.png (.p8 can be thought of as dev files).
There isn't any way to get data into the Lua machine during runtime -- it only executes whatever is in the code section of the cartridge on startup and that's it. But I hope the token scheme and larger code side will be sufficient.
I went with a 32k ascii limit to address this to some degree -- but anyway I think it's cool that there is a way to stuff a bit more data into the cartridge for emergencies! There will be a substr() to get the values back out. In this case, you'd also need to manually watch that the compressed code size doesn't go over 15360 bytes as you add more data.
Really looking forward to this change, thanks for the work on it! I'm already half full with very little content and I've been spending a huge amount of dev time on refactoring...
Hm, interesting reasons behind tokens. Though I'm burning through them like there's no tomorrow. Do you think it couldn't be upped to 16k? This would allow for more complicated stuff like rpgs. As for p8.png thing, I think it could be dealt with by physically making cart image bigger (as in bigger pixel area), say 2x as big as it is now.
//edit: Or just resign of png idea altogether - while I admire idea and tech behind it, I think same could be done by making possible to just load text version of the cart, possibly utilizing tool similar to pastebin like few online dcpu-16 emulators do. Granted, users would have easier time to just hack up text version of cart, but without map/gfx/music editors it would be extremely hard and online player won't have the tools.
Old systems got around memory space limitations with in-cartridges mappers. I like all the "runtime" restrictions, but in terms of "ROM" space, I think the cap should be something like 512K, i.e. effectively infinite given the platform's needs.
What do you think about an option to use "shared data" (typically used for sprite/map data) to extend the code space?
Also, it may be worth upping the filesize to 6k or 8k; and just doubling the size of the "shared data" along with that option... from what I understand, a lot of us are hitting token walls, code/token walls, or sprite/map walls (at least one or the other) before we're getting projects finished right now. We can have "fun constraints" still, but if the belt is too tight to push major content, that's kind of a problem. We don't need a lot more, but we need a little more to work with; and I think that does include being able to up code space; and this will keep the project nice and tight while still being able to make "readable" and adequately commented code while still being able to finalize projects. And more flexible space (as opposed to more static space) will also increase the variation of the projects as well.
Stick with PNG carts though, that's such a fantastic thing. And 6k is still a (relatively) microscopic filesize. In retrospect, Super Mario Land (for B/W GameBoy) is SIXTY FOUR FUCKING KAY. More than ten of said Picos. Well, currently more than fifteen of them.
EDIT: While we're talking about restructuring shared data... it might be a good idea to make "map data" extended into the shared data begin from the bottom of the sprite data up; so that the sprites and the map can remain in sequence-friendly formats without overlapping. We'll usually finish spriting and mapmaking before needing to extend code (I think!!), so the code extension can then slot into committed row-blocks of data between them (like 4 screens/sprites each)... but that in the sprite/map editor, those can be copied and pasted like the sprites themselves to manipulate them more easily... copy/paste the first block, and the whole code chunk pastes where you put it. You can even label the code blocks like: [C][O][D][E]
Extending code = extending tokens, too. This way, you only add as many tokens as it takes to approximately finish the extended code. Let's say, every 4 blocks you commit to code extension adds another 512 or 1048 tokens?
another point I'd like to make is old school developers didn't have source code size constraints, it's a bit silly to restrict source code size to the degree that people have to make it illegible even to themselves...
FREE PICO
It also occurred to me: you can make a temporary "PATCH" command from the command menu for compatibility if you do this duplication of the shared data - just have it copy any map data and paste it in the extra buffer in backwards order of blocks! It'll keep the current games functional with the new structure; and even though you'll still see (sprites in corrupted map space, or map tiles in corrupted sprite space); you can simply overwrite them or ignore them entirely.
BOOM. All old games would be compatibile with the new format using that.
[Please log in to post a comment]
