Hi, people. I like this project, but I wonder if there is some complete list of the carts for easy download like TIC-80 has: https://tic80.com/api?fn=dir&path=play/Games

When I want to download all the carts for my Batocera installation, I have to scrape whole website what is resource consuming. Would it be possible to make some complete list to simplify download? Thanks!

Nothing?

Are you sure you want ALL the carts?

It is easier to choose when I see all, at least all the games. Why not?

I think there's a zipped collection of them somewhere called PICO Awesome or something along those lines, but I don't know the link. It's definitely not a complete list of every cart in existence (it leaves out demos, music carts, WIP, etc), just several thousand of them, so you'll have to manually find the rest probably.

I know, but I prefer up to date collection on my system, to see what is new. I have set up TIC-80, WASM-4, working on LowresNX and Pico-8 is the next in order, so far the most complicated because of the number of games.

There are a lot of partial lists people make... should be enough to keep you busy. But none of them are comprehensive.

How are you scraping the data now?
Can you grab all of the carts ever 'featured'? That way you only grab finished, quality games and there should be a lot.

Not scraping Pico-8 yet, want to finish LowresNX and then try Pico-8. I still hope there will be some simple list that people can easily use.

I kind of did it, I think

I made a script that :

• gets the last post's id
• go through all IDs from last post's one to 1, generates an URL and checks if it contains a Pico-8 cartridge (using a div called 'breadcrumbs'). Excludes anything that is not "pico-8" and "cartridges" so no Voxatron/Picotron neither Snippet, Work in Progress...
• if it's a Pico 8 Cartridge, the script checks if it's already in the carts file, if yes, compares ID to keep only the higher one (most recent update of the game)
• carts file has ID, Title and Author name

I passed the carts file through a duplicates check and didn't found any

Both scrape and check dup scripts have parallel processes option, and with 8 jobs, it took roughly 5 hours to build the carts file but now that the big thing is done, only have to update from time to time ...

By doing other checks on the carts files, found (and removed) ~150 errors. Some posts are flagged as pico-8 carts without having any (that's how I came here aha), others got deleted

Also, i only checked for "pid" not "tid" (in the URL) so I may be missing some carts ?

I'm pretty sure there's other flaws but well ... I have a list of 9200+ carts and I can launch a random one whenever I want on my phone so I'm happy with it :D

I really don't know if I'm allowed to share such a list and also, don't know how to do it. Also don't know if I should share the md "pretty" file or the csv "ugly" one (but really easy to parse!)

Hope that helps, have a nice day!

That sounds good. It would be great to upload that list to some github so other people can use it.

There you go :

https://gist.github.com/bunigost/f04635191bbcce0a99d3988c2ab8e67d

It's the CSV file used for doing things (launching carts, checking for duplicates...). Special characters and spaces are removed

I have a MD file with the same data, but spaces and special characters are kept, the format is almost a table, more human-readable

Since I'm a beginner, it might have errors, duplicates or flaws ...

edit :
could be nice to use it as a base to add another field with maybe cart name, description...

9000 URLs to scrape is way more manageable than 165000+

I made a lowresnx cart list using github actions here https://github.com/WizzardSK/gameflix/blob/main/lowresnx.txt so using the same approach I may create pico-8 list. If you would like to share your code, I may use is and run it periodically too.

Made a git repo to share carts files and the script :

https://github.com/bunigost/P8-cartgen#

That script looks quite impressive, however, it does not save the most important thing, the cart file name. I am sure it is not hard to add it, anyway, I am working on more simple script, maybe will use also a part of yours. Thank you very much for the inspiration!

I made a better and faster script. It writes all the cart names and file names in a few minutes here: https://github.com/WizzardSK/gameflix/blob/main/pico8.sh
It runs every day and updates this list: https://github.com/WizzardSK/gameflix/blob/main/pico8.txt