User blog:Hseiken/Saving massive amounts of data in petitcom

So I'm working on a game where you can edit levels and such and the issue was always 'how do I save this stuff?'

In Petitcom, MEM$ is a string of 256 characters or less and you save by SAVE "MEM:MYFILE". It's good for high scores or for, say, progress in an RPG where you create a string of values such as what you're carrying and what not. For instance if there's 20 items in the game and I want to save location of the player and what bosses he's beaten and how much experience and gold he has, I could do this by converting the flags for what items have been collected into strings (either 1 or 0...where 1 is you have the item and 0 is you don't have the item). So that's 20 characters. So far so good. Then you have the location of the player. There may be, say, 10 different places you can save your game at. So that's a single digit: 9. We're at 21 characters in our string, etc. etc. As you can see, there's plenty of room before you hit the maximum of 256 characters!

The problem comes when trying to save LARGE amounts of data. Let's look at the game I'm working on, a game with a map editor and much information attached to each tile of the map.

I have for each tile:

Tile graphic (values from 0-255)

Tile color 1 (values from 0-15)

Tile color 2 (values from 0-15)

Obstruction Tile (value 0 or 1)

Height of Tile (values from 0-9)

Tile affects player height (values 0-2)

Tile forces player to move (values 0-5)

Tile is Destructable (value 0 or 1)

So that's a lot of data for ONE tile. Now lets consider how many tiles I can put on screen, which is 32x24 8*8 tiles, which is 768. Already, if I'm to just write out the tiles as a string, I'm WELL over my limit! This is where creative thought processes come into play! :)

So my desired map format is going to be 32 screeens of 768 tiles with all that data stuck in there but MEM$ only holds up to 256 characters. However, Petitcom can save several other file formats and we can 'hide' that data in those formats. You may have already checked out some of the programs from Japanese users who already use this method. For instance, one programmer created a text editor so he could program on Petit with better copy/paste options and be able to change between fonts on a whim, etc. etc. and he uses GRP images. Each pixel of the GRP image is a character in one ginormous string! So this method seems like it's plausible for my save file.

So lets look at the GRP format. When you create an image for GRP, you're using GPSET, GPLINE and other drawing commands. We are only concerned with GPSET. GPSET x_coordinate,y_coordinate,color is the format for this particular command, so we need to pass valid data to into this command before saving. The value of x_coordinate cannot be more than 255, y_coordinate has a maximum value of 191 and color has a maximum value of 255. So first, let's look at how many pixels we can write to:  191*255 = 48705. Cool so lets see if just the basic map tiles can be written:  32*24*32 = 24576. So we're well within the limits. Half way, to be precise. However, if we write one pixel for each of the values associate with each tile, you begin to see that there's definitely not enough room to save in the GRP file. Again, time for some creativity.

There's two solutions with this that we can look at. The first is number stacking using binary conversions. Basically, what this means is you will 'hide' several numbers in a single number. When you 'explode' a value into binary, it makes it easier to select just the parts of that number you need. What I mean is this:

Let's combine two variables, Color 1 and Color 2 into a single number. First, we convert color 1 and color 2 into binary. We'll use the values 12 for color1 and 7 for color 2.

In binary form they'd look like this: COLOR1 COLOR2 0111   1100 So if you convert them into a string$ and then SUBST$ them into a single string, you have 01111100. Now let's convert that back to decimal, which is 19. So you can hide two separate values well below the 255 limit of the number of colors a pixel can be.

It's important to know that you cannot exeed in binary 11111111 when combining values. Doing this when converting back to decimal would create a value that is not accepted by the color variable of GPSET!

So if you're savvy, you can figure out which values you can stack together in order to maximize the GRP format using this method. However, there's a second method, which is a bit unorthadox. The previous method is used quite frequently by programmers and is good stuff to learn to do anyway.

This next method is one I decided to come up with to avoid the 255 ceiling and increase reading/writing speed when saving/loading map data. Let's consider the values we'll be storing in each pixel again.

ITEM 1 = 0-255

ITEM 2 = 0-15

ITEM 3 = 0-15

ITEM 4 = 0-1

ITEM 5 = 0-9

ITEM 6 = 0-2

etc. etc.

So let's consider this specific situation. In the program, these values are all stored in an array as a string. In other words, I have LEVEL$(32,24) and for each string stored, all of that data is read like this "2550712191" etc. Pretty messy, but when using MID$(STRING$,Char_to_read_from,number_of_characters_to_read) and then convert the string using VAL(STRING$) I can easily extract or replace data. But that number is WAAAAAYYY to large for a single pixel. However, we can arrange that data in the string so that when chopped into groups of 3 characters and converted with VAL(STRING$), it will never be over 255. As long as keep track of what data is in what pixel and cycle through the pixels correctly, you can use MID$ to extract each group of 3 values, convert it with VAL, saving it into a dummy variable to be used by GPSET. Using this method, I can save a maximum of 3 different variables from my map array per pixel. But to be clear, they have to be arranged so that you never exceed 255 when converting to a value from a string. For instance, I can put COLOR1 and WALK as a single pixel because the maximum it can be is 151 (15 = highest color value and 1 = highest WALK) value. So if I'm going to be able to save values that can be up to 9, I need to make sure the first digit can only be 0 or 1.

With good organization, you can squash massive amounts of data into a GRP resource. So let's look at the numbers again and see if I can still fit all this into GRP.

I've rearranged my numbers so that it takes 4 pixels per tile...32*24*4*32 = 98304. 48705 is our maximum so we didn't make it, however, it's much better with 2 GRP's for a complete game data save than 1 GRP for each value (upwards of 10 GRPs!). And the coolest thing is that these files are transferrable between other users, which is great for me because my game relies heavily on user generated content! :)  Once you get done making your level/scenario, you can upload it to the internet via QR codes!  :D

When we want to read it, we simply use the command GSPOIT x,y and then reconstruct the 3 numbers into the program by placing them into the variables they originally came from. :)

I hope this has helped and opens the door for more utilitarian programming in PetitCom for everyone. It's been an adventure learning about this stuff so I hope everyone else finds it useful...and now back to typing with a stylus...*sigh*