Javascript String Compression

About this tutorial

Category: Javascript
Difficulty: Expert
Javascript String Compression.zip

Results

Saving is an important aspect of incremental games, and is covered in another tutorial, but what if your save data is quite large? With the advent of local storage, you are now given 5MB of storage space per domain. Never assume, however, that you have access to that total amount, as there may be other applications that are using this storage space.

So how do you work around these limitations? Compression, along with JSON, is key. In this tutorial, we will use a script available for free usage to compress our JSON string and save it to local storage, then load it back and uncompress it to display the information.

If you haven't yet learnt how to save data with JSON and classes, check out the Javascript Saving tutorial first.

We'll build on our previous tutorial, so you'll need to use either the files you created, or the downloadable package from that tutorial. We're going to add a script available from here. The script is free to use for your projects, and is the most efficient I've been able to find thus far. Note that we are not using the script in the main body, but a revision at the bottom of the screen in the comments.

The exact script we'll use is the following:

// LZW-compress a string
function lzw_encode(s) {
    var dict = {};
    var data = (s + "").split("");
    var out = [];
    var currChar;
    var phrase = data[0];
    var code = 256;
    for (var i=1; i<data.length; i++) {
        currChar=data[i];
        if (dict['_' + phrase + currChar] != null) {
            phrase += currChar;
        }
        else {
            out.push(phrase.length > 1 ? dict['_'+phrase] : phrase.charCodeAt(0));
            dict['_' + phrase + currChar] = code;
            code++;
            phrase=currChar;
        }
    }
    out.push(phrase.length > 1 ? dict['_'+phrase] : phrase.charCodeAt(0));
    for (var i=0; i<out.length; i++) {
        out[i] = String.fromCharCode(out[i]);
    }
    return out.join("");
}
 
// Decompress an LZW-encoded string
function lzw_decode(s) {
    var dict = {};
    var data = (s + "").split("");
    var currChar = data[0];
    var oldPhrase = currChar;
    var out = [currChar];
    var code = 256;
    var phrase;
    for (var i=1; i<data.length; i++) {
        var currCode = data[i].charCodeAt(0);
        if (currCode < 256) {
            phrase = data[i];
        }
        else {
           phrase = dict['_'+currCode] ? dict['_'+currCode] : (oldPhrase + currChar);
        }
        out.push(phrase);
        currChar = phrase.charAt(0);
        dict['_'+code] = oldPhrase + currChar;
        code++;
        oldPhrase = phrase;
    }
    return out.join("");
}
 
function encode_utf8(s) {
  return unescape(encodeURIComponent(s));
}
 
function decode_utf8(s) {
  return decodeURIComponent(escape(s));
}

You'll need to load this into your javascript file. It will handle everything for you, but you need to remember that we will be using UTF8 formatting, so if your users are going to be exporting, you'll need to let them know that they need to change their save file type or they may lose their data.

Implementing this is pretty simple. We take the JSON of our Game variable and first of all encode to UTF8 (just to be safe) then compress. Replace the following line:

window.localStorage['SaveName'] = JSON.stringify(Game);

With this:

var SaveGame = JSON.stringify(Game);
SaveGame = encode_utf8(SaveGame);
SaveGame = lzw_encode(SaveGame);
window.localStorage['SaveName'] = SaveGame;

The above will create your JSON string, encode it as UTF8, then compress it and save it to local storage. To get it back, you need to do the reverse. Replace the following:

window.GameTwo = JSON.parse(window.localStorage['SaveName']);

With this:

SaveGame = window.localStorage['SaveName'];
SaveGame = lzw_decode(SaveGame);
SaveGame = decode_utf8(SaveGame);
window.GameTwo = JSON.parse(SaveGame);

The above retrieves your compressed data, uncompresses it, decodes from UTF8, then parses the string into the GameTwo variable. If you leave the rest of the file the same, you should find that the file works perfectly.