AIWatch.

Release: condense-json 1.0

I'm trying to get braver at releasing 1.0 versions. This little library is a year and a half old now - I've applied some sensible and non-disruptive fixes and shipped the big 1.0 for it.

Here's an example of what it can do, lifted from the README:

{
  "foo": {
    "bar": {
      "string": "This is a string with foxes in it",
      "nested": {
        "more": ["Here is a string", "another with foxes in it too"]
      }
    }
  }
}

Combine that with a replacements object:

{"1": "with foxes in it"}

And condense_json(input_json, replacements) produces the following:

{
  "foo": {
    "bar": {
      "string": {"$r": ["This is a string ", {"$": "1"}]},
      "nested": {
        "more": ["Here is a string", {"$r": ["another ", {"$": "1"}, " too"]}]
      }
    }
  }
}

It scans for strings or substrings that are present in that replacements object and replaces those with a special {"$r": ...} syntax in the output.

You can reverse the effect with uncondense_json(condensed, replacements).

The idea is to make it easier to store JSON that includes duplicated data from other related structures. I use it to save space in the SQLite logs generated by LLM - see PR #1586 for the latest iteration of that.

Tags: json, projects, python, llm