I'm making a new tag: #devski. I'll be posting future development updates to #devski and specifically not #hubski, unless it's sufficiently user-facing. So if you care about code updates, follow #devski. If not…carry on.

I expect a lot of my work to be not–user facing. I also expect a lot of my work to be on Github. Starting now.

https://github.com/hubski/hubski

What is this??! That's the C trigraph for a vertical bar. But that's not important right now. That is what I've been working on for the last couple weeks. Don't get too excited—it's an internal API, not a public one (see my previous to-do list).

Specifically, this code:

1. converts the internal s-expression publication data to PostgreSQL

2. serve the PostgreSQL publication data over an internal REST JSON API, for raw internal publications, and which accepts both reads and writes via GET/POST.

3. adds Racket functions to read and write from the REST interface, which can be called directly by the primary Hubski Arc service.

At this point, it's about 95% complete. The big things that need done now are converting static data, and doing tests and timings for converting the live data. The s-expressions have some eccentricities which I'm not 100% sure the conversion functions handle perfectly. Hence, tests.

I'm not a Racket expert, so if you are, feel free to suggest improvements. We're not really looking for pull requests at the moment; maybe in the future, but right now, as small as this project is, it's probably about as fast to write as to review.

My to-do list is now:

1. Write tests validating sexpr–sql–json conversions.

2. Test converting the full publication file 'database' to SQL.

3. Convert hubski.com to the internal API service. This involves both converting to SQL, and converting the primary service to read from REST.

4. Convert more data. Publications are by far the biggest data we have, but there's a lot more. Hopefully the publications API will act as a framework, and the rest will go faster.

5. Create a public API, based on the internal API.

My long-term timeline hasn't changed. I'm conservatively estimating: 1,2 in the next couple weeks, 3 in a month or so, 4 by October or November, and 5 by the new year.

I have a few nonspecific things on my list too, like changing markdown→html parsing to use a library, and reviewing and publishing the Arc code. At some point, these to-do lists will appear as Github issues.