Morning Hubski! Welcome to a Thursday edition.
We're still trying to get the configuration settings for our new "sync" (think SQL replication without all the benefits of having it done for you) functionality pinned down tight. Which essentially means that you have have to go each table, column by column and determine how a 'key' can be derived from the data inside (its really tricky! Sometimes you can forget exactly what users can change and your left with a bunch of 'dupes' once your keys go bad). Luckily, I've been doing this kind of thing in the database now for over eight months; I've becoming pretty good at being able to eye what a good key will be. Duplicates are a man made construct anyway.
Bugs have been steadily rolling in. We went live with around 20 rigs in the past four months. After doing my little part for the overall sync configuration, I've been tracking this one bug that is spewing orphaned records into a couple of tables at a pretty high rate. As of right now I suspect 1/3rd (530,000 some odd rows) of the table is actually garbage. I plan to exterminate the bad data, and correct the module(s) that are causing it. I think its just a stored procedure with a couple of joins that are not explicit enough.
On the side, I'm working on a database driven web scraper that I'm hoping will be able to be pretty dynamic and configurable. The overall idea being that you'd be able to configure the scraper to go to websites,and it will go and try to scrape out whatever you've defined as the domain model for it. If the website changes, no big deal, just go back and re-evaluate your xpaths and redefine the node commands for it and viola, its back up and running. I'm hoping that I can spring up dynamic restful services to serve as an ad hoc API for websites that don't have one. It has a long way to go.
What are you working on today?
Rather to my surprise, I wrote a hundred lines of code between getting up and going to work this morning. I'm building an environment to help teach my students programming, which consists of panels where you can define functions on the left, and a repl-like thingy on the right where you can call functions, see their results, click on the results to expand a trace of computations, zoom in and out of the trace. Eventually I hope to be able to make an edit to a function and automatically rerun all the commands in the 'session', flagging any changes or errors. Automatic unit tests for a fraction of the effort. As of this morning, the left side is done. Screenshot: I'm going to start wiring up the right side next. (The repl and trace browser are already done as standalone commands.)
I'm just about to make something similar! Every Saturday, I teach a class of 10-18 kids aged 10-15. Because the younger ones do not have the patience for more conventional methods(Some refuse to read anything longer than a tweet...), I prepare a .pdf more focused on younger people, here's my most recent one. What do you think? Anyway, it takes about 40 minutes to make a page, so my plan is to either make a text editor or a plugin for an IDE/text editor that acts kind of like genius.com, but for code. I would write code, and then annotate it with descriptions so that when you hover over the code, it's description would display. And some other things like a mode that shows you the order that each statement is read in, to help explain functions, though I'm not sure how plausible that is without writing my own JavaScript interpreter.
This is great! Is it a common approach to go over a non-trivial program like this? I've been ranting for months that we should do more of this. Less blank page, more tweaking programs that actually do something, getting feedback along the way. In case you haven't seen it, this is great as well.
Thanks! Yeah, when I started teaching these kids, I remembered that generating a list of prime numbers between zero and n wasn't too appealing to me when I was learning C. Since the start I've been using this approach. What are you doing with your students? I kind of get to cheat with JavaScript, it's not my favourite language but if you want results fast it's definitely up there, and that is very helpful because as soon as a kid starts experimenting and having lots of fun, that's when you've got them hooked.
And thank you very much for that link, it makes so much sense and I had no idea Khan Academy was as good for this as it is. Definitely getting the kids on it this Saturday.
I currently have two students (a 45 year old and his 10 year old son). They came expressly to learn Arc lisp. So far my approach with the younger one has been to give him a skeleton of a program to play tic-tac-toe, and then we're gradually filling it in. Right now we're three sub-problems deep learning about recursion, and tic tac toe probably seems much harder than when we started :) My general approach has been to give him a day or two to solve a problem before working with him on the solution. The idea being that he'll remember more once he's struggled with it by himself. But several times now I realized that the problem I gave him was too hard and spent the next session working on something simpler. So I'm learning a lot.
I'm currently working on trying to figure out how to actually use the newish communication protocol another department wrote. There's only rudimentary documentation and a lot of the core knowledge owners left, so it is a lot of banging my head against NullPointers in code I can't see and running around and asking the few people who aren't afraid of it for help. Once I have that to a fairly stable point I'll go back to putting the finishing touches on a shell script that has grown from just calling Java to parsing files and piping things between sed and awk.
I'm trying to create a training system for a large company that uses the same mechanics as a skill tree from video games. We have four departments in this wing of the company. Each starts out with the same "tutorial" on how to get your desk setup and how to access the intranet, that kind of thing. Then there are two trees - one for technical prowess and one for social prowess. Account managers are largely social, but have to do the first couple of rungs of the technical tree to get an understanding of our software and of excel. Analysts, on the other hand, have to complete the technical tree with only the first rung of the social tree, should they find themselves on the phone with a client for some reason. It's an interesting challenge and trying to simplify the massive amounts of knowledge our current employees and and future employees need is a proving to be rather difficult.
Attempting to learn Julia. The recommended IDE is very cool, but it's not as frictionless as I'd like, and I'm struggling with some of the details.
Not today, but nowadays I'm working on a object database as a semantic file system.