challenges and choices

I recently got a silent promotion* to Senior Engineer here, which I thought was funny, until they put me in charge of the Pets strike force, and I now understand how deadly serious it is. I'm heading up the engineering effort to add pets to our MMO platform, and now all of a sudden I have to do things like "code reviews" and "assigning tasks," which is always a challenge for me, because I often have very specific ideas about how everything should be done, but delegating means letting go to some extent.

Actually the best practice I ever had in delegating was my time as work frosh for Interhovse, back in the day. I found that people really need a clear task, that suits their interests and capabilities, and in some cases a lot of support in terms of tools and advice, but given that they will take ownership and really shine. It was such a great experience, being a part of that process, that I chased it all throughout college. The dolphins, the HME, all that good stuff.

Well anyway it's tricky, making a shift now from being a programmer to being a team leader. I'll still be doing a lot of coding, but more and more of my time is probably going to be spent on other people's code, which frankly I am not thrilled about. But on the other hand it means I can tackle bigger problems, which is fantastic. I know that a lot of programmers have trouble making this transition, and I can see why. So I hope I can beat that curve.

And also I might have to buy some nicer shirts. :-/

*I know right? But in a small office, almost nobody ever uses anyone's actual job title. So to announce it would be rather conspicuous and frankly probably not that great for the morale of people who haven't been promoted so, I can see the point. Of course, it's possible that everyone else was already a senior engineer, and I just didn't know it. :-p

getting serious about it

So the whole weight-loss thing. Sam keeps mentioning this Hacker's Diet thing, and on Friday I finally looked it up. It's a free online book and a set of free tools to go along with it. I found it, as promised*, extremely compelling.

The approach the book takes is that weight is a problem to be solved, just like any other problem, and it examines the problem from an engineering and management perspective. The thesis is that people who have chronic weight problems (e.g. the author, e.g. this author) have a broken appetite. Some people have bad eyesight, some have faulty appetite signals. No big deal. Except that you can't just go to the CVS and buy corrective appetite-glasses.

So the thing to do is to build a set of external tools that can do what the appetite should be doing. In building the set of tools, he brings up a few key points:

1. Your weight varies day-to-day by much more than the amount of fat you gained or lost, because of water. But, it is mathematically simple to calculate a trend line from your weight data that will give you real, solid data that you can rely on week after week. In other words, we can accurately measure the results of our actions.

2. People (or at least, people who need to lose weight) are assumed to be terrible at estimating calories. Look it up, plan it out, know what you eat. You can also control the input.

3. If you have bad eyesight, you will need glasses for the rest of your life (or until you get laser surgery.) If you have a faulty appetite, you have to apply external tools for the rest of your life**. If you do, you can keep the weight off.

And the rest is details, such as knowing that the first 48-72 hours of a diet are the most painful, such as forming a habit to weigh in, such as knowing how to counts calories, and all that other common sense (!) stuff.

I've been on a diet for about three weeks now, but over the past two days I read the book and I'm ready now to get really serious. I've been unhappy with my weight basically, forever. But I've always tried to manage it intuitively, and I've always denied, at some level that my appetite was broken. Admitting that it's broken is actually really liberating. Seeing a path to correct it is really empowering***. 

I don't know if this book would work as well for non-engineers as it seems to have worked for me, but hey it's free and it's online.

*Nothing was actually promised.
**But you DO NOT have to be hungry for the rest of your life. Getting past the worst first few days has made a new man of me.
***Yes, I will have to weigh myself every morning, for the rest of my life. That now seems a small price to pay, which is interesting because I've balked at the same idea many times before. But knowing that the only thing I care about is the trend line, not the number itself, is really comforting to me. Removing the noise from the signal and shortening the feedback loop is an extremely good optimization for any control system. Weight is a control system. My natural weight control system is broken, but I can fix it with tools.


Today, Wii Fit told me that I am no-longer obese. I feel simultaneously proud and insulted.

BMI. Pretty annoying.

Also, thanks to everyone who encouraged me. The hunger pain has indeed gone down somewhat, and the results feel good.

marking the moment

I now think of myself as an adult. And I think I'm ok with that, really. It doesn't mean I stop having fun or anything, but there are simply a lot of subtle cues all around my life, that tell me that when it comes to classifying myself, adult is what I am.

A little rough around the edges? sure. Eccentric? hopefully. But fundamentally, no-longer a "young person."

smartphone research, and seeking advice

my phone research let me show you it.

So... I want a smartphone. And thanks to a recent windfall, I can now almost justify getting one. But, I have serious, serious reservations about jumping in to the deep end of the pool in terms of monthly contracts. For reference, with my pay-as-you-go plan from Virgin Mobile, right now I spend $20-30 a month. (I make few calls, I send few text messages, and I don't do twitter.) So if I buy a smartphone that will be a pretty big cost difference.

Anyway I've been researching and waiting. I've come to the conclusion that what I really want -- a pay-as-you-go smartphone -- does not exist, and isn't going to anytime soon. So then, what are my options, and how do they stack up?

HTC IncredibleHTC EVOiPhone 4Google Nexus One unlockedGoogle Nexus One
Up Front:200200200530180
Total cost 1 year:11001040104012501139
Total cost 2 years:20001880188019702099
Total cost 3 years:29002720272026903059
what?very well reviewedno battery life, too much like a monster trucktoo Apple, but.. probably a really good contract, but also less support = ??too much $.

As you can tell I'm deciding between an Android phone and an iPhone, and my preference is leaning slightly towards the Android, but only slightly. 

It turns out they all cost about the same. Especially since, if I'm going to be honest with myself, I can be pretty hard on my devices, and a phone I buy might not last 3, or even 2 years. So there's the replacement/repair cost to be considered as well, and that makes an unlocked phone relatively less attractive.

It's striking to compare the cost of a phone with the cost of, for example, a TV. A television lasts maybe 5-10 years. If it lasts just 5 years, and I spend $700 on it, then my cell phone is more than 6 times as expensive as my TV. (Of course, that doesn't count netflix or cable subscriptions... but it also doesn't count phone overage or apps.)

No decision has been reached. Does anyone have any words of wisdom?

*Raw research:
android phones

Verizon - HTC Incredible
$200 + $75/mo
($39.99 plan + $29.99 data + $5 text (200 messages))

Sprint HTC EVO
$200 + $70/mo

AT&T - iPhone 4
$200 + $70/mo (down to $60/mo with 200MB data limit... interesting but annoying...)

T-Mobile - Nexus One
$530 + $60/mo
Nexus One Contract
$179 + $80/mo

database design for revision history

So, I was working on a personal project yesterday, and I came across a bullet point in my notes that looked like this:
  • implement item revisions
And it stopped me cold in the middle of a pretty productive session.

I originally designed my database to represent the "current" state of the system. So my central table, Item, doesn't yet have a concept of "history" or "latest revision." And I don't mind changing it, it's still a young system and very much work in progress, but the question is, what's the right way to represent an item that's been through an arbitrary number of revisions?

The constraints are:
  • The solution should be clean (normalized or nearly normalized)
  • The solution should be simple to code against
  • The solution should be high performance and very scalable
  • The solution should be applicable to other tables if/as needed
So after reading around and thinking, a couple of options pop up.

1. Rebuild it on the fly:

Never update anything, just add a new "RevisionItem" that points to the original item. When you fetch an item, also fetch its latest RevisionItem, and in most cases render that instead of the original. This will cost you performance, but it does make the revision history absolutely clear, and it means not modifying the Item table. As a side note, this method fits well into the design pattern of this particular project, where items generally have other items (comments, tasks, sponsorships) associated with them.

2. Log the old versions:

You can keep a "ItemRevisionLog" table, which stores old versions of the items, while the Item table always stores the most up-to-date. The advantage is that you don't have to change the Item table at all, and you keep it from bloating. The disadvantage is that you have two tables with the same columns-- not great design, and if you change one you have to change the other.

3. Add an isLatest flag to your data:

You can just keep every version of the Item in the Item table, and use a flag to find the current version. This runs into problems because every Item query now has to do a lot of extra work, and also you need to think very carefully about your primary key. (i.e. when other tables link to an item, they are now linking to many item revisions, not a single row.) Still, it's not unworkable. Just a little ugly.

4. Treat revisions as first-class data, and normalize them:

Make a table called ItemRevision, and put into it everything about an item that you want to be able to revise, and add a couple of columns for date, editor, and item_id. Then take those columns out of Item, and instead give it an ItemRevision reference called "latestRevision." Each revision is now available when needed, and that data is only written once. Retrieval is also a simple join by id. The downside is you have to be very explicit about what is revisable, and you have to change your data design every time you change your mind about what can be revised (e.g. right now).


So I'm going to cut to the chase. Number 4 is really the right answer, it seems. Anyway that's what wikipedia does, and their capacity for revision management is [unquestioned? unassailable? practically idiomatic?]. Also I found this great chart of the wikipedia schema, which, if you're into that kind of thing, is pretty cool. It's pretty clean, for all the work it does. Color me impressed.

You can see how they handle page revisions right in the middle of the graph. The Page table points to a Latest Revision, while each Revision table points to Page. A separate Text table stores the actual page text, for reasons that are not immediately clear to me, but that I will assume don't really apply to my much smaller and simpler system. In any case the chart, working in combination with a good night's sleep, has convinced me.

This has been a fun puzzle for me, and once I stop grumbling about rewriting my Item data access to go through a "revision" object, it will make me a better person.

programmable keypad update

(see here)

It's the best!

I filled up all 24 keys. Only 3 with single-keystroke keys (tab, enter, delete), the rest with all kinds of wacky keyboard shortcuts.

I'm pretty happy with it. I might buy another one for home if I start uh... using my computer more.

funny thing about biology

So I'm trying to eat less and exercise more and all that good stuff, but one of the funny side effects of eating smaller portions, is that it turns "the story of Nate working on some stuff" into "the story of Nate trying not to eat stuff." It's amazing how distracting hunger in particular can be for me. Part of the problem is that when I'm working hard, I'll often pacify my body with a large meal. Really not a very good habit when "working hard" means "sitting perfectly still at my desk with a pained expression on my face." But changing the habit is proving difficult and disruptive.

Sigh. :-p

where's the beef (web design)

I've got some personal problems with internet technologies. The central problem is that, as a game programmer, I've never really respected web tech. The web is static and unresponsive. Most of the code is bad. Most of the languages used are (dynamically typed?) and unresponsive.

But as I keep pushing forward anyway, it seems to me that my central problem has been that I don't even understand where the work happens. Where's the beef, as it were.

A game, for example, is written in one programming language, and is tightly bound to itself, and to a single machine. It's easy, in principle, to follow the path of execution from user input, to simulation, to display. And in between there's a mass of c++ or what-have-you that makes everything go. To me, that makes sense.

But on the web, I face a daunting situation. The user sits on machine A, in browser B, makes a request to server C, which hits database D, and when the server gets the data, it renders a page, and returns it to the client. The browser then renders the page, and then the client side code takes over and manages some of the interaction going forward, talking to the server as necessary.

Just to take one example, count the number of languages, or distinct syntaxes, involved in this chain of events:

  1. URI/URL, (HTTP)
  2. ASP*
  3. C#
  4. SQL
  5. HTML
  6. CSS
  7. Javascript
  8. XML (webservices, optional)
  9. Flash (optional, special purpose)

So, as a newcomer to the field I say, great, where do I put my application? Where does the code go? And the sad answer is, it has to go everywhere. In order to make a high performance site (the only kind I can even bear to contemplate), you need spend time with each of these technologies and syntaxes. The connections between most of the different languages are very loose. There aren't a lot of good tools for looking at more than one or three of the languages at a time. And they all matter.

So my essential personal problem with web development, is that I find it absolutely disgusting. It literally triggers my "unclean" emotional response. And then I shut my brain off and try to hold my nose as I program, and it's not very easy to work one-handed.

Well anyway I have these websites that I want to build. So, I need to get over it. I need to get over my tightly-bound, completely-integrated elitism. And so part of what I've been learning lately, is that I've been looking at the stack in the wrong way.

Now that I've figured out that nobody designs websites in html and css (they design in photoshop or illustrator), things are starting to make a lot more sense. I can redraw my big list like this:

how does it look?
1. Photoshop/Illustrator
3. CSS
4. ASP (optional)

what does it do?
1. SQL
2. C#
3. ASP

how does it feel?
1. Javascript
2. XML (webservices, optional)
3. Flash (optional, special purpose)

So, there are three distinct jobs here. (Four, if you count the job of drawing the lines in between the categories.) Each job has good tools associated with it. Each job covers a small enough group of technologies that it's possible to work effectively. The trick, I think, is to only work on one job at a time. Put on your architect hat and draw the divisions. Put on your UI art hat and render the page. Put on yous system engineer hat and define the services. Put on your game programmer hat and write some javascript.

Do not, under any circumstances, try to design your page layout with C#. That sort of procedural art is technically possible, but it's incredibly time consuming, and the tools for it are terrible or nonexistent.

I know this isn't novel, but I feel so much better for writing it down.

*I'll use the ASP.NET C# stack because that's what I've been using for my personal projects. I strongly suspect that it's not much better or worse than anything else. I know I personally like it better than Ruby on Rails. I've never really tried PHP.

the dialog box is a lame duck

file under: UI Pontification

Firefox Ditches the Dialog Box | Webmonkey�|

Modal dialogs are confusing and frustrating. I hate the way they insistently blink at me when I try to click outside, demanding my attention before I can do what I want, forcing me to think on their rails. Outside of a few rare, special cases, modal interactions are a crutch that should be avoided wherever possible.

Non-modal boxes are... clunky. They get lost easily or they get in the way. They probably still have their uses, but as a design paradigm, they will be eclipsed.

Doing better may not be easy, but the sign of the dialog box is declining. Human interfaces are maturing, slowly, painfully.

single-purpose objects

There's a fetish in western consumer culture for the single-purpose object. What I mean is, a tool, container, or item that has exactly 1 use. For example, a potato masher or a salad spinner, as opposed to a fork, or a bowl. The potato masher is utterly useless for anything except mashing, but it's a pretty good masher -- much better than a fork is. Single-purpose tools fit into a funny category because they're always perilously close to being zero-purpose knick-knacks, if their niche disappears. You'll get a lot of use out of that hammer, but maybe not so much out of that post-holer.

I think that as our society gets richer, we see more and more of these single-purpose items in our daily lives. They're not really very economically efficient unless you use them pretty often, but I know that I have an emotional weak spot for the perfect tool, the tool that makes one particular job really easy. And I'm finding, as I look around, that more often than not the right tool is out there*.

I think the same trend is true across all manufactured items, across all sectors of industry and commerce. In an abundant ecosystem, the drive towards specialization is strong. You see the same thing in rain forests, where many species specialize on eating just a few things, and you see the opposite in tough or turbid environments like cities and deserts, where a few animal species survive by being generalists. If you only have a $20 budget for your kitchen supplies, you're not going to blow it on a salad spinner. But if you already have plenty of bowls, it starts to look pretty useful.

I'm not sure if I think this kind of specialization is good or bad. I find it pretty easy to make arguments either way. Certainly I think people can take it too far, in what they choose to acquire. But I don't think I'd blame the tool for that, I'd blame the collector, if they clutter up their house or blow their budget. It's not the pipe wrench's fault that you're not a plumber.

Anyway, I like to have an awareness what tools I choose and why, and this has been on my mind a lot lately as I try to clean and organize the house, and yes, as I ponder new acquisitions. ;-)

*Funnily enough, I think this is part of the reason I get so angry about software that doesn't suit my needs; I've grown used to the idea that if I need an egg scrambler, I can order one from the home shopping channel. But when I want the perfect keyboard or IDE, I'm left in the cold. Maybe my expectations are unrealistically high? Or maybe the software industry is still young when compared to say, manufacturing, or kitchen ware...