So I've been working on the distributed network quite a lot at work. One of our build steps is rather slow and involves compressing a lot of data, and I've been working on farming that out to the network. I got it working at one point, but ended up turning it off again due to a bottleneck I hadn't considered (thirty computers reading uncompressed bitmaps uses a *lot* of hard drive bandwidth.) After going back and solving the bottleneck (by bzipping all the bitmaps :P) I had some more work that needed to be done, plus one or two glitches that needed to be fixed. Turning it on produced *another* batch of glitches (so it went off again), but I'd fixed those and felt it was ready for another real-world test . . . which was good timing, all things considered.
boss> This is *incredibly* slow. I've been running this build for half an hour now, and it's not even a quarter done. me> You know, if I turned the distributed network back on, I bet it'd be faster to restart the build than to leave it running. I think I've got the latest batch of bugs worked out. boss> You think so? me> Yeah. There we go, it's enabled. Go for it. boss> Okay, let's give it a try.
It finished in ten minutes.
Of course, with my luck it'll be turned off again by the time I get in tomorrow, but maybe I finally got the thing working entirely :)
Well . . . until the next set of improvements . . . I think I can decrease disk bandwidth by another factor of eight. *grin*