Staffan Malmgrens blogg

Lately, I’ve only been blogging about what’s going on in my world. Time to go through the last few days of blog postings that I found interesting for one reason or another:

• pHatidic writes about the fact that Wikipedia does work, despite all the theoretical arguments detailing why it cannot. It’s like the bumblebee, which by the law of physics cannot fly (well, actually of course they can), but since the bumblebees have never taken even remedial physics, they are blissfully unaware of this fact.
• At first, I thought Microsoft was patenting su. After further examination, it’s not as bad as it looks. Still, if you subscribe to the newly granted Microsoft patents RSS feed, you’ll get several examples each week of why software patentability is not a good idea.
• Spamamusment continues to churn out brilliant spam-inspired comics.
• A new beta of the next-generation windows shell (Monad) is out. I no longer have MSDN beta access, but I’m really looking forward to trying this out when it becomes more widely available. I just hope they focus as much on the accompanying programs as the core shell technology. If one has do download Monad-compliant utilities from all over the place, much of the value of the shell will be lost. I’m looking forward to see how the functionality of tools like sed and awk could be expressed in a .Net object environment.
• Raymond is shocked to find out that Swedes are mis-using the right to sick leave. “Everyone” has known that the sick-leave system is widely abused, but this is the maybe the first time that the politicans in charge admit it. But I’m mainly linking to this since I, like most Swedes, am deeply curious about how this country looks through the eyes of someone outside of it.
• Scoble continues to act as the Kibo of the blogosphere. This post is yet another cheap attempt to get mentioned on his blog.
• This story about scientific experiments with swimming in syrup has been seen everywhere today, but I first read it in jwz’s livejournal, as usual.
• Cedric’s talking about his experiences after 6+ months of working with JDK 5.0. Interesting stuff.
• Edward W. Felten has an interesting take on the recent story about online-poker-playing bots. “There’s a sucker at every poker table, and if you can’t see him, it’s probably you”

And some links to interesting swedish blog posts. If you’re not a swedish speaker, take a page from Raymond Chen’s book

• My friend and colleguage Erik has published his report on the state of the web security of swedish political parties. It has sparked some debate, and Erik has also published a follow-up article detailing why it’s important to fix security problems, even if they can’t be exploited right away.
• Karl Jonsson writes about the latest controversy involving swedish bittorent site The Pirate Bay. The TPB guys come off more immature than ever in their email exchange with Uppsala Universitet. Those advocating stronger copyright protection must love TPB, since they provide so many examples of how irresponsible the filesharing community can behave.
• Niklas Lundblad is holding a lecture about information security and philosophy, and has put up his slides with comments. Certainly a unique perspective on security, I would have loved to attend the lectures.

Ok, so I’m busily parsing lawtext with EBNF grammars, but the resulting parse tree is somewhat cumbersome to work with. Each node is a 4-tuple, [tagname,startindex,endindex,childnodes], where childnodes is a list of similar 4-tuples:

  for n in r[3]:
      print "%s: %s" % (n[0],indata[n[1]:n[2])
      if n[0] == 'refs':
          for sn in n[3]:
              print "  %s: %s" % (sn[0],indata[sn[1]:sn[2]])

So, late at night I begin to ponder if it would be possible to write some sort of OO-wrapper around this structure of tuples in tuples, so that I could write somewhat more readable code:

  for n in r.nodes:
      print "%s: %s" % (n.tag,n.text)
      if n.tag == 'refs':
          for sn in n.nodes:
              print "  %s: %s" % (sn.tag,sn.text)

Turns out it takes less than 20 lines of code:

class NodeTree:
    def __init__(self,root,data,offset=0,isRoot=True):
        self.data = data
        self.root = root
        self.isRoot = isRoot
        self.offset = offset

    def __getattr__(self,name):
        if name == "text":
            return self.data
        elif name == "tag":
            return (self.isRoot and 'root' or self.root[0])
        elif name == "nodes":
            res = []
            for p in (self.isRoot and self.root[1] or self.root[3]):
                res.append(NodeTree(p,self.data[p[1]-self.offset:p[2]-self.offset],p[1],False))
            return res
        else:
            raise AttributeError

A good python programmer could probably trim down the above to ten lines, but still. I don’t know if it’s the language, or the particular problem I’m trying to solve, but I’ve been programming a whole lot more recursively lately.

An interesting side effect of deciding to quit programming for food is that I’ve started to program for fun again. Since a set of laws is, at it’s core, just a gigantic set of large documents, much interesting stuff can be done by textual processing of this body of work. And so I’m writing some python code to fetch all current swedish laws, the preparation documents for them, and verdicts referencing them, in order to semantically mark them all up and cross-reference the hell out of them. It’s been surprisingly fun so far.

So, why python? Since I’m more proficient in C# or Perl, it would make more sense to use any of these languages. Well, C# in particular is a very sensible language, with a very useful class library. But, you know, it’s just not that fun. There’s something about the whitespace-sensitivity of python that just feels good. And since I no longer have to worry about marketable programming skills, that’s what I’m going to use.

But one aspect of the sensibility of C# and the .Net programming platform is that the tools are really really (really) good. Visual Studio’s integration of the class libraries (incl intellisense), documentation and debugging is first-class. Sure, the editor might not be as powerful as Emacs, but that’s really only a small part of the puzzle.

And so I’ve been searching for a good IDE for python development. So far I’ve tried, and rejected, the following:

PythonWin

Free with a download of ActivePython. No integrated class browsing or intellisense-like features, and no easy access to documentation.

Visual Python

This is a plugin to Visual Studio, which to me makes a lot of sense, and it does a lot of things right. What brings it down for me is the lack of help and autocompletion for built-in types like strings and the lack of an Immediate pane during debugging. It’s also way to expensive for me to buy personally.

Komodo

This is a standalone commercial IDE with an affordable personal use price. Here, the main dealbreaker is the lack of class library integration and a slow editor. The class browser also leaves much to be desired (it’s more of a symbol browser, relly).

So, if none of these passed my test, what am I using? Well, I gave up on the whole IDE thing and decided to just use Emacs. With some help from Pontus, I got python-mode.el to behave enough that I’m comfortable enough to get some stuff done. The py-help-at-point command does most of what integrated documentation would do, and for the time being I’ll have to browse the class library over here with Mozilla instead. And use printf-debugging. It worked ten years ago, it should work now as well.

I do intend to test WingIDE and BlackAdder, but since they’re both commercial and not that affordable for a student, they have to be really really good for me to choose them over my current Emacs solution.

In another forum, there has been a lot of talk about the book “Hacker’s Delight“, a book focusing on low-level computer arithmetic and logic. In the sample chapter available online, the author shows a large collection of bit manipulation formulas to achieve all kinds of complex procedures on bitfields.

The text is fairly compact, and for someone like me, who isn’t that familiar with thinking in terms of bits, it takes a while to understand all the examples, but I think it is very useful to once in a while step away from our high-level API’s, frameworks and design patterns, and get down to the bit level, the level where stuff actually gets done.

So, I’m setting out to do interesting stuff on the PocketPC/Windows Mobile platform, mostly to do with hardware integration, particularly bluetooth and IrDA communication, so I’m not interested in that sissy Compact Framework stuff that MSDN seems to want to push. Through MSDN Home > Mobile & Embedded Developer Center > Product & Technology Information > Platforms > Windows Mobile, I finally get to “Windows Mobile Developer Resource Kit“, which sounds about right:

The Windows Mobile Developer Resource Kit is essential for developers seeking knowledge about the latest platform advances for mobile technologies.

Bring it on!, I say. But it turns out that the Windows Mobile Developer Resource Kit (WMDRK) is only available on DVD, you can’t download it. Hello? This is 2004, why do I have to wait for a physical disc (and pay $16 for S&H) to get the bits? This is so Amish!

Of course, i understand that the entire WMDRK is a big download, but I really don’t need it all. I have the IDE’s and the SDK’s already, I just want to have a look at the case studies, technical articles and developer power toys. Of course, I’m not sure of any of these actually are, as there is hardly any information about them at all. Hell, those things might already be available for download somewhere else (the SDK’s are), but where?

It feels like I’m back developing for the Epoc/Symbian platform, circa 2000. And that’s not a compliment.

Noone in the Microsoft development space can have missed the big change that has taken place within Microsoft during the last year. The company goes further in trying to build ties with the developers, employees are blogging, they are trying to get people involved.

Now, I have a healthy amount of suspicion towards Microsoft. I feel I’ve been fucked over by MS a number of times, from when I tried to use Perl with ASP 1.0, through when I tried to make sense of CDO (sparsely documented, 300+ byte IDs,…), to when I tried to use Pocket Outlook Object Model (POOM) on PocketPC for real-world scenarios (ID’s that change when record contents change — you’ve got to be KIDDING me??!).

Posts like Chris Pratley’s post about Word file format disclosure: “We don’t do that because it is our intellectual property. People who want to work with us can get it by contacting us; people who want to compete with us need to work harder. That’s business. We might change our minds […] but really it is our prerogative” do nothing to change my suspicion

Microsoft, look; if you’re going to be both platform provider and competitor, you’ll have to accept that people care more about your position as a competitor rather than your position as a platform provider. In that sense, Robert Scoble’s post about “Mozilla should adapt Longhorn tech” was spectacularly naive considering the history of Internet Explorer and Netscape.

I don’t take anything that Microsoft says at face value any longer. I always factor in what they have to loose and what they have to win. In the POOM case, we were doing a product that competed with Microsoft, so it makes sense (from a shareholder-earning standpoint) that they wouldn’t go to any great length to make my life easier. I’m thinking about this when reading the email thread that wasn’t “created with public consumption in mind“. I can’t help to think that it was — it seems pretty sanitized to me.

When it comes to redefining Microsoft’s public image, comment’s like Chris Pratley’s don’t help. You’re an established monopoly, now deal with the new rules and expectations. If you want to be a platform vendor, then document your platform. This includes full documentation of the Office formats, CDO, Extended MAPI, and Pocket Outlook Object Model (POOM). This will undermine your position as a application vendor on that platform. Deal with it. Don’t make us rely on undocumented properties and sites like CDOLive. Don’t just skirt the issue by saying that “it’s your prerogative” to keep things secret.

I’ve read lots of interesting things the last week, but haven’t sat down to go through my “Items for Review” folder until now. By the way, that RSSBandit feature rocks!

• Gunnar Kudrjavets has an interesting discussion about wheter to have assertions turned on in Release code (Part I and Part II). At first, one would agree with Schneier and Ferguson in that “That [Release code] is the only place where you really need it!”. The problem is that release code has performance demands. If liberal usage of assertions slow down release code, you’ll be tempted to use them less, ultimately rendering you with a less solid system.
• Those entries from Gunnar are several weeks old, but I didn’t find out about his blog until today through a post from Cyrus (whose last name I cannot seem to find). Cyrus has plenty of interesting stuff as well, particularly this post about some new stuff coming out of Microsoft Research. AsmL is an interesting system that reminds me of both automata theory and literate programming. Polyphonic C# is a extension of C# for asynchronous concurrent programming, which sound similar to how Erlang works.
• Another interesting post from Cyrus is where he enters a well-written bug report to the Mono guys (regarding anonymous delegate support). It warms my heart to see Microsoft-Open source cooperation at this level, between the actual developers, and serves as a nice example that we all actually CAN get along.
• Jimmy Nilsson writes a little about the current state of AOP on the .Net platform. Now, if the AOP fans are correct in that this is the next big thing on the same scale as OOP is the big thing today, you all should expect to deal with it in the real world in about 10 years

• Google being irrational in the best of ways: Google expects to bring in $eB ($2,718,281,828) in their IPO. If you ask me, they’re worth at least $πB.
• There has been a lot of buzz about the newest publication from Microft’s Pattern and Practices group, “Improving .NET Application Performance and Scalability”. Since I’m not finished with “Improving Web Application Security” yet, I’ve only skimmed this one, but it seems to approach it’s subject with similar throughoutness. However, can I ask for a PDF version?
• Keith Pitty gathers introductory links to Java Server Faces, which for me as a .Net developer seem to correspond nicely to ASP.NET WebForms. While I really only focus on the .Net platform in my day-to-day development, I make a point of staying somewhat in touch in what happens in the Java community, if for no other reason to be able to contrast the platforms and knowing the strength and weaknesses in their differing approaches. Since I’ve been looking into Code access security, I should probably read up a little on the Java security manager. After all, the Java camp has been thinking about these issues for ten years now.
• Omar Shahine tells how to get PDF’s to open in Acrobat Reader instead of in the browser. As added bonus, he gives pointers on how to make recent versions of Acrobat Reader start quicker.
• L-G from Entombed is apparently a guest on Eurodisco star E-Type’s new record. E-Type has already enlisted Motörhead drummer Mikey Dee, at least for performances, but I still don’t think this is going to get him respect in heavy metal circles.

In the comments section to an earlier post  Göran Krampe brought up some excellent points, backed by experience, on my remark that more dynamic languages increases risk in projects, especially if all developers in the project are not on the same level. Here are some of my comments on those, but please do read Göran’s entire comment for more context. Since I find the discussion to be very interesting, I thought I’d bring it up as a new article.

Just read your article at IDG regarding Alan Kay/Smalltalk - and it was good. For once not a single glaring misconception regarding Smalltalk - I am a bit jaded by all people throwing around “truths” about Smalltalk without having actual experience themselves. So kudos for that.

Thank you! However, I must confess that my Smalltalk experience is limited to playing around with Squeak in my spare time, so I have never used it in a large project. Bear this in mind when you read my response…

Above though it sounds like you are saying that Java would mean a lower risk than Smalltalk for larger projects or projects in which the developers are of “average” or “mixed” level. That I definitely do NO agree with on the other hand.

[…]

So my experience is the exact opposite. I claim that less experienced developers can more easily be made productive in Smalltalk than in Java. I have repeatedly taught OO *and* the basics of Smalltalk in a single day including practice and then had these pupils find bugs and complement a working small system the next day.

I suspected that a smalltalker might not agree with that Please not that I’m only talking about risk, not programmer effiency. Risk isn’t inherently bad, since it usually goes hand in hand with opportunity. Your points about new programmers learning Smalltalk quicker than Java is obviously backed up by experience, but a little beside the point.

The reason that I feel projects done in more dynamic environments (I’m going to lump Smalltalk, Python and Lisp together here, which might not be entirely fair) to have a higher level of risk is that the dynamic properties of the language will allow your system to take on aspects of a domain specific language, especially if some members of your team understands the Zen of Smalltalk/Lisp/Python. For example, in both Lisp and Smalltalk you can essentially redefine parts of the syntax, like the if/then/else statements, and if it makes sense in your design, you might very well want to do that. But if another programmer, who has yet to be enlightened (sticking with the Zen theme) on the expects the syntax to work like default, this might trip him up (Operator overloading in C++ and .Net gives us essentially the same problem). This risk increases if the programmer has been taught more traditional imperative programming in the style of Java/C# and have not been given a good introduction to Smalltalk.

Things like tail recursion, higher order functions, metaclasses and even polymorphism cam be difficult to wrap your head around, especially since you can get a lot of stuff done without them, and in not-so-dynamic environments, you usually do.

Or to put it more soundbite-friendy way: The more dynamic and open-ended a language is, the more choices you have at any given time. Every choice introduces opportunity — and risk.

However, since I haven’t done any large projects in either Smalltalk or Lisp (I did a elevator control system in Scheme in school, though :-)), maybe my opionions should be taken with a grain of salt. I’d be happy to hear your opinions on why the things I’ve mentioned above aren’t such big risk factors after all.

Also, the idea that there is something inherent in Smalltalk making it unsuitable for large scale development is simply not true either. And there is ample evidence for that.

There are very large systems having been built in Smalltalk with great success over a long period of time. And I mean *large*. 14000 classes, 65 developers/staff, 500 users etc, see for example the “Kapital” system at JP Morgan Bank, https://secure.cwheroes.org/briefingroom_2004/pdf_frame/index.asp?id=4909)

I don’t think I’ve implied anything like that. It is interesting to hear about these big successful projects, but I keep wondering why we don’t hear more about them. Paul Graham published an essay about how good Lisp was for building ViaWeb, and most people that are familiar with XP has at least heard about the C3 project, but there must be more stories, right?

The point of my column that started the debate was that while many of the inventions and discoveries that come from the Smalltalk community are now used everywhere (MVC, JIT, design patterns…, XP), the language itself isn’t. I don’t believe it’s as simple as Sun and Microsoft having a larger marketing budget, but I’m very interested in a discussion of what the real reason could be, as I still think there are more that the general programmer community could learn from Smalltalk.

Alan Kay recently won the Turing award for his work on Smalltalk and subsequent projects. I spent some time with the free smalltalk implementation Squeak this weekend, and was particularly impressed with the very dynamic nature of things — being able to modify the actual root Object is not something that’s common in more mainstream environments. Felt very much like a Lisp environment like Emacs, but object oriented. Anyway, it all led to me writing a column on Smalltalk and it’s impact for IDG.se. As usual, in Swedish.

Update: Oh, look, it made the frontpage