How things sometimes need a little "kick"

It's amazing how some things need a little kick to get enough of my attention and motivation to make me spend extra time exploring them. It must be a special kind of a kick since some "forgotten" things get to wait years.

Almost always when I find a tool or technology that intrigues me I jump right on exploring it. Right away I'd give it a few minutes to a few hours of my time, I'd read about it, play around with it, do something with it. When I could not devote instant attention immediately I would save it for later, often print a few pages and take it home to read, or just put it on hold with a thought to get back to it the very next available moment. The time usually comes, sooner or later, though something of course I never get back to...

Powershell had to wait a long time to finally get through to me. I looked at it briefly when it first came out in 2006 and I thought it's great and definitely worth looking at. I downloaded it but then put it "on hold" ("refrigerate or freeze after opening") and never got back to it. I knew it was out there, I thought I knew what it was (it turned out I did not), but until very recently I had no motivation to try it out and see for myself.

Last week my son brought home this nasty stomach flu virus and we had a few very rough days watching him recover. Health care is extremely powerful these days compared to 100 year ago but we still can't do much about viruses. Too bad. Human body has to get over it by itself, the only help it can get is in fighting symptoms.

It's all long gone now, but when virus was still alive, Denis wouldn't sleep well. I had to spend half the night close to him just watching him breathe, being ready to help if he wakes up. They say, there are three things a person can keep staring at for hours, fascinated: a burning fire, a flowing water, and other people working :) I now know that a parent can do the same just watching his child sleep...

Then, I figured, this extra time staying up at night was a great opportunity to catch up on some of the technology innovations that I left behind unexplored and thus unknown to me. This is how I ended up spending a few nights watching podcasts and reading blogs - a luxury I rarely get.

This year PDC announced a few very interesting innovations. I already knew what's coming in C# 4.0 from Eric Lippert's blog though listening to Anders Hejlsberg was definitely worth the time. Then there was a demo of CCR and DSS from robotics. This one impressed me a lot. Very nice and clean approach to true concurrent programming (Erlang, beware!) that is not done on a language level but instead delivered as a library, pure managed code! The guys said it would make its way into the core framework one day. There were a few other very interesting videos including introduction to F#, and then I spotted "Powershell" in the keywords (I was selecting what video to watch by tags that appealed to me from this list).

- Kick!

There was 2.0 to the familiar name that I thought I knew but never really looked at.

- Kick!

It's not fair to watch Powershell 2.0 video if I didn't look at the original stuff; I figured I could not appreciate new features unless I knew old ones.

- Kick!

Enough kicks. Poweershell indeed wanted me to look at it. I had motivation and I had time too.

--

I knew that Powershell was a much more powerful shell language than the old good MSDOS batch, I knew it had pipelines similar to Unix shell, and this was pretty much all I knew about it. When I needed something really simple scripted I could use batch and do *.cmd. For more complex stuff I would use JScript as I did once for bulk visio backwards compatibility conversion automation. Powershell never came out of the freezer I put it in and I was missing 2 out of 3 its key features. I knew the pipeline but without the other 2 it was "just" a pipeline. We have a tiny little pipelie in regular cmd today (netstat | find "80"), so why get too much excited about a more powerful one? here's why:

• Powershell script operates cmdlets [command-lets]. It does not really have a "language" to it. Each command has a cmdlet behind it. And then cmdlet is just a .NET class that fulfills a few contracts. One can build a new one in a matter of minutes and register it with Powershell runtime


• but more importantly cmdlets operate with objects, "real" objects with attributes and methods. One command pipes in objects into another one and off it goes. The command implementation can read incoming objects, run them, do anything a language cmdlet is written in lets you do with objects.
  

Let me show you. As you can imagine Powershell does provide equivalents of well known DOS commands, ones like "dir". In Powershell it's now the Get-ChildItem cmdlet that can do any listings thru providers, not only filesystem, but it does not matter for what I want to show you. And one can do "dir" as Powershell supports aliasing:

PS > dir | get-member

what I asked Powershell to do is to explain me the objects that "dir" cmdlet returns. here's the output:



you see? now we can exploit the pipe, join a few other cmdlets, and do stuff like this:

PS > Get-ChildItem C:\2send\4Max | Select-Object FullName, Extension, Length | Where-Object {$_.Length -gt 1000}

or in a much simpler form

PS > dir C:\2send\4Max | select FullName, Extension, Length | where {$_.Length -gt 1000}

and then one can do:

PS > dir [..] | where [..] | copy [..]

and this is all very basic stuff! I bet one can do a much, much better use out of it. I now smile when I look at this post where I truly thought the dir /a /b /s > filelisting.txt was the nicest way of doing recursive filelistings :)

Next time I need to script something I will definitely try Powershell. If it turns out to be not as good as I now think it is I will make sure I blog about it. stay tuned.

Mastering Language

I am very passionate about mastering languages. It is true with both spoken and programming languages though this post is about mastering language we speak.

I admire people who can use the language in a way that freezes my attention, makes me read the text over and over again tasting phrases, feeling sounds and intonations, almost hearing the author reading the text back to me. As if I was listening with my eyes.

My native language and the language I speak at home with my family is Russian. First foreign language I studied was German. English came naturally with the profession and became my language passion. I live in the US since last year and all this time I've been practicing my language skills, working on they way I sound, and mastering vocabulary I use. The more time I spend with the language the better I feel how far I am from the place I want to be. Surprisingly, it's rather motivating and keeps me going.

I enjoy good reading. I don't have much free time to read and with a thick stack of professional things to catch up on I have even less time to read. Mix two lovely kids in (Denis is 6 and Alice just turned 1 month :), and here I am, constantly trading sleep time for at least some reading.

Fiction, I figured, is not always a good source of language enjoyment, at least not the enjoyment I wanted. While it might be great written it's often not the language I would use in my day to day life. Another thing is short time intervals during the day that I can fill with it. Do you like watching a movie with five interruptions when you pick it up from where you left it off the next day only? Neither do I.

So I picked up memoirs. It is apparently a popular type of writing these days and there are talented authors writing about their lives; some do it in a way that makes me come back to the same book over and over again. It's often not an easy reading but the language is well worth it. Some of the authors who I enjoyed reading lately are Augusten Burroughs (you may know him as an author of Running with Scissors though Dry is what I liked the most), his older brother John Elder Robinson and his look me in the eye, and Lucy Grealy with her Autobiography of a Face. You may question the contents and "Why on earth am I reading this stuff?" (my wife does :) but you can't argue the language quality. It's great, full of life, deep, entertaining, sharp, honest, Real.

--

When I come across a blog author who can do equal quality writing about technical stuff, who can give me both professional reading and fun reading, I feel jealous. It's a rare talent to be very good with technology and be able to use the language to describe what you're passionate about in a form that makes it fun to read for everybody while still delivering a unique value in content.

There's a list of Top 100 blogs for Development Managers. Almost everything falls into "my" category. But there's much more out there. This whole post was triggered by the blog of Eric Lippert who is working on the C# compiler team. Read his last series about C# 4.0features and reasoning behind it, excellent stuff.

--

I wish I had more time to write about things that I'm working on. I plan to do a series of posts about a modeling/configuration engine that I architected and built a few years ago. It got some attention recently and we decided to give it some uplift and maybe opensource it. We will see how it goes.

Personal Wiki [Continued]

A few days ago I wrote about an idea to set up a personal wiki space and how I was stuck at apparent big variety of wiki tools to choose from.

The choice really came down to hosted (thus available from anywhere) vs. local (thus available offline). I considered it for a while and figured that ability to work with my personal knowledgebase offline (say on a plane) was more important. And then if I need to I can host my knowledgebase out. Deal.

Next step was much easier than I thought. To pick a wiki tool for local deployment from a huge list of options I followed a simple selection process:

1. filter the list down to only tools that have been out there for a while (a few year I figured is Ok to secure product roadmap and stability) and have recently had fresh stable release (this year was recent enough for me)


2. must be open source and free


3. better not require anything I don't have installed (database, app server, etc.)

DokuWiki was my instant choice. It's been there for about 4 years, it's file-based so no database setup/configuration hassle, and it only requires Apache + PHP. Apache is always around and PHP was apparently in place too.

I have installed it 15 minutes ago and I am already using it. Dan gave me a good advice (though I didn't pick the suggested wiki tool) - "just start typing".

The best thing to do after you came across a thought to have a personal wiki space is to go ahead and create it!

Requirements

Martin Fowler has recently blogged on what he called observed requirement emphasizing the idea of web sites' users behavior observation as a powerful but yet not widely used driver of identifying and testing requirements. This post triggered two parallel thoughts in my head that I figured might be worth writing about.

#1: WebTrends and alike
#2: Different engagements warrant different approaches

#1. WebTrends and alike

The testing and observing part is not really new. "WebTrends and alike" is usually my first thought when I hear my customers want to analyze how their users behave on their web sites. The tool seems to be out there for about 15 years and for a good 2/3 of its lifetime we've been doing WebTrends-ready applications. What the companies then usually did with this was often limited to reporting. They would report on how did users behave, where did they drop off of the site, and what content was the most often requested/viewed. Sometimes they were even interested to see the most common user paths thru the site as they thought it might help building the right advertisement strategy. I recall one company did in fact want to include WebTrends(-like) support in order to test their original set of features and see if they could figure the best way to enhance their product thru their customers' footprints on their website. I can't tell if the "enhance" part ever happened (and my guess is it did not), but they had been working on content that they thought was "good" but apparently not proven to be so by the actual visits statistic.

I don't think there's an Approach to how to convert users' behavior into the site's feature-driver. It's just you can't see what's not there so you can't build a step-by-step guideline. The advice may be easier to give: leverage the instrument, especially if you are a company running your business on the web and competing in the SaaS market. Approach should be tangible, Advice may not be :)

#2. Different engagements warrant different approaches

Another thought that Martin Fowler's post triggered is not related to analytics. It's related to the intro quote Martin used to open his post. Martin quoted a "waterfalish" approach to requirements management and challenged it with the agile approach that is "...intending to discover the 'requirements' during construction and after delivery".

During last few years we as a company get more and more exposed to agile methods, agile projects, and customers who want to go agile (sounds close to the "go green" buzz). This whole concept of doing things agile and following agile principles in software engineering and software projects management is Great but is also often misinterpreted and misunderstood. The concept of "working software over comprehensive documentation" often gets translated into "agile means zero documentation". Does it? working software comes first. Yes! but who said zero documentation? The concept of "responding to change over following a plan" often gets translated to "we don't need requirements and we don't need to plan". Do we not? isn't sprint backlog a plan?

Different engagements warrant different approaches. One of the ideal environments for the agile project management (say SCRUM) is a product development project when the team takes on building a non-existing (or a new version of an existing one) product that gets elaborated on as the team progresses. It just naturally assumes close interaction with product management and demo-based turn-around. Integration project in a large highly-bureaucratic corporation where ten vendors have to cooperate and manage inter dependencies will naturally get very formal, and though can (and should) be delivered iteratively will have a lot of waterfall smell.

Consider how contract type affects the game plan. Another characteristic of the environment that well fits agile management practices is when the team is either in-house (e.g. business company running an internal project, ISV building software products for sale) or when performing organization runs on a T&M contract (e.g. outsourcing partner practicing Agile). Progressive elaboration to the extent when requirements get identified (not just clarified) as the system is being built is open ended in nature. Iteration cycle completes and produces an increment of the working product; the next cycle(s) will be driven by the priorities as of by then. Would you as a seller be happy to sign a fixed price contract without at least some level of due diligence on requirements? Would you rather want to build some requirements boundaries and document assumptions that would let you later be "flexible" and approach changes with a great agile attitude knowing your contract watches your back? I know what you think. I too wish I could explain all benefits of the "true agile" approach to all different businesses I get to work with. What an ideal world it would be.

Of course agile engineering practices are much less affected by the external environmental factors and writing unit tests as well as integrating continuously is like breathing regardless of whether you're up to building an internal product or a contracted vendor for one of the Blue Cross and Blue Shield licensees.

I'm slowly getting to what I was about to say here under number 2. Best practices are considered best in context. Requirements clarification and to some extent identification will be happening during construction regardless of your thoughts about it. It's the "art" part of the software construction. Though depending on the project environment you may well want to do a fair level of requirements due diligence upfront. Short on-site visit may be all you need. It will pay off.

And then finally a "requirement" can be very different. Without some level of "requirements" you just can't start the project, when existence of another level of "requirements" will only become apparent after final delivery.

Competence

...Core competence does not diminish with use. Unlike physical assets, which do deteriorate over time, competencies are enhanced as they are applied and shared. But competencies still need to be nurtured and protected; knowledge fades if it is not used. Competencies are the glue that binds existing businesses...

(c) Hamel, Gary & Prahalad, C. K. The Core Competence of the Corporation. Harvard Business Review, 1990.

Personal Wiki

I figured I need a personal wiki, a knowledge management and self-organization tool. I imagined a small personal portal that is a mixture of a wiki, a file drive, and a to-do lists management. Ideally a built-in mind mapping tool too but this one is a nice to have. I also considered whether I needed Outlook integration and decided I did not. I don't want to mix priorities. Outlook is my do-right-now-all-about-work tool and this one I target to organize my writings, readings, ideas, R&D topics, learning priorities, etc.

ok. where do I start? if you faced with a multi-criteria choice problem and you didn't know where to start what would you do? I thought I weight my criteria and start with the most important one. Wiki is definitely the key and if powerful enough may well end up being all I need. Great. what do you do next? Google.

do you know how many different wiki tools there are? I thought there would be a dozen commercial platforms and a few open source, some of which I thought might target personal usage. Who needs more, right? Take a look:

Comparison of wiki software

so where do I start now ?