Improving the security of mobiles and services

After the recent debacle where pictures has been leaked from Apple's iCloud, by using iBrute and EPPB to attack specific users iCloud acounts there is clearly a need to address weak security in synchronization protocols used by mobiles for backing up and synchronizing data to the cloud.

Since these apps, our built in services, on the phone relies on static information to identify and authenticate seamlessly it is relatively easy to reverse engineer the protocol and emulate an individual device. These apps and services operates on the user's behalf and can not rely on multi factor authentication (MFA) requiring user intevention.

I will here propose a solution based on a integrated chip (IC) for generating OTPs (one time passwords) to introduce some randomness to the identity information submitted from the mobile.

When the user installs and app or activates a service on the phone, the app/service registers itself as a OTP consumer to the mobile OS. The OS must be able to uniquely identify an app/service in order to prevent impersonation from other apps. No token should be required to store by the app. Most app developers are not security experts and will probably not be able to secure such a token properly.

When the app logs in to the remote service for the first time, an OTP sequence is set up in the onboard chip. The chip generates touples of passwords, one part as the actual password, and a second to use as salt for a hashing algorithm. The remote side must set up the same OTP sequence, and the process should be confirmed by the user who identifies with username, password and ideally a MFA token entered by the user. This could be a OTP generated by Google Authenticator or similar. Steve Gibsons SQRL will also be an excellent confirmation. This requires the user to create the account beforehand, or the app does this during first time login. Devices added to the account can be revoked access if they are stolen, and when devices are added the user is properly notified in a separate channel (e.g. email, SMS)

When the app or service want to communicate with the remote end this sequence will occur:

1. App/service request OTP from IC
2. IC generates OTP touple and hashes the first part using salt from the second part
3. The hash is returned to the app (the app never sees the real OTP)
4. The app supplies the hash value as part of communication setup process
5. The remote side also generates same OTP touple as the mobile and calculates the hash
6. Hashes are compared.
7. A succesful compare allow for proceeding the operation.

 This will make it impossible to emulate a device by using static information. The hashing is there to obfuscate the original OTP, in order to make it even harder to guess the next password.

The proposed solution may also be used where the user is active, but as a added layer of security. Recently Tesla Motors announced that the iPhone could be used as a FOB (and probably Android/Windows devices later, guessing). If the setup process used the onboard screen of the car, scanning a QR code on the screen, the risk of illegitimate access and use of the car is minimized. This is probably just the beginning of use cases for mobiles giving access to valuable items, and security issues lies ahead.

This solution only help protect to use of a service from a mobile, not the content stored on the service providers storage. These will to varying degrees be vulnerable to other attacks, so don't put anything on a service providers storage that must absolutely not leak into the wrong hands.

This idea is free to use, but oh, I'm saving money for a Model S ;)

New blog dedicated to systems thinking

I have created a new blog where I will focus on systems thinking from a programmers perspective here http://flowshaped.io.

This blog may continue to contain various ramblings on software and the industry.

Real world authentication and transaction protocol semantics

A real world story explained in authentication and transaction protocol semantics to provide some insight into the relationships between real world information flow and failure demand that is so important when managing IT dependent service organizations.

A couple of years ago me and my family went on a trip to Mallorca. In the confusion of the arrival terminal baggage area we left one backpack on the pickup trail. The backpack contained artifacts of high value, but not critical to our stay. A human error started an obscure process enduring the our whole stay on the island.

Note: As most histories of failure demand this one started with the customer making a mistake. In systems thinking terms, this is probably so frequent in the charter business that it should be considered as mere variation.

On the bus to the hotel I discovered we where missing the backpack and asked the guide what to do. The guide told me to contact the guides located at the hotel as he was just having responsibility for transport to and from the airport.

Note: this must be happening all the time, why doesn't charter operators have procedures for handling this immediately? This is in effect a redirection to another service (endpoint), much like a http code 303 See other. This is also an example of standardized work that drives costs and complexity up.

I attended to the guides on the hotel on a daily basis. This resulted in using estimated 30 minutes per day in waiting and talking to the guides. Each day they promised to look into it, they contacted the airport, waited for response from the lost property department.

Note: I was polling the guides, which was in a constant loop answering me and polling the lost property department. This consumed resources and was of course a source of irritation to us.

In the day we where leaving, the guide having responsibility for the returning bus transport followed me to the lost property department of the airport. When asking for the backsack the clerk just shaked his head. Nothing like my backpack was found. I pulled my passport to eliminate any doubt of who I was, and provide correct information. The clerk quickly disappeared and returned with my sack. Huh? It appeared the guides was not informed of the required authentication to pick up left property.

Note: The airport clerks pretended it did not exist until I showed up in person and provided proper authentication. Much like a properly invented security measure in IT, but remember to inform your users (guides) about it. Failing to inform results in failure demand in the flow and bad service seen from the customer (which in this case will associate the travelling destination with bad service (cognitive availability).

Since the time before checkin was nearing fast I did not have time to check the contents of the sack before later. It appeared that mobile chargers and iPods where missing.

Note: Almost no security measure can stop unloyal employees from taking what they want.

After returning I claimed the loss to my credit card company. Travels payed with the card automatically has travel insurance. They presented a bureaucratic and cumbersome process, and in general did not cover lost property. Huh? So much for travel insurance. Our house contents (NO: innbo) insurance though covered lost property during travel. The process was very straightforward and used average prices on artifacts calculated from real prices. I got a security token that I could use to "buy" the artifacts again from their webpage or get money refund. -> Happy travelling insurance customer. I will stick with these guys for a while, as my availablity heuristic brain tells me they provides good service.

Note: providing a simple process with clear semantics reduces pain and resource consumption and great service.

Content and technology cycles

Most content outlives any presentation and distribution technologies. Content valuable enough will be converted to new formats and prepared for distribution on new carriers. This is nothing new and has been ongoing since humans started to draw and write.

Content creation involves effort and possibly considerable amounts of economical investments. Professional content creators live off their work, and naturally wants to be paid for providing their works.

Somewhere along the road, movie-, music- and publisher industry acquired exclusive distribution rights. When they where distributing content in a physical form (in atoms), this added considerable value to the product. Distributors got involved in all kinds of coordination and promotion activities related to their exclusive rights. As long as the distributors controlled the technology this went along fine, but with the invention of the music cassette the foundation for a copy culture was laid. Content consumers wanted to have copies of music in their cars, walkmans and in the living room. The VCR allowed copying of films. A growing pirate industry, mainly based in Asia was founded, but the economical problems was limited as they had to move atoms around the globe and copied material was not more accessible than the originals.

For music and movies this led both to direct loss for content creators, but it also provided culture sharing. When more people heard new music or films, some bought originals while some copied. It will be impossible to know if the entertainment industry have lost or gained in this game. A copy today may result in future sales of releases of music and movies.

When computers moved into the home copying of software soon became common as the physical burden was shrinking. We started to move bits on physical carriers (diskettes). When PCs was introduced copying flourished. Whether this favored or hurt the software industry is not quite clear. Microsofts success with Office was laid with the massive distribution of Windows which was heavily copied and distributed amongst users. I would guess the software industry as a whole, especially platform vendors, have gained opportunities because of copying. This is not a defense of piracy, but my estimation of the outcome. The software industry has slowly adjusted to the new realities. Software can be bought and downloaded or provided as as service (SaaS). Open Source developers uses the moving of bits their full advantage.

In the late 90-ies the Internet was introduced for consumers. The Internet made moving atoms obsolete. Content could now be moved as pure bits globally, with the speed of the network. The Internet provides us with a completely new way of distributing and consuming content. It is also a foundation for collaboration, creativity and provides us with vast new opportunities for doing business.

The elimination of the need for moving atoms also reduces costs. There is little value in moving bits, and the concept of the moving and copying of bits breaks the limitations of physical distribution. It also breaks exclusivity as content can be made available for anyone instantly at a global scale. The entertainment industry seems not have understood or do not welcome this. They seem to be willing to go all the way in their efforts to stop the evolution. They lobby for and support legislations like SOPA, PIPA and EU Data Retention Directive.

Technology follows very rapid cycles of invention, while content has a completely different cycle. The most valuable content will inevitably be converted to new presentation- and carrier technologies. If the copyright owners does not do it, consumers will. And why should they not? If you have bought a product shouldn't you be allowed to use it on gadgets not yet invented? To me it is not crystal clear copying is theft. Copying is also culture sharing, and generates future sales. The Internet gives us great possibilities for sharing culture and this could be lost, or at least very restricted if the entertainment industry will get it's will with lawmakers.

So why would anyone tie exclusive distribution rights to specific distribution technologies? This gives distributors no incentives for adjusting to new realities and opportunities. These industries are now lobbying for draconic laws that is protecting their relatively outdated business models. They refuse to meet the demand of their customers and stubbornly gives Internet the blame for everything not going their way. The difference now is that consumers (and pirates) have all the tools and infrastructure to fill in the gaps. When the gap left open by the entertainment industry is wide enough and demand is high, massive piracy is the inevitable result.

Instead of exploiting vast opportunities for culture sharing and selling a lot more (for a lower price per item, reflecting reduced costs) they want laws that will seriously impede democratic freedoms hard won through recent centuries. The Internet has breathed new life to democracies (video of Al Gore talking about SOPA was quickly deleted from Youtube...). Whether the lawmakers does not understand this, or do not want more democracy will be mere speculation. It is probably a mix of incompetence and using piracy as an excuse for their own agenda.

Another consequence of the internet is that there is only one market: the global market. Trying to limit releases to restricted areas will fail and only cause problems. It has become de facto Cargo Culting in the media industry. In the digital distribution world there is no borders or physical restrictions that creates exclusivity. A better strategy will be global releases and making use of social media buzz to spread the word and attract consumers to buy content from copyright holders.

Hack your phone

Recently my old HTC Hero started to get notoriously slow and frequently report "low on space". Even after removing all apps memory was still almost exhausted. The phone has been running on MoDoCo 2.1 since the last r5 release.

I could have just bought a new phone, which of course would be better by all means, but since the risk of bricking the phone was low (low probability and the impact of the risk was just to by a new one) I decided to find what the latest update around for an old phone like the Hero.

I found far more than I expected in Cyanogen Mod 7.1. Now I have a far newer OS than HTC has ever released for the phone, a lot of new functionality plus a much more stable and responsive phone. Extended standby time included. In addition to saving (or at least postponing replacing the Hero) it was a satisfying feeling to successfully tinker with the device.

When the risk is low the threshold for hacking your phone should be low. Just as people should know how to switch light bulbs and connect the pieces of a surround system, they should also have some knowledge about their phones and how to repair them. It is not hard, it is fun and you'll learn a lot.

My data

We are doing it all wrong. Empowering people with technology is currently centered around the technology only. The technology is important of course, but it is totally useless without any data. Technology is just for playing with the data.

So do we empower the user with any control over data? Can we manage data digitally as in real life? Well of course not, digital information is about making data more fluid. But making data more fluid does not mean doing it to all data. The principles if privacy should be valid independent of format.

• Private data should be as private as in a bank box. If the individual chooses to destroy the data, it is his or her choice.
• Shared secrets should be as secret as though it was shared with real friends. Sometimes information will leak.
• Published data should assimilate. As in real life. It would be impossible to control the flow
• Observations can be recorded, but no personal information should be stored without consent

Given these simple rules, it should be the individual deciding which risks to take. Facebook, Google or governments should be in no position to decide what risks to take with any data.

That does not mean any data can be gathered by trusted parties, but what and how data is collection should be open. It must always be possible to opt out. That does not exclude companies and governments from monitoring activity. It should just be recorded as impersonal observations. As in real life.

Information industry battles

Evidence that we are now witnessing one of the greatest battles in the information media industry is emergent. The giants of the industry is either actively pursuing ever more control or is being pushed to take defensive steps. Through the last century many similar battles has taken place: AT&T, the Hollywood Filmthrust and RCA vs Armstrong serving here as prominent examples.

The roles of the battle is not new: weak and unprepared governments, capitalism serving shareholders, and eager consumers. The goal of the current battle is to dominate so as to cut [exclusive] deals with content owners. As often seen before patents is used by the big guys to limit innovation from competitors, and push around those big enough to pose a competitive threat.

In earlier battles a very limited set of patents was used a weapons. Now the giants has to collect patent portfolios in order to gain sufficient control over/defense against competitors. A large number of consolidated patents is powerful when  one tries to suffocate innovation and limit the innovative freedom of others.

Innovation has always been important in the Information media industry. Innovation could make the industry more or less self regulating. If a conglomerate/cartel manages to gain control over innovation that could be a threat and even ultimately replace them, they have also gained invulnerability. The loosers if the battle is the content consumers, that will have less choice. The free market can easily commit suicide, especially in the information industry.

The governments, and especially the USA, has much to easily given the giants the weapon they need: patents. It is maybe a bit counterintuitive, but patents is a construction for limiting further innovation. Governments are also generally weak at regulating the information media industry. This creates the opportunity to create an empire. The more powerful, the easier it is to get allies either by fear or business. But, there is a but, when governments has seen trough it's fingers of this battle yielding consolidated giants, they have also created a soft underbelly on those. The giants knows it, and parts of the governments knows it too. The parts of the governments that has understood, also know how to exploit it. Ultimately the information industry stiffens, only casting static shadows of its former dynamic nature. This is when capitalism stops working.

Recently the US and EU has implemented legislation that let them get access to the giants business records. In the name of the fight against terrorism and child porn, they have adopted draconian laws, that removes our digital privacy. This just get easier the bigger the giants gets, because the stakes get higher with size. No giant wants to be defeated up by an anti-trust case.

The current battle is more destructive than ever. The real stakeholders is not shareholders, but us. Our privacy is at stake. Information media industry collects private information and serves public information. It is a unbalanced game. We loose as consumers because without competetion, the giants will be lazy but almost impossible to replace. They have their patent portfolios. This equilibrium is exploited by our governments.

Patent dowry

Patents, that are originally meant to promote innovation and protect investments, has become a strategical asset in the game of alliances. For Google it has become an urgent matter to strengthen the Android ecosystem with a patent portfolio, not innovations. I guess it is important to the Android partners that they are "protected" by patents. Not necessarily things they have invented. Just any patents that can be used in the war against the other players.

Google providing a patent portfolio is merely a dowry to make the Android ecosystem attractive and protect it's inhabitants. Innovation has become a minor variable in the equation. Innovating mostly pays off in lawsuits these days, because there is always someone who has bought or patented something the innovation resembles. The genius working alone that need to protect her or his ideas is a myth. Innovation happens in teams and cooperation with other entities, and is mostly empiric.

What really happens now is patents are collected in portfolios and presented as a deck, either to be attractive or frightening. Entities are forced to join conglomerates and consortiums in search of protection. If this game is allowed to proceed any longer it will be hard, and even impossible to enter the market. An idea that competes with the existing products will not have a chance. Innovation and the free market suffers. More or less willingly cartels are born through consolidation as a result of the patent wars.

Apple is already in bed with media industry, network providers, so we already have conglomerate of corporations controlling a large part of media consumption. Now with Google buying Motorola hardware a manufacturer is the same entity controlling the largest switch (search engine) of the Internet. These entities become very powerful. As long as they behave nicely, this is not a serious problem. But this system is vulnerable in two ways. First, how can such powerful entities restrict themselves so that this power is not misused? Secondly they have a soft underbelly, as they will probably be investigated in terms of antitrust. But governments have recently become very interested in how the Internet and electronic communications can be surveilled and even controlled.

Will governments regulate or exploit the opportunity? As long as the patent war proceeds, the conglomerates will not dissolve. It is their survival strategy. But it undermines the original design of the Internet with distributed control. It does not matter if the Internet is technically controlled in a distributed manner, when the information flow is centralized.

The situation will then resemble some of the pre antitrust cases in the information technology industry. But this time it is driven by patents.

To begin with, selling and buying patents should not be allowed as this fuels the patent war. But I guess it is much more complicated to fix this problem now than ever. Big patent owners will not like the idea of their patent portfolio, expensively procured, should only be used to protect innovative ideas for a short period of time.

PS! I listened to the JavaPosse #360 Newscast while writing this. It has some interesting points about these issues, recorded almost a week before Google buying Motorola, and as such is free of speculations over why.

Further reading
Patents, Schmatents!

Chaos is good - and why we should trust Google

Google, the search engine, does not own content it directs users to, with some exceptions. I do not trust Google everything, and I do not say they always adheres to their own "do no evil" mantra. But when it comes to limiting the powers of traditional content owners it does a decent job. Google is the net neutrality's best bet. I will explain why I think this is so.

Google plays the same role as the phone catalog, but in a much broader sense. It helps users find what they look for. If the dispatcher(s) of the Internet is agnostic to newcomers and established content providers this will help innovation, education and general informing the global community. At the same time this degree of centralization represented by the mighty Google,  is vulnerable. Misused or controlled neutral dispatching is broken and value limited.

A neutral dispatcher is important in ensuring net neutrality. Except from paid search rankings, Google's page ranking algorithm favors paths to content users prefers. I guess we are lucky to have Google. Google is no saint, and is not defending net neutrality out of pure values and standards. But it is important to their business and current position. They depend on users looking for content in a chaotic ever growing amount of content. Google creates value from chaos. Chaos is to their advantage. Content providers and owners, the kind that is lobbying against net neutrality, want control and order. E.g. Apple, married to AT&T in the US, wants you to find what you look for inside iTunes, and is does a heck of a good job providing a streamlined user experience.

Google, a strong player that does not have specific interests in providing owned content (Youtube beeing an important exception, but it is a free service and content is user provided) and with no network preferences they effectively have become kind of net neutrality guarantor. Because of their unique position in the information industry, it is their interest that the net is neutral towards all content providers.

As a consequence Google has never been closely related to content producers and network providers. Content providers even accuse Google of copyright infringements, even traffic is directed their way via the search engine.

With Android Google have become related to device manufacturers. But one should note that none of these, or at least to a small degree, has, produces or provides content. They compete on producing devices suitable for content consumption and to some degree content production.

Apple and Sony are examples of the opposite type of device manufacturers, with Sony Ericsson in a limbo position providing Android handsets. Sony Ericsson's Android handsets are typically more customized than other Android handsets. Apple is closely related to content providers through iTunes and AT&T on the network side for exclusive deal on the iPhone.

Content owners will want to shortcut the neutral dispatcher to gain advantages over other content providers. Ultimately main players in the information industry want to control distribution, and even consumer devices. By controlling the networks used for distribution and devices, they can direct, and even filter content available. This is called walled gardens, and what traditional content providers want. I interpret what happens in OECD as a content provider control coup towards network services. Content providers utilize their powers over governments to gain some (initial?) control over network services.

The architecture of the Internet, as Licklider and the other founders designed it premiered distributed media control. The design was in direct opposite of how AT&T and telephone networks was designed, and AT&T still struggles with this.

But Google can not alone defend against the forces trying to divide and conquer the Internet. Even Google will have to change business strategy if net neutrality is lost. Google is wholly dependant on the prosumer (producing consumers) and others betting on the open and neutral web (which of course is in their own interest too).

The World Wide Web and http protocol is used for providing, finding and consuming content. An open WWW is mutual dependant on net neutrality. Net neutrality is an important foundation for WWW as we know it, and at the same time helps defending against centralized control. The reason for this is the hyperlinking nature of WWW.

So what can net neutrality defenders do? The decisions needed is often counterintuitive, since you often will have to choose chaos over order and not first class content providers that only accepts exclusive deals. Prosumers must put an effort in putting all kind of common knowledge on the web under a sharing license e.g. Creative Commons to prohibit evasive copyrighting of stuff of interest to everyone. Digital tool makers must strive to give prosumers tools to mass produce good quality content. Prosumers should link to relevant content, that help glue the Internet. Network service providers must not make exclusive deals with content providers or let them to close to their operations.

Content makers should also consider what will serve them best in the long run: a close marriage with media conglomerates that was formed during the 90's or a model with room for all players in the content industries. In contrast to the job
market 20 years ago, the options are no much more dynamic. Lock-in is not a good thing when tools are democratized to a level where everybody can produce something (talent or not). A large part of the content industries will benefit from net neutrality in the long run, but established entities will often fight against it.

Governments must regulate to by intervening using antitrust  laws, but when one studies the history of media industries it is revealed that they are slow movers and too vulnerable to lobbying. The book The Masterswitch, by Tim Wu, describes the phenomena of media- and content industries since the inception of telephony and is an important contribution in the net neutrality debate.

Can patents and intellectual property rights put a deadlock on the information society?

What is information? Or what does information make us capable of doing? Information put into context is knowledge. Information exchange is the basis for empiric evolvment and great inventions. Without relatively free flow of information society can not evolve and prosper. Historically things would have turned out pretty differently of the alphabet was patented, or mathematics was protected as intellectual property.

During listening to a podcast interview with Robert Laughlin, that (I thought was only) about carbon future and climate, the talk also mentioned the topic of patents and intellectual property rights (at 50:35). Protection of informaton has restricted US to some extent to empirically evolve, and jobs is outsourced to e.g. Japan and now China. The patents is kept by American companies, but production is not in the US. It is however unclear how this affects employment and innovation in the long run, but there is a high probability that a connection is present. It started my thinking on how such protective measures affects our society. Laughlin mentions a book he has authored: Crime of Reason that rationalizes over this subject.

Just think how some cities and whole nations became recognized mariners in the era of sailships. By sharing knowledge, and demolishing the churchs false demagogy saying the world was flat, they conquered the earth. Little or no knowledge was patented before industrialization. At least not commodity knowledge.

The invention of the internet has let loose massive flows of information. Our society and daily lives is packed with technology. Information technology is ubiquitous and indespensable in the parts of the world calling themselves information societies. What disturbs me is that the tools we are so dependant on is illegal to tinker with to an increasing degree. Apple is the forefront of this development, but they are not alone. Given that a lot of smart people, buying products, sees ways to improve them it is a waste of talent not to let them. The knowledge of how the tools that we depend upon works should be available. Reverse engineering should not be am act of crime. The products themselves is just as valuable with the knowledge available, if not even more. When products can be extended in ways the manufacturer did not think of, the usefulness and usablility increases.

This is especially true for software, that increasingly becomes the inner workings of our tools. Did you know that the average car has software with over 10 million lines of code? How many knows how that code works, opposed to traditional home mechanic doing maintenance works on his own car? Recently it has been shown that wireless pressure sensors are vulnerable for malicous hacker attacks. Patents can not protect you from criminals, but people with good intentions (and I am fairly convinced they outnumber criminals) could reveals such things. The most capable could even provided fixes. Software should not be patented. The value is in goods that can be traded and valueadding services using the knowledge. Using information correctly is complex (instantiation of knowledge) and will always be in demand.

My point with arguing that knowledge about how our tools works is that this is how it has been most of the time during our civilization, and probably before that too. When there are too many patents and intellectual properties protected from reuse and tinkering, the information society may be deadlocked. If this is true, it is a slow process. It is like the story of boiled frogs, that do not recognize that their beein boiled when let into cold water slowly heating up.

The issues of protecting knowledge may eventually restrict desired and needed innovation. In the context of the interview with Laughlin, one can derive that it slows down or prohibit much needed concensus over what environmental challenges we are really facing and how they can be solved. To make it clear where I am going with this: Environmental challenges are global, the internet is made for global information exchange. The tools we use, that largely is the cause of (our perceived) environmental challenges, are protected from tinkering. Information protection and patents is not helping us in figuring what we have to do.

Has Oracle killed innovation on the Java platform

Most of the innovation the software world is derivative works. The whole software innovation ecosystem is empiric, and new offsprings sees the daylight with knowledge originating from other successes and failures. Further, a lot of innovative products is based on commodity software, that saves innovators from the tedious and costly work of doing everything themselves. If these mechanisms breaks apart, the whole innovation ecosystem in the software world would crumble. I think this article explains how innovation happens today, and it lists som famous everyday innovations that where never patented. Things everyone of us uses almost everyday. Had they been patented, some things would be quite differently today.

So, Oracle sues Google over some patent infringments and IP rights. Because they own Java they can do just that. Google has been clever omitting these patents and IP rights and Sun did not see any interest in pursuing any possible infringements in court. I guess they saw Googles works as strengtening Javas overall position, even if the code was not portable, but knowledge is. And knowledge is very easily transferrable. Further, it can result in new offsprings and innovations.

The Java Virtual Machine is a commodity that a lot of business and open source projects relies upon. The desktop and server editions are open source under GPLv2. The mobile edition is not fully open source, and it is here that the Oracle lawyers (hyeanas are more appropriate me thinks) are seeing legal meat to dig into. How they axactly will argue is not yet revealed, but it could be they will argue lost business. I would say it is downright unethical to sue a another party on such basis. Just as Apples iPhone would not have had Androids market share, if it did not exist? There are no other real competitors to Apple just now, and is Oracle producing phones? Can they show a prototype? Can Oracle provide any proof that Java ME has lost any market share (as far as I know Java ME has not played any significant role in a market sense)? They are just hurting the JVM and Java language as a commodity by inserting insecurity and fright into the ecosystem. Maybe this will be the event that triggers completely new programming languages or strengthenes some new ones in the pipeline? Languages with absolutely no patent strings attached and potential misuse by the "owner".

The consequences could be devastating for innovation on the Java platform. Those who want to use Java the platform must from now on be extremely careful to not irritate Oracles lawyers. What does the Scala people think now? Will Oracle try to sue them for lost revenues on Java IDE's or do some ridicoulous changes to stop Scala?

In addition to damage innovation on the Java platform, this lawsuit will drain energy and time from managamenet at Google and Oracle. Who gains on that? Yes Apple and Microsoft. While Oracle bashes Google with stone age business models, competitors can exploit their distraction from the scene where innovation happen.

What feels so completly wrong with this lawsuit, apart from totally missing the point of the software industry works, is that Google has probably been one of the biggest contributors to the diffusion of Java language on the mobile platform, namele Android. Android provides no or little revenue for Google, but sees at as mere innovation and business platform that suits them. I think Oracle will have a hard time arguing for any economic losses as Google does not make money on Android directly and Oracle is certainly not in the ad-business.

Friday 13th  , August, 2010 is a sad day in software history, and James Gosling did foresee it in the aquiring negotiations with Oracle. He must have felt like Albert in We, the drowned by Carsten Jensen. Albert can see war victims beeing killed in his dreams before it actually happen.

Update 18.08.2010
Charles Nutter has written a thorough blogpost on the issue.

Check your brakes

This week I had several almost-accidents when riding my bike because my brakes was worn out. For every incident I got more careful and lowered my speed.

I changed my brakes, and I can tell you, it was a revelation. I could go much faster, and could stop almost instantly when required. The effects are only positive: I can go faster, safer and it is more fun. The safer part means I don't risk hurting myself and others. Good brakes kind of set you free.

During my first ride with the new brakes it struck me that this has analogies to many professional life as a programmer. When programming it is very easy to assume you have understood requirements, the technology I am using and all kinds of other assumptions.

Developers, projects and organizations should integrate feedback and brakes in their work. Good brakes is useless without proper signal to use them.

A brake in programming can be incarnated in several forms, which I will try to show here.

A programmer should listen to all feedback provided from compiler warnings, automatic- and peer code reviews. When struggling to name a software artifact, stop coding and take a break. Find a colleague that can act as a rubberduck or ask for advice. Do pair programming.

Unit tests provide concrete signal (when done adequately), and will in some ways act as a brake. They can let you refactor, delete and improve code more freely. A good test can also stop you from implement plain wrong functionality when you can not satisfy correct asserts in any way.

Likewise performance tests, security reviews, coding guidelines etc can act as signal to brake when things are going in the wrong direction. The more you can automate of these things, the faster it will let you go.

I think the principle of integrating feedback and brakes in all work done in an organization is valuable. For software projects frequent releases will provide feedback, and tell you occasionaly to clutch the brakes. Open discussions in the project can reveal bottlenecks and inadequate ways of doing things. No topic should be avoided, as avoidance can hide potential problems.

Requirements must be asserted too with prototypes and design can be explored and refined with CRC

A promising new technique that is beeing developed for decision making in software architecture can provide a much tighter feedback, brake and correction loop. I think the boardgame approach can be adopted for a lot of complex decision making processes.

A Policy Advisory Board (PAB) will be responsible for overseeing obedience of common rules and update outdated rules. At the organizational level the same yields as for projects regarding open discussion. Let people discuss freely, use wikis, and microblogging to unlock knowledge about inefficiencies.

The key observation of brakes, is not only the slowing down aspect. It is also about changing direction. Changing direction at high speeds may be impossible, risky or it has to be done with a large turn radius. Slowing down for a moment improves your quickness.

Now after a lot of talk about brakes it is important to state brakes are essentially a hook into work processes. These hooks should let people intervene to change direction. The brakes does not provide the required decisions that must be made, just an opportunity to make them before too much damage is done. They also provide excellent opportunities to learn. Failures is valuable learning and is an integral part of an learning organization. Learning is, amongst other things, effectively brake checking and the loop is complete.

Can tech obstruct your fundamentals?

This Easter I came across this blog post about The Value Of Fundamentals (highly recommended reading), and it made me think about how and why we adopt new technology. (Maybe I am fond by this way of thinking because I was a martial arts practitioner myself, and highly respect them still.) In the adoption process the problem it is supposed to solve gets lost, and it is used everywhere as a one-solution-fits-all technology. Fundamentals is not as well understood as they should be in many professions, and software development is no exception.

Technology can prevent you from using your basic skills, as "advanced" technology may look like it can solve several of your problems at once. This is sometimes true, but more often just smoke and mirrors. In addition technology brings on a new set of problems (that you will try to solve with workarounds), and sometimes they become a hammer that you'll try to apply to any problem you stumble upon.

This occurs frequently in software development, but it happens elsewhere too, often with computer technology that is supposed to solve multiple problems. Way too often this obstructs the use of basic skills to solve problems. The challenges here can be mapped to other professions, e.g. physicians on large hospitals relying too much on all the available technology rather their basic skills.

Software development has always been ridden by the Silver Bullet Syndrome, and Silver bullets often infers technology that in effect makes us rely on factors out of control of the individual, project, and most important the stakeholders. But it is not only silver bullet technologies that can make you loose sight of real simple and elegant solutions to the most important problems you are trying to solve. Even wellproven and widely used technology may obstruct the view of the problems.

Here are some examples of categories of technologies that often obfuscates simple solutions:

• Frameworks
• Code generators
• Integration technology, e.g. OR-mapping
  

Most of these adds complexity to the solutiuon (but promises an overall general complexity reduction), as they often are very general solutions trying to solve multiple problems. Solving complex problems by adding multiple frameworks to the solution make a very complex solution. This is where fundamentals becomes valuable, and decisions on whether to use a framework should be balanced with using basic skills.

When basic skills are forgotten or not practiced often you rely too much on technology to solve your problems. What happens then is that socalled advanced technology is applied to simple problems, that should be solved applying fundamentals.

What is really bad about using technology instead of basic skills in a software development context is that technology must be maintained throughout a products lifetime. When a technology vendor stops supporting a certain technology, all products relaying on it implicitly receive a death sentence. Products implemented mostly using basic skills has a better survival rate, as it often can be ported to new platforms where technologies on the legacy platform is unavailable. It can be very complex to update 3rd party frameworks if they are interdependant, or the code is invaded by the used technology. The maintenance cost can rise, and if ignored create serious technical debt.

Additionally well written software brings on best practices to newcomers reading the code, and when a skill/practice is questioned it triggers lot more valuable discussions than whether to throw out/introduce some technology. Evolving basic skills is a whole lot more valuable than decisions on the latest fad. Evolving basic programming skills empowers the individual , project, organization and profession. Technology brings on a substantial number of abbrevitions that is impossible to communication with users and stakeholders. Independant of what technologies are involved in your project, keep it out of non-technical discussions. Users don't care, and is certainly not impressed. Software that works, and that evolves with the users impresses. Wellcrafted software is easier to evolve in the long run, rather than fighting with framework/code generators.

So what fundamentals should all software developers master? I can easily come up with this shortlist, but it is not comprehensive or prioritized in any way:

• Central design patterns
• Know at least 2-3 programming languages, and 1-2 of these deeply and you should be acquainted with a scripting language.
  
• Create readable and maintainable code
  
• What is robust code?
• Important concurrency concepts
• Coupling theory
  

So this is a post that promotes the Not-invented-here syndrome? Not at all! Just let programming fundamentals weigh in heavier against e.g. frameworks and code generators. When cost and/or time constraints favor use of these, look at the source code (not only the documentation) and evaluate amongst other things:

• it is well written
• will not invade your code
• not get in the way of creating elegant solutions to your problems.
  

As a pleasant sideeffect you might even learn a technique you are currently unaware of. When you understand how it is written it will be far easier to communicate with the vendor if changes must be made.

When you think about it, technologies come and go but the software industry in general changes at a much slower pace. It is time to shift this balance from praising the latest fads towards improving basic skills in the software development profession. This will make both the software itself, and the programmers, more valuable. Investing in fundamentals should also be incorporated in organizations' learning circle. Finally programming more consciusly with basic skills will contribute to less monocoltures in software, which in my opinion results healthier software.

Stone age business models

It is sad to observere that patentbased lawsuits against competitors seems to be part of major mobile manufacturers business strategy these days. They have started to bang each other in the head using lawyers equipped with patents.

Nokia sues Apple, who sues HTC on UI technology. They've started to dig trenches instead of trying to beat the competion by innovating. High profile lawsuits like this require a lot of attention from the companies leaders, that distract them from making real business decisions. Especially Apple seems to have already forgotten about how fast they've recently grown in the smartphone market, only because they where innovative (and maybe already had a cool reputation). Well these times seems to have passed, and they've started to protect their innovations.

What Apple and Nokia is forgetting here is that this will force competion to do something uniquely new. Somewhere someone will come up with innovations making the iPhone look outdated. Innovation in this space often happens outside the big corporations, and this is becoming particularly true nowadays. The cloud offer startups vast computing resources, open source provide building blocks to start with and finally social media gives rapid feedback. No patent can protect an investment from this. See more on this here https://sites.google.com/a/webstep.no/openinnovation/Home/news-about-open-innovation/guykawasakioninnovationandthemythoflightningboltinspiration

I think patents in software is an anomaly that must be burried and forgotten. They can not help protect software investments, and only gives the patent grantees a false feeling of safety.

This week a new way of doing UI, Skinput, was presented, and that from the patent borg in Redmond, or at least a Microsoft driven University. I guess this will be patented too since it is a Microsoft-led innovation, but nevertheless it comes from a "unexpected" source, as Microsoft has lately been accused of not innovating much

Update: Very interesting podcast on the Techrepublic on this subject

Update 12.03.2010: Some very interesting and relevant blogposts:
The New Paradigm of Advantage and Jonathan Schwartz on Patent Litigation

Major- and Minor Tyrannies in software

Serviceorientated Architecture (SOA) can enable/support redesign of business processes that can help organizations tap more of their potential or even providing uniquely new products and services.

But there are som pitfalls that might show up later in the process as obstacles, caused by lack of/forced- or accidental decisions. The pitfalls I will elaborate on here are those that dictate projects to use inapropriate technologies for the problem they are supposed to solve. I call these majority and minority software architecture tyrannies. Software architects must spot these and handle them properly.

Majorities
Majorities often force inappopriate solutions on other projects in an organization, disquising this as standard solutions to be used. This eliminates good and qualified decisions in projects that is doing something different than previous project. Doing something different is the norm in software projects, as it often is part of renewal/change in business/technology.

Such standards is often invasive and impossible, or at least very hard, to change later.
Majority tyrannies must be met with knowledge about better alternatives, and how they can contribute to better supporting requirements and business strategy. In cases where standard noninvasive technology is forced, an isolation layer can be introduced to prevent unwanted dependency diffusion into architecture and code.

Standards often appears as a relieve, liberating the project architect from making decisions. Not staying alert can prove fatal to the project later on.

Minorities
Survivor projects clinging to old or inappropriate technology can prevent others from moving on to better or more suitable technologies. Measures dependant projects can take is to create a Integration Anti Corruption Layer (orginal definition from DDD Anti Corruption) layer, so it will be easier to replace later.

A subset of minority tyranny cases is lack of proper versioning of dependencies, making releases of different projects interdependant. In SOA this is amplified and has become a runtime challenge, as opposed to earlier where this was primarily a build time problem. Several versions of shared services (and components) must be supported simultaneously to enable independent and smooth releases. In SOA the most flexible way of version handling is using the Evolving Endpoint pattern.

Consequences and how to deal with this
The concequence of making wrong/forcing/avoiding architecture decisions is high complexity in release management and ineffective software for supporting business processes. Since both organizations and software technology changes continously, decisions can not be be written on stone tablets. Previous decisions must be challenged, and discarded as they are
a) proven wrong
b) outdated
c) proved unecessary

This is part of SOA Governance and must be handled by a Center Of Excellence or Policy Advisory Board

Especially majorites often lead to Architectural monocultures, which is bad both seen from innovation (evolution) and security point of views. These are strong motivators for evaluating and make architectural decisions based on business requirements rather than what has worked before.

This post may appear as a anti standard manifest, but that is not my intention. Good standards have been reevaluated many many times, and survived these evaluations. This "process" will run the test-of-time on standards and give feedback for refinement. I think it can be viewed as variation of natural selection, where the fittest survive. Natural selection, by the way does, not apply in monocultures. Monocultures can produce odd mutations and eventually they collapse.

On a per service this manifests itself as providing multiple endpoints, that makes it as accesible, usable and flexible as possible. A service consumed by many clients gives it a strong position in the organization(s) using it, and thus it may itself become a well proven standard.

Excel and grep is the logfile analyzers best friend

Lately I have been analyzing some logfiles for invocation times of remote services. Luckily the most problematic part of the system logs time consumption together with a remote system identifier and some other stuff.

Some years ago I was working on SmartLearn, implementing analytics for learning accountables. Through that work I got to know Excels Pivoting capabilities, but back then I used Microsoft Analysis Services for creating the Pivot tables. I have also been a user of Linux-, Unix- and Cygwin's sharp commandline tools for a long time. Seperately I know strength of the tools from both of these worlds, but I did not recognize how I could use them together.

One of my project peers showed me how Excel could be used to extract data from flat files and present it as Pivot tables in Excel with very few steps. The key premise is that the interesting dimensions of the Pivot-table is logged on the same lines as the interesting numbers/text. The technnique described here will let you visualize counting of things. Even without any numbers the frequence of things can be very interesting. Most system log files contains a timestamp, and this can be combined with almost anything *. Using *nix-commandline tools it is of course possible to extract whatever information you like from flat files.


17:48:05,168 DEBUG Task1:29 - end call, Task0 duration=401

To extract only these lines I use Cygwin's grep command like this:

grep Task.*duration <logfilename> > extractedlog.txt

From Excel open the extracted log file. Excel will recognize the file as a textfile that you might want to split in columns. Choose between splitting at fixed positions and characters. Spaces can be a viable option in some cases.

When you have imported the file it may be necessary to split columns manually using the Text To Columns tool in Data toolbar.

Now you must insert a row at the top of the dataset, and add header names of the interesting columns. Cut & paste the interesting columns so they are adjacent to each other.

Now select the Insert tool in Excel, and Pivot Table (leftmost button in my installtion of Excel 2007 under Insert)

Choose either Table or Chart, and select the interesting fact and dimension columns from the spreadsheat. When you click OK, you can start dragging & dropping columns into Axes, Values and Legend containers on the right side.

You can now twist the logdata as you want, and find relations between them you would spend enormous amount of time finding manually in the log file. Some simple examples:


To create the dataset I used this Java code that produces differentiated exection trends in a number of threads (which is not meant as a educational example of how to write multithreaded code):

package com.webstep.logfilegenerator;

import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

public class LogGenerator {

/**
* @param args
*/
public static void main(String[] args) {
  ExecutorService exSvc =  Executors.newFixedThreadPool(3);

  List<Future> futures = new ArrayList<Future>();
  final Random r = new Random();
  for (int i = 0; i < 10; i++) {
      futures.add(exSvc.submit(new Task1("Task"+i, r.nextInt(i+1*500))));
  }  
  try {
      for (Future f : futures) {
          System.out.println(f.get(10000, TimeUnit.MILLISECONDS));
      }

  } catch (InterruptedException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
  } catch (ExecutionException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
  } catch (TimeoutException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
  } finally {
      exSvc.shutdownNow();
  }
}
}

import java.util.Random;
import java.util.concurrent.Callable;

import org.apache.log4j.Logger;


public class Task1 implements Callable {
private static Logger log = Logger.getLogger(Task1.class.getName());
private final static Random r = new Random();
private final int maxTimeout;
private final String name;

public Task1(final String name, final int maxTimeout) {
  super();  
  this.maxTimeout = maxTimeout;
  this.name = name;
  log.debug(name + " maxtimeout=" + maxTimeout);
}


public Object call() throws Exception {
  while (true) {
      log.debug("start call " + name);

      long start = System.currentTimeMillis();
      Thread.sleep(r.nextInt(maxTimeout));
      log.debug("end call, " + name + " duration=" + (System.currentTimeMillis() - start));
  }
    
}
}

Some sample output from this code:

17:47:56,798 DEBUG Task1:19 - Task0 maxtimeout=98
17:47:56,803 DEBUG Task1:19 - Task1 maxtimeout=12
17:47:56,803 DEBUG Task1:25 - start call Task0
17:47:56,804 DEBUG Task1:19 - Task2 maxtimeout=17
17:47:56,804 DEBUG Task1:25 - start call Task1
17:47:56,806 DEBUG Task1:19 - Task3 maxtimeout=51
17:47:56,807 DEBUG Task1:19 - Task4 maxtimeout=92
17:47:56,807 DEBUG Task1:19 - Task5 maxtimeout=47
17:47:56,808 DEBUG Task1:19 - Task6 maxtimeout=8
17:47:56,809 DEBUG Task1:19 - Task7 maxtimeout=87
17:47:56,809 DEBUG Task1:19 - Task8 maxtimeout=66
17:47:56,810 DEBUG Task1:19 - Task9 maxtimeout=39
17:47:56,808 DEBUG Task1:25 - start call Task2
17:47:56,876 DEBUG Task1:29 - end call, Task2 duration=65
17:47:56,878 DEBUG Task1:25 - start call Task2
17:47:57,030 DEBUG Task1:29 - end call, Task2 duration=151
17:47:57,031 DEBUG Task1:25 - start call Task2
17:47:57,174 DEBUG Task1:29 - end call, Task0 duration=369
17:47:57,175 DEBUG Task1:25 - start call Task0
17:47:57,296 DEBUG Task1:29 - end call, Task1 duration=490
17:47:57,297 DEBUG Task1:25 - start call Task1
17:47:57,323 DEBUG Task1:29 - end call, Task0 duration=146
17:47:57,324 DEBUG Task1:25 - start call Task0
17:47:57,390 DEBUG Task1:29 - end call, Task1 duration=91
17:47:57,391 DEBUG Task1:25 - start call Task1
17:47:57,448 DEBUG Task1:29 - end call, Task2 duration=415
17:47:57,449 DEBUG Task1:25 - start call Task2
17:47:57,549 DEBUG Task1:29 - end call, Task1 duration=157
17:47:57,550 DEBUG Task1:25 - start call Task1
17:47:57,729 DEBUG Task1:29 - end call, Task2 duration=278
17:47:57,730 DEBUG Task1:25 - start call Task2
17:47:57,802 DEBUG Task1:29 - end call, Task0 duration=477
17:47:57,803 DEBUG Task1:25 - start call Task0
17:47:57,886 DEBUG Task1:29 - end call, Task1 duration=334
17:47:57,887 DEBUG Task1:25 - start call Task1
17:47:58,048 DEBUG Task1:29 - end call, Task2 duration=316
17:47:58,048 DEBUG Task1:25 - start call Task2
17:47:58,068 DEBUG Task1:29 - end call, Task0 duration=264
17:47:58,069 DEBUG Task1:25 - start call Task0
17:47:58,087 DEBUG Task1:29 - end call, Task1 duration=199
17:47:58,092 DEBUG Task1:25 - start call Task1
17:47:58,098 DEBUG Task1:29 - end call, Task1 duration=5
17:47:58,098 DEBUG Task1:25 - start call Task1
17:47:58,099 DEBUG Task1:29 - end call, Task2 duration=48
17:47:58,099 DEBUG Task1:25 - start call Task2
17:47:58,115 DEBUG Task1:29 - end call, Task1 duration=17
17:47:58,115 DEBUG Task1:25 - start call Task1
17:47:58,245 DEBUG Task1:29 - end call, Task2 duration=146
17:47:58,245 DEBUG Task1:25 - start call Task2
17:47:58,339 DEBUG Task1:29 - end call, Task0 duration=267
17:47:58,340 DEBUG Task1:25 - start call Task0
17:47:58,398 DEBUG Task1:29 - end call, Task0 duration=57
17:47:58,399 DEBUG Task1:25 - start call Task0
17:47:58,412 DEBUG Task1:29 - end call, Task0 duration=12
17:47:58,412 DEBUG Task1:25 - start call Task0
17:47:58,542 DEBUG Task1:29 - end call, Task1 duration=427
17:47:58,543 DEBUG Task1:25 - start call Task1
17:47:58,573 DEBUG Task1:29 - end call, Task2 duration=326
17:47:58,574 DEBUG Task1:25 - start call Task2
17:47:58,741 DEBUG Task1:29 - end call, Task1 duration=197
17:47:58,742 DEBUG Task1:25 - start call Task1
17:47:58,783 DEBUG Task1:29 - end call, Task0 duration=369
17:47:58,784 DEBUG Task1:25 - start call Task0
17:47:58,835 DEBUG Task1:29 - end call, Task2 duration=260
17:47:58,835 DEBUG Task1:25 - start call Task2
17:47:58,873 DEBUG Task1:29 - end call, Task0 duration=89
17:47:58,873 DEBUG Task1:25 - start call Task0
17:47:58,923 DEBUG Task1:29 - end call, Task2 duration=87
17:47:58,923 DEBUG Task1:25 - start call Task2
17:47:58,942 DEBUG Task1:29 - end call, Task0 duration=67
17:47:58,942 DEBUG Task1:25 - start call Task0
17:47:59,084 DEBUG Task1:29 - end call, Task1 duration=342
17:47:59,084 DEBUG Task1:25 - start call Task1
17:47:59,091 DEBUG Task1:29 - end call, Task2 duration=166
17:47:59,091 DEBUG Task1:25 - start call Task2
17:47:59,115 DEBUG Task1:29 - end call, Task0 duration=172
17:47:59,115 DEBUG Task1:25 - start call Task0
17:47:59,348 DEBUG Task1:29 - end call, Task2 duration=257
17:47:59,348 DEBUG Task1:25 - start call Task2
17:47:59,473 DEBUG Task1:29 - end call, Task2 duration=124
17:47:59,474 DEBUG Task1:25 - start call Task2
17:47:59,505 DEBUG Task1:29 - end call, Task0 duration=389
17:47:59,505 DEBUG Task1:25 - start call Task0
17:47:59,506 DEBUG Task1:29 - end call, Task2 duration=31
17:47:59,507 DEBUG Task1:25 - start call Task2
17:47:59,534 DEBUG Task1:29 - end call, Task1 duration=450
17:47:59,535 DEBUG Task1:25 - start call Task1
17:47:59,579 DEBUG Task1:29 - end call, Task1 duration=44
17:47:59,580 DEBUG Task1:25 - start call Task1
17:47:59,629 DEBUG Task1:29 - end call, Task2 duration=121
17:47:59,630 DEBUG Task1:25 - start call Task2
17:47:59,667 DEBUG Task1:29 - end call, Task0 duration=160
17:47:59,668 DEBUG Task1:25 - start call Task0
17:47:59,747 DEBUG Task1:29 - end call, Task1 duration=167
17:47:59,748 DEBUG Task1:25 - start call Task1
17:47:59,852 DEBUG Task1:29 - end call, Task2 duration=221
17:47:59,852 DEBUG Task1:25 - start call Task2
17:48:00,153 DEBUG Task1:29 - end call, Task0 duration=485
17:48:00,154 DEBUG Task1:25 - start call Task0
17:48:00,156 DEBUG Task1:29 - end call, Task1 duration=408
17:48:00,156 DEBUG Task1:25 - start call Task1
17:48:00,185 DEBUG Task1:29 - end call, Task2 duration=332
17:48:00,185 DEBUG Task1:25 - start call Task2
17:48:00,316 DEBUG Task1:29 - end call, Task2 duration=130
17:48:00,317 DEBUG Task1:25 - start call Task2
17:48:00,435 DEBUG Task1:29 - end call, Task2 duration=117
17:48:00,436 DEBUG Task1:25 - start call Task2
17:48:00,542 DEBUG Task1:29 - end call, Task1 duration=385
17:48:00,543 DEBUG Task1:25 - start call Task1
17:48:00,629 DEBUG Task1:29 - end call, Task0 duration=474
17:48:00,631 DEBUG Task1:25 - start call Task0
17:48:00,720 DEBUG Task1:29 - end call, Task2 duration=283
17:48:00,721 DEBUG Task1:25 - start call Task2
17:48:00,809 DEBUG Task1:29 - end call, Task1 duration=266
17:48:00,810 DEBUG Task1:25 - start call Task1
17:48:00,886 DEBUG Task1:29 - end call, Task1 duration=74
17:48:00,887 DEBUG Task1:25 - start call Task1
17:48:00,908 DEBUG Task1:29 - end call, Task0 duration=276
17:48:00,909 DEBUG Task1:25 - start call Task0
17:48:01,023 DEBUG Task1:29 - end call, Task2 duration=302
17:48:01,023 DEBUG Task1:25 - start call Task2
17:48:01,140 DEBUG Task1:29 - end call, Task0 duration=230
17:48:01,140 DEBUG Task1:25 - start call Task0
17:48:01,349 DEBUG Task1:29 - end call, Task1 duration=462
17:48:01,350 DEBUG Task1:25 - start call Task1
17:48:01,378 DEBUG Task1:29 - end call, Task2 duration=355
17:48:01,378 DEBUG Task1:25 - start call Task2
17:48:01,518 DEBUG Task1:29 - end call, Task2 duration=139
17:48:01,518 DEBUG Task1:25 - start call Task2
17:48:01,599 DEBUG Task1:29 - end call, Task1 duration=249
17:48:01,599 DEBUG Task1:25 - start call Task1
17:48:01,603 DEBUG Task1:29 - end call, Task0 duration=461
17:48:01,603 DEBUG Task1:25 - start call Task0
17:48:01,939 DEBUG Task1:29 - end call, Task0 duration=335
17:48:01,940 DEBUG Task1:25 - start call Task0
17:48:01,997 DEBUG Task1:29 - end call, Task2 duration=476
17:48:01,998 DEBUG Task1:25 - start call Task2
17:48:02,014 DEBUG Task1:29 - end call, Task1 duration=414
17:48:02,015 DEBUG Task1:25 - start call Task1
17:48:02,146 DEBUG Task1:29 - end call, Task1 duration=131
17:48:02,147 DEBUG Task1:25 - start call Task1
17:48:02,261 DEBUG Task1:29 - end call, Task0 duration=320
17:48:02,262 DEBUG Task1:25 - start call Task0
17:48:02,343 DEBUG Task1:29 - end call, Task2 duration=342
17:48:02,343 DEBUG Task1:25 - start call Task2
17:48:02,378 DEBUG Task1:29 - end call, Task1 duration=230
17:48:02,378 DEBUG Task1:25 - start call Task1
17:48:02,520 DEBUG Task1:29 - end call, Task0 duration=257
17:48:02,520 DEBUG Task1:25 - start call Task0
17:48:02,597 DEBUG Task1:29 - end call, Task2 duration=253
17:48:02,598 DEBUG Task1:25 - start call Task2
17:48:02,734 DEBUG Task1:29 - end call, Task1 duration=355
17:48:02,735 DEBUG Task1:25 - start call Task1
17:48:02,786 DEBUG Task1:29 - end call, Task0 duration=265
17:48:02,787 DEBUG Task1:25 - start call Task0
17:48:02,959 DEBUG Task1:29 - end call, Task2 duration=361
17:48:02,959 DEBUG Task1:25 - start call Task2
17:48:03,017 DEBUG Task1:29 - end call, Task0 duration=228
17:48:03,018 DEBUG Task1:25 - start call Task0
17:48:03,027 DEBUG Task1:29 - end call, Task2 duration=66
17:48:03,030 DEBUG Task1:25 - start call Task2
17:48:03,214 DEBUG Task1:29 - end call, Task1 duration=478
17:48:03,215 DEBUG Task1:25 - start call Task1
17:48:03,299 DEBUG Task1:29 - end call, Task2 duration=268
17:48:03,300 DEBUG Task1:25 - start call Task2
17:48:03,305 DEBUG Task1:29 - end call, Task1 duration=90
17:48:03,306 DEBUG Task1:25 - start call Task1
17:48:03,324 DEBUG Task1:29 - end call, Task1 duration=17
17:48:03,325 DEBUG Task1:25 - start call Task1
17:48:03,345 DEBUG Task1:29 - end call, Task2 duration=44
17:48:03,345 DEBUG Task1:25 - start call Task2
17:48:03,456 DEBUG Task1:29 - end call, Task1 duration=130
17:48:03,456 DEBUG Task1:25 - start call Task1
17:48:03,484 DEBUG Task1:29 - end call, Task2 duration=138
17:48:03,484 DEBUG Task1:25 - start call Task2
17:48:03,491 DEBUG Task1:29 - end call, Task2 duration=6
17:48:03,492 DEBUG Task1:25 - start call Task2
17:48:03,496 DEBUG Task1:29 - end call, Task0 duration=477
17:48:03,496 DEBUG Task1:25 - start call Task0
17:48:03,696 DEBUG Task1:29 - end call, Task0 duration=199
17:48:03,696 DEBUG Task1:25 - start call Task0
17:48:03,796 DEBUG Task1:29 - end call, Task1 duration=340
17:48:03,809 DEBUG Task1:25 - start call Task1
17:48:03,841 DEBUG Task1:29 - end call, Task2 duration=348
17:48:03,841 DEBUG Task1:25 - start call Task2
17:48:03,845 DEBUG Task1:29 - end call, Task1 duration=35
17:48:03,847 DEBUG Task1:25 - start call Task1
17:48:03,876 DEBUG Task1:29 - end call, Task0 duration=179
17:48:03,876 DEBUG Task1:25 - start call Task0
17:48:03,952 DEBUG Task1:29 - end call, Task2 duration=111
17:48:03,952 DEBUG Task1:25 - start call Task2
17:48:03,999 DEBUG Task1:29 - end call, Task0 duration=123
17:48:04,000 DEBUG Task1:25 - start call Task0
17:48:04,220 DEBUG Task1:29 - end call, Task1 duration=372
17:48:04,221 DEBUG Task1:25 - start call Task1
17:48:04,348 DEBUG Task1:29 - end call, Task2 duration=395
17:48:04,349 DEBUG Task1:25 - start call Task2
17:48:04,385 DEBUG Task1:29 - end call, Task0 duration=384
17:48:04,392 DEBUG Task1:25 - start call Task0
17:48:04,633 DEBUG Task1:29 - end call, Task2 duration=284
17:48:04,634 DEBUG Task1:25 - start call Task2
17:48:04,678 DEBUG Task1:29 - end call, Task1 duration=457
17:48:04,679 DEBUG Task1:25 - start call Task1
17:48:04,704 DEBUG Task1:29 - end call, Task1 duration=25
17:48:04,704 DEBUG Task1:25 - start call Task1
17:48:04,766 DEBUG Task1:29 - end call, Task0 duration=373
17:48:04,766 DEBUG Task1:25 - start call Task0
17:48:04,912 DEBUG Task1:29 - end call, Task2 duration=277
17:48:04,912 DEBUG Task1:25 - start call Task2
17:48:04,928 DEBUG Task1:29 - end call, Task2 duration=15
17:48:04,928 DEBUG Task1:25 - start call Task2
17:48:04,943 DEBUG Task1:29 - end call, Task1 duration=238
17:48:04,944 DEBUG Task1:25 - start call Task1
17:48:05,072 DEBUG Task1:29 - end call, Task2 duration=143
17:48:05,072 DEBUG Task1:25 - start call Task2
17:48:05,132 DEBUG Task1:29 - end call, Task2 duration=59
17:48:05,132 DEBUG Task1:25 - start call Task2
17:48:05,135 DEBUG Task1:29 - end call, Task1 duration=191
17:48:05,135 DEBUG Task1:25 - start call Task1
17:48:05,168 DEBUG Task1:29 - end call, Task0 duration=401
17:48:05,168 DEBUG Task1:25 - start call Task0
17:48:05,239 DEBUG Task1:29 - end call, Task1 duration=103
17:48:05,240 DEBUG Task1:25 - start call Task1
17:48:05,357 DEBUG Task1:29 - end call, Task2 duration=224
17:48:05,357 DEBUG Task1:25 - start call Task2
17:48:05,526 DEBUG Task1:29 - end call, Task1 duration=286
17:48:05,526 DEBUG Task1:25 - start call Task1
17:48:05,555 DEBUG Task1:29 - end call, Task2 duration=197
17:48:05,555 DEBUG Task1:25 - start call Task2
17:48:05,600 DEBUG Task1:29 - end call, Task2 duration=44
17:48:05,600 DEBUG Task1:25 - start call Task2
17:48:05,616 DEBUG Task1:29 - end call, Task0 duration=447
17:48:05,616 DEBUG Task1:25 - start call Task0
17:48:05,650 DEBUG Task1:29 - end call, Task2 duration=49
17:48:05,652 DEBUG Task1:25 - start call Task2
17:48:05,772 DEBUG Task1:29 - end call, Task2 duration=120
17:48:05,773 DEBUG Task1:25 - start call Task2
17:48:05,775 DEBUG Task1:29 - end call, Task1 duration=248
17:48:05,775 DEBUG Task1:25 - start call Task1
17:48:05,782 DEBUG Task1:29 - end call, Task2 duration=9
17:48:05,782 DEBUG Task1:25 - start call Task2
17:48:05,865 DEBUG Task1:29 - end call, Task2 duration=83
17:48:05,866 DEBUG Task1:25 - start call Task2
17:48:05,870 DEBUG Task1:29 - end call, Task2 duration=4
17:48:05,870 DEBUG Task1:25 - start call Task2
17:48:05,878 DEBUG Task1:29 - end call, Task0 duration=261
17:48:05,879 DEBUG Task1:25 - start call Task0
17:48:06,105 DEBUG Task1:29 - end call, Task2 duration=234
17:48:06,105 DEBUG Task1:25 - start call Task2
17:48:06,239 DEBUG Task1:29 - end call, Task0 duration=360
17:48:06,239 DEBUG Task1:25 - start call Task0
17:48:06,258 DEBUG Task1:29 - end call, Task2 duration=152
17:48:06,258 DEBUG Task1:25 - start call Task2
17:48:06,260 DEBUG Task1:29 - end call, Task1 duration=485
17:48:06,260 DEBUG Task1:25 - start call Task1
17:48:06,452 DEBUG Task1:29 - end call, Task0 duration=213
17:48:06,452 DEBUG Task1:25 - start call Task0
17:48:06,586 DEBUG Task1:29 - end call, Task1 duration=326
17:48:06,586 DEBUG Task1:25 - start call Task1
17:48:06,619 DEBUG Task1:29 - end call, Task0 duration=166
17:48:06,620 DEBUG Task1:25 - start call Task0
17:48:06,672 DEBUG Task1:29 - end call, Task0 duration=52
17:48:06,672 DEBUG Task1:25 - start call Task0
17:48:06,747 DEBUG Task1:29 - end call, Task2 duration=489
17:48:06,747 DEBUG Task1:25 - start call Task2
17:48:06,792 DEBUG Task1:29 - end call, Task1 duration=205
17:48:06,792 DEBUG Task1:25 - start call Task1

* Updated 2009-11-29

Open Source will never get out of stock

It is like running out of light or wind. Software can be copied as many times as needed. Why haven't Microsoft and Apple understood this yet?

Some random examples of Windows 7 beeing out of stock
http://crave.cnet.co.uk/software/0,39029471,49303067,00.htm
http://www.digi.no/827047/windows-7-utsolgt-i-sverige

How can software be sold out? The only commodity invlolved is the software to be copied. To produce a physical copy a USB drive or recordable CD or DVD must be used in the process. But basically software can be copied between computers over the network. The required production equipment is computers, electric power and a network. These resources are abundant in the enterprise and in the average european/american home.

It seems like it is mostly pirates and the Open Source movement that have understood this distribution model. Commercial vendors seems to try to ignore or deny it. That is too bad, because they could profit from the extremely low distribution costs.

• Pirates exploits the low distribution costs to distribute goods illegally
  
• Open Source have relied on this distribution model for a long time. This has resulted in a very broad open source software usage, and distribution costs is approximately zero.

While Open Source and pirated digitial goods has very different usage profiles, they share the common properties of extremely effective and almost non-existing distribution costs. Open Source is mainly used by software developers, and consumers download software and digital media content for free.

In the last couple of years a new distribution patterns has gained foothold, beyond the most simple forms: SaaS. SaaS leverages software that can be used as-is instantly. Since there is no significant download and installation, the costs/effort required from the end user is even lower than with Open Source. For example the latest release of Ubuntu aims to provide a OS as a service by leveraging Amazon EC2 compatible images.

Software vendors, and other digital content providers, should free themselves from physical medias. Soon physical medias will be of interest of the entusiast. The rest of us will prefer instant gratification over the physical media.

Last weekend I was in a record shop where they of course played some music. I was very surprised when I saw the PC with Spotify was used instead of a CD player. That says it all about the state of physical distribution of digital goods.

Monocultures are unhealthy - even in software

Making the assumption that every project should be based on the same software stack, is just another variation over Silver Bullet.

In the software world the temptation to default to an architeture that has worked before is unhealthy. The result is almost always a constant struggle for the project to overcome limitations and find workarounds for architetural monstrosities.

Monocultures fosters very little learning in the organisation and leads to forcing inappropiate solutions to problems. According to epidemology theories monocultures is very vulnerable and is unable to evolve to tackle changing environment. Thinking of epidemology naturally leads to thinking about security issues as well. Monocultures is only able to resist specific types of threats, and given that the threats is certainly evolving at a blazing speed there is a obvious need to have variation.

In the longer term Monoculture will hinder innovation, and this can disastrous for an organization whose business model is e.g. developing and selling software.

But of course not all variations will be good, and should be dismissed. When experimenting with a technology new to the organization or a given project, do it in the small before going full scale. The opposite of genetical monoculture is diversity. Healthy software architecture in an organisation is probably best grown in an evolutionary way, allowing varations, promote the things that work well (but not restrict to only that). More important: things that do not work well must die.

All systems should be architected with the "right set" of technologies for the problem it is suppose to solve. One should start with what you know for sure, and make as few assumptions about the future as possible. The Cantara Software foundation has a wiki discussing these issues among other things, and I will try to post more in this topic there.

Yesterday I came across a blog post in Cutter Consortium, about uncertainty in a leadership perspective. This applies to software architects too. This should be in the architect's mind when evolving the software arcitecture for the organization.

DDD and Ubiquitous language

A recurring subject on several sessions on the 1st day of Javazone 09 was Domain Driven Design and Ubiquitous Language. I do agree with those believes it is important to establish a language that can be used by domain experts and developers to ensure consistency.

Just before I had to leave I attended the DDD panel discussion where it was discussed whether this language should be used in the code as well. Well that is completely natural for English speaking developers. For developers in countries where English is not a native language, this raises some questions:

• The code can not be maintained by people not familiar with the language the code is written in. This is very relevant in outsourcing.
  
• It will pose problems when companies from different countries merges and must update the code and integrate to reflect the new situation.
• Code that will or can be exposed as open source or in company partnerships will be of less value if it is written in non English

My personal opinion is that Norwegian (my native language) does not look well in code. It feels unnatural.

Is it responsible to write code in a non English language? I think this should be discussed with the stakeholders. On the other hand when there is a language impedance, how do you deal with it? Is it the developers responsibility?

I wanted to ask the panel these questions, but had to leave to catch a bus.

Updated: Corrected Domain Driven Development to Domain Driven design and added useful links.

Do you master social media?

This post is almost "off topic", and does not follow up on the series I announced in my previous post. Think I've forgot what I was thinking of then, but I'll surely get back to software and architecture later.

The background for this post is that I can not help myself getting frustrated over uninformed criticism over social media that a lot of old hat media seems to love these days.

Social media is, by many, viewed as a tornado of (useless/disconnected) information. On the other hand it is easy to find examples of constructive use of social media, that provides extremely useful information and knowledge to it's readers. For some examples look at my blogger following on the side of my blog. Sometimes I get irritated when people has strong negative opinions against social media (especially microblogging), stating it makes us dumber and unable to concentrate anymore. Well that is up to the individual to decide what to make of it, and decide to pick up a book instead of e.g. "hypertweeting".

I have stumbled upon extremely useful knowledge on Twitter, blogs and more professional web publishers, reading and discussing it through social media's unique features. My opinion is that social media is a whole lot better way of staying informed and increasing your knowledge than watching TV. Well at least a lot of what is distributed via TV is not exactly what tickles your brain to think.

To be really interesting in the social media space you will have to consume considerable amount of information, and possess the capability to transform and magnify this information to your readers. This requires deep concentration and is exactly how it always has been. Just think of ancient Greek philosophers. Some have large information processing capability and others not. Given that some has better capabilities than others, everyone can improve their skills. Social media provides the best training ground ever for improving the skill to express your knowledge. As Chris Anderson says in The Long Tail, the tools for expressing yourself in public writing has been democratized.

In my view expressing and discussing your knowledge is probably the most valuable thing a person can do for oneself and society in general. The ability to express your knowledge through writing is more important than ever. I am a strong believer of knowledge is key to improve individuals and mankind. Each and everyone should acquire the knowledge they need to make informed decisions for themselves or in a position as a decision maker for others such as leaders and politicians. With social media it is even possible to discuss things on a completely democratic way before things is realized.