I have been hearing for a while the “mobile app development is a lottery” proclamation and largely agreeing with it. People telling me that making money on the app store is like having a hit song.

I couldn’t disagree with it, it seemed to make sense… until I plugged in my sorry excuse for brain.

It’s time to hit this messed up, insanely brain-dead FUD with the stupid stick

Given that they think at least 60% of apps don’t break even and there are over 500,000 apps in the store, this means circa 200,000 apps are breaking even or better.

So let’s think about this. Just shy of half of the “businesses” are doing good, and some of those are doing great. Where else can you say that? Did it apply to desktop software? No. Does it apply to internet app services? No. Does it apply to any category of business, really? If so please let me know where I can have close guaranteed 50% chance of break even or better in a business. Business is a about investment and risk and in reality a great many businesses struggle for a long time, and many lose huge amounts of investment (in time and money) even if they struggled for years.

Next up: a reality check. Of the 500,000 apps in the store I will posit that less than 5,000 of those (1% for the arithmetic challenged) are great apps worth paying for. You know the kind of stuff that is really good and you are happy to shell out for, and feel no “buyer’s remorse” after clicking BUY and finding it doesn’t really do what you want.

It is certainly not true that 40% of the apps in the store are worth what you pay for them. Therefore there’s a whole metric buttload of apps in that 200,000 that are making money but largely suck.

Final fact check: Who the hell is running a lottery where you have a 40% chance of winning? If so, please let me know where I can buy a fistful of tickets. “App development is like a lottery” is the biggest bullshit ever, if you engage your brain.

Here’s a top tip to app developers: If you’re not making money, your app is not good enough. Perhaps you should consider that making mobile apps is not a gold rush that you can make a quick buck in with any old app.

P.S. Apple would love to make the App Store contain only great apps. The fact that probably 99% are crappy must make them wince. They will love it if this “gold rush” ends and app submissions start to plateau and only people who make great apps like Instagram, Angry Birds, Paper by 53, GarageBand, OmniFocus, Tweetbot, Infinity Blade etc submit apps. The entire ecosystem would get much better. The sooner people realise writing an app is not a golden ticket the better

It is my day to day work to create free tools for web developers and help my clients build web applications, so please do not make the mistake of thinking I am waging some anti-Web war here. This is years of web development talking.

I was musing about the nature of the modern web-app stack the other day and suddenly it hit me.

I think I finally got to the root of what makes me feel so uneasy about using the web to develop applications (applications as opposed to pure document presentation which is what the web is pretty good for).

We have such amazing advanced tools now for web apps. You pick a powerful framework like Grails or Rails or Node, you use tools like Less, Moustache, Twitter Bootstrap, you write your JS in something like CoffeeScript or maybe even Objective-J etc.

The diagram above shows what a web app used to be, and really what it should be.

The reality of modern web apps is very, very different and looks something more like this:

What I realised is that we are now in some evil kind of place where we are using a great many code generators in an attempt to simplify our working lives. Why is this happening? Because it is too hard to make the kind of apps that people deserve with standard web tech. The standards always, always, always lag well behind real world needs.

So we may typically have at least 5 code generators in our app stacks now:

• A tool like Less or Sass to generate one or more CSS files from a CSS-like language that preserves our sanity. Don’t bother to tell me that CSS variables are coming soon. This outputs CSS files.
• A tool like CoffeeScript to make, in some peoples’ opinions, JavaScript more maintainable and productive – outputting JS
• Some kind of JS and HTML friendly templating tool like Moustache – generating JS and/or HTML output
• A view layer like GSP, JSP, HAML or whatever else you like – which is generating HTML output
• Often a view aggregation/decoration layer like Sitemesh – which is generating HTML output from multiple HTML inputs

This is not even the end of it. CSS and JS optimisers and bundlers are a form of code generation. CSS Sprite generators create images and/or CSS files. Some of these are code generators that process the output of other generators. Its like the dev toolchain from hell.

We’re not even talking here about client-side JS that generates markup or DOM nodes, or that your server-side app stack is likely generating byte code from your source code, and that itself runs inside a VM.

Look at this powerful stack of tools we have now, and teleport yourself back to the year 2000.

Someone says to you that they’ve got this great web app stack you can use and it needs at least 5 different code generation tools, and you’ll have to write in at least 4 but probably 5 or 6 languages. You. Have. Got. To. Be. Kidding. Me.

There’s nothing they could have shown you that would have persuaded you it was worth this crazy complexity or investment. What have we done? We’ve flogged this broken “solution” for making cross platform apps so hard and for so long that we cannot see the wood for the trees.

Obviously, in terms of what we can do with web apps this is indeed progress. But only in the context of a completely, utterly broken platform for making applications.

No wonder Flash was so popular.

But here is the question: How complex does this stack have to get for the perceived cost trade-off vs. native for mobile and/or desktop to fall down?

You need to factor in the type of web application – most are not complex, perhaps easier to implement with native UIs – and revenue generation models. Also the availability of people with the skills. At what point does it become easier to hire people with native skills than people with ‘hit the ground runnings’ skills in say 5 different languages.

Remember that the web is supposed to be our “single language” cross platform solution, even when single means a minimum of 4; app code e.g. PHP/Java/Groovy/Ruby/Python, and UI with CSS/JS/HTML. The fact we can’t do anything “good” with it these days without an extra handful of processing and languages on top is a pretty sad indictment.

In the old days when I’d trawled through the minutiae of the SMTP, NNTP, NTP, POP3, HTTP and MIME RFCs, the web was not touted as the solution to cross platform app development in any way whatsoever.

In fact the response of that time to the problems we have now would have been to create something new. Something fit for purpose. A real cross platform UI based-application RFC and new protocols and standards to hang off it. Something that didn’t require a zillion complex tools and dependencies.

Something more like a native environment. But that’s not possible because all the O/S vendors need to differentiate.

So it is because of them that we are stuck with this stack from hell – or we choose a native platform or two and go with it.

Even though it is a milestone release, it has very functional Configuration and Security APIs that are already in use in some production apps.

To pique your interest, here are a few of my favourite things available in this release of it:

1. the pluginConfig variable added to all artefacts declared in plugins. This receives the namespaced Config values for the plugin, tying into the cool new doWithConfigOptions hook that lets plugins declare the configuration values that are valid – as well as default values and validators. See the docs
2. The displayMessage method added to controllers and the g:displayMessage tag that renders the messages set by controllers. Along with their flash scope variants this eliminates more boiler plate from controllers, and provides a uniform mechanism for plugins and apps to use – your app views don’t need to know how a plugin passes a message to the view if the plugin uses these methods. See the docs
3. The securityIdentity variable added to all Controllers, Services, Domains and TagLibs to provide the identity of the current logged in user, independent of the security plugin your code is using. This opens up app-defined security integration to all plugins – and using something like this would make the recent dependency injection exploits less likely. No need to inject any services into your domains to get the current user. See the docs

There’s a lot more to come, we’re really excited about the M2 release which will expose the public API for Navigation and Events.

Note that if you want to use the security API now in your apps and plugins, you will need to implement the security bridge for your chosen security provider.

I wanted to explain this in a little more depth so that everybody knows how to tell whether or not their current systems are vulnerable to it and what this might mean. It is worth noting that new releases of Grails 1.3.x and 2.0.x have already been made by the team – upgrading to those now and redeploying your app will close this particular hole for you.

To know if you must take some rapid action to address this issue, some detail is required on the nuance of the problem.

In summary: if you have a command or domain object that receives property values from param binding that is not protected adequately by include/exclude lists, and that command or domain receives dependency injected beans from the application context, you may be at risk.

There is an extra dimension here that further narrows down the risk, more of that in a moment, but suffice to say there are apps out there right now that can effectively be taken out of service by a single malicious request – sometimes only from a registered user of the site, but not exclusively. So yes, this is serious and yes you do need to understand it and audit your code if you are not in a position to immediately upgrade Grails and redeploy your app.

On to the details. Say you have a command or domain object that needs to store the current user id as a property of the object before saving. To do this you may inject a service and call a method on the service during a persistence event. The injection is the part that matters:

class UserProfile {
def securityService
...
}

Now it should be pretty obvious by now that there is the opportunity here to trash the value of the securityService field if the request includes a parameter with the same value.

As the property is untyped, binding can stick anything it likes in there, so you can replace the injected service with a String from the request params.

That’s actually not so terrible. Usually if you use the service it will crash with a missing method error.

If the injected property is explicitly typed nothing will happen at all as binding won’t know how to convert a String to that type. However that won’t save you from the real threat.

The threat comes from the fact that Spring beans injected like this are often singletons – there is just one shared instance in the whole app – and that beans often inject into themselves other beans.

So now imagine that our securityService also injects into itself a cacheService:


class SecurityService {
def cacheService
...
}


Because this is injected untyped, binding can reach this cacheService singleton bean and trash its value. All you have to do is pass in a query parameter like securityService.cacheService=boom and your security service bean is now trashed as any access to the cache service will likely result in missing method or missing property errors – for all future requests until your app is restarted.

This should show that actually the problem is not that common in code, and requires knowledge of your application to some extent, which mitigates risk. However if you use plugins that are open source and people can reasonably guess you used certain plugins / probe for them, then your risks are higher.

There is a final nuance to this. If the security service has public typed properties that can be bound from String i.e. booleans, numbers, dates and perhaps more, those values can be trashed on the singleton security service also – or worse changed without your knowledge.

Again, in terms of open source plugins this opens up risk – if you inject beans from plugins that have such fields, attackers can find out about this and use it.

Luckily the scope of this is realistically likely to be small, but it is sufficiently nuanced that the only recommendations at this time can be:

1. read Jeff’s blog on binding safely
2. upgrade to one of the new grails releases and redeploy now
3. or remove all injected properties in commands and domains
4. or use includes/excludes in every single binding call you make

Remember that you may use plugin controllers or domains that are also susceptible. For example Weceem is currently vulnerable but only where users have content edit rights, and apps using Spring Security Core may be vulnerable as it is not uncommon to inject springSecurityService into domains and that service has a number of corruptible dependencies injected.

The quick fix is to use option 2. Do it.

Thanks to the SpringSource guys for working so hard to get fixes in place. It is heartening that such efforts are taken to respect such problems and deal with them correctly when, truth be told, they are just really annoying and time consuming.

Businesses can get that stuff, free for that last ~10 years or so, but in reality the quality is very variable whether they know it or not. Also, because it is free people typically do not have the high bar they should apply to something they purchase.

Furthermore, it is hard to really feel that you value something if it cost you nothing.

It is also pretty intuitive that people who get stuff free usually don’t want to pay for it later, especially if it is still available for free.

People all over the world make free stuff.

But surely businesses don’t really want to base their commercially important systems on stuff created by hobbyists to which they have no actual connection or provenance, do they? That wold be crazy right?

Of course a lot of free open source is created by people at work. These people are usually paid to work on stuff the company needs or sells – but sometimes they are paid by a company that produces open source and sells consulting and support. Sometimes they just do it under the radar.

Sometimes, just sometimes, some big organisation may bankroll/sponsor some free stuff for a while.

What is the end result here?

How does it affect the quality of the software we use, and what code is developed?

How does it affect our forward progress in this field of software development?

It certainly means most companies get to build their products without paying for a lot of the code they use. However the cost of this software (that they are not paying) split among all the people worldwide using it, would be miniscule.

By jumping on the no-cost open software model, we developers have basically stymied hardcore progress on quality globally.

This is because hobby projects must be sufficiently small  for hobbyists to manage at a high quality – or it becomes low quality, perhaps only later down the line. Companies the world over are using the hobbyist stuff that is between both ends of that spectrum – usually without even thinking about it.

Bigger projects that cannot be sustained by hobbyists or become commercially important to somebody, or an attractive bit of commercial bling, may survive through team acquisition / sponsorship and related commercials mentioned above.

But what does that mean?

It inevitably means the direction of those bigger projects follow a course commensurate with the commercial interests of the backers, whatever that may be. It often means increased quality and robustness, which is good.

However it also means that effectively the large high-quality successful projects that we all have access to for free are implicitly filtered and chosen by those backers.

And we really have no way to change this by choosing to pay for stuff to support developers to make it their full time occupation to develop, refine and support their users.

Why? Because someone else will always be doing it free and everyone’s become used to free.

Of course commercial software does not guarantee quality. But without financial returns you cannot get the dedicated minds of smart people to make “insanely great” code for you to use, on an ongoing basis. At some point it will end.

I miss the days when people would pay you ~$50 for a library and get support from you bundled in.

You could dedicate yourself to your users.

[Note that I make a good living from consulting, and I am floating the Grailsrocks support product too. What I'm saying is I'm not whining about how I can't make money - I care about quality, innovation and strong community]

There’s only one way to say this: what a load of bollocks.

If you don’t agree with the statement “the retina display on the new iPad is a game changer” you need to consider this:

1. When you cannot see the individual pixels, on a screen of this size, it will no longer seem like you are looking at a screen. This has a massive effect on the way the user feels and perceives the product and the software that runs on it. If you don’t believe this, throw away your laser printer and get a 100 dpi dot matrix from the 1980s. While you don’t normally think about it your brain and perception is aware of the tiny black grid separating the pixels and the “unnatural” jagged edges on things.
2. Once the public see this screen, all other tablet screens (and in turn laptop screens) will be judged by this standard. Where can competitors go to compete with this? Brighter? More saturation? More battery life? All of those are possible, but they are not differentiators – and nobody can compete with iPad battery life. And they have to be retina as well. Effectively this new screen will be THE screen. THE future of displays. There is no point increasing resolution any further, and resolution of a device no longer becomes important at retina ppi – all that matters is the physical size of the screen. And if you’re thinking about 7″ tablets, in this new market all that means is less space to display stuff.
3. This screen is by all accounts very challenging to manufacture and very expensive. Only Apple can achieve this right now, perhaps for a couple of years – they’re using their buying power and cash reserves to achieve the “impossible”. This probably puts all other tablet makers in the shitter with inferior screen resolution for the next couple of years.
4. Still to this day nobody can compete with iPad on price for the same features and quality. Now the gap between Apple and those tablets has got a lot bigger. It may make it impossible for others to catch them now.

Anyone I eventually managed to launch Grailsrocks. Please take a look. Some great value commercial support plans for my Grails plugins.

I also released a new plugin to accompany this, but not just for customers, called Rocks. Its pretty cool, and hints at some of the future ideas I have for the Grails development.

The great thing is that actually, with only a little knowledge, doing this stuff is really easy. The Arduino board designs and software are open source, which is a good thing. However as usual this means the docs can be problematic.

There’s a confusing array of Arduino hardware out there. The boards have the ATmega micro controller chips on them that you program, and lots of digital and analog input/outputs so you can control and measure things. There are boards with Ethernet, Wi-fi, USB and even LED matrix drivers.

Basic Arduino (UNO)

Using this Arduino which has just a USB port and the I/O connectors, plus a bread board and 4-digit 7-segment LED display from an Earthshine starter kit, I managed after digging out the spec sheet on the display, to create a counter from 0001 to 9999. This sounds trivial, but it has a few challenges:

1. The LED display is basically 4 x 8 LEDs (7 segment + decimal point)
2. There are only 12 pins on it. Huh?
3. If you don’t send power to one of the LEDs, it goes off.

The spec sheet showed that 4 pins control which of the 4 digits you are addressing – set them LOW (common cathode) to enable the corresponding digit.

Then you set some of the other 8 pins HIGH to represent the digit or symbol you want to show.

So to set all 4 digits without any shift register chips or other smarts to “remember” the state of segments for each digit, you have program the Arduino to continually loop through the 4 selector pins, set the segment pins HIGH, wait a few milliseconds, then set the selector pin for that digit HIGH (to turn it off), and then repeat the same process for the next digit’s selector pin. What happens is that each digit is illuminated for say 5ms and then it moves to the next one.

They are continually going on and off, but because you constantly loop at high speed over them, it appears to be a continuous display.

Much more fun – the Rainbowduino

The Rainbowduino is a self contained Arduino board and the latest v3.0 one has built in mini-USB. So its just like an Arduino UNO but has a bunch of other controllers to drive an 8×8 RGB LED matrix with constant current (which I think means when lots of LEDs are on, they stay the same brightness). CAUTION: the LEDs are also seemingly as bright as the sun and staring at them is likely to leave you with a disturbing 8×8 grid of dots burnt into your retina.

So I bought the Rainbowduino plus an 8×8 RGB LED Matrix. The Rainbowduino is built to have the matrix directly plugged into the top of it, which is very handy.

Rainbowduino board plus LED matrix mounted on top

There are some tricks to getting this working however:

1. Unlike the Arduino UNO, my Mac would not recognise the Rainbowduino when plugged in to USB. I had to install the “FTDI” drivers from here. Thanks to @andydavies for the tip there
2. The docs on Rainbowduino are sketchy, and seem to be mainly targeted at an LED 3D cube rather than the matrix, and using the Processing language (which is not what the Arduino IDE uses).
3. It turns our there are several options for controlling this matrix. You can flash your own Arduino “sketches” (programs) directly onto the Rainbowduino, or you can use the I2C serial protocol to drive the Rainbowduino from another Arduino – I am guessing this means the Rainbowduino ATMega has a custom boot loader that supports serial comms to drive the display. I may be wrong.

To solve (2) I had to find the driver code that would work with the Arduino IDE. The v3 Rainbowduino driver is here. If you save that to ~/Documents/Arduino/Libraries/Rainbowv3 or similar (for Mac OS X at least that is where it is) and restart Arduino IDE, you will see you can now insert it into your Arduino sketch using the “Import Library…” menu option.

You will then find it does not compile, complaining it cannot find “WProgram.h”. This is I think because the library is built to run with “Wiring” programming language. To fix this you need to edit the Rainbowduino.h file from the driver that you saved, and change the #include <WProgram.h> to <Arduino.h>.

After that you can use the driver, which is incredibly simple:

1. in setup() call Rb.init()
2. in loop() call some Rb methods such as: Rb.blankDisplay() and Rb.setPixelXY(x, y, rgbcolour)

Rainbowduino with diagonal spectrum

Why so dark? LEDs are too bright to photograph without a filter. I put lots of tissue paper over them.

So here’s an example to apply the colour spectrum diagonally to the entire grid in action:

Note that if you find your reds are not working you probably have a C typing problem, and need to explicitly cast your values to uint32_t to avoid the RR of your RRGGBB being lost to 16bit accuracy.

And seeing as most of my life is spent in Grails… this wouldn’t be complete without a video of the Grails cup shining in all its 8×8 glory.

And here’s the code, if you want some Grails glory yourself:

We started off trying to watch it on the tv via AirPlay to the apple tv, from the iPad. That one sentence tells you that apple are certain to raid the tv market. Remove two components from that list.

Anyway it was fail. Every three minutes or so the 4OD app stopped streaming to apple tv. BBC iPlayer in Safari streams fine so it’s either the 4od app not liking AirPlay or a new iOS 5 bug as we did this fine before 5.0.1

We resorted to watching on the MacBook pro as we had done in the past fine.

Harbinger of what was to follow – channel 4 site demanded I upgrade Flash first. Fine

We start watching. Subjected to 3 mandatory viewing ads at the start all is well for a while.

Another ad break, another 3 ads. Some repeated from before. Watch some more.

Oh noes! Pop up message saying there’s an error and the content can’t be played. Video stops. Hmm. Press play again. Starts from beginning.

Watch first three ads I enjoyed so much again. Unmissable, really.

Fast forward to next mandatory ad section to try to get to where we were.

Watch 3 more ads a second time. Its awesome.

Watch some more misfits at last.

Oh noes! Error message again. Video carries on though. Message goes away by itself after many seconds. Hooray! Watch some more.

Error message again! Video stops. Press play again. Yay! Carries on from 2 ad breaks back, but not from the start. We’re blessed! Skip to ad break before last error point. Bonus! Only see 1 repeat ad!

Oh noes! The missing 2 ads are undead and playing in the background somehow – audio is mixed with programme audio. Press space to try to pause to work out what is going on. Pressing space doesn’t pause. It opens a new browser window with an advert for Assassins Creed – one of the ads we were not currently viewing. Soon discover clicking anywhere on the player window launches last ad’s website. Just what I wanted. Especially 5 minutes after avidly watching an ad.

I have no other browser tabs or windows open with the video streams of the ads. Player has simply gone mad. We wait for ads we cannot see but can hear to finish.

Unpause the player, full screen it again, watch more misfits.

Oh noes! Another error message. Video carries on, message obscuring screen for minutes.

Bonus! Audio from ads we cannot see becomes mixed in again! Now I have mastered this streaming tv technology I know to pause the video, mute the audio, wait a few minutes, unmute, resume video. Perfect!

We carry on watching, we’re 38 minutes into a 45 minute programme – but its taken over an hour to get this far. The magic of tv is somewhat eroded. Only in part due to more ads than content.

Oh noes! A few minutes before the end another error message, video stops. Press play… back at the beginning! YES! I had forgotten those original ads at the start that were repeated throughout, and wanted to catch up on those again. Thank you Channel 4.

The final insult, though perhaps merciful: the laptop battery runs out and it all shuts down.

I pick up the iPad, open 4od app, press play on Misfits. Have to watch one short ad, then can instantly jump to the last few minutes and watch it flawlessly while holding the iPad aloft.

This experience speaks volumes about the state of current computing, tv, media, advertising and software development.

If it weren’t for the iOS AirPlay bug none of this would have happened.

Fear the Apple TV chaps.

Oh, and there must be a separate team doing 4OD for iPad, cos the experience is infinitely better than the web one. Far fewer ads. Rapid seeking. No upgrades. No Flash. Great content. Anywhere.

That’s all you need – happy consumers.

And believe it or not that’s what advertisers need.

I’ve been good so far. Here the NSFW summary: OH FOR FUCK’S SAKE WHAT ON EARTH DO THOSE WANKERS THINK THEY ARE DOING?

I’ve been picking mushrooms for about 15 years now, though not that regularly. We were blessed with finding a good amount of Ceps and a lot of Wood Blewit this time, along with some Blushing Wood Mushroom and some small Horse Mushroom. Just before cooking my wife asked me to confirm the young white Horse mushrooms were what I thought and I said yes of course. I’m experienced after all. I know how to identify Agaricus species.

I cooked some ceps, blewits, wood mushroom and the horse mushroom with some spaghetti, garlic and thyme for me, my wife and kids. It was well received by all.

After the meal I started to wonder. My wife’s questioning made me suddenly anxious as to whether my Horse Mushroom identification was correct. I thought: they were a bit small actually for horse mushroom, and the “cog” pattern on the ring wasn’t so well defined. They didn’t stain brightly yellow, I double checked.

I was feeling a bit tense, wondering if I felt truly sick. The abject fear that perhaps I had fed Death Caps or Destroying Angel to my entire family alone started making me feel ill. You simply cannot *feed* your children a mushroom that you cannot identify – this is a potentially lethal mistake. I went over the id of all the really poisonous white/yellow mushrooms again and my memory of how they looked (another foolish mistake – not keeping one back from cooking).

I ended up the evening pretty certain that what we had was indeed an Agaricus, the Wood mushroom Agaricus Silvicola – a fine eating mushroom at that. Not – thankfully due to my in-built “Agaricus identification skills” but sadly not through real conscious identification – a Destroying Angel.

I can only say how genuinely mortifying it is to be in this kind of situation – even if just through a bought of self-doubt – wondering if in 8-12 hours your entire family will start suffering the dreadful symptoms of often fatal Amanita poisoning that begins with diarrhoea and ends with kidney failure.

Moral of the story: even if you’re experienced at picking mushrooms don’t let excitement get the better of you – remember the perils of identifying the “small white mushroom”. In future I’ll be steering clear of them. I never want to feel that fear again.