Category Archives: Julia

Stats in bed, part 1: Ubuntu Touch

Round at the RSS Statistical Computing committee, we were having a chuckle at the prospect of a meeting about Stats In Bed. By which I mean analysis on mobile devices, phones and tablets (henceforth phablets), not some sort of raunchy performance indicator. This is something that has been nagging at the back of my mind for a while. Why, in this day and age, can’t we just run any software anywhere? Well, it’s because the major manufacturers have narrowed the scope for tinkering on phablets.

Although it seems silly, I know there is some time each day when I could do little useful tasks with a tablet that I can’t do with a laptop, even a very small lightweight one. One of the issues is time to start up or come out of hibernation in Windows, so I turned my attention to Linux, and in particular Ubuntu Touch.

I had just acquired a Nexus 10 tablet (comes with Android installed) for this purpose, and was also a Linux noob, so I was stumbling about as I experienced the first rays of dawn, like Bertie Wooster after a particularly libatious evening at the Drones. In this post, I’ll describe my first line of attack, which didn’t prove successful, but could be at some point in the future when the software develops further. My goal was to be able to run R, Julia and ideally Stan (therefore C++ compilation).

Ubuntu Touch is a work in progress, a new version of Ubuntu Linux designed for touch screens. Converting an operating system to work with touch screens is not a trivial job, although Ubuntu seem to have set themselves a dangerously optimistic timescale for this collective effort. A Linux smartphone has been waved around, and the USP is supposedly the goal of using the phablet on the move, then docking it in the office, where it carries on but scales the apps to work on your monitor*, keyboard and mouse. This is kind of cool, but I don’t feel a great need for it. I need a desktop that handles unpleasant computation quickly and without distracting itself, and frankly, although I groan every time my various versions of Windows grind to a halt to do… what? something obscure and, I suspect, unnecessary in the so-called background, I can’t live in the real world with other people and ditch Windows any time soon. If I was a hackathon hipster then I could, but I am somewhere in a hospital in the London suburbs, with employers and colleagues who require Windows.

* – my father-in-law actually used the term VDU the other day. I laughed, rather unfairly.

As Ubuntu Touch stands, it can be installed on a desktop for you to try apps out and contribute to the development, but installing on the phablet is rather more limited. Still, it’s not hard to do. The first step is to invalidate your warranty and root the phablet. Like all other first-timers, I had a vision of trying to sell it on for a couple of bucks after I had totally killed it, but it turns out to be much safer than that. If anything goes wrong, you can just replace the Android exactly as it came from the factory. You hold down all the buttons on the phablet together until the alarming sight appears of the little green robot on its back, with its guts opened up for some Android surgery. (I felt guilty at taking risks with the little guy.) You need a computer with Linux, you connect the phablet, and follow the (now deprecated) instructions from the link above. It’s amazingly simple. Then you have a Linux phablet, like this:


The username and password are phablet, which serves to remind you that this is a kind of preview version. And yet it works pretty well, despite some odd shadows lingering after windows have been swiped off to one side. The home screen looks like this:


and there are some other apps, none of which interested me:


But you can’t install new ones at this stage, even though it looks like you can. There is a dash search, and indeed a terminal app appears:


Which has tiny font, but whatever…


At this stage, I was getting pretty excited. From the terminal I could install R… maybe. It didn’t come with the Touch installation. It behaves like the desktop should when you issue an apt-get command to update your software, prior to installing the new R and Julia and so on.


But then fails. You see, a large part of the memory is not writeable, because this is an evaluation version. I just hope that is a temporary measure, and not yet another OS for phablets that blocks the user from poking around under the bonnet. Time will tell when (if?) the real Ubuntu Touch comes out.


So, that was an interesting diversion, but it was back to the drawing board and back to Android. One day it might be the choice to go with if you want to run R and friends on a phablet, but the small space given to the terminal wasn’t enough, and the keyboard was too chunky. Hopefully someone could write a full-screen terminal app, and a better keyboard, and hopefully the OS will allow it. Until then, you can read how I made friends with the little green Android dude in the second installment.


Filed under Julia, R

A room full of Julians

Despite winter rain, I was delighted to head uptown last week to Skills Matter on the old Goswell Road for the first ever London Julia meetup. The first thing I learnt was that Julia’s friends are called Julians.

If you don’t know it yet, Julia is a pretty new (v 0.3 is current) programming language for fast numerical computing. Everything is designed from the ground up for speed, by some very clever people. They claim speeds consistently close to compiled machine code, which is generally the upper limit, like the speed of light. But a few facts make it potentially Revolutionary Computing: you don’t have to compile it before running, you can mess about in a command-line interface to learn it, it’s free and open source, you can directly call C functions from inside normal Julia code – and vice versa, and the syntax is LISP-ish and light as eiderdown (there are some nice comparative examples of this on the homepage).

Arise, ye Julians

Arise, ye Julians

The focus was on getting started, and the room was packed. Personally, I spent some time playing with it last year and then let it lapse, but now with v0.3 out there it seems to be time to get back up to speed.

For stats people, there are a few important packages to install: Distributions, Stats, DataFrames, HypothesisTests, and possibly Optim, MCMC, depending on your own interests. That’s all pretty straightforward, but when you start up Julia or load one of the packages like this:


it takes a noticeable while to get ready. This is an artefact of the just-in-time compiler and open source programming. Almost all of the packages and the standard library are written in Julia itself. When you first need it, it gets compiled, and after that it should be superfast. Apparently a package is on the way to supply a pre-compiled standard library, to increase startup speeds.

Here’s a little power simulation I tried out afterwards:

for (i in 1:100000)
 sig= pvalue(EqualVarianceTTest(xx,yy))<0.05 ? 1 : 0;
 nsig = nsig+sig;

This does 100,000 simulations of independent-samples t-tests with sample size 10 per group, means 140 and 135, and SD 15, and took 5.05 seconds on a teeny weeny Samsung N110 ‘netbook’ with 1.6GHz Atom CPU and 1GB RAM (not what you would normally use!) once the package was loaded.

In R, you could do this at least two ways. First a supposedly inefficient looped form:

for (i in 1:100000) {
 if(t.test(xx,yy)$p.value<0.05) {

Next, a supposedly more efficient vectorized form:

tp<-function(x) {


In fact, the first version was slightly quicker at 2 minutes 3 seconds, compared to 2 minutes 35. While we’re about it, let’s run it in Stata too:

clear all
timer on 1
set obs 10
local p = 0
gen x=.
gen y=.
forvalues i=1/1000 {
qui replace x=rnormal(140,15)
qui replace y=rnormal(135,15)
qui ttest x==y, unpaired
if r(p)<0.05 local p = `p'+1
dis `p'
timer off 1
timer list

That took 30 seconds so we’re looking at 50 minutes to do the whole 100,000 simulations, but Stata black belts would complain that the standard language is not the best tool for this sort of heavy duty number-crunching. I asked top clinical trial statistician Dan Bratton for some equivalent code in the highly optimised Mata language:

timer clear 1
timer on 1
reps = 100000
n = (10 \ 10)
m = (140 , 135)
s = (15 , 15)
pass = 0
for (i=1;i<=reps;i++) {
 X = rnormal(10,1,m,s)
mhat = mean(X)
 v = variance(X)
df = n[1]+n[2]-2
 t = (mhat[1]-mhat[2])/sqrt((1/n[1]+1/n[2])*((n[1]-1)*v[1,1]+(n[2]-1)*v[2,2])/df)
p = 2*ttail(df,t)
if (p<0.05) pass = pass+1
timer off 1
timer list 1

… which clocked in at 7 seconds. I’m not going to try anything more esoteric because I’m interested in the speed for those very pragmatic simulations such as sample size calculations, which the jobbing statistician must do quite often. (Actually, there is an adequate approximation formula for t-tests that means you would never do this simulation.)

That time difference surprised me, to say the least. It means that Julia is an option to take very seriously indeed for heavy-duty statistical calculations. It really isn’t hard to learn. However, I don’t know of any scientific papers published yet that used Julia instead of any more established software. Perhaps the version 0.x would worry editors and reviewers, but surely v1.0 is not far away now.


Filed under Julia, R, Stata