I'm afraid the time has come when I have to admit that I won't be able to update this blog anymore. My final year at university is about to begin and I simply won't have time to write anything substantial here throughout that period, so for now I am putting the blog on hold.

But: this is certainly not the end of the road for InsideReality! I have already made progress this summer on a major revamp of the website. The site has gone through a complete remodelling a few times over the years, and another one has always been inevitable. Most of the existing content will remain under the new site, along with plenty of new stuff.

The blog will also probably continue. The format may be slightly different this time since technology and trends have moved on from where they were in 2004, when I started doing this.

I have other projects which are related to the new InsideReality, too, but I'm not letting out many details at the moment. None of this is likely to go live for at least 6 months, and may even have to wait for summer 2009, but I will gladly inform everybody who is interested via email when 'normal' services do resume. Until then, there is still the archive, and various other articles of interest on InsideReality, so do continue hanging around.

If you want to be included in the updates mailing list, then let me know through the comments here or email updates at insidereality dot net with "JOIN" in the subject (you can leave at any time by emailing that same address with "LEAVE" in the subject).

Thanks very much to all who have read, commented on, and supported this website over the years. Hopefully you won't be disappointed with what is coming next.

I'm building a personal server out of an old AMD Duron computer. Here are some experimental things I have planned for it:

1) Stream my music collection to any computer I happen to be using.

2) Run an SVN repository to keep hold of all my source code, so I can potentially work on projects from anywhere, and keep track of old versions of code.

3) Distributed filesystems: to synchronise my desktop settings and home directory across various machines. rsync seems to be the way to go with this, but I've heard of people keeping their whole home dir in SVN...

4) Automatically download backups from Inside Reality at a set interval.

5) Run eyeOS, a web-based 'operating system' which includes word processing and spreadsheet applications. Essentially the same features as Google apps and similar services, except that all the content is stored on my own machine... and no advertisements.

6) Apache + php + MySql for web development.
6.1) Tomcat to play a bit more with Java web services.
6.2) Play with Haskell and Lisp application servers - yes they do exist, and that is exceptionally cool.

7) Running long computational tasks to free up my laptop. Currently I can't actually think of anything I'd want to run that would take longer than about 5 minutes, though.

8) Attach a webcam and make it accessible (securely) over the Web. However driver issues might spoil that idea.

9) Participate in one of the various distributed computing project out there.

10) Keep track of my schedule, providing a calender interface over the Web and sending me email reminders. This requires finding a decent application to do it, or maybe writing my own. I've been trying out a program called BORG, however I'm not fond of the user interface. Any other suggestions are most welcome.

Just when you thought cows couldn't be any more awesome, new research has shown that they may have a sense of the Earth's magnetic poles, as they tend to prefer to face in a north-south direction.

And yes I fully admit that the quality of this blog is dropping. But it wasn't all that high to begin with, was it?

There seems to be a trend towards centralising larger amounts of personal data. For example right now you can use Google's services to store all your email, contacts, documents, newsgroup subscriptions, photos, and your calender.

These 'free' services are provided because they allow Google to sell targeted advertising placements to its real customers. But should we really trust Google with all that personal data? what if it were abused, or leaked, or what happens when the courts are able to force certain data to be disclosed?

This is one of several reasons I have for advocating more decentralisation, in order to counter the trend. I firmly believe that my data should remain on my computer alone, 'just in case'. I'm even going to move most of my email back to my webhosting provider and use Gmail less. Now of course, that may be less secure there than at Google (easier for some sysadmin to abuse and not get noticed), but at least it isn't being held indefinitely in a massive database, and sold on as a product to advertisers. More to the point, the messages will only stay on the server until next time I retrieve them through POP3 and delete the remote copies, meaning that the permanent copies reside on my computer alone, and placing me in control of them.

Likewise the data given away to social networking services is worrying. What's interesting is that people have started using these as an alternative to email, which is barely usable now thanks to spam. And the problem with that is that unlike email, these services are closed and proprietary, which would mean giving up control once more to large companies.

Before we had these, we had similar tools, like finger on Unix. Of course finger isn't searchable, which is one problem with decentralising: people like the convenience of being able to search for data, emails, people, etc.

Another aspect of this comes from DRM and similar efforts to control what can and can't be done with certain files. I'm not going to get into a long debate regarding music piracy, mostly because all of my music is paid for anyway. Right now there exist 3 digital copies of my whole collection, and I can make as many more as I like thanks to avoiding these sort of restrictions. I think that using open source software helps here, as DRM isn't exactly popular in the open source community.

Indeed I see Linux and other open source software as the perfect means of seizing back what is rightfully mine - my computer, and my data.

Throughout 2007 and 2008 I have been acquiring more and more taste for non-popular music, for want of a better description, or 'classical' in its more broad - and less correct - sense. I've always had a deep love for music, and after starting to listen to jazz a few years back, this was the inevitable next step; I'm working my way backwards through musical history, by the looks of it.

This all started in tandem with my attempts to start playing the piano again. I used to mess around on this instrument when I was younger, but never made a real attempt to be able to play it. The inspiration to do so came from listening to jazz, and also to certain rock music (Muse works in some lovely piano, some of which, as I was later to learn, being based on Rachmaninoff).

With this ambition under way I soon needed some real pieces to learn, and I started by attempting Erik Satie's Gnossiennes, which were the only suitable pieces I had heard at the time. In my simple and inaccurate personal taxonomy at the time, Satie was 'classical music'. As the piano study continued, I also explored some other music, mostly by browsing videos on Youtube. I focused on piano music, but curiosity would often lead me to some orchestral material as well.

Soon I had added Edvard Grieg, Sergei Rachmaninoff and Edward Elgar to my music collection. Yet it was only last week that I decided to learn something about the history behind this music (this merely being an interesting afterthought, as such learning is irrelevant to the enjoyment of the music itself). I knew from school music lessons that though it is often used as an umbrella term, the term 'classical' really refers to one of several periods in European art history. I had no idea where any composers fell in that scale, and so was amused to realise that most of this music I had been exploring is in fact not classical music at all: the four composers I've mentioned so far are all part of the romantic movement/era.

Romanticism was an artistic movement which began during the 18th century. It emphasised emotion and experience (of the art). Romantic music (as I understand) was a departure from the strong formal structure used in classical music. There was even some exchange with jazz near the end of the movement, with Rachmaninoff's 4th piano concerto including some jazz influences, and Rachmaninoff himself being a great fan of jazz pianist Art Tatum.

I've decided to follow the BBC Proms this summer, which is a great way for someone who knows very little of this music to hear a lot of it. I've already taken a liking to several more composers, some of whom I had never even heard of before, such as Sergei Prokofiev and Vaughan Williams. In addition to this there have been programmes for world music and English folk, and performances of recently composed works.

As for the piano, I've learnt some of Satie's Gnossiennes, but not all yet. I've been playing with a few other pieces but I've got a long way to go. When I can play jazz, I'll be happy.

I've finally got around to installing Fedora 9. The update install (from Fedora 7) didn't work, and I suppose I didn't really expect it to, so I did a fresh install in the end. Impressions so far:

Things which work (which didn't in fc7 on my laptop)

• Wireless (drivers seem a little buggy though)


• Audio input

Nice things

• Various system tools


• Network manager


• Filesystem encryption option in the installer


• SE Linux troubleshooter (but the popup messages are irritating)

Things which don't work

• The installer (Anaconda) would randomly crash during package installation. It eventually worked but only by not customizing and only installing those selected by default, and even then it took two attempts. This was very frustrating and not the first time I've had problems with Anaconda.


• Keyboard stops working after suspend.


• Package manager doesn't have 'Fedora disk' as a choice of software source. Am I the only one who prefers to try installing something from the disk before downloading it? especially when I have to manually install packages that I skipped to stop Anaconda crashing.

Of course, I've been a couple of versions behind, so I'm not even sure which things are new to FC9 and which have been around for longer.

Over the last couple of years I have started to realise (or learnt the hard way) a few things about the process of studying mathematics which have gone against the assumptions I held before starting university. Maths isn't my main subject of study, but it is a big part of it, and since I enjoy it, I also dedicate a fair bit of my own time studying it further.

First off, memorizing formulas is a waste of time. You may pass an exam that way, but it's far better in the long run to take the time to really understand the maths behind a formula. For example, I've been reviewing a lot of statistics recently, and for the first time I can recite the formulae for the binomial probability distribution. Not because I've memorised it after copying it out 1000 times, but because (also for the first time) I actually understand what it means and so can reconstruct it in my head easily. I was never very good at probability, and this is something I'm now trying to correct.

I think that distinction between rote memorization, and reconstruction, is important. For a lot of people it's difficult to memorize things in maths for long, and moreover, if the initial understanding isn't there to begin with, then it is simply a wasted effort.

For when you actually need a formula, there are whole books filled with them (as well as series expansions and the like). And the most common ones seem to be printed in the back of every textbook I've used.

Second, it seems that however much experience you have, stupid arithmetic and algebraic mistakes still happen. Yesterday I was doing exercises on differential equations, and got the wrong answer twice over before realising I had missed a subtraction.

Additionally, mental/paper arithmetic skills aren't as important as school teachers insist, although they do help to provide a stronger mental feel for mathematics, they can speed things up, and are handy for quickly seeing if a solution makes sense.

Third, what was once difficult will become trivially simple eventually. The more maths you learn and practice, the more your existing knowledge and skills strengthen. Very early in school I found it hard to learn all the steps for solving a quadratic equation, which seems about as simple as breathing now.

Finally, the textbooks sometimes get the answers wrong, but more often than not it's your mistake (as much as I like to try and blame it on the book...)

Does anyone else have something to add to the list?

So earlier today I needed to look up cumulative probabilities over a binomial distribution with n = 30... as you do. Problem was I couldn't find any tables online which went that far.

The obvious solution was to use a spreadsheet, which I did, and it even had the function in-built. But then I was curious as to how to actually write an algorithm for it. More specifically, how to write a fast algorithm to do it.

The binomial distribution is used in a situation in which there is some event which can have one of two outcomes. The event is usually termed a 'trial', and one of the outcomes is defined to be 'success' while the other is 'failure' (though the terminology shouldn't be taken too literally). A probability p is assigned to a successful outcome, and consequently, q = 1 - p is the probability of failure.

If a trial is repeated n times then we can ask the question "what is the probability of k successes out of those n trials?". To work this out, the following reasoning is used:

1. There is an event which has probability p of occurring. The chance of it happening k times in a row is p^k.
2. If that event does happen k times during n trials, then the probability that it doesn't happen in any of the remaining n-k trials is q^(n-k) where q = 1 - p.
3. During n trials there are many different ways of arranging the k successes. For example, they could all happen in a row, or they could happen alternately in-between failures. In total there are n! / (n-k)!k! ways/orderings in which k successes may be selected from n trials. We will write this as nCk (the binomial coefficient).

Putting that together we get:

Pr(K = k) = nCk * p^k * q^(n-k)

Now for a more interesting question: "what is the probability that there are at most k successes?".

The solution is to add together the probabilities of 0 successes, 1 success, 2 successes, and so on, up to k successes, and this is called the cumulative probability. Naturally it would be far too tedious to calculate by hand, and that is where the computer program comes in handy.

As for the algorithm, it turns out (as can be expected) that computing nCk for each probability is what causes the most problems.

The most elegant way, in my opinion, of finding nCk is to use the following recursive function, where binom(n, c) is the same as nCk.

binom(n, 0) = 1
binom(n, n) = 1
nCn = binom(n - 1, k - 1) + binom(n - 1, k)

Which is based on Pascal's triangle. But though this looks very nice, it also becomes very slow for a computer as n - k gets larger.

Instead the solution used here is non-recursive and works by repeatedly doing a multiplication followed by a division. This is not only much faster than a recursive solution but avoids overflow by keeping the intermediate result small.

The source code is available here.

Following the release of Firefox 3, Microsoft sent Mozilla a cake to congratulate them. Isn't that nice of them? I'm pretty sure Microsoft didn't see any Firefox cake after IE 7 shipped.

I suppose it beats a giant IE logo on your lawn.

...having solved 4 out of 196 problems on Project Euler.

Yeah that's right. Within hours of finishing my last exam (algorithm design and analysis), I had to find something to occupy the empty space in my mind left by having no more studying to do. So I started on Project Euler, which is a series of maths-based programming challenges of varying difficulty.

From the website:

"Project Euler is a series of challenging mathematical/computer programming problems that will require more than just mathematical insights to solve. Although mathematics will help you arrive at elegant and efficient methods, the use of a computer and programming skills will be required to solve most problems."

Creating an account allows you to keep track of your own score, check the correctness of your solutions, see other people's solutions and access a PDF file with a deeper exposition of each problem (the latter two only being accessible after you submit a correct solution), although there's nothing to stop you viewing and solving the problems without having an account. Currently mine tells me I'm "2% genius". Seems I've got a long way to go - but then again, university doesn't start again until October...

The problems are listed here: http://projecteuler.net/index.php?section=problems

More exams on the way so I won't be posting anything new here until June. I will also be in Milton Keynes in June, then back in Newcastle by July.

In other things, Fedora 9 is released.

It [should] go without saying that the world we experience has some correspondance with the world we really inhabit... but not all that much. When we make vague references to reality, more often than not we are actually talking about our mental perceptions of it.

Naturally, it is not as though we could ever escape our bodies and break past that veil of perception. Of course this is nothing new; people have been aware of these things for probably as long as they have engaged in any kind of introspection.

However, given that our views of reality are so tainted by dreams and imagination as well as instinct, expectation and prejudice, it can be of great fascination to explore the way in which another person perceives things.

There are fundamentals that are common to almost everyone's perception - e.g. most people will agree on quantities, shape, relative size and distance. These are uninteresting (at least in this context). Some things which people do not tend to agree on include meaning, purpose, utility, and aesthetic, and also colour, texture, sound, though the latter three are more down to biological differences in the sense organs.

One person cannot (yet) directly experience another's mental 'world', but we seem to have found many ways in which to communicate it regardless. I refer to art, which I feel is a very powerful medium for communicating aspects of one person's inner mental world to other people.

Surrealism is a word thrown around a lot without much meaning. Its actual meaning is very interesting and relevent to the above discussion. André Breton is generally credited as the founder of Surrealist art, and wrote the following:

"Psychic automatism in its pure state, by which one proposes to express -- verbally, by means of the written word, or in any other manner -- the actual functioning of thought. Dictated by the thought, in the absence of any control exercised by reason, exempt from any aesthetic or moral concern."

Psychic automatism is a kind of spontaneous self-expression, without regard to logic, conventions, aesthetics or morality. Whether that means behaving amorally is, I suppose, a matter of choice, but in a purely artistic sense there is surely no need for morals anyway.

The strictest definition of surrealist art, then, is any kind of art generated by psychic automatism - i.e. spontaneous and free. This includes automatic writing/drawing, and even improvised jazz. The idea has evolved and expanded quite a bit since its founding of course, but the purpose (as if surrealism should even have a purpose) is to coax out the contents of the mind, and record them through some external medium to be shared by others.

Another definition could be 'simulated insanity'.

Naturally most or all surrealist art ends up being meaningless to everyone except the creator. How you perceive and interpret it is just as subjective as how you perceive and interpret objective reality, making the whole exercise brilliantly futile.

Now go stroke a horse you lunatic.

I wrote recently about the stagnation of the 3port project, and said that most of the problems stem from working with an outdated code structure.

Well, it has been agreed (between me and the only other developer), that we will start again from scratch. Although some of the old code will still be reused. I am in the process of designing a completely object-oriented framework. Modules will then act as applications running on top of the framework. The modules will include article management, blogging and forums, and I would assume Mr M's image gallery as well.

The framework

The framework takes advantage of PHP 5's object oriented programming features, which though available in php 4, were severely limited back then. For this reason, 3port will no longer be compatible with php 4. Since the start of 2008, php 4 was no longer being developed, and given that 3port still won't be released for a while, it doesn't seem worth maintaining support for it at all.

The basic idea is to break the code into 3 areas: system, libraries, and modules. Libraries provide all the useful functionality and are themselves broken into two subgroups - core, and extra. The core libraries provide database access, sessions, caching, presentation (i.e. the user interface), user services, and support for modules.

Modules are, as mentioned earlier, like applications. Since this is a content management system, each module provides some sort of content to users. This could be displaying articles, or managing a forum.

The system code is just the part which ties it all together. It loads libraries and modules, and co-ordinates a page request from the start right up to generating the html for the finished page.

Use of design patterns

To take full advantage of php 5's oop features, several design patterns have been applied. Design patterns are particularly important in providing libraries with a consistent interface for use by module developers.

One problem we encountered in on the old code was that many classes really should only have one instance active at run-time. Now, all such classes will follow the singleton pattern, looking something like this:

class Singleton

 {

     private static $instance;

     private function __construct(){}

     public static function getInstance()

     {

         return isset(self::$instance) ? self::$instance : self::$instance = new self();

     }

 }

Many classes should not be instantiated at all. These are termed utility classes and will contain only static functions, using a private constructor to prevent instantiation.

Finally, the factory pattern should prove very useful. There are many cases where we want to provide several alternative implementations of something, and allow the user/administrator to choose which one to use. For example, we can provide multiple database drivers, rather than restricting administrators to mySql only.

This is achieved by having a general interface 'Database', with implementations such as 'MySqlDatabase' and 'PostgreSqlDatabase'. When the system wants a database connection, it calls a factory method, 'DatabaseFactory::getDatabaseDriver()', which provides an instance of whichever driver has been specified in the configuration file.

Development progress

I've started a development wiki with details of the framework design. It's still a work in progress of course, and it has a long way to go, but hopefully implementation can begin sometime in June.

A spectre haunts Newcastle University, the spectre of exams...

They begin quite early this year, in the last two weeks of May, so unfortunately that means I've now started proper revision and will be increasingly anti-social up to the end of exams.

Following that I'll probably be in Milton Keynes for a few weeks in June, but I will be coming back to Newcastle by July when I start my summer job at a financial software company.

I think I will have a week free at the end of September before uni starts again, but don't know if I will be going anywhere during that week.

Now, things to do for summer (while not working, which has most of my attention):

1) Finish 3port, followed by the Inside Reality redesign (if there's time)
2) Learn Erlang. This looks really interesting and I've been planning to try it out for some time now.
3) Hone my c/c++ skills. Any suggestions for small projects?
4) Maths: brush up on differential equations, and learn to handle partial differential equations (for a course next year).
5) Other projects including doing some code on a project of Tomzini's.

Things to read:

1) Finish the Structure and Interpretation of Computer Programs.
2) Get and read Design Patterns.
3) Some philosophy stuff I've had idling on my bookshelf for far too long.

For all you stalkers and other sad individuals who care, I'm finally done selecting my modules to study next year:

Operating systems - sounds easy, so it probably isn't
Understanding programming languages - on how languages are designed, their syntax and semantics. Also Cliff Jones has a Wikipedia page
Real-time programming in Java - I saw a practical for this last week and they were playing with robot arms. That's all the convincing I need. Apparently very difficult challenging, though.
Evolution of complex systems - I hope I'm not the only one doing this
Graphics - hard but fun
Advanced graphics and virtual environments - even harder, but probably still fun
Software architectures - I find this kind of thing a little boring at times, but it's worthwhile
Numerical and computational modelling - probably pretty hard considering it's taught by maths and not comp sci. I'll need to learn partial differential equations and Fortran this summer.

All in all next year is going to be intense, but being the final year, that's to be expected. Also my dissertation subject will probably involve cellular automata. I haven't got more specific than that yet.